We Keep Coding

Python overload operator = [duplicate]

Is there a magic method that can overload the assignment operator, like __assign__(self, new_value)?
I'd like to forbid a re-bind for an instance:
class Protect():
def __assign__(self, value):
raise Exception("This is an ex-parrot")
var = Protect() # once assigned...
var = 1 # this should raise Exception()
Is it possible? Is it insane? Should I be on medicine?

The way you describe it is absolutely not possible. Assignment to a name is a fundamental feature of Python and no hooks have been provided to change its behavior.
However, assignment to a member in a class instance can be controlled as you want, by overriding .__setattr__().
class MyClass(object):
def __init__(self, x):
self.x = x
self._locked = True
def __setattr__(self, name, value):
if self.__dict__.get("_locked", False) and name == "x":
raise AttributeError("MyClass does not allow assignment to .x member")
self.__dict__[name] = value
>>> m = MyClass(3)
>>> m.x
3
>>> m.x = 4
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 7, in __setattr__
AttributeError: MyClass does not allow assignment to .x member
Note that there is a member variable, _locked, that controls whether the assignment is permitted. You can unlock it to update the value.

No, as assignment is a language intrinsic which doesn't have a modification hook.

I don't think it's possible. The way I see it, assignment to a variable doesn't do anything to the object it previously referred to: it's just that the variable "points" to a different object now.
In [3]: class My():
...: def __init__(self, id):
...: self.id=id
...:
In [4]: a = My(1)
In [5]: b = a
In [6]: a = 1
In [7]: b
Out[7]: <__main__.My instance at 0xb689d14c>
In [8]: b.id
Out[8]: 1 # the object is unchanged!
However, you can mimic the desired behavior by creating a wrapper object with __setitem__() or __setattr__() methods that raise an exception, and keep the "unchangeable" stuff inside.

Inside a module, this is absolutely possible, via a bit of dark magic.
import sys
tst = sys.modules['tst']
class Protect():
def __assign__(self, value):
raise Exception("This is an ex-parrot")
var = Protect() # once assigned...
Module = type(tst)
class ProtectedModule(Module):
def __setattr__(self, attr, val):
exists = getattr(self, attr, None)
if exists is not None and hasattr(exists, '__assign__'):
exists.__assign__(val)
super().__setattr__(attr, val)
tst.__class__ = ProtectedModule
The above example assumes the code resides in a module named tst. You can do this in the repl by changing tst to __main__.
If you want to protect access through the local module, make all writes to it through tst.var = newval.

Using the top-level namespace, this is impossible. When you run
var = 1
It stores the key var and the value 1 in the global dictionary. It is roughly equivalent to calling globals().__setitem__('var', 1). The problem is that you cannot replace the global dictionary in a running script (you probably can by messing with the stack, but that is not a good idea). However you can execute code in a secondary namespace, and provide a custom dictionary for its globals.
class myglobals(dict):
def __setitem__(self, key, value):
if key=='val':
raise TypeError()
dict.__setitem__(self, key, value)
myg = myglobals()
dict.__setitem__(myg, 'val', 'protected')
import code
code.InteractiveConsole(locals=myg).interact()
That will fire up a REPL which almost operates normally, but refuses any attempts to set the variable val. You could also use execfile(filename, myg). Note this doesn't protect against malicious code.

I will burn in Python hell, but what's life without a little fun.
Important disclaimers:
I only provide this example for fun
I'm 100% sure I don't understand this well
It might not even be safe to do this, in any sense
I don't think this is practical
I don't think this is a good idea
I don't even want to seriously try to implement this
This doesn't work for jupyter (probably ipython too)*
Maybe you can't overload assignment, but you can (at least with Python ~3.9) achieve what you want even at the top-level namespace. It will be hard doing it "properly" for all cases, but here's a small example by hacking audithooks:
import sys
import ast
import inspect
import dis
import types
def hook(name, tup):
if name == "exec" and tup:
if tup and isinstance(tup[0], types.CodeType):
# Probably only works for my example
code = tup[0]
# We want to parse that code and find if it "stores" a variable.
# The ops for the example code would look something like this:
# ['LOAD_CONST', '<0>', 'STORE_NAME', '<0>',
# 'LOAD_CONST', 'POP_TOP', 'RETURN_VALUE', '<0>']
store_instruction_arg = None
instructions = [dis.opname[op] for op in code.co_code]
# Track the index so we can find the '<NUM>' index into the names
for i, instruction in enumerate(instructions):
# You might need to implement more logic here
# or catch more cases
if instruction == "STORE_NAME":
# store_instruction_arg in our case is 0.
# This might be the wrong way to parse get this value,
# but oh well.
store_instruction_arg = code.co_code[i + 1]
break
if store_instruction_arg is not None:
# code.co_names here is: ('a',)
var_name = code.co_names[store_instruction_arg]
# Check if the variable name has been previously defined.
# Will this work inside a function? a class? another
# module? Well... :D
if var_name in globals():
raise Exception("Cannot re-assign variable")
# Magic
sys.addaudithook(hook)
And here's the example:
>>> a = "123"
>>> a = 123
Traceback (most recent call last):
File "<stdin>", line 21, in hook
Exception: Cannot re-assign variable
>>> a
'123'
*For Jupyter I found another way that looked a tiny bit cleaner because I parsed the AST instead of the code object:
import sys
import ast
def hook(name, tup):
if name == "compile" and tup:
ast_mod = tup[0]
if isinstance(ast_mod, ast.Module):
assign_token = None
for token in ast_mod.body:
if isinstance(token, ast.Assign):
target, value = token.targets[0], token.value
var_name = target.id
if var_name in globals():
raise Exception("Can't re-assign variable")
sys.addaudithook(hook)

No there isn't
Think about it, in your example you are rebinding the name var to a new value.
You aren't actually touching the instance of Protect.
If the name you wish to rebind is in fact a property of some other entity i.e
myobj.var then you can prevent assigning a value to the property/attribute of the entity.
But I assume thats not what you want from your example.

Yes, It's possible, you can handle __assign__ via modify ast.
pip install assign
Test with:
class T():
def __assign__(self, v):
print('called with %s' % v)
b = T()
c = b
You will get
>>> import magic
>>> import test
called with c
The project is at https://github.com/RyanKung/assign
And the simpler gist: https://gist.github.com/RyanKung/4830d6c8474e6bcefa4edd13f122b4df

Generally, the best approach I found is overriding __ilshift__ as a setter and __rlshift__ as a getter, being duplicated by the property decorator.
It is almost the last operator being resolved just (| & ^) and logical are lower.
It is rarely used (__lrshift__ is less, but it can be taken to account).
Within using of PyPi assign package only forward assignment can be controlled, so actual 'strength' of the operator is lower.
PyPi assign package example:
class Test:
def __init__(self, val, name):
self._val = val
self._name = name
self.named = False
def __assign__(self, other):
if hasattr(other, 'val'):
other = other.val
self.set(other)
return self
def __rassign__(self, other):
return self.get()
def set(self, val):
self._val = val
def get(self):
if self.named:
return self._name
return self._val
#property
def val(self):
return self._val
x = Test(1, 'x')
y = Test(2, 'y')
print('x.val =', x.val)
print('y.val =', y.val)
x = y
print('x.val =', x.val)
z: int = None
z = x
print('z =', z)
x = 3
y = x
print('y.val =', y.val)
y.val = 4
output:
x.val = 1
y.val = 2
x.val = 2
z = <__main__.Test object at 0x0000029209DFD978>
Traceback (most recent call last):
File "E:\packages\pyksp\pyksp\compiler2\simple_test2.py", line 44, in <module>
print('y.val =', y.val)
AttributeError: 'int' object has no attribute 'val'
The same with shift:
class Test:
def __init__(self, val, name):
self._val = val
self._name = name
self.named = False
def __ilshift__(self, other):
if hasattr(other, 'val'):
other = other.val
self.set(other)
return self
def __rlshift__(self, other):
return self.get()
def set(self, val):
self._val = val
def get(self):
if self.named:
return self._name
return self._val
#property
def val(self):
return self._val
x = Test(1, 'x')
y = Test(2, 'y')
print('x.val =', x.val)
print('y.val =', y.val)
x <<= y
print('x.val =', x.val)
z: int = None
z <<= x
print('z =', z)
x <<= 3
y <<= x
print('y.val =', y.val)
y.val = 4
output:
x.val = 1
y.val = 2
x.val = 2
z = 2
y.val = 3
Traceback (most recent call last):
File "E:\packages\pyksp\pyksp\compiler2\simple_test.py", line 45, in <module>
y.val = 4
AttributeError: can't set attribute
So <<= operator within getting value at a property is the much more visually clean solution and it is not attempting user to make some reflective mistakes like:
var1.val = 1
var2.val = 2
# if we have to check type of input
var1.val = var2
# but it could be accendently typed worse,
# skipping the type-check:
var1.val = var2.val
# or much more worse:
somevar = var1 + var2
var1 += var2
# sic!
var1 = var2

In the global namespace this is not possible, but you could take advantage of more advanced Python metaprogramming to prevent multiple instances of a the Protect object from being created. The Singleton pattern is good example of this.
In the case of a Singleton you would ensure that once instantiated, even if the original variable referencing the instance is reassigned, that the object would persist. Any subsequent instances would just return a reference to the same object.
Despite this pattern, you would never be able to prevent a global variable name itself from being reassigned.

As mentioned by other people, there is no way to do it directly. It can be overridden for class members though, which is good for many cases.
As Ryan Kung mentioned, the AST of a package can be instrumented so that all assignments can have a side effect if the class assigned implements specific method(s). Building on his work to handle object creation and attribute assignment cases, the modified code and a full description is available here:
https://github.com/patgolez10/assignhooks
The package can be installed as: pip3 install assignhooks
Example <testmod.py>:
class SampleClass():
name = None
def __assignpre__(self, lhs_name, rhs_name, rhs):
print('PRE: assigning %s = %s' % (lhs_name, rhs_name))
# modify rhs if needed before assignment
if rhs.name is None:
rhs.name = lhs_name
return rhs
def __assignpost__(self, lhs_name, rhs_name):
print('POST: lhs', self)
print('POST: assigning %s = %s' % (lhs_name, rhs_name))
def myfunc():
b = SampleClass()
c = b
print('b.name', b.name)
to instrument it, e.g. <test.py>
import assignhooks
assignhooks.instrument.start() # instrument from now on
import testmod
assignhooks.instrument.stop() # stop instrumenting
# ... other imports and code bellow ...
testmod.myfunc()
Will produce:
$ python3 ./test.py
POST: lhs <testmod.SampleClass object at 0x1041dcc70>
POST: assigning b = SampleClass
PRE: assigning c = b
POST: lhs <testmod.SampleClass object at 0x1041dcc70>
POST: assigning c = b
b.name b

Beginning Python 3.8, it is possible to hint that a value is read-only using typing.Final. What this means is that nothing changes at runtime, allowing anyone to change the value, but if you're using any linter that can read type-hints then it's going to warn the user if they attempt to assign it.
from typing import Final
x: Final[int] = 3
x = 5 # Cannot assign to final name "x" (mypy)
This makes for way cleaner code, but it puts full trust in the user to respect it at runtime, making no attempt to stop users from changing values.
Another common pattern is to expose functions instead of module constants, like sys.getrecursionlimit and sys.setrecursionlimit.
def get_x() -> int:
return 3
Although users can do module.get_x = my_get_x, there's an obvious attempt on the user's part to break it, which can't be fixed. In this way we can prevent people from "accidentally" changing values in our module with minimal complexity.

A ugly solution is to reassign on destructor. But it's no real overload assignment.
import copy
global a
class MyClass():
def __init__(self):
a = 1000
# ...
def __del__(self):
a = copy.copy(self)
a = MyClass()
a = 1

Is there a more 'pythonic' way to write this closure?

In JavaScript I might write a function like this:
function counter() {
foo = 0;
function increment() {
foo += 1
console.log(foo);
}
function printVal() {
console.log(foo);
}
return {
increment: increment,
printVal: printVal,
}
}
func = counter();
func.increment()
func.increment()
func.printVal()
I want to try to achieve a similar function using python. Particularly how I can access both inner functions via the outer function's return.
Here is a working, but funny looking, version in python:
def counter():
foo = {'bar': 0}
def increment():
foo['bar'] += 1
return foo['bar']
def printVal():
return foo['bar']
return {'increment': increment, 'printVal': printVal}
func = counter()
func['increment']()
func['printVal']()
Is there some sort of more elegant or 'pythonic' way of writing a closure like this?

Python is not as strong as other languages in closures. First, it supports only reading the "closed" variables, and second, the return statement makes it a bit more awkward.
On the other hand, it is very concise with classes, so if you want a data member with two functions, I would do it with a class:
class Counter:
def __init__(self, c=0):
self.count = c
def increment(self):
self.count += 1
def printVal(self):
return self.count
c = Counter()
c.increment()
print(c.printVal())
I dare say that in this case, this would be the pythonic way.
EDIT
After seeing your comment about tracking function calls, I add this. You can make it by a closure, like this:
# this is the tracked function
def add2(a, b):
return a + b
# this is the count tracker
def counterize(func):
c = [0]
def counter_func(*args, **kw):
c[0] += 1
counter_func.count = c[0]
return func(*args, **kw)
return counter_func
cadd2 = counterize(add2)
print(cadd2(1, 2))
print(cadd2(3, 4))
print('Called %s times' % cadd2.count)
>>>
3
7
Called 2 times
But this is not idiomatic in Python. Also keeping count inside the function object is a nice trick, but that's what it is. A trick. In LISP or Scala on the other hand a closure would be more natural, but then, I don't think it would be possible to keep count as a field, but, rather, return it together with the result.
I would say that in this case, again, the idiomatic Python code would be via classes, it is more comprehensible IMO and is of the same length of code:
class Counterize:
def __init__(self, func):
self.func = func
self.count = 0
def __call__(self, *args, **kwargs):
self.count += 1
return self.func(*args, **kwargs)
cadd2 = Counterize(add2)
print(cadd2(1, 2))
print(cadd2(3, 4))
print('Called %s times' % cadd2.count)
And the output would be the same. The purpose of __call__ is to allow treating the object as a function by means of calling with parenthesis.

Simply implement __getitem__
class Foo:
class_attributes = {'a': 3,
'b': 5}
def __getitem__(self, name):
return Foo.class_attributes[name]
f = Foo()
print f['a']
output:
3

I think that what you want to achieve is something like the following example :
def makeInc(x):
def inc(y):
# x is "attached" in the definition of inc
return y + x
return inc
incOne = makeInc(1)
incFive = makeInc(5)
incOne (5) # returns 6
incFive(5) # returns 10
You want the pythonic way of creating closures. So the above example demonstrate how to do that.

How to get builtin_function_or_method objects copied in ram without wrapping? [duplicate]

I would like to make a deepcopy of a function in Python. The copy module is not helpful, according to the documentation, which says:
This module does not copy types like module, method, stack trace, stack frame, file,
socket, window, array, or any similar types. It does “copy” functions and classes (shallow
and deeply), by returning the original object unchanged; this is compatible with the way
these are treated by the pickle module.
My goal is to have two functions with the same implementation but with different docstrings.
def A():
"""A"""
pass
B = make_a_deepcopy_of(A)
B.__doc__ = """B"""
So how can this be done?

The FunctionType constructor is used to make a deep copy of a function.
import types
def copy_func(f, name=None):
return types.FunctionType(f.func_code, f.func_globals, name or f.func_name,
f.func_defaults, f.func_closure)
def A():
"""A"""
pass
B = copy_func(A, "B")
B.__doc__ = """B"""

My goal is to have two functions with the same implementation but with different docstrings.
Most users will do this, say the original function is in old_module.py:
def implementation(arg1, arg2):
"""this is a killer function"""
and in new_module.py
from old_module import implementation as _implementation
def implementation(arg1, arg2):
"""a different docstring"""
return _implementation(arg1, arg2)
This is the most straightforward way to reuse functionality. It is easy to read and understand the intent.
Nevertheless, perhaps you have a good reason for your main question:
How can I make a deepcopy of a function in Python?
To keep this compatible with Python 2 and 3, I recommend using the function's special __dunder__ attributes. For example:
import types
def copy_func(f, name=None):
'''
return a function with same code, globals, defaults, closure, and
name (or provide a new name)
'''
fn = types.FunctionType(f.__code__, f.__globals__, name or f.__name__,
f.__defaults__, f.__closure__)
# in case f was given attrs (note this dict is a shallow copy):
fn.__dict__.update(f.__dict__)
return fn
And here's an example usage:
def main():
from logging import getLogger as _getLogger # pyflakes:ignore, must copy
getLogger = copy_func(_getLogger)
getLogger.__doc__ += '\n This function is from the Std Lib logging module.\n '
assert getLogger.__doc__ is not _getLogger.__doc__
assert getLogger.__doc__ != _getLogger.__doc__
A commenter says:
This can’t work for built‑in functions
Well I wouldn't do this for a built-in function. I have very little reason to do this for functions written in pure Python, and my suspicion is that if you are doing this, you're probably doing something very wrong (though I could be wrong here).
If you want a function that does what a builtin function does, and reuses the implementation, like a copy would, then you should wrap the function with another function, e.g.:
_sum = sum
def sum(iterable, start=0):
"""sum function that works like the regular sum function, but noisy"""
print('calling the sum function')
return _sum(iterable, start)

from functools import partial
def a():
"""Returns 1"""
return 1
b = partial(a)
b.__doc__ = """Returns 1, OR DOES IT!"""
print help(a)
print help(b)
Wrap it as a partial?

def A():
"""A"""
pass
def B():
"""B"""
return A()

The others answers do not allow for serialization with pickle. Here a code that I am using to clone a function and allow for serialization for python3:
import pickle
import dill
import types
def foo():
print ('a')
oldCode=foo.__code__
name='IAmFooCopied'
newCode= types.CodeType(
oldCode.co_argcount, # integer
oldCode.co_kwonlyargcount, # integer
oldCode.co_nlocals, # integer
oldCode.co_stacksize, # integer
oldCode.co_flags, # integer
oldCode.co_code, # bytes
oldCode.co_consts, # tuple
oldCode.co_names, # tuple
oldCode.co_varnames, # tuple
oldCode.co_filename, # string
name, # string
oldCode.co_firstlineno, # integer
oldCode.co_lnotab, # bytes
oldCode.co_freevars, # tuple
oldCode.co_cellvars # tuple
)
IAmFooCopied=types.FunctionType(newCode, foo.__globals__, name,foo.__defaults__ , foo.__closure__)
IAmFooCopied.__qualname__= name
print ( 'printing foo and the copy', IAmFooCopied, foo )
print ( 'dill output: ', dill.dumps(IAmFooCopied ))
print ( 'pickle Output: ', pickle.dumps (IAmFooCopied) )
Output:
printing foo and the copy <function IAmFooCopied at 0x7f8a6a8159d8> <function foo at 0x7f8a6b5f5268>
dill output: b'\x80\x03cdill._dill\n_create_function\nq\x00(cdill._dill\n_load_type\nq\x01X\x08\x00\x00\x00CodeTypeq\x02\x85q\x03Rq\x04(K\x00K\x00K\x00K\x02KCC\x0ct\x00d\x01\x83\x01\x01\x00d\x00S\x00q\x05NX\x01\x00\x00\x00aq\x06\x86q\x07X\x05\x00\x00\x00printq\x08\x85q\t)X\x10\x00\x00\x00testCloneFunc.pyq\nX\x0c\x00\x00\x00IAmFooCopiedq\x0bK\x05C\x02\x00\x01q\x0c))tq\rRq\x0ec__builtin__\n__main__\nh\x0bNN}q\x0ftq\x10Rq\x11.'
pickle Output: b'\x80\x03c__main__\nIAmFooCopied\nq\x00.'
You may encounter problem with the qualname attribute if you try this snippet with class methods (I think pickle should fail to find your function). I never tried it, however it should be easily fixable. Just check the doc about qualname

It's quite easy to do using lambda and rest parameters:
def my_copy(f):
# Create a lambda that mimics f
g = lambda *args: f(*args)
# Add any properties of f
t = list(filter(lambda prop: not ("__" in prop),dir(f)))
i = 0
while i < len(t):
setattr(g,t[i],getattr(f,t[i]))
i += 1
return g
# Test
def sqr(x): return x*x
sqr.foo = 500
sqr_copy = my_copy(sqr)
print(sqr_copy(5)) # -> 25
print(sqr_copy(6)) # -> 36
print(sqr_copy.foo) # -> 500
print(sqr_copy == sqr) # -> False
Try it online!

put it in a function:
def makefunc( docstring ):
def f():
pass
f.__doc__ = docstring
return f
f = makefunc('I am f')
g = makefunc("I am f too")

Adjusted for python3
import types
def copy_func(f, name=None):
return types.FunctionType(f.__code__, f.__globals__, name or f.__name__,
f.__defaults__, f.__closure__)
def func1(x):
return 2*x
func2=copy_func(func1)
print(func2(7))

I've implemented a general-purpose function copy in haggis, a library which I wrote and maintain (available with pip but probably not conda). haggis.objects.copy_func makes a copy on which you can not only reassign the __doc__ attribute, but also modify the module and __globals__ attributes effectively.
from haggis.objects import copy_func
def a(*args, **kwargs):
"""A docstring"""
a2 = copy_func(a)
a2.__doc__ = """Another docstring"""
>>> a == a2
False
>>> a.__code__ == a2.__code__
True
>>> a.__doc__
'A docstring'
>>> a2.__doc__
'Another docstring'

Can a function in python behave like a class?

In python a function is a first class object. A class can be called. So you can replace a function with a class. But can you make a function behave like a class? Can you add and remove attributes or call inner functions( then called methods) in a function?
I found a way to do this via code inspection.
import inspect
class AddOne(object):
"""class definition"""
def __init__(self, num):
self.num = num
def getResult(self):
"""
class method
"""
def addOneFunc(num):
"inner function"
return num + 1
return addOneFunc(self.num);
if __name__ == '__main__':
two = AddOne(1);
two_src = '\n'.join([line[4:] for line in inspect.getsource(AddOne.getResult).split('\n')])
one_src = '\n'.join([line[4:] for line in two_src.split('\n')
if line[:4] == ' ' and line[4:8] == ' ' or line[4:8] == 'def '])
one_co = compile(one_src, '<string>', 'exec')
exec one_co
print addOneFunc(5)
print addOneFunc.__doc__
But is there a way to access the local variables and functions defined in a class in a more direct way?
EDIT
The question is about how to access the inner structure of python to get a better understanding. Of course I wouldn't do this in normal programming. The question arose when we had a discussion about private variables in python. My opinion was this to be against the philosophy of the language. So someone came up with the example above. At the moment it seems he is right. You cannot access the function inside a function without the inspect module, rendering this function private. With co_varnames we are awfully close because we already have the name of the function. But where is the namespace dictionary to hold the name. If you try to use
getResult.__dict__
it is empty. What I like to have is an answer from python like
function addOneFunc at <0xXXXXXXXXX>

You can consider a function to be an instance of a class that only implements __call__, i.e.
def foo(bar):
return bar
is roughly equivalent to
class Foo(object):
def __call__(self, bar):
return bar
foo = Foo()
Function instances have a __dict__ attribute, so you can freely add new attributes to them.
Adding an attribute to a function can be used, for example, to implement a memoization decorator, which caches previous calls to a function:
def memo(f):
#functools.wraps(f)
def func(*args):
if args not in func.cache: # access attribute
func.cache[args] = f(*args)
return func.cache[args]
func.cache = {} # add attribute
return func
Note that this attribute can also be accessed inside the function, although it can't be defined until after the function.
You could therefore do something like:
>>> def foo(baz):
def multiply(x, n):
return x * n
return multiply(foo.bar(baz), foo.n)
>>> def bar(baz):
return baz
>>> foo.bar = bar
>>> foo.n = 2
>>> foo('baz')
'bazbaz'
>>> foo.bar = len
>>> foo('baz')
6
(although it's possible that nobody would thank you for it!)
Note, however, that multiply, which was not made an attribute of foo, is not accessible from outside the function:
>>> foo.multiply(1, 2)
Traceback (most recent call last):
File "<pyshell#20>", line 1, in <module>
foo.multiply(1, 2)
AttributeError: 'function' object has no attribute 'multiply'
The other question addresses exactly what you're trying to do:
>>> import inspect
>>> import new
>>> class AddOne(object):
"""Class definition."""
def __init__(self, num):
self.num = num
def getResult(self):
"""Class method."""
def addOneFunc(num):
"inner function"
return num + 1
return addOneFunc(self.num)
>>> two = AddOne(1)
>>> for c in two.getResult.func_code.co_consts:
if inspect.iscode(c):
print new.function(c, globals())
<function addOneFunc at 0x0321E930>

Not sure if the following is what you're thinking about, but you can do this:
>>> def f(x):
... print(x)
...
>>> f.a = 1
>>> f.a
1
>>> f(54)
54
>>> f.a = f
>>> f
<function f at 0x7fb03579b320>
>>> f.a
<function f at 0x7fb03579b320>
>>> f.a(2)
2
So you can assign attributes to a function, and those attributes can be variables or functions (note that f.a = f was chosen for simplicity; you can assign f.a to any function of course).
If you want to access the local variables inside the function, I think then it's more difficult, and you may indeed need to revert to introspection. The example below uses the func_code attribute:
>>> def f(x):
... a = 1
... return x * a
...
>>> f.func_code.co_nlocals
2
>>> f.func_code.co_varnames
('x', 'a')
>>> f.func_code.co_consts
(None, 1)

How to get self into a Python method without explicitly accepting it

I'm developing a documentation testing framework -- basically unit tests for PDFs. Tests are (decorated) methods of instances of classes defined by the framework, and these are located and instantiated at runtime and the methods are invoked to execute the tests.
My goal is to cut down on the amount of quirky Python syntax that the people who will write tests need to be concerned about, as these people may or may not be Python programmers, or even very much programmers at all. So I would like them to be able to write "def foo():" instead of "def foo(self):" for methods, but still be able to use "self" to access members.
In an ordinary program I would consider this a horrible idea, but in a domain-specific-languagey kind of program like this one, it seems worth a try.
I have successfully eliminated the self from the method signature by using a decorator (actually, since I am using a decorator already for the test cases, I would just roll it into that), but "self" does not then refer to anything in the test case method.
I have considered using a global for self, and even come up with an implementation that more or less works, but I'd rather pollute the smallest namespace possible, which is why I would prefer to inject the variable directly into the test case method's local namespace. Any thoughts?

My accepted answer to this question was pretty dumb but I was just starting out. Here's a much better way. This is only scantily tested but it's good for a demonstration of the proper way to do this thing which is improper to do. It works on 2.6.5 for sure. I haven't tested any other versions but no opcodes are hardcoded into it so it should be about as portable as most other 2.x code.
add_self can be applied as a decorator but that would defeat the purpose (why not just type 'self'?) It would be easy to adapt the metaclass from my other answer to apply this function instead.
import opcode
import types
def instructions(code):
"""Iterates over a code string yielding integer [op, arg] pairs
If the opcode does not take an argument, just put None in the second part
"""
code = map(ord, code)
i, L = 0, len(code)
extended_arg = 0
while i < L:
op = code[i]
i+= 1
if op < opcode.HAVE_ARGUMENT:
yield [op, None]
continue
oparg = code[i] + (code[i+1] << 8) + extended_arg
extended_arg = 0
i += 2
if op == opcode.EXTENDED_ARG:
extended_arg = oparg << 16
continue
yield [op, oparg]
def write_instruction(inst):
"""Takes an integer [op, arg] pair and returns a list of character bytecodes"""
op, oparg = inst
if oparg is None:
return [chr(op)]
elif oparg <= 65536L:
return [chr(op), chr(oparg & 255), chr((oparg >> 8) & 255)]
elif oparg <= 4294967296L:
# The argument is large enough to need 4 bytes and the EXTENDED_ARG opcode
return [chr(opcode.EXTENDED_ARG),
chr((oparg >> 16) & 255),
chr((oparg >> 24) & 255),
chr(op),
chr(oparg & 255),
chr((oparg >> 8) & 255)]
else:
raise ValueError("Invalid oparg: {0} is too large".format(oparg))
def add_self(f):
"""Add self to a method
Creates a new function by prepending the name 'self' to co_varnames, and
incrementing co_argcount and co_nlocals. Increase the index of all other locals
by 1 to compensate. Also removes 'self' from co_names and decrease the index of
all names that occur after it by 1. Finally, replace all occurrences of
`LOAD_GLOBAL i,j` that make reference to the old 'self' with 'LOAD_FAST 0,0'.
Essentially, just create a code object that is exactly the same but has one more
argument.
"""
code_obj = f.func_code
try:
self_index = code_obj.co_names.index('self')
except ValueError:
raise NotImplementedError("self is not a global")
# The arguments are just the first co_argcount co_varnames
varnames = ('self', ) + code_obj.co_varnames
names = tuple(name for name in code_obj.co_names if name != 'self')
code = []
for inst in instructions(code_obj.co_code):
op = inst[0]
if op in opcode.haslocal:
# The index is now one greater because we added 'self' at the head of
# the tuple
inst[1] += 1
elif op in opcode.hasname:
arg = inst[1]
if arg == self_index:
# This refers to the old global 'self'
if op == opcode.opmap['LOAD_GLOBAL']:
inst[0] = opcode.opmap['LOAD_FAST']
inst[1] = 0
else:
# If `self` is used as an attribute, real global, module
# name, module attribute, or gets looked at funny, bail out.
raise NotImplementedError("Abnormal use of self")
elif arg > self_index:
# This rewrites the index to account for the old global 'self'
# having been removed.
inst[1] -= 1
code += write_instruction(inst)
code = ''.join(code)
# type help(types.CodeType) at the interpreter prompt for this one
new_code_obj = types.CodeType(code_obj.co_argcount + 1,
code_obj.co_nlocals + 1,
code_obj.co_stacksize,
code_obj.co_flags,
code,
code_obj.co_consts,
names,
varnames,
'<OpcodeCity>',
code_obj.co_name,
code_obj.co_firstlineno,
code_obj.co_lnotab,
code_obj.co_freevars,
code_obj.co_cellvars)
# help(types.FunctionType)
return types.FunctionType(new_code_obj, f.func_globals)
class Test(object):
msg = 'Foo'
#add_self
def show(msg):
print self.msg + msg
t = Test()
t.show('Bar')

little upgrade for aaronasterling's solution( i haven't enough reputation to comment it ):
def wrap(f):
#functools.wraps(f)
def wrapper(self,*arg,**kw):
f.func_globals['self'] = self
return f(*arg,**kw)
return wrapper
but both this solutions will work unpredictable if f function will be called recursively for different instance, so you have to clone it like this:
import types
class wrap(object):
def __init__(self,func):
self.func = func
def __get__(self,obj,type):
new_globals = self.func.func_globals.copy()
new_globals['self'] = obj
return types.FunctionType(self.func.func_code,new_globals)
class C(object):
def __init__(self,word):
self.greeting = word
#wrap
def greet(name):
print(self.greeting+' , ' + name+ '!')
C('Hello').greet('kindall')

Here's a one line method decorator that seems to do the job without modifying any Special attributes of Callable types* marked Read-only:
# method decorator -- makes undeclared 'self' argument available to method
injectself = lambda f: lambda self: eval(f.func_code, dict(self=self))
class TestClass:
def __init__(self, thing):
self.attr = thing
#injectself
def method():
print 'in TestClass::method(): self.attr = %r' % self.attr
return 42
test = TestClass("attribute's value")
ret = test.method()
print 'return value:', ret
# output:
# in TestClass::method(): self.attr = "attribute's value"
# return value: 42
Note that unless you take precautions to prevent it, a side-effect of the eval() function may be it adding a few entries -- such as a reference to the __builtin__ module under the key __builtins__ -- automatically to the dict passed to it.
#kendall: Per your comment about how you're using this with methods being in container classes (but ignoring the injection of additional variables for the moment) -- is the following something like what you're doing? It's difficult for me to understand how things are split up between the framework and what the users write. It sounds like an interesting design pattern to me.
# method decorator -- makes undeclared 'self' argument available to method
injectself = lambda f: lambda self: eval(f.func_code, dict(self=self))
class methodclass:
def __call__():
print 'in methodclass::__call__(): self.attr = %r' % self.attr
return 42
class TestClass:
def __init__(self, thing):
self.attr = thing
method = injectself(methodclass.__call__)
test = TestClass("attribute's value")
ret = test.method()
print 'return value:', ret
# output
# in methodclass::__call__(): self.attr = "attribute's value"
# return value: 42

The trick is to add 'self' to f.func_globals. This works in python2.6. I really should get around to installing other versions to test stuff like this on. Sorry for the wall of code but I cover two cases: doing it with a metaclass and doing it with a decorator. For your usecase, I think the metaclass is better since the whole point of this exercise is to shield users from syntax.
import new, functools
class TestMeta(type):
def __new__(meta, classname, bases, classdict):
for item in classdict:
if hasattr(classdict[item], '__call__'):
classdict[item] = wrap(classdict[item])
return type.__new__(meta, classname, bases, classdict)
def wrap(f):
#functools.wraps(f)
def wrapper(self):
f.func_globals['self'] = self
return f()
return wrapper
def testdec(f):
#functools.wraps(f)
def wrapper():
return f()
return wrapper
class Test(object):
__metaclass__ = TestMeta
message = 'You can do anything in python'
def test():
print self.message
#testdec
def test2():
print self.message + ' but the wrapper funcion can\'t take a self argument either or you get a TypeError'
class Test2(object):
message = 'It also works as a decorator but (to me at least) feels better as a metaclass'
#wrap
def test():
print self.message
t = Test()
t2 = Test2()
t.test()
t.test2()
t2.test()

This might be a use case for decorators - you give them a small set of lego bricks to build functions with, and the complicated framework stuff is piped in via #testcase or somesuch.
Edit: You didn't post any code, so this is going to be sketchy, but they don't need to write methods. They can write ordinary functions without "self", and you could use decorators like in this example from the article I linked:
class myDecorator(object):
def __init__(self, f):
print "inside myDecorator.__init__()"
f() # Prove that function definition has completed
def __call__(self):
print "inside myDecorator.__call__()"
#myDecorator
def aFunction():
print "inside aFunction()"