In python a function is a first class object. A class can be called. So you can replace a function with a class. But can you make a function behave like a class? Can you add and remove attributes or call inner functions( then called methods) in a function?
I found a way to do this via code inspection.
import inspect
class AddOne(object):
"""class definition"""
def __init__(self, num):
self.num = num
def getResult(self):
"""
class method
"""
def addOneFunc(num):
"inner function"
return num + 1
return addOneFunc(self.num);
if __name__ == '__main__':
two = AddOne(1);
two_src = '\n'.join([line[4:] for line in inspect.getsource(AddOne.getResult).split('\n')])
one_src = '\n'.join([line[4:] for line in two_src.split('\n')
if line[:4] == ' ' and line[4:8] == ' ' or line[4:8] == 'def '])
one_co = compile(one_src, '<string>', 'exec')
exec one_co
print addOneFunc(5)
print addOneFunc.__doc__
But is there a way to access the local variables and functions defined in a class in a more direct way?
EDIT
The question is about how to access the inner structure of python to get a better understanding. Of course I wouldn't do this in normal programming. The question arose when we had a discussion about private variables in python. My opinion was this to be against the philosophy of the language. So someone came up with the example above. At the moment it seems he is right. You cannot access the function inside a function without the inspect module, rendering this function private. With co_varnames we are awfully close because we already have the name of the function. But where is the namespace dictionary to hold the name. If you try to use
getResult.__dict__
it is empty. What I like to have is an answer from python like
function addOneFunc at <0xXXXXXXXXX>
You can consider a function to be an instance of a class that only implements __call__, i.e.
def foo(bar):
return bar
is roughly equivalent to
class Foo(object):
def __call__(self, bar):
return bar
foo = Foo()
Function instances have a __dict__ attribute, so you can freely add new attributes to them.
Adding an attribute to a function can be used, for example, to implement a memoization decorator, which caches previous calls to a function:
def memo(f):
#functools.wraps(f)
def func(*args):
if args not in func.cache: # access attribute
func.cache[args] = f(*args)
return func.cache[args]
func.cache = {} # add attribute
return func
Note that this attribute can also be accessed inside the function, although it can't be defined until after the function.
You could therefore do something like:
>>> def foo(baz):
def multiply(x, n):
return x * n
return multiply(foo.bar(baz), foo.n)
>>> def bar(baz):
return baz
>>> foo.bar = bar
>>> foo.n = 2
>>> foo('baz')
'bazbaz'
>>> foo.bar = len
>>> foo('baz')
6
(although it's possible that nobody would thank you for it!)
Note, however, that multiply, which was not made an attribute of foo, is not accessible from outside the function:
>>> foo.multiply(1, 2)
Traceback (most recent call last):
File "<pyshell#20>", line 1, in <module>
foo.multiply(1, 2)
AttributeError: 'function' object has no attribute 'multiply'
The other question addresses exactly what you're trying to do:
>>> import inspect
>>> import new
>>> class AddOne(object):
"""Class definition."""
def __init__(self, num):
self.num = num
def getResult(self):
"""Class method."""
def addOneFunc(num):
"inner function"
return num + 1
return addOneFunc(self.num)
>>> two = AddOne(1)
>>> for c in two.getResult.func_code.co_consts:
if inspect.iscode(c):
print new.function(c, globals())
<function addOneFunc at 0x0321E930>
Not sure if the following is what you're thinking about, but you can do this:
>>> def f(x):
... print(x)
...
>>> f.a = 1
>>> f.a
1
>>> f(54)
54
>>> f.a = f
>>> f
<function f at 0x7fb03579b320>
>>> f.a
<function f at 0x7fb03579b320>
>>> f.a(2)
2
So you can assign attributes to a function, and those attributes can be variables or functions (note that f.a = f was chosen for simplicity; you can assign f.a to any function of course).
If you want to access the local variables inside the function, I think then it's more difficult, and you may indeed need to revert to introspection. The example below uses the func_code attribute:
>>> def f(x):
... a = 1
... return x * a
...
>>> f.func_code.co_nlocals
2
>>> f.func_code.co_varnames
('x', 'a')
>>> f.func_code.co_consts
(None, 1)
Related
I would like to make a deepcopy of a function in Python. The copy module is not helpful, according to the documentation, which says:
This module does not copy types like module, method, stack trace, stack frame, file,
socket, window, array, or any similar types. It does “copy” functions and classes (shallow
and deeply), by returning the original object unchanged; this is compatible with the way
these are treated by the pickle module.
My goal is to have two functions with the same implementation but with different docstrings.
def A():
"""A"""
pass
B = make_a_deepcopy_of(A)
B.__doc__ = """B"""
So how can this be done?
The FunctionType constructor is used to make a deep copy of a function.
import types
def copy_func(f, name=None):
return types.FunctionType(f.func_code, f.func_globals, name or f.func_name,
f.func_defaults, f.func_closure)
def A():
"""A"""
pass
B = copy_func(A, "B")
B.__doc__ = """B"""
My goal is to have two functions with the same implementation but with different docstrings.
Most users will do this, say the original function is in old_module.py:
def implementation(arg1, arg2):
"""this is a killer function"""
and in new_module.py
from old_module import implementation as _implementation
def implementation(arg1, arg2):
"""a different docstring"""
return _implementation(arg1, arg2)
This is the most straightforward way to reuse functionality. It is easy to read and understand the intent.
Nevertheless, perhaps you have a good reason for your main question:
How can I make a deepcopy of a function in Python?
To keep this compatible with Python 2 and 3, I recommend using the function's special __dunder__ attributes. For example:
import types
def copy_func(f, name=None):
'''
return a function with same code, globals, defaults, closure, and
name (or provide a new name)
'''
fn = types.FunctionType(f.__code__, f.__globals__, name or f.__name__,
f.__defaults__, f.__closure__)
# in case f was given attrs (note this dict is a shallow copy):
fn.__dict__.update(f.__dict__)
return fn
And here's an example usage:
def main():
from logging import getLogger as _getLogger # pyflakes:ignore, must copy
getLogger = copy_func(_getLogger)
getLogger.__doc__ += '\n This function is from the Std Lib logging module.\n '
assert getLogger.__doc__ is not _getLogger.__doc__
assert getLogger.__doc__ != _getLogger.__doc__
A commenter says:
This can’t work for built‑in functions
Well I wouldn't do this for a built-in function. I have very little reason to do this for functions written in pure Python, and my suspicion is that if you are doing this, you're probably doing something very wrong (though I could be wrong here).
If you want a function that does what a builtin function does, and reuses the implementation, like a copy would, then you should wrap the function with another function, e.g.:
_sum = sum
def sum(iterable, start=0):
"""sum function that works like the regular sum function, but noisy"""
print('calling the sum function')
return _sum(iterable, start)
from functools import partial
def a():
"""Returns 1"""
return 1
b = partial(a)
b.__doc__ = """Returns 1, OR DOES IT!"""
print help(a)
print help(b)
Wrap it as a partial?
def A():
"""A"""
pass
def B():
"""B"""
return A()
The others answers do not allow for serialization with pickle. Here a code that I am using to clone a function and allow for serialization for python3:
import pickle
import dill
import types
def foo():
print ('a')
oldCode=foo.__code__
name='IAmFooCopied'
newCode= types.CodeType(
oldCode.co_argcount, # integer
oldCode.co_kwonlyargcount, # integer
oldCode.co_nlocals, # integer
oldCode.co_stacksize, # integer
oldCode.co_flags, # integer
oldCode.co_code, # bytes
oldCode.co_consts, # tuple
oldCode.co_names, # tuple
oldCode.co_varnames, # tuple
oldCode.co_filename, # string
name, # string
oldCode.co_firstlineno, # integer
oldCode.co_lnotab, # bytes
oldCode.co_freevars, # tuple
oldCode.co_cellvars # tuple
)
IAmFooCopied=types.FunctionType(newCode, foo.__globals__, name,foo.__defaults__ , foo.__closure__)
IAmFooCopied.__qualname__= name
print ( 'printing foo and the copy', IAmFooCopied, foo )
print ( 'dill output: ', dill.dumps(IAmFooCopied ))
print ( 'pickle Output: ', pickle.dumps (IAmFooCopied) )
Output:
printing foo and the copy <function IAmFooCopied at 0x7f8a6a8159d8> <function foo at 0x7f8a6b5f5268>
dill output: b'\x80\x03cdill._dill\n_create_function\nq\x00(cdill._dill\n_load_type\nq\x01X\x08\x00\x00\x00CodeTypeq\x02\x85q\x03Rq\x04(K\x00K\x00K\x00K\x02KCC\x0ct\x00d\x01\x83\x01\x01\x00d\x00S\x00q\x05NX\x01\x00\x00\x00aq\x06\x86q\x07X\x05\x00\x00\x00printq\x08\x85q\t)X\x10\x00\x00\x00testCloneFunc.pyq\nX\x0c\x00\x00\x00IAmFooCopiedq\x0bK\x05C\x02\x00\x01q\x0c))tq\rRq\x0ec__builtin__\n__main__\nh\x0bNN}q\x0ftq\x10Rq\x11.'
pickle Output: b'\x80\x03c__main__\nIAmFooCopied\nq\x00.'
You may encounter problem with the qualname attribute if you try this snippet with class methods (I think pickle should fail to find your function). I never tried it, however it should be easily fixable. Just check the doc about qualname
It's quite easy to do using lambda and rest parameters:
def my_copy(f):
# Create a lambda that mimics f
g = lambda *args: f(*args)
# Add any properties of f
t = list(filter(lambda prop: not ("__" in prop),dir(f)))
i = 0
while i < len(t):
setattr(g,t[i],getattr(f,t[i]))
i += 1
return g
# Test
def sqr(x): return x*x
sqr.foo = 500
sqr_copy = my_copy(sqr)
print(sqr_copy(5)) # -> 25
print(sqr_copy(6)) # -> 36
print(sqr_copy.foo) # -> 500
print(sqr_copy == sqr) # -> False
Try it online!
put it in a function:
def makefunc( docstring ):
def f():
pass
f.__doc__ = docstring
return f
f = makefunc('I am f')
g = makefunc("I am f too")
Adjusted for python3
import types
def copy_func(f, name=None):
return types.FunctionType(f.__code__, f.__globals__, name or f.__name__,
f.__defaults__, f.__closure__)
def func1(x):
return 2*x
func2=copy_func(func1)
print(func2(7))
I've implemented a general-purpose function copy in haggis, a library which I wrote and maintain (available with pip but probably not conda). haggis.objects.copy_func makes a copy on which you can not only reassign the __doc__ attribute, but also modify the module and __globals__ attributes effectively.
from haggis.objects import copy_func
def a(*args, **kwargs):
"""A docstring"""
a2 = copy_func(a)
a2.__doc__ = """Another docstring"""
>>> a == a2
False
>>> a.__code__ == a2.__code__
True
>>> a.__doc__
'A docstring'
>>> a2.__doc__
'Another docstring'
I would like to write a decorator for a python class method that can determine if the method was called from a public context or private context. For example, given the following code
def public_check_decorator(f):
def wrapper(self):
if self.f is `called publicly`: # <-- how do I make this line work correctly?
print 'called publicly'
else:
print 'called privately'
return f(self)
return wrapper
class C(object):
#public_check_decorator
def public_method(self):
pass
def calls_public_method(self):
self.public_method()
runtime execution would ideally look something like this:
>>> c = C()
>>> c.public_method()
called publicly
>>> c.calls_public_method()
called privately
Is there any way to do this in python? That is, alter the line
if self.f is `called publicly`: # <-- how do I make this line work correctly?
to give the desired output?
Given the name of the package decides whether a function is being called from a private context or public one:
import inspect
import re
def run():
package_name = '/my_package/'
p = re.match(r'^.*' + package_name, inspect.stack()[0].filename).group()
is_private_call = any(re.match(p, frame.filename) is not None for frame in inspect.stack()[1:])
print(is_private_call)
Try running from within the package and then from outside the package!!!
see inspect.stack()
Some of this seems like trying to swim against the current of "python". Is that appropriate?
Do you know about the double-unscore standard? It makes methods "more private":
>>> class C(object):
... def __hide_me(self):
... return 11
... def public(self):
... return self.__hide_me()
...
>>> c = C()
>>> c.__hide_me()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'C' object has no attribute '__hide_me'
>>> c.public()
11
>>> c._C__hide_me()
11
>>>
Is that private enough? And using this technique is pythonic.
What is the best way to create a new member function of a class with function name contained as string? Also, this new function is merely as pass-through for another object(helper class), which has the same function name but with variable arguments. I use lambda to achieve this, but I don't know how to handle the scenario, where my pass-through wrapper would be more than one-statement (which is my requirement)
# This is a helper class
class Compensation:
def bonus(self):
return 10000
def salary(self):
# Do something
def stack(self):
# Do something
# This is a employer class
class employee:
def __init__(self):
self.compensation = Compensation()
# This is a wrapper that creates the function
def passThru(funcName):
fn = "employee."+funcName+"=" + "lambda self, *arg: self.compensation." + funcName +"(*arg)"
exec(fn)
fnNames = ["bonus", "salary", "stocks"]
for items in fnNames: passThru(items)
emp = employee()
emp.bonus() # returns 1000
All that trickery with exec gives me a headache ;-) I'm not exactly clear on what you want to do, but adding a new method with a name given by a string is really quite easy. For example,
class employee:
pass
# Some multiline-function.
def whatever(self, a, b):
c = a + b
return c
e = employee()
# Make `whatever` an `employee` method with name "add".
setattr(employee, "add", whatever)
print e.add(2, 9)
Whenever you're reaching for exec, you're probably missing a straightforward way.
EDIT: an oddity here is that if someone tries to display e.add, they'll get a string claiming its name is whatever. If that bothers you, you can add, e.g.,
whatever.__name__ = "add"
Fleshing it out
Is this closer to what you want? Note that #gnibbler suggested much the same, although more telegraphically:
class Compensation:
def bonus(self, a):
return 10000 + a
def salary(self):
return 20000
def stack(self, a=2, b=3):
return a+b
class employee:
def __init__(self):
self.comp = Compensation()
e = employee()
for name in "bonus", "salary", "stack":
def outer(name):
def f(self, *args, **kw):
return getattr(self.comp, name)(*args, **kw)
f.__name__ = name
return f
setattr(employee, name, outer(name))
print e.bonus(9)
print e.salary()
print e.stack(b="def", a="abc")
That displays:
10009
20000
abcdef
All that said, you might want to re-think your architecture. It's strained.
You want setattr. Let's say you have:
>>> inst = Foo(10)
>>> class Foo(object):
def __init__(self, x):
self.x = x
>>> inst = Foo(10)
>>> inst2 = Foo(50)
If you want to add a method to all instances of the class, then setattr on the class. This function will end up being an unbound method on the class, becoming bound in each instance, so it will take the self param:
>>> setattr(inst.__class__, "twice_x", lambda self: self.x * 2)
>>> inst.twice_x()
20
>>> inst2.twice_x()
100
If you want to add the function to just one instance of the class, then setattr on the instance itself. This will be a regular function which will not take the implicit self argument:
>>> setattr(inst, "thrice_x", lambda: inst.x * 3)
>>> inst.thrice_x()
30
>>> inst2.thrice_x()
Traceback (most recent call last):
File "<pyshell#16>", line 1, in <module>
inst2.thrice_x()
AttributeError: 'Foo' object has no attribute 'thrice_x'
You are looking for setattr/getattr.
for func_name in fnNames:
setattr(employee, func_name, (lambda self, *args:getattr(self.compensation, func_name)(*args)))
This still has a problem because you need the lambda function to be closed over func_name. While you could create a closure with another lambda, I'll pull it out into another function for readability
for func_name in fnNames:
def f(func_name): # close the lambda over "func_name"
return lambda self, *args:getattr(self.compensation, func_name)(*args)
setattr(employee, items, f(func_name))
In this piece of code:
def fa(a,b):
print a
print b
print " fa called"
class A:
#classmethod
def fa(a,b):
print a
print b
print " A.fa called"
class B:
def __init__(s,a):
s.a = a
obj1 = B(A.fa)
obj1.a("xxxx")
obj2 = B(fa)
obj2.a("xxxx")
Output:
__main__.A
xxxx
A.fa called
Traceback (most recent call last):
File "test.py", line 20, in <module>
obj2.a("xxxx")
TypeError: fa() takes exactly 2 arguments (1 given)
Why is the free method "fa" not receiving the "self" as a first parameter? The bound method A.fa behaves as expected.
The bound method A.fa receives A as the first parameter because it is a class method of A. No matter how you call this function it will always receive A as the first parameter.
The free method fa is not bound, so the only arguments it will receive are the ones that are passed in. No matter how you call this function, it will never receive parameters other than the ones that are passed in.
This behavior is different from a language like JavaScript, where how the method is called determines the context. In Python the implicit argument passing (similar to JavaScript context) is determined at the function definition time, and that binding or lack thereof will always be used for that function regardless of how it is called.
If you want to dynamically bind a free method you can do this using types.MethodType, for example:
def fa(x):
print x
class B: pass
>>> obj1 = B()
>>> obj1.a = MethodType(fa, obj1)
>>> obj1.a() # obj1.a behaves like an instance method bound to obj1
<__main__.B instance at 0x7f0589baf170>
>>> obj1.a2 = MethodType(fa, B)
>>> obj1.a2() # obj1.a2 acts like a class method bound to B
__main__.B
Because doing obj2.a = fa does not make a (fa) a method of obj2:
>>> class A(object):
... def meth(self, x, y):
... print x, y
...
>>>
>>> a = A()
>>>
>>> a.meth
<bound method A.meth of <__main__.A object at 0x10e281950>> # Method
>>>
>>> def fn(x, y):
... print x, y
...
>>>
>>> fn
<function fn at 0x10e287140>
>>> a.fn = fn
>>>
>>> a.fn
<function fn at 0x10e287140> # Not a method, still a function
Let's start with some code:
def func(*x):
print('func:', x)
class ABC:
def __init__(self, f):
self.f1 = f
def f2(*x):
print('f2:', x)
Now we do some tests:
>>> a = ABC(func)
>>> a.f1(10)
func: (10,)
>>> a.f2(10)
f2: (<__main__.ABC object at 0xb75381cc>, 10)
>>> a.f3 = func
>>> a.f3(10)
func: (10,)
>>> a.f1
<function func at 0xb74911ec>
>>> a.f2
<bound method ABC.f2 of <__main__.ABC object at 0xb75381cc>>
>>> a.f3
<function func at 0xb74911ec>
Note that func is a normal function and we are making it a method f1 of the class.
We can see that f2 is getting the class instance as the first argument, but f1 and f3 are not, even though all functions are called as class methods. We can also see that if we call a normal function as a method of a class, Python does not make a bound method from it.
So why is f1 or f3 NOT getting a class instance passed to it even when we are calling it as a method of a class? And also, how does Python know that we are calling an outer function as a method so that it should not pass an instance to it.
-- EDIT --
OK, so basically what I am doing wrong is that I am attaching the functions on the instance and NOT on the class object itself. These functions therefore simply become instance attributes. We can check this with:
>>> ABC.__dict__
... contents...
>>> a.__dict__
{'f1': <function func at 0xb74911ec>, 'f3': <function func at 0xb74911ec>}
Also note that this dict can not be assigned to:
>>> ABC.__dict__['f4'] = func
TypeError: 'dict_proxy' object does not support item assignment
You kind of partially answered your own question inspecting the object. In Python, objects behave like namespaces, so the first attribute points to a function and the second points to a method.
This is how you can add a method dynamically:
from types import MethodType
def func(*x):
print('func:', x)
class ABC:
def __init__(self, f):
self.f1 = MethodType(f, self, self.__class__)
def f2(*x):
print('f2:', x)
if __name__ == '__main__':
a = ABC(func)
print a.f1(10)
print a.f2(10)
a.f3 = MethodType(func, a, ABC)
print a.f3(10)
Note that it will bind the method to your instance, not to the base class. In order to monkeypatch the ABC class:
>>> ABC.f4 = MethodType(func, None, ABC)
>>> a.f4(1)
('func:', (<__main__.ABC instance at 0x02AA8AD0>, 1))
Monkeypatching is usually frowned upon in the Python circles, despite being popular in other dynamic languages (notably in Ruby when the language was younger).
If you ever resort to this powerful yet dangerous technique, my advice is:
never, ever override an existing class method. just don't.
That's because f1 and f3 are not class method they are just references to a global function defined in __main__:
In [5]: a.f1
Out[5]: <function __main__.func>
In [8]: a.f3
Out[8]: <function __main__.func>
In [9]: a.f2
Out[9]: <bound method ABC.f2 of <__main__.ABC instance at 0x8ac04ac>>
you can do something like this to make a global function a class method:
In [16]: class ABC:
def __init__(self,f):
ABC.f1=f
def f2(*x):
print('f2',x)
....:
In [17]: a=ABC(func)
In [18]: a.f1(10)
('func:', (<__main__.ABC instance at 0x8abb7ec>, 10))