Let's start with some code:
def func(*x):
print('func:', x)
class ABC:
def __init__(self, f):
self.f1 = f
def f2(*x):
print('f2:', x)
Now we do some tests:
>>> a = ABC(func)
>>> a.f1(10)
func: (10,)
>>> a.f2(10)
f2: (<__main__.ABC object at 0xb75381cc>, 10)
>>> a.f3 = func
>>> a.f3(10)
func: (10,)
>>> a.f1
<function func at 0xb74911ec>
>>> a.f2
<bound method ABC.f2 of <__main__.ABC object at 0xb75381cc>>
>>> a.f3
<function func at 0xb74911ec>
Note that func is a normal function and we are making it a method f1 of the class.
We can see that f2 is getting the class instance as the first argument, but f1 and f3 are not, even though all functions are called as class methods. We can also see that if we call a normal function as a method of a class, Python does not make a bound method from it.
So why is f1 or f3 NOT getting a class instance passed to it even when we are calling it as a method of a class? And also, how does Python know that we are calling an outer function as a method so that it should not pass an instance to it.
-- EDIT --
OK, so basically what I am doing wrong is that I am attaching the functions on the instance and NOT on the class object itself. These functions therefore simply become instance attributes. We can check this with:
>>> ABC.__dict__
... contents...
>>> a.__dict__
{'f1': <function func at 0xb74911ec>, 'f3': <function func at 0xb74911ec>}
Also note that this dict can not be assigned to:
>>> ABC.__dict__['f4'] = func
TypeError: 'dict_proxy' object does not support item assignment
You kind of partially answered your own question inspecting the object. In Python, objects behave like namespaces, so the first attribute points to a function and the second points to a method.
This is how you can add a method dynamically:
from types import MethodType
def func(*x):
print('func:', x)
class ABC:
def __init__(self, f):
self.f1 = MethodType(f, self, self.__class__)
def f2(*x):
print('f2:', x)
if __name__ == '__main__':
a = ABC(func)
print a.f1(10)
print a.f2(10)
a.f3 = MethodType(func, a, ABC)
print a.f3(10)
Note that it will bind the method to your instance, not to the base class. In order to monkeypatch the ABC class:
>>> ABC.f4 = MethodType(func, None, ABC)
>>> a.f4(1)
('func:', (<__main__.ABC instance at 0x02AA8AD0>, 1))
Monkeypatching is usually frowned upon in the Python circles, despite being popular in other dynamic languages (notably in Ruby when the language was younger).
If you ever resort to this powerful yet dangerous technique, my advice is:
never, ever override an existing class method. just don't.
That's because f1 and f3 are not class method they are just references to a global function defined in __main__:
In [5]: a.f1
Out[5]: <function __main__.func>
In [8]: a.f3
Out[8]: <function __main__.func>
In [9]: a.f2
Out[9]: <bound method ABC.f2 of <__main__.ABC instance at 0x8ac04ac>>
you can do something like this to make a global function a class method:
In [16]: class ABC:
def __init__(self,f):
ABC.f1=f
def f2(*x):
print('f2',x)
....:
In [17]: a=ABC(func)
In [18]: a.f1(10)
('func:', (<__main__.ABC instance at 0x8abb7ec>, 10))
Related
assume following class definition:
class A:
def f(self):
return 'this is f'
#staticmethod
def g():
return 'this is g'
a = A()
So f is a normal method and g is a static method.
Now, how can I check if the funcion objects a.f and a.g are static or not? Is there a "isstatic" funcion in Python?
I have to know this because I have lists containing many different function (method) objects, and to call them I have to know if they are expecting "self" as a parameter or not.
Lets experiment a bit:
>>> import types
>>> class A:
... def f(self):
... return 'this is f'
... #staticmethod
... def g():
... return 'this is g'
...
>>> a = A()
>>> a.f
<bound method A.f of <__main__.A instance at 0x800f21320>>
>>> a.g
<function g at 0x800eb28c0>
>>> isinstance(a.g, types.FunctionType)
True
>>> isinstance(a.f, types.FunctionType)
False
So it looks like you can use types.FunctionType to distinguish static methods.
Your approach seems a bit flawed to me, but you can check class attributes:
(in Python 2.7):
>>> type(A.f)
<type 'instancemethod'>
>>> type(A.g)
<type 'function'>
or instance attributes in Python 3.x
>>> a = A()
>>> type(a.f)
<type 'method'>
>>> type(a.g)
<type 'function'>
To supplement the answers here, in Python 3 the best way is like so:
import inspect
class Test:
#staticmethod
def test(): pass
isstatic = isinstance(inspect.getattr_static(Test, "test"), staticmethod)
We use getattr_static rather than getattr, since getattr will retrieve the bound method or function, not the staticmethod class object. You can do a similar check for classmethod types and property's (e.g. attributes defined using the #property decorator)
Note that even though it is a staticmethod, don't assume it was defined inside the class. The method source may have originated from another class. To get the true source, you can look at the underlying function's qualified name and module. For example:
class A:
#staticmethod:
def test(): pass
class B: pass
B.test = inspect.getattr_static(A, "test")
print("true source: ", B.test.__qualname__)
Technically, any method can be used as "static" methods, so long as they are called on the class itself, so just keep that in mind. For example, this will work perfectly fine:
class Test:
def test():
print("works!")
Test.test()
That example will not work with instances of Test, since the method will be bound to the instance and called as Test.test(self) instead.
Instance and class methods can be used as static methods as well in some cases, so long as the first arg is handled properly.
class Test:
def test(self):
print("works!")
Test.test(None)
Perhaps another rare case is a staticmethod that is also bound to a class or instance. For example:
class Test:
#classmethod
def test(cls): pass
Test.static_test = staticmethod(Test.test)
Though technically it is a staticmethod, it is really behaving like a classmethod. So in your introspection, you may consider checking the __self__ (recursively on __func__) to see if the method is bound to a class or instance.
I happens to have a module to solve this. And it's Python2/3 compatible solution. And it allows to test with method inherit from parent class.
Plus, this module can also test:
regular attribute
property style method
regular method
staticmethod
classmethod
For example:
class Base(object):
attribute = "attribute"
#property
def property_method(self):
return "property_method"
def regular_method(self):
return "regular_method"
#staticmethod
def static_method():
return "static_method"
#classmethod
def class_method(cls):
return "class_method"
class MyClass(Base):
pass
Here's the solution for staticmethod only. But I recommend to use the module posted here.
import inspect
def is_static_method(klass, attr, value=None):
"""Test if a value of a class is static method.
example::
class MyClass(object):
#staticmethod
def method():
...
:param klass: the class
:param attr: attribute name
:param value: attribute value
"""
if value is None:
value = getattr(klass, attr)
assert getattr(klass, attr) == value
for cls in inspect.getmro(klass):
if inspect.isroutine(value):
if attr in cls.__dict__:
bound_value = cls.__dict__[attr]
if isinstance(bound_value, staticmethod):
return True
return False
Why bother? You can just call g like you call f:
a = A()
a.f()
a.g()
In Python, if I define a simple class as follows:
>>> class myClass:
... z = zip('abc', (97, 98, 99))
... def meth(self):
... print(self)
...
>>>
and then enter the following at the prompt:
>>> myClass.meth
<function myClass.meth at 0x00639540>
>>> myClass.z
<zip object at 0x006432B0>
>>>
I find that the function object is presented by Python as belonging to the class (myClass.meth), but the attribute's string representation is not (its name is not preceded by myClass.) and is no different than if it were defined outside the class. What's the reasoning for this?
This appears to be a side effect of defining a function inside a class statement. Defining the method at the global scope and calling type explicitly doesn't add the class name to the function's name.
Consider the following code:
class myClass:
def meth(self):
print(self)
print("meth:", repr(meth))
x = lambda self: print(self)
print("x:", repr(x))
def mymeth(self):
print(self)
print("mymeth:", repr(mymeth))
C = type('C', (object,), {'mymeth': mymeth})
print("mymeth:", repr(C.mymeth))
whose output is
meth: <function myClass.meth at 0x1055c6b90>
x: <function myClass.<lambda> at 0x1055c6dd0>
mymeth: <function mymeth at 0x1055c6b00>
mymeth: <function mymeth at 0x1055c6b00>
Things to note:
Inside the class statement, the class name is added to the function name, whether you use def or a lambda expression. This indicates that the behavior is not part of the def statement. Also, this output is produced before the class object itself is actually created.
Outside the class statement, the call to type does not add the class name to the function name. This suggests that it's the class statement itself, not the act of creating a class, that modifies the function name.
As to why this is the case? Presumably, it was decided that it would be useful to add a "fully qualified" method name in the output of function.__repr__ for debugging purposes, but I'm just speculating.
z is just a name, the value it holds is the builtin zip class, which is not part of myClass.
It's because by calling myclass.meth, you are simply pointing to the memory location where the function is located.
But for myClass.z, you are just calling the contents returned by z.
Similar to if it were:
class myClass:
x = 2
z = zip('abc', (97, 98, 99))
...
Then myClass.x would be 2
SIDENOTE: In your code snippet, z is a part of myClass. For simplicity's sake, it is the same as if you'd define self.z = zip('abc', (97, 98, 99)) with in the constructor.
To get a clear understanding of why if really not like doing that, I'd check
Class vs Instance attributes
ADDED EXAMPLE:
Adding another example that might shine some light on the issue, consider this
def some_fn():
return 'hello world'
class A(object):
p = some_fn
Then
A.p
>> <function some_fn at 0x7f00fefbef80>
and NOT
>>> <function A.some_fn at ... >
In this piece of code:
def fa(a,b):
print a
print b
print " fa called"
class A:
#classmethod
def fa(a,b):
print a
print b
print " A.fa called"
class B:
def __init__(s,a):
s.a = a
obj1 = B(A.fa)
obj1.a("xxxx")
obj2 = B(fa)
obj2.a("xxxx")
Output:
__main__.A
xxxx
A.fa called
Traceback (most recent call last):
File "test.py", line 20, in <module>
obj2.a("xxxx")
TypeError: fa() takes exactly 2 arguments (1 given)
Why is the free method "fa" not receiving the "self" as a first parameter? The bound method A.fa behaves as expected.
The bound method A.fa receives A as the first parameter because it is a class method of A. No matter how you call this function it will always receive A as the first parameter.
The free method fa is not bound, so the only arguments it will receive are the ones that are passed in. No matter how you call this function, it will never receive parameters other than the ones that are passed in.
This behavior is different from a language like JavaScript, where how the method is called determines the context. In Python the implicit argument passing (similar to JavaScript context) is determined at the function definition time, and that binding or lack thereof will always be used for that function regardless of how it is called.
If you want to dynamically bind a free method you can do this using types.MethodType, for example:
def fa(x):
print x
class B: pass
>>> obj1 = B()
>>> obj1.a = MethodType(fa, obj1)
>>> obj1.a() # obj1.a behaves like an instance method bound to obj1
<__main__.B instance at 0x7f0589baf170>
>>> obj1.a2 = MethodType(fa, B)
>>> obj1.a2() # obj1.a2 acts like a class method bound to B
__main__.B
Because doing obj2.a = fa does not make a (fa) a method of obj2:
>>> class A(object):
... def meth(self, x, y):
... print x, y
...
>>>
>>> a = A()
>>>
>>> a.meth
<bound method A.meth of <__main__.A object at 0x10e281950>> # Method
>>>
>>> def fn(x, y):
... print x, y
...
>>>
>>> fn
<function fn at 0x10e287140>
>>> a.fn = fn
>>>
>>> a.fn
<function fn at 0x10e287140> # Not a method, still a function
Consider a simple function like
def increment(self):
self.count += 1
which is run through Cython and compiled into an extension module. Suppose now I'd like to make this function a method on a class. For example:
class Counter:
def __init__(self):
self.count = 0
from compiled_extension import increment
Counter.increment = increment
Now this will not work, as the calling convention at the C level will be broken. For example:
>>> c = Counter()
>>> c.increment()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: increment() takes exactly one argument (0 given)
But in Python 2, we can convert the function to an unbound method by doing:
Counter.increment = types.MethodType(increment, None, Counter)
How can I accomplish this same thing in Python 3?
One simple way is to use a slim wrapper:
from functools import wraps
def method_wraper(f):
def wrapper(*args, **kwargs):
return f(*args, **kwargs)
return wraps(f)(wrapper)
Counter.increment = method_wrapper(increment)
Is there a more efficient way to do it?
First thing is getting the names correctly:
>>> def increment(obj):
... obj.count += 1
...
>>> class A(object):
... def __init__(self):
... self.count = 0
...
>>> o = A()
>>> o.__init__
<bound method A.__init__ of <__main__.A object at 0x0000000002766EF0>>
>>> increment
<function increment at 0x00000000027797C8>
So proper names are functions and bound methods. Now you can look for how to Bind an Unbound Method and you will probably end up reading about descriptors:
In general, a descriptor is an object attribute with "binding
behavior", one whose attribute access has been overridden by methods
in the descriptor protocol. Those methods are __get__, __set__, and
__delete__. If any of those methods are defined for an object, it is said to be a descriptor.
You can easily transform function to method by just using different invocation of __get__
>>> increment.__get__(None, type(None))
<function increment at 0x00000000027797C8>
>>> increment.__get__(o, type(o))
<bound method A.increment of <__main__.A object at 0x00000000027669B0>>
And it works like a charm:
>>> o = A()
>>> increment.__get__(None, type(None))(o)
>>> o.count
1
>>> increment.__get__(o, type(o))()
>>> o.count
2
You can easily add these newly bounded methods to objects:
def increment(obj):
obj.count += 1
def addition(obj, number):
obj.count += number
class A(object):
def __init__(self):
self.count = 0
o = A()
o.inc = increment.__get__(o)
o.add = addition.__get__(o)
print(o.count) # 0
o.inc()
print(o.count) # 1
o.add(5)
print(o.count) # 6
Or create your own descriptor that will will convert function to bound method:
class BoundMethod(object):
def __init__(self, function):
self.function = function
def __get__(self, obj, objtype=None):
print('Getting', obj, objtype)
return self.function.__get__(obj, objtype)
class B(object):
def __init__(self):
self.count = 0
inc = BoundMethod(increment)
add = BoundMethod(addition)
o = B()
print(o.count) # 0
o.inc()
# Getting <__main__.B object at 0x0000000002677978> <class '__main__.B'>
print(o.count) # 1
o.add(5)
# Getting <__main__.B object at 0x0000000002677978> <class '__main__.B'>
print(o.count) # 6
And you also can see that this is nicely consistent with function/bound method principles:
Class dictionaries store methods as functions. In a class definition, methods are written using def and lambda, the usual tools for creating functions. The only difference from regular functions is that the first argument is reserved for the object instance. By Python convention, the instance reference is called self but may be called this or any other variable name.
To support method calls, functions include the __get__() method for binding methods during attribute access. This means that all functions are non-data descriptors which return bound or unbound methods depending whether they are invoked from an object or a class.
And functions becomes bound method during instance initialization:
>>> B.add
# Getting None <class '__main__.B'>
<function addition at 0x00000000025859C8>
>>> o.add
# Getting <__main__.B object at 0x00000000030B1128> <class '__main__.B'>
<bound method B.addition of <__main__.B object at 0x00000000030B1128>>
Import the extension like this:
import compiled_extension
In your class you write:
def increment: return compiled_extension.increment()
This seems more readable and might be more efficient.
For example, I need for class calling returns string.
class Foo(object):
def __init__(self):
self.bar = 'bar'
def __call__(self):
return self.bar
Foo calling returns Foo object.
Foo()
<__main__.Foo object at 0x8ff6a8c>
What should I do to class returns string or other? How to use __call__?
With your example (of limited usefulness), you have a class of callable objects.
You can do now, as you have done,
>>> o = Foo()
>>> o
<__main__.Foo object at 0x8ff6a8c>
>>> o()
'bar'
I. e., __call__() does not make your class callable (as it is already), but it gives you a callable object.
In Python, everything is an object. Even classes.
Classes, furthermore, are callable objects. You don't have to do anything to make this happen, they just are. Calling the class creates an instance.
Setting up a __call__ method makes the instances also callable. You call the instances the same way you called the class (or any other function).
In [1]: class A:
...: def __init__(self):
...: print "init"
...:
...: def __call__(self):
...: print "call"
...:
...:
In [2]: a = A()
init
In [3]: a()
call