This question already has answers here:
Get defining class of unbound method object in Python 3
(5 answers)
Closed 7 years ago.
I need to:
Receive a method as argument (directly from the class, without instance)
Create an instance
Execute said method from instance
The thing is, how can I reliably get the class out of the method? I've tried researching something in the inspect module, but because there is no instance, it thinks of the method as a function rather than a method.
Here's some example of what I'm trying to do:
def execute_method(method):
cls = get_class_from_method(method)
obj = cls()
getattr(obj, method.__name__)()
def get_class_from_method(method):
pass # how?
execute_method(HelloView.say_hello)
A Python 2 method objects have a im_class attribute referencing the class of the instance they are bound to:
cls = method.im_class
For an unbound method, in Python 3, you'll be out of luck (the function is returned entirely unchanged, with no reference to the class). Python 2 returns an unbound method type with the im_class attribute still there:
>>> class Foo(object):
... def bar(self): pass
...
>>> Foo.bar.im_class
<class '__main__.Foo'>
In Python 3, your options are far more limited and are error-prone. You could look at the __qualname__ attribute of the function, and deduce from that what the class might be bound to in the global namespace:
>>> class Foo:
... def bar(self): pass
...
>>> Foo.bar
<function Foo.bar at 0x10e97b268>
>>> Foo.bar.__qualname__
'Foo.bar'
>>> Foo.bar.__qualname__.rpartition('.')[0]
'Foo'
>>> Foo.bar.__globals__.get(Foo.bar.__qualname__.rpartition('.')[0])
<class '__main__.Foo'>
However, if the class was created in a function (and is thus a local), or the class was replaced via a class decorator or simply renamed in the global namespace, the above trick at best won't work, at worst give you an entirely different object, and you can't know if this is the case.
Related
Consider a trivial example:
class C:
#staticmethod
def my_static_method():
print("static")
def my_instance_method(self):
print("self")
When I call C().my_static_method(), python doesn't pass the instance of C into my_static_method, and the descriptor that my_static_method references doesn't expect an instance of C, either.
This makes sense.
But then when I call C().my_instance_method(), how does python know to pass the instance of C that I'm calling my_instance_method from in as an argument, without me specifying anything?
As the link explains, function objects are descriptors! Just like staticmethod objects.
They have a __get__ method which returns a bound-method object, which essentially just partially applies the instance itself as the first positional argument. Consider:
>>> def foo(self):
... return self.bar
...
>>> class Baz:
... bar = 42
...
>>> baz = Baz()
>>> bound_method = foo.__get__(baz, Baz)
>>> bound_method
<bound method foo of <__main__.Baz object at 0x7ffcd001c7f0>>
>>> method()
42
By adding the #staticmethod decorator to my_static_method, you told python not to pass the calling instance of C into the function. So you can call this function as C.my_static_method().
By calling C() you created an instance of C. Then you called the non static function my_instance_method() which Python happily passed your new instance of C as the first parameter.
What happens when you call C.my_instance_method() ?
Rhetorical: You'll get a "missing one required arg self" exception -- since my_instance_method only works when calling from an instance unless you decorate it as static.
Of course you can still call the static member from an instance C().my_static_method() but you don't have a self param so no access to the instance.
The key point here is that methods are just functions that happen to be attributes of a class. The actual magic, in Python, happens in the attribute lookup process. The link you give explains earlier just how much happens every time x.y happens in Python. (Remember, everything is an object; that includes functions, classes, modules, type (which is an instance of itself)...)
This process is why descriptors can work at all; why we need explicit self; and why we can do fun things like calling a method with normal function call syntax (as long as we look it up from the class rather than an instance), alias it, mimic the method binding process with functools.partial....
Suppose we have c = C(). When you do c.my_instance_method (never mind calling it for now), Python looks for my_instance_method in type(c) (i.e., in the C class), and also checks if it's a descriptor, and also if it's specifically a data descriptor. Functions are non-data descriptors; even outside of a class, you can write
>>> def x(spam): return spam
...
>>> x.__get__
<method-wrapper '__get__' of function object at 0x...>
Because of the priority rules, as long as c doesn't directly have an attribute attached with the same name, the function will be found in C and its __get__ will be used. Note that the __get__ in question comes from the class - but it isn't using the same process as x.__get__ above. That code looks in the class because that's one of the places checked for an attribute lookup; but when c.my_instance_method redirects to C.my_instance_method.__get__, it's looking there directly - attaching a __get__ attribute directly to the function wouldn't change anything (which is why staticmethod is implemented as a class instead).
That __get__ implements the actual method binding. Let's pretend we found x as a method in the str class:
>>> x.__get__('spam', str)
<bound method x of 'spam'>
>>> x.__get__('spam', str)()
'spam'
Remember, although the function in question takes three arguments, we're calling __get__, itself, as a method - so x gets bound to it in the same way. Equivalently, and more faithful to the actual process:
>>> type(x).__get__(x, 'spam', str)
<bound method x of 'spam'>
>>> type(x).__get__(x, 'spam', str)()
'spam'
So what exactly is that "bound method", anyway?
>>> bound = type(x).__get__(x, 'spam', str)
>>> type(bound)
<class 'method'>
>>> bound.__call__
<method-wrapper '__call__' of method object at 0x...>
>>> bound.__func__
<function x at 0x...>
>>> bound.__self__
'spam'
>>> type(bound)(x, 'eggs')
<bound method x of 'eggs'>
Pretty much what you'd expect: it's a callable object that stores and uses the original function and self value, and does the obvious thing in __call__.
This question already has answers here:
Schrödinger's variable: the __class__ cell magically appears if you're checking for its presence?
(2 answers)
Closed 2 years ago.
I found this quirk while checking out how to use super.
In [1]: super?
Init signature: super(self, /, *args, **kwargs)
Docstring:
super() -> same as super(__class__, <first argument>)
...
Note that the first example uses __class__ directly.
And somehow, __class__ can be used inside instance methods:
class Test():
def __init__(self):
print(__class__)
def foo(self):
print(__class__)
def bar(self):
print(bar)
t = Test() # <class '__main__.Test'>
t.foo() # <class '__main__.Test'>
t.bar() # NameError: name 'bar' is not defined
Can anyone explain why this is the case?
So, it is a quirk for the compiler to find the parent class.
From Python Data Model:
__class__ is an implicit closure reference created by the compiler if any methods in a class body refer to either __class__ or super. This
allows the zero argument form of super() to correctly identify the
class being defined based on lexical scoping, while the class or
instance that was used to make the current call is identified based on
the first argument passed to the method.
I just stumbled over this strange behavior when the type of a method changes during subclassing:
class A:
def f(self, x):
return x**2
class B(A):
#classmethod
def f(cls, x):
return x**2
If I now ask for the type of B.f, I'll get the (supposedly) wrong answer:
In [37]: type(B.f)
Out[37]: method
Whereas this works as expected:
In [39]: type(B.__dict__["f"])
Out[39]: classmethod
(Seen in Python 3.4 and 3.6.)
Is this just a bug or is there a specific reason for this?
What's the difference between the attribute f and the .__dict__["f"] item? I thought they were the same.
In a testing suite, I was trying to support both types of methods inside a class to be tested. To be able to do that, I need to know the type in order to pass the correct number of arguments. If it's a normal method (i.e. self is the first argument), I'd just pass None explicitly, which by design shouldn't be used inside the method anyway, since it's not instance-dependent.
Maybe there's a better way to do this, like duck typing the call to the method. But there might be cases where this is not so easy to do, like if the method had *args and **kwargs... Therefore I went with the explicit type check, but got stuck at this point.
No, this is not a bug, this is normal behaviour. A classmethod produces a bound method when accessed on a class. That's exactly the point of a classmethod, to bind a function to the class you access it on or the class of an instance you access it on.
Like function and property objects, classmethod is a descriptor object, it implements a __get__ method. Accessing attributes on an instance or a class is delegated to the __getattribute__ method, and the default implementation of that hook will not just return what it found in object.__dict__[attributename]; it will also bind descriptors, by calling the descriptor.__get__() method. This is a hugely important aspect of Python, it is this mechanism that makes methods and attributes and loads of other things work.
classmethod objects, when bound by the descriptor protocol, return a method object. Method objects are wrappers that record the object bound to, and the function to call when they are called; calling a method really calls the underlying method with the bound object as first argument:
>>> class Foo:
... pass
...
>>> def bar(*args): print(args)
...
>>> classmethod(bar).__get__(None, Foo) # decorate with classmethod and bind
<bound method bar of <class '__main__.Foo'>>
>>> method = classmethod(bar).__get__(None, Foo)
>>> method.__self__
<class '__main__.Foo'>
>>> method.__func__
<function bar at 0x1056f0e18>
>>> method()
(<class '__main__.Foo'>,)
>>> method('additional arguments')
(<class '__main__.Foo'>, 'additional arguments')
So the method object returned for a classmethod object references the class (the second argument to __get__, the owner), and the original function. If you use a class method on an instance, the first argument is still ignored:
>>> classmethod(bar).__get__(Foo(), Foo).__self__ # called on an instance
<class '__main__.Foo'>
Functions, on the other hand, want to bind only to instances; so if the first argument to __get__ is set to None, they simply return self:
>>> bar.__get__(None, Foo) # access on a class
<function bar at 0x1056f0e18>
>>> bar.__get__(Foo(), Foo) # access on an instance
<bound method bar of <__main__.Foo object at 0x105833a90>>
>>> bar.__get__(Foo(), Foo).__self__
<__main__.Foo object at 0x105833160>
If accessing ClassObject.classmethod_object would return the classmethod object itself, like a function object would, then you could never actually use the class method on a class. That'd be rather pointless.
So no, object.attribute is not always the same thing as object.__dict__['attribute']. If object.__dict__['attribute'] supports the descriptor protocol, it'll be invoked.
What arguments do types.MethodType expect and what does it return?
https://docs.python.org/3.6/library/types.html doesn't say more about it:
types.MethodType
The type of methods of user-defined class instances.
For an example, from https://docs.python.org/3.6/howto/descriptor.html
To support method calls, functions include the __get__() method for
binding methods during attribute access. This means that all functions
are non-data descriptors which return bound or unbound methods
depending whether they are invoked from an object or a class. In pure
python, it works like this:
class Function(object):
. . .
def __get__(self, obj, objtype=None):
"Simulate func_descr_get() in Objects/funcobject.c"
if obj is None:
return self
return types.MethodType(self, obj)
Must the first argument self of types.MethodType be a callable object? In other words, must the class Function be a callable type, i.e. must Function have a method __call__?
If self is a callable object, does it take at least one argument?
Does types.MethodType(self, obj) mean giving obj as the first argument to the callable object self, i.e. currying self with obj?
How does types.MethodType(self, obj) create and return an instance of types.MethodType?
Thanks.
Usually you don't need to create instance of types.MethodType yourself. Instead, you'll get one automatically when you access a method on an instance of a class.
For example, if we make a class, create an instance of it, then access a method on the instance (without calling it), we'll get an instance of types.MethodType back:
import types
class Foo:
def bar(self):
pass
foo = Foo()
method = foo.bar
print(type(method) == types.MethodType) # prints True
The code you excerpt in your question is trying to show how this normally happens. It's not something you usually have to do yourself, though you can if you really want to. For instance, to create another instance of types.MethodType equivalent to method above, we could do:
method_manual = types.MethodType(Foo.bar, foo)
The first argument to MethodType is a callable object (normally a function, but it can be something else, like an instance of the Function class in the example you were reading). The second argument what we're binding the function to. When you call the method object (with e.g. method()), the bound object will be passed into the function as the first argument.
Usually the object the method gets bound to is an instance, though it can be something else. For instance, a classmethod decorated function will bind to the class it is called on, rather than an instance. Here's an example of that (both getting a method bound to a class automatically, and doing it manually ourselves):
class Foo2:
#classmethod
def baz(cls):
pass
foo2 = Foo2()
method2 = Foo2.baz
method2_via_an_instance = foo2.baz
method2_manual = types.MethodType(method2.__func__, Foo2)
All three of the method2-prefixed variables work exactly the same way (when you call them, they'll all call baz with Foo2 as the cls argument). The only wonky thing about the manual approach this time is that it's hard to get at the original baz function without getting a bound method instead, so I fished it out of one of the other bound method objects.
A final note: The name types.MethodType is an alias for the internal type used for bound methods, which doesn't otherwise have an accessible name. Unlike many classes, the repr of an instance is not an expression to recreate it (it will be something like "<bound method Foo.bar of <__main__.Foo object at 0x0000...>>"). Nor is the repr of the type a valid name to access the type by (the repr is "method").
Short Answer:
Must the first argument self of types.MethodType be a callable object?
In other words, must the class Function be a callable type, i.e. must
Function have a method __call__?
Yes
If self is a callable object, does it take at least one argument?
Depends
Does types.MethodType(self, obj) mean giving obj as the first argument
to the callable object self, i.e. currying self with obj?
Yes
How does types.MethodType(self, obj) create and return an instance of
types.MethodType?
It doesn't work like that.
Long Answer:
the code
class Function(object):
. . .
def __get__(self, obj, objtype=None):
"Simulate func_descr_get() in Objects/funcobject.c"
if obj is None:
return self
return types.MethodType(self, obj)
As Daniel explained is mainly to demonstrate for
To support method calls, functions include the __get__() method for
binding methods during attribute access. This means that all functions
are non-data descriptors which return bound or unbound methods
depending whether they are invoked from an object or a class. In pure
python, it works like this:
The types.MethodType() works when the Function has an object.
if obj is None would be False
Then it's a method of some object aka. bound method.
It explains how Python grammar work. As a function, it could be called in the
following two ways.
some_func_() or some_class.some_func()
The former part https://docs.python.org/3.6/howto/descriptor.html#invoking-descriptors explained.
For objects, the machinery is in object.__getattribute__() which
transforms b.x into type(b).__dict__['x'].__get__(b, type(b)). The
implementation works through a precedence chain that gives data
descriptors priority over instance variables, instance variables
priority over non-data descriptors, and assigns lowest priority to
__getattr__() if provided.
Here it's some demonstrate code
>>> import types
>>> types.MethodType
<type 'instancemethod'>
>>> def a(self):
... print(1)
...
>>> class B:
... pass
...
>>> types.MethodType(a,B)
<bound method ?.a of <class __main__.B at 0x7f4d3d5aa598>>
>>> B.t = types.MethodType(a,B)
>>> B.t()
1
>>> def s():
... print(3)
...
>>> B.r = types.MethodType(s,B)
>>> B.r
<bound method ?.s of <class __main__.B at 0x7f4d3d5aa598>>
>>> B.r()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: s() takes no arguments (1 given)
See also dynamically adding callable to class as instance "method"
Documentation doesn't say much, but you can always check its source code. The signature of MethodType constructor is:
def __init__(self, func: Callable[..., Any], obj: object) -> None: ...
It accepts a callable and object, and returns None.
MethodType can be used to add an instance method to an object, instead of a function; here's an example:
from types import MethodType
class MyClass:
language = 'Python'
# a function is bound to obj1
obj1 = MyClass()
obj1.say_hello = lambda: 'Hello World!'
print(type(obj1.say_hello)) # type is class 'function'
obj1.say_hello()
# a method is bound to obj2
obj2 = MyClass()
# this is used to bind a "method" to a specific object obj2, rather than a function
obj2.say_hello = MethodType(lambda self: f'Hello {self.language}!', obj2)
print(type(obj2.say_hello)) # type is class 'method'
obj2.say_hello()
It's not something you would ever call. Like most of the classes in the types module, it's more for comparing with existing objects (for example in isinstance).
I always hear this statement in Python (for topics such as decorators, etc. when you are passing functions, etc.) but have never really seen an elaboration on this.
For example is it possible to create a class c that has only one abstract method that is called with a set of opened and closed brackets.
i.e class c:
#abstractmethod
def method_to_be_called_by():
...
so you can have
c(whatever parameters are required)
I could be way off the mark with my understanding here, I was just curious about what people meant by this.
You are looking for the __call__ method. Function objects have that method:
>>> def foo(): pass
...
>>> foo.__call__
<method-wrapper '__call__' of function object at 0x106aafd70>
Not that the Python interpreter loop actually makes use of that method when encountering a Python function object; optimisations in the implementation jump straight to the contained bytecode in most cases.
But you can use that on your own custom class:
class Callable(object):
def __init__(self, name):
self.name = name
def __call__(self, greeting):
return '{}, {}!'.format(greeting, self.name)
Demo:
>>> class Callable(object):
... def __init__(self, name):
... self.name = name
... def __call__(self, greeting):
... return '{}, {}!'.format(greeting, self.name)
...
>>> Callable('World')('Hello')
'Hello, World!'
Python creates function objects for you when you use a def statement, or you use a lambda expression:
>>> def foo(): pass
...
>>> foo
<function foo at 0x106aafd70>
>>> lambda: None
<function <lambda> at 0x106d90668>
You can compare this to creating a string or an integer or a list using literal syntax:
listobject = [1, 'two']
The above creates 3 objects without ever calling a type, Python did that all for you based on the syntax used. The same applies to functions.
Creating one yourself can be a little more complex; you need to have a code object and reference to a global namespace, at the very least:
>>> function_type = type(lambda: None)
>>> function_type
<type 'function'>
>>> function_type(foo.__code__, globals(), 'bar')
<function bar at 0x106d906e0>
Here I created a function object by reusing the function type, taking the code object from the foo function; the function type is not a built-in name but the type really does exist and can be obtained by calling type() on an existing function instance.
I also passed in the global namespace of my interpreter, and a name; the latter is an optional argument; the name is otherwise taken from the code object.
One simple way to see this is to create a function in the Python interpreter def bar(x): return x + 1 and then use dir(bar) to see the various magic attributes including __class__.
Yes, python functions are full objects.
For another approach, objects are functions if they have a magic __call__() method.