I always hear this statement in Python (for topics such as decorators, etc. when you are passing functions, etc.) but have never really seen an elaboration on this.
For example is it possible to create a class c that has only one abstract method that is called with a set of opened and closed brackets.
i.e class c:
#abstractmethod
def method_to_be_called_by():
...
so you can have
c(whatever parameters are required)
I could be way off the mark with my understanding here, I was just curious about what people meant by this.
You are looking for the __call__ method. Function objects have that method:
>>> def foo(): pass
...
>>> foo.__call__
<method-wrapper '__call__' of function object at 0x106aafd70>
Not that the Python interpreter loop actually makes use of that method when encountering a Python function object; optimisations in the implementation jump straight to the contained bytecode in most cases.
But you can use that on your own custom class:
class Callable(object):
def __init__(self, name):
self.name = name
def __call__(self, greeting):
return '{}, {}!'.format(greeting, self.name)
Demo:
>>> class Callable(object):
... def __init__(self, name):
... self.name = name
... def __call__(self, greeting):
... return '{}, {}!'.format(greeting, self.name)
...
>>> Callable('World')('Hello')
'Hello, World!'
Python creates function objects for you when you use a def statement, or you use a lambda expression:
>>> def foo(): pass
...
>>> foo
<function foo at 0x106aafd70>
>>> lambda: None
<function <lambda> at 0x106d90668>
You can compare this to creating a string or an integer or a list using literal syntax:
listobject = [1, 'two']
The above creates 3 objects without ever calling a type, Python did that all for you based on the syntax used. The same applies to functions.
Creating one yourself can be a little more complex; you need to have a code object and reference to a global namespace, at the very least:
>>> function_type = type(lambda: None)
>>> function_type
<type 'function'>
>>> function_type(foo.__code__, globals(), 'bar')
<function bar at 0x106d906e0>
Here I created a function object by reusing the function type, taking the code object from the foo function; the function type is not a built-in name but the type really does exist and can be obtained by calling type() on an existing function instance.
I also passed in the global namespace of my interpreter, and a name; the latter is an optional argument; the name is otherwise taken from the code object.
One simple way to see this is to create a function in the Python interpreter def bar(x): return x + 1 and then use dir(bar) to see the various magic attributes including __class__.
Yes, python functions are full objects.
For another approach, objects are functions if they have a magic __call__() method.
Related
Consider a trivial example:
class C:
#staticmethod
def my_static_method():
print("static")
def my_instance_method(self):
print("self")
When I call C().my_static_method(), python doesn't pass the instance of C into my_static_method, and the descriptor that my_static_method references doesn't expect an instance of C, either.
This makes sense.
But then when I call C().my_instance_method(), how does python know to pass the instance of C that I'm calling my_instance_method from in as an argument, without me specifying anything?
As the link explains, function objects are descriptors! Just like staticmethod objects.
They have a __get__ method which returns a bound-method object, which essentially just partially applies the instance itself as the first positional argument. Consider:
>>> def foo(self):
... return self.bar
...
>>> class Baz:
... bar = 42
...
>>> baz = Baz()
>>> bound_method = foo.__get__(baz, Baz)
>>> bound_method
<bound method foo of <__main__.Baz object at 0x7ffcd001c7f0>>
>>> method()
42
By adding the #staticmethod decorator to my_static_method, you told python not to pass the calling instance of C into the function. So you can call this function as C.my_static_method().
By calling C() you created an instance of C. Then you called the non static function my_instance_method() which Python happily passed your new instance of C as the first parameter.
What happens when you call C.my_instance_method() ?
Rhetorical: You'll get a "missing one required arg self" exception -- since my_instance_method only works when calling from an instance unless you decorate it as static.
Of course you can still call the static member from an instance C().my_static_method() but you don't have a self param so no access to the instance.
The key point here is that methods are just functions that happen to be attributes of a class. The actual magic, in Python, happens in the attribute lookup process. The link you give explains earlier just how much happens every time x.y happens in Python. (Remember, everything is an object; that includes functions, classes, modules, type (which is an instance of itself)...)
This process is why descriptors can work at all; why we need explicit self; and why we can do fun things like calling a method with normal function call syntax (as long as we look it up from the class rather than an instance), alias it, mimic the method binding process with functools.partial....
Suppose we have c = C(). When you do c.my_instance_method (never mind calling it for now), Python looks for my_instance_method in type(c) (i.e., in the C class), and also checks if it's a descriptor, and also if it's specifically a data descriptor. Functions are non-data descriptors; even outside of a class, you can write
>>> def x(spam): return spam
...
>>> x.__get__
<method-wrapper '__get__' of function object at 0x...>
Because of the priority rules, as long as c doesn't directly have an attribute attached with the same name, the function will be found in C and its __get__ will be used. Note that the __get__ in question comes from the class - but it isn't using the same process as x.__get__ above. That code looks in the class because that's one of the places checked for an attribute lookup; but when c.my_instance_method redirects to C.my_instance_method.__get__, it's looking there directly - attaching a __get__ attribute directly to the function wouldn't change anything (which is why staticmethod is implemented as a class instead).
That __get__ implements the actual method binding. Let's pretend we found x as a method in the str class:
>>> x.__get__('spam', str)
<bound method x of 'spam'>
>>> x.__get__('spam', str)()
'spam'
Remember, although the function in question takes three arguments, we're calling __get__, itself, as a method - so x gets bound to it in the same way. Equivalently, and more faithful to the actual process:
>>> type(x).__get__(x, 'spam', str)
<bound method x of 'spam'>
>>> type(x).__get__(x, 'spam', str)()
'spam'
So what exactly is that "bound method", anyway?
>>> bound = type(x).__get__(x, 'spam', str)
>>> type(bound)
<class 'method'>
>>> bound.__call__
<method-wrapper '__call__' of method object at 0x...>
>>> bound.__func__
<function x at 0x...>
>>> bound.__self__
'spam'
>>> type(bound)(x, 'eggs')
<bound method x of 'eggs'>
Pretty much what you'd expect: it's a callable object that stores and uses the original function and self value, and does the obvious thing in __call__.
What are the advantages of using MethodType from the types module? You can use it to add methods to an object. But we can do that easily without it:
def func():
print 1
class A:
pass
obj = A()
obj.func = func
It works even if we delete func in the main scope by running del func.
Why would one want to use MethodType? Is it just a convention or a good programming habit?
In fact the difference between adding methods dynamically at run time and
your example is huge:
in your case, you just attach a function to an object, you can call it of course but it is unbound, it has no relation with the object itself (ie. you cannot use self inside the function)
when added with MethodType, you create a bound method and it behaves like a normal Python method for the object, you have to take the object it belongs to as first argument (it is normally called self) and you can access it inside the function
This example shows the difference:
def func(obj):
print 'I am called from', obj
class A:
pass
a=A()
a.func=func
a.func()
This fails with a TypeError: func() takes exactly 1 argument (0 given),
whereas this code works as expected:
import types
a.func = types.MethodType(func, a) # or types.MethodType(func, a, A) for PY2
a.func()
shows I am called from <__main__.A instance at xxx>.
A common use of types.MethodType is checking whether some object is a method. For example:
>>> import types
>>> class A(object):
... def method(self):
... pass
...
>>> isinstance(A().method, types.MethodType)
True
>>> def nonmethod():
... pass
...
>>> isinstance(nonmethod, types.MethodType)
False
Note that in your example isinstance(obj.func, types.MethodType) returns False. Imagine you have defined a method meth in class A. isinstance(obj.meth, types.MethodType) would return True.
This question already has answers here:
Get defining class of unbound method object in Python 3
(5 answers)
Closed 7 years ago.
I need to:
Receive a method as argument (directly from the class, without instance)
Create an instance
Execute said method from instance
The thing is, how can I reliably get the class out of the method? I've tried researching something in the inspect module, but because there is no instance, it thinks of the method as a function rather than a method.
Here's some example of what I'm trying to do:
def execute_method(method):
cls = get_class_from_method(method)
obj = cls()
getattr(obj, method.__name__)()
def get_class_from_method(method):
pass # how?
execute_method(HelloView.say_hello)
A Python 2 method objects have a im_class attribute referencing the class of the instance they are bound to:
cls = method.im_class
For an unbound method, in Python 3, you'll be out of luck (the function is returned entirely unchanged, with no reference to the class). Python 2 returns an unbound method type with the im_class attribute still there:
>>> class Foo(object):
... def bar(self): pass
...
>>> Foo.bar.im_class
<class '__main__.Foo'>
In Python 3, your options are far more limited and are error-prone. You could look at the __qualname__ attribute of the function, and deduce from that what the class might be bound to in the global namespace:
>>> class Foo:
... def bar(self): pass
...
>>> Foo.bar
<function Foo.bar at 0x10e97b268>
>>> Foo.bar.__qualname__
'Foo.bar'
>>> Foo.bar.__qualname__.rpartition('.')[0]
'Foo'
>>> Foo.bar.__globals__.get(Foo.bar.__qualname__.rpartition('.')[0])
<class '__main__.Foo'>
However, if the class was created in a function (and is thus a local), or the class was replaced via a class decorator or simply renamed in the global namespace, the above trick at best won't work, at worst give you an entirely different object, and you can't know if this is the case.
The following Python code executes normally without raising an exception:
class Foo:
pass
class Foo:
pass
def bar():
pass
def bar():
pass
print(Foo.__module__ + Foo.__name__)
Yet clearly, there are multiple instances of __main__.Foo and __main__.bar. Why does Python not raise an error when it encounters this namespace collision? And since it doesn't raise an error, what exactly is it doing? Is the first class __main__.Foo replaced by the second class __main__.Foo?
In Python everything is an object - instance of some type. E.g. 1 is an instance of type int, def foo(): pass creates object foo which is an instance of type function (same for classes - objects, created by class statement are instances of type type). Given this, there no difference (at the level of name binding mechanism) between
class Foo:
string = "foo1"
class Foo:
string = "foo2"
and
a = 1
a = 2
BTW, class definition may be performed using type function (yeah, there is type type and built-in function type):
Foo = type('Foo', (), {string: 'foo1'})
So classes and functions are not some different kind of data, although special syntax may be used for creating their instances.
See also related Data Model section.
The Foo class is effectively being re-defined further down the script (script is read by the interpreter from top to bottom).
class Foo:
string = "foo1"
class Foo:
string = "foo2"
f = Foo()
print f.string
prints "foo2"
The second definition replaces the first one, as expected if you think at classes as elements in the "types dictionary" of the current namespace:
>>> class Foo:
... def test1(self):
... print "test1"
...
>>> Foo
<class __main__.Foo at 0x7fe8c6943650>
>>> class Foo:
... def test2(self):
... print "test2"
...
>>> Foo
<class __main__.Foo at 0x7fe8c6943590>
>>> a = Foo()
>>> a.test1()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: Foo instance has no attribute 'test1'
>>> a.test2()
test2
>>>
here you can clearly see that the "definition" of Foo changes (Foo points to different classes in memory), and that it's the last one that prevails.
Conceptually this is just rebinding a name. It's no different from this:
x = 1
x = 2
and I'm sure you would not want that to be an error.
In compiled and some interpreted languages there is a clear seperation between definition, declaration and execution. But in python it's simpler. There are just statements!
Python EXECUTES your script/program/module as soon as it is invoked. It may help, to see def and class as "syntactic sugar". E.g. class is a convenient wrapper around Foo = type("class-name", (bases), {attributes}).
So python executes:
class Foo #equivalent to: Foo = type("class-name", (bases), {attributes})
class Foo
def bar
def bar
print(Foo.__module__ + Foo.__name__)
which boils down to overwriting the names Fooand bar with the latest "declaration". So this just works as intended from a python-pov - but maybe not as you intended it! ;-)
so it's also a typical error for developers with a different background to misunderstand:
def some_method(default_list = []):
...
default_list is a singleton here. Every call to some_method usese the same default_list, because the list-object is created at first execution.
Python doesn't enter the function-body, but only executes the signature/head as soon as it begins parsing.
Really sorry for the extremely stupid title, but if I know what it is, I wouldn't write here (:
def some_decorator( func ):
# ..
class A:
#some_decorator
def func():
pass
#func.some_decorator # this one here - func.some_decorator ?
def func():
pass
some_decorator decorates func - that's OK. But what is func.some_decorator and how some_decorator becomes a member ( or something else ? ) of func?
P.S. I'm 90% sure, that there's such question here (as this seems something basic), but I don't know how to search it. If there's a exact duplicate, I'll delete this question.
Note : It's not typo, nor accident, that both member functions are named func. The decorator is for overloading: the question is related to Decorating method (class methods overloading)
Remember that the function definition with decorator is equivalent to this:
def func():
pass
func = some_decorator(func)
So in the following lines, func doesn't refer to the function you defined but to what the decorator turned it into. Also note that decorators can return any object, not just functions. So some_decorator returns an object with a method (it's unfortunate that the names some_decorator and func are reused in the example - it's confusing, but doesn't change anything about the concept) that is itself a decorator. As the expression after the # is evaluated first, you still have a reference to the first decorator's method after you defined another plain function func. That decorator is applied to this new function. The full example is then equivalent to this:
class A:
def func():
pass
func = some_decorator(func)
_decorator = func.some_decorator
def func():
pass
func = _decorator(func)
One way to clarify this is to demonstrate it with a concrete example that behaves like this, the builtin property descriptor:
class C(object):
#property
def x(self):
"This is a property object, not a function"
return self._x
#x.setter
def x(self, val):
self._x = val
>>> c = C()
>>> c.x = 1
>>> c.x
1
>>> C.x
<property object at 0x2396100>
>>> C.x.__doc__
'This is a property object, not a function'
>>> C.x.getter.__doc__
'Descriptor to change the getter on a property.'
>>> C.x.setter.__doc__
'Descriptor to change the setter on a property.'
>>> C.x.deleter.__doc__
'Descriptor to change the deleter on a property.'
The first invocation of property (as a decorator) means that x is not a function - it is a property descriptor. A feature of properties is that they allow you to initially define just the fget method, and then provide fset and fdel later by using the property.setter and property.deleter decorators (although since each of these creates a new property object, you do need to make sure to use the same name each time).
Something similar will usually be the case whenever you see code using this kind of pattern. Ideally, the naming of the decorators involved will make it reasonably clear what is going on (e.g. most people seem to grasp the idiom for defining property attributes reasonably easily).