Related
I just stumbled over this strange behavior when the type of a method changes during subclassing:
class A:
def f(self, x):
return x**2
class B(A):
#classmethod
def f(cls, x):
return x**2
If I now ask for the type of B.f, I'll get the (supposedly) wrong answer:
In [37]: type(B.f)
Out[37]: method
Whereas this works as expected:
In [39]: type(B.__dict__["f"])
Out[39]: classmethod
(Seen in Python 3.4 and 3.6.)
Is this just a bug or is there a specific reason for this?
What's the difference between the attribute f and the .__dict__["f"] item? I thought they were the same.
In a testing suite, I was trying to support both types of methods inside a class to be tested. To be able to do that, I need to know the type in order to pass the correct number of arguments. If it's a normal method (i.e. self is the first argument), I'd just pass None explicitly, which by design shouldn't be used inside the method anyway, since it's not instance-dependent.
Maybe there's a better way to do this, like duck typing the call to the method. But there might be cases where this is not so easy to do, like if the method had *args and **kwargs... Therefore I went with the explicit type check, but got stuck at this point.
No, this is not a bug, this is normal behaviour. A classmethod produces a bound method when accessed on a class. That's exactly the point of a classmethod, to bind a function to the class you access it on or the class of an instance you access it on.
Like function and property objects, classmethod is a descriptor object, it implements a __get__ method. Accessing attributes on an instance or a class is delegated to the __getattribute__ method, and the default implementation of that hook will not just return what it found in object.__dict__[attributename]; it will also bind descriptors, by calling the descriptor.__get__() method. This is a hugely important aspect of Python, it is this mechanism that makes methods and attributes and loads of other things work.
classmethod objects, when bound by the descriptor protocol, return a method object. Method objects are wrappers that record the object bound to, and the function to call when they are called; calling a method really calls the underlying method with the bound object as first argument:
>>> class Foo:
... pass
...
>>> def bar(*args): print(args)
...
>>> classmethod(bar).__get__(None, Foo) # decorate with classmethod and bind
<bound method bar of <class '__main__.Foo'>>
>>> method = classmethod(bar).__get__(None, Foo)
>>> method.__self__
<class '__main__.Foo'>
>>> method.__func__
<function bar at 0x1056f0e18>
>>> method()
(<class '__main__.Foo'>,)
>>> method('additional arguments')
(<class '__main__.Foo'>, 'additional arguments')
So the method object returned for a classmethod object references the class (the second argument to __get__, the owner), and the original function. If you use a class method on an instance, the first argument is still ignored:
>>> classmethod(bar).__get__(Foo(), Foo).__self__ # called on an instance
<class '__main__.Foo'>
Functions, on the other hand, want to bind only to instances; so if the first argument to __get__ is set to None, they simply return self:
>>> bar.__get__(None, Foo) # access on a class
<function bar at 0x1056f0e18>
>>> bar.__get__(Foo(), Foo) # access on an instance
<bound method bar of <__main__.Foo object at 0x105833a90>>
>>> bar.__get__(Foo(), Foo).__self__
<__main__.Foo object at 0x105833160>
If accessing ClassObject.classmethod_object would return the classmethod object itself, like a function object would, then you could never actually use the class method on a class. That'd be rather pointless.
So no, object.attribute is not always the same thing as object.__dict__['attribute']. If object.__dict__['attribute'] supports the descriptor protocol, it'll be invoked.
What arguments do types.MethodType expect and what does it return?
https://docs.python.org/3.6/library/types.html doesn't say more about it:
types.MethodType
The type of methods of user-defined class instances.
For an example, from https://docs.python.org/3.6/howto/descriptor.html
To support method calls, functions include the __get__() method for
binding methods during attribute access. This means that all functions
are non-data descriptors which return bound or unbound methods
depending whether they are invoked from an object or a class. In pure
python, it works like this:
class Function(object):
. . .
def __get__(self, obj, objtype=None):
"Simulate func_descr_get() in Objects/funcobject.c"
if obj is None:
return self
return types.MethodType(self, obj)
Must the first argument self of types.MethodType be a callable object? In other words, must the class Function be a callable type, i.e. must Function have a method __call__?
If self is a callable object, does it take at least one argument?
Does types.MethodType(self, obj) mean giving obj as the first argument to the callable object self, i.e. currying self with obj?
How does types.MethodType(self, obj) create and return an instance of types.MethodType?
Thanks.
Usually you don't need to create instance of types.MethodType yourself. Instead, you'll get one automatically when you access a method on an instance of a class.
For example, if we make a class, create an instance of it, then access a method on the instance (without calling it), we'll get an instance of types.MethodType back:
import types
class Foo:
def bar(self):
pass
foo = Foo()
method = foo.bar
print(type(method) == types.MethodType) # prints True
The code you excerpt in your question is trying to show how this normally happens. It's not something you usually have to do yourself, though you can if you really want to. For instance, to create another instance of types.MethodType equivalent to method above, we could do:
method_manual = types.MethodType(Foo.bar, foo)
The first argument to MethodType is a callable object (normally a function, but it can be something else, like an instance of the Function class in the example you were reading). The second argument what we're binding the function to. When you call the method object (with e.g. method()), the bound object will be passed into the function as the first argument.
Usually the object the method gets bound to is an instance, though it can be something else. For instance, a classmethod decorated function will bind to the class it is called on, rather than an instance. Here's an example of that (both getting a method bound to a class automatically, and doing it manually ourselves):
class Foo2:
#classmethod
def baz(cls):
pass
foo2 = Foo2()
method2 = Foo2.baz
method2_via_an_instance = foo2.baz
method2_manual = types.MethodType(method2.__func__, Foo2)
All three of the method2-prefixed variables work exactly the same way (when you call them, they'll all call baz with Foo2 as the cls argument). The only wonky thing about the manual approach this time is that it's hard to get at the original baz function without getting a bound method instead, so I fished it out of one of the other bound method objects.
A final note: The name types.MethodType is an alias for the internal type used for bound methods, which doesn't otherwise have an accessible name. Unlike many classes, the repr of an instance is not an expression to recreate it (it will be something like "<bound method Foo.bar of <__main__.Foo object at 0x0000...>>"). Nor is the repr of the type a valid name to access the type by (the repr is "method").
Short Answer:
Must the first argument self of types.MethodType be a callable object?
In other words, must the class Function be a callable type, i.e. must
Function have a method __call__?
Yes
If self is a callable object, does it take at least one argument?
Depends
Does types.MethodType(self, obj) mean giving obj as the first argument
to the callable object self, i.e. currying self with obj?
Yes
How does types.MethodType(self, obj) create and return an instance of
types.MethodType?
It doesn't work like that.
Long Answer:
the code
class Function(object):
. . .
def __get__(self, obj, objtype=None):
"Simulate func_descr_get() in Objects/funcobject.c"
if obj is None:
return self
return types.MethodType(self, obj)
As Daniel explained is mainly to demonstrate for
To support method calls, functions include the __get__() method for
binding methods during attribute access. This means that all functions
are non-data descriptors which return bound or unbound methods
depending whether they are invoked from an object or a class. In pure
python, it works like this:
The types.MethodType() works when the Function has an object.
if obj is None would be False
Then it's a method of some object aka. bound method.
It explains how Python grammar work. As a function, it could be called in the
following two ways.
some_func_() or some_class.some_func()
The former part https://docs.python.org/3.6/howto/descriptor.html#invoking-descriptors explained.
For objects, the machinery is in object.__getattribute__() which
transforms b.x into type(b).__dict__['x'].__get__(b, type(b)). The
implementation works through a precedence chain that gives data
descriptors priority over instance variables, instance variables
priority over non-data descriptors, and assigns lowest priority to
__getattr__() if provided.
Here it's some demonstrate code
>>> import types
>>> types.MethodType
<type 'instancemethod'>
>>> def a(self):
... print(1)
...
>>> class B:
... pass
...
>>> types.MethodType(a,B)
<bound method ?.a of <class __main__.B at 0x7f4d3d5aa598>>
>>> B.t = types.MethodType(a,B)
>>> B.t()
1
>>> def s():
... print(3)
...
>>> B.r = types.MethodType(s,B)
>>> B.r
<bound method ?.s of <class __main__.B at 0x7f4d3d5aa598>>
>>> B.r()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: s() takes no arguments (1 given)
See also dynamically adding callable to class as instance "method"
Documentation doesn't say much, but you can always check its source code. The signature of MethodType constructor is:
def __init__(self, func: Callable[..., Any], obj: object) -> None: ...
It accepts a callable and object, and returns None.
MethodType can be used to add an instance method to an object, instead of a function; here's an example:
from types import MethodType
class MyClass:
language = 'Python'
# a function is bound to obj1
obj1 = MyClass()
obj1.say_hello = lambda: 'Hello World!'
print(type(obj1.say_hello)) # type is class 'function'
obj1.say_hello()
# a method is bound to obj2
obj2 = MyClass()
# this is used to bind a "method" to a specific object obj2, rather than a function
obj2.say_hello = MethodType(lambda self: f'Hello {self.language}!', obj2)
print(type(obj2.say_hello)) # type is class 'method'
obj2.say_hello()
It's not something you would ever call. Like most of the classes in the types module, it's more for comparing with existing objects (for example in isinstance).
I understand there are at least 3 kinds of methods in Python having different first arguments:
instance method - instance, i.e. self
class method - class, i.e. cls
static method - nothing
These classic methods are implemented in the Test class below including an usual method:
class Test():
def __init__(self):
pass
def instance_mthd(self):
print("Instance method.")
#classmethod
def class_mthd(cls):
print("Class method.")
#staticmethod
def static_mthd():
print("Static method.")
def unknown_mthd():
# No decoration --> instance method, but
# No self (or cls) --> static method, so ... (?)
print("Unknown method.")
In Python 3, the unknown_mthd can be called safely, yet it raises an error in Python 2:
>>> t = Test()
>>> # Python 3
>>> t.instance_mthd()
>>> Test.class_mthd()
>>> t.static_mthd()
>>> Test.unknown_mthd()
Instance method.
Class method.
Static method.
Unknown method.
>>> # Python 2
>>> Test.unknown_mthd()
TypeError: unbound method unknown_mthd() must be called with Test instance as first argument (got nothing instead)
This error suggests such a method was not intended in Python 2. Perhaps its allowance now is due to the elimination of unbound methods in Python 3 (REF 001). Moreover, unknown_mthd does not accept args, and it can be bound to called by a class like a staticmethod, Test.unknown_mthd(). However, it is not an explicit staticmethod (no decorator).
Questions
Was making a method this way (without args while not explicitly decorated as staticmethods) intentional in Python 3's design? UPDATED
Among the classic method types, what type of method is unknown_mthd?
Why can unknown_mthd be called by the class without passing an argument?
Some preliminary inspection yields inconclusive results:
>>> # Types
>>> print("i", type(t.instance_mthd))
>>> print("c", type(Test.class_mthd))
>>> print("s", type(t.static_mthd))
>>> print("u", type(Test.unknown_mthd))
>>> print()
>>> # __dict__ Types, REF 002
>>> print("i", type(t.__class__.__dict__["instance_mthd"]))
>>> print("c", type(t.__class__.__dict__["class_mthd"]))
>>> print("s", type(t.__class__.__dict__["static_mthd"]))
>>> print("u", type(t.__class__.__dict__["unknown_mthd"]))
>>> print()
i <class 'method'>
c <class 'method'>
s <class 'function'>
u <class 'function'>
i <class 'function'>
c <class 'classmethod'>
s <class 'staticmethod'>
u <class 'function'>
The first set of type inspections suggests unknown_mthd is something similar to a staticmethod. The second suggests it resembles an instance method. I'm not sure what this method is or why it should be used over the classic ones. I would appreciate some advice on how to inspect and understand it better. Thanks.
REF 001: What's New in Python 3: “unbound methods” has been removed
REF 002: How to distinguish an instance method, a class method, a static method or a function in Python 3?
REF 003: What's the point of #staticmethod in Python?
Some background: In Python 2, "regular" instance methods could give rise to two kinds of method objects, depending on whether you accessed them via an instance or the class. If you did inst.meth (where inst is an instance of the class), you got a bound method object, which keeps track of which instance it is attached to, and passes it as self. If you did Class.meth (where Class is the class), you got an unbound method object, which had no fixed value of self, but still did a check to make sure a self of the appropriate class was passed when you called it.
In Python 3, unbound methods were removed. Doing Class.meth now just gives you the "plain" function object, with no argument checking at all.
Was making a method this way intentional in Python 3's design?
If you mean, was removal of unbound methods intentional, the answer is yes. You can see discussion from Guido on the mailing list. Basically it was decided that unbound methods add complexity for little gain.
Among the classic method types, what type of method is unknown_mthd?
It is an instance method, but a broken one. When you access it, a bound method object is created, but since it accepts no arguments, it's unable to accept the self argument and can't be successfully called.
Why can unknown_mthd be called by the class without passing an argument?
In Python 3, unbound methods were removed, so Test.unkown_mthd is just a plain function. No wrapping takes place to handle the self argument, so you can call it as a plain function that accepts no arguments. In Python 2, Test.unknown_mthd is an unbound method object, which has a check that enforces passing a self argument of the appropriate class; since, again, the method accepts no arguments, this check fails.
#BrenBarn did a great job answering your question. This answer however, adds a plethora of details:
First of all, this change in bound and unbound method is version-specific, and it doesn't relate to new-style or classic classes:
2.X classic classes by default
>>> class A:
... def meth(self): pass
...
>>> A.meth
<unbound method A.meth>
>>> class A(object):
... def meth(self): pass
...
>>> A.meth
<unbound method A.meth>
3.X new-style classes by default
>>> class A:
... def meth(self): pass
...
>>> A.meth
<function A.meth at 0x7efd07ea0a60>
You've already mentioned this in your question, it doesn't hurt to mention it twice as a reminder.
>>> # Python 2
>>> Test.unknown_mthd()
TypeError: unbound method unknown_mthd() must be called with Test instance as first argument (got nothing instead)
Moreover, unknown_mthd does not accept args, and it can be bound to a class like a staticmethod, Test.unknown_mthd(). However, it is not an explicit staticmethod (no decorator)
unknown_meth doesn't accept args, normally because you've defined the function without so that it does not take any parameter. Be careful and cautious, static methods as well as your coded unknown_meth method will not be magically bound to a class when you reference them through the class name (e.g, Test.unknown_meth). Under Python 3.X Test.unknow_meth returns a simple function object in 3.X, not a method bound to a class.
1 - Was making a method this way (without args while not explicitly decorated as staticmethods) intentional in Python 3's design? UPDATED
I cannot speak for CPython developers nor do I claim to be their representative, but from my experience as a Python programmer, it seems like they wanted to get rid of a bad restriction, especially given the fact that Python is extremely dynamic, not a language of restrictions; why would you test the type of objects passed to class methods and hence restrict the method to specific instances of classes? Type testing eliminates polymorphism. It would be decent if you just return a simple function when a method is fetched through the class which functionally behaves like a static method, you can think of unknown_meth to be static method under 3.X so long as you're careful not to fetch it through an instance of Test you're good to go.
2- Among the classic method types, what type of method is unknown_mthd?
Under 3.X:
>>> from types import *
>>> class Test:
... def unknown_mthd(): pass
...
>>> type(Test.unknown_mthd) is FunctionType
True
It's simply a function in 3.X as you could see. Continuing the previous session under 2.X:
>>> type(Test.__dict__['unknown_mthd']) is FunctionType
True
>>> type(Test.unknown_mthd) is MethodType
True
unknown_mthd is a simple function that lives inside Test__dict__, really just a simple function which lives inside the namespace dictionary of Test. Then, when does it become an instance of MethodType? Well, it becomes an instance of MethodType when you fetch the method attribute either from the class itself which returns an unbound method or its instances which returns a bound method. In 3.X, Test.unknown_mthd is a simple function--instance of FunctionType, and Test().unknown_mthd is an instance of MethodType that retains the original instance of class Test and adds it as the first argument implicitly on function calls.
3- Why can unknown_mthd be called by the class without passing an argument?
Again, because Test.unknown_mthd is just a simple function under 3.X. Whereas in 2.X, unknown_mthd not a simple function and must be called be passed an instance of Test when called.
Are there more than three types of methods in Python?
Yes. There are the three built-in kinds that you mention (instance method, class method, static method), four if you count #property, and anyone can define new method types.
Once you understand the mechanism for doing this, it's easy to explain why unknown_mthd is callable from the class in Python 3.
A new kind of method
Suppose we wanted to create a new type of method, call it optionalselfmethod so that we could do something like this:
class Test(object):
#optionalselfmethod
def optionalself_mthd(self, *args):
print('Optional-Self Method:', self, *args)
The usage is like this:
In [3]: Test.optionalself_mthd(1, 2)
Optional-Self Method: None 1 2
In [4]: x = Test()
In [5]: x.optionalself_mthd(1, 2)
Optional-Self Method: <test.Test object at 0x7fe80049d748> 1 2
In [6]: Test.instance_mthd(1, 2)
Instance method: 1 2
optionalselfmethod works like a normal instance method when called on an instance, but when called on the class, it always receives None for the first parameter. If it were a normal instance method, you would always have to pass an explicit value for the self parameter in order for it to work.
So how does this work? How you can you create a new method type like this?
The Descriptor Protocol
When Python looks up a field of an instance, i.e. when you do x.whatever, it check in several places. It checks the instance's __dict__ of course, but it also checks the __dict__ of the object's class, and base classes thereof. In the instance dict, Python is just looking for the value, so if x.__dict__['whatever'] exists, that's the value. However, in the class dict, Python is looking for an object which implements the Descriptor Protocol.
The Descriptor Protocol is how all three built-in kinds of methods work, it's how #property works, and it's how our special optionalselfmethod will work.
Basically, if the class dict has a value with the correct name1, Python checks if it has an __get__ method, and calls it like type(x).whatever.__get__(x, type(x)) Then, the value returned from __get__ is used as the field value.
So for example, a trivial descriptor which always returns 3:
class GetExample:
def __get__(self, instance, cls):
print("__get__", instance, cls)
return 3
class Test:
get_test = GetExample()
Usage is like this:
In[22]: x = Test()
In[23]: x.get_test
__get__ <__main__.Test object at 0x7fe8003fc470> <class '__main__.Test'>
Out[23]: 3
Notice that the descriptor is called with both the instance and the class type. It can also be used on the class:
In [29]: Test.get_test
__get__ None <class '__main__.Test'>
Out[29]: 3
When a descriptor is used on a class rather than an instance, the __get__ method gets None for self, but still gets the class argument.
This allows a simple implementation of methods: functions simply implement the descriptor protocol. When you call __get__ on a function, it returns a bound method of instance. If the instance is None, it returns the original function. You can actually call __get__ yourself to see this:
In [30]: x = object()
In [31]: def test(self, *args):
...: print(f'Not really a method: self<{self}>, args: {args}')
...:
In [32]: test
Out[32]: <function __main__.test>
In [33]: test.__get__(None, object)
Out[33]: <function __main__.test>
In [34]: test.__get__(x, object)
Out[34]: <bound method test of <object object at 0x7fe7ff92d890>>
#classmethod and #staticmethod are similar. These decorators create proxy objects with __get__ methods which provide different binding. Class method's __get__ binds the method to the instance, and static method's __get__ doesn't bind to anything, even when called on an instance.
The Optional-Self Method Implementation
We can do something similar to create a new method which optionally binds to an instance.
import functools
class optionalselfmethod:
def __init__(self, function):
self.function = function
functools.update_wrapper(self, function)
def __get__(self, instance, cls):
return boundoptionalselfmethod(self.function, instance)
class boundoptionalselfmethod:
def __init__(self, function, instance):
self.function = function
self.instance = instance
functools.update_wrapper(self, function)
def __call__(self, *args, **kwargs):
return self.function(self.instance, *args, **kwargs)
def __repr__(self):
return f'<bound optionalselfmethod {self.__name__} of {self.instance}>'
When you decorate a function with optionalselfmethod, the function is replaced with our proxy. This proxy saves the original method and supplies a __get__ method which returns a boudnoptionalselfmethod. When we create a boundoptionalselfmethod, we tell it both the function to call and the value to pass as self. Finally, calling the boundoptionalselfmethod calls the original function, but with the instance or None inserted into the first parameter.
Specific Questions
Was making a method this way (without args while not explicitly
decorated as staticmethods) intentional in Python 3's design? UPDATED
I believe this was intentional; however the intent would have been to eliminate unbound methods. In both Python 2 and Python 3, def always creates a function (you can see this by checking a type's __dict__: even though Test.instance_mthd comes back as <unbound method Test.instance_mthd>, Test.__dict__['instance_mthd'] is still <function instance_mthd at 0x...>).
In Python 2, function's __get__ method always returns a instancemethod, even when accessed through the class. When accessed through an instance, the method is bound to that instance. When accessed through the class, the method is unbound, and includes a mechanism which checks that the first argument is an instance of the correct class.
In Python 3, function's __get__ method will return the original function unchanged when accessed through the class, and a method when accessed through the instance.
I don't know the exact rationale but I would guess that type-checking of the first argument to a class-level function was deemed unnecessary, maybe even harmful; Python allows duck-typing after all.
Among the classic method types, what type of method is unknown_mthd?
unknown_mthd is a plain function, just like any normal instance method. It only fails when called through the instance because when method.__call__ attempts to call the function unknown_mthd with the bound instance, it doesn't accept enough parameters to receive the instance argument.
Why can unknown_mthd be called by the class without passing an
argument?
Because it's just a plain function, same as any other function. I just doesn't take enough arguments to work correctly when used as an instance method.
You may note that both classmethod and staticmethod work the same whether they're called through an instance or a class, whereas unknown_mthd will only work correctly when when called through the class and fail when called through an instance.
1. If a particular name has both a value in the instance dict and a descriptor in the class dict, which one is used depends on what kind of descriptor it is. If the descriptor only defines __get__, the value in the instance dict is used. If the descriptor also defines __set__, then it's a data-descriptor and the descriptor always wins. This is why you can assign over top of a method but not a #property; method only define __get__, so you can put things in the same-named slot in the instance dict, while #properties define __set__, so even if they're read-only, you'll never get a value from the instance __dict__ even if you've previously bypassed property lookup and stuck a value in the dict with e.g. x.__dict__['whatever'] = 3.
The Python 3.x language reference described two ways to create a method object:
User-defined method objects may be created when getting an attribute of a class (perhaps via an instance of that class), if that attribute is a user-defined function object or a class method object.
When an instance method object is created by retrieving a user-defined function object from a class via one of its instances, its self attribute is the instance, and the method object is said to be bound. The new method’s func attribute is the original function object.
When a user-defined method object is created by retrieving another method object from a class or instance, the behaviour is the same as for a function object, except that the func attribute of the new instance is not the original method object but its func attribute.
When an instance method object is called, the underlying function (func) is called, inserting the class instance (self) in front of the argument list. For instance, when C is a class which contains a definition for a function f(), and x is an instance of C, calling x.f(1) is equivalent to calling C.f(x, 1).
When an instance method object is derived from a class method object, the “class instance” stored in self will actually be the class itself, so that calling either x.f(1) or C.f(1) is equivalent to calling f(C,1) where f is the underlying function.
In different ways, they both have different __func__ and __self__ values, but I'm not very aware of these two different ways, could someone explain it to me?
Python Language Reference | The Standard Type Hierarchy:
https://docs.python.org/3/reference/datamodel.html#the-standard-type-hierarchy
I'm not 100% sure that I completely understand your question, but maybe looking at an example will be helpful. To start, lets create a class which has a function in the definition:
>>> class Foo(object):
... def method(self):
... pass
...
>>> f = Foo()
User-defined method objects may be created when getting an attribute of a class (perhaps via an instance of that class), if that attribute is a user-defined function object or a class method object.
Ok, so we can create a method object by just accessing the attribute on an instance (if the attribute is a function). In our setup, f is an instance of the class Foo:
>>> type(f.method)
<class 'method'>
Compare that with accessing the method attribute on the class:
>>> type(Foo.method)
<class 'function'>
When an instance method object is created by retrieving a user-defined function object from a class via one of its instances, its __self__ attribute is the instance, and the method object is said to be bound. The new method’s __func__ attribute is the original function object.
This is just telling us what attributes exist on instance methods. Let's check it out:
>>> instance_method = f.method
>>> instance_method.__func__ is Foo.method
True
>>> instance_method.__self__ is f
True
So we see that the method object has a __func__ attribute which is just a reference to the actual Foo.method function. It also has a __self__ attribute that is a reference to the instance.
When an instance method object is called, the underlying function (func) is called, inserting the class instance (self) in front of the argument list. For instance, when C is a class which contains a definition for a function f(), and x is an instance of C, calling x.f(1) is equivalent to calling C.f(x, 1).
Basically, in reference to our example above, this is just saying that If:
instance_method = f.method
Then:
instance_method(arg1, arg2)
executes the following:
instance_method.__func__(instance_method.__self__, arg1, arg2)
For completeness and as an addendum to the excellent answer provided by #mgilson, I wanted to explain the remaining 2 paragraphs referenced in the original question.
First let's create a class with a classmethod:
>>> class Foo(object):
... #classmethod
... def cmethod(cls):
... pass
...
>>> f = Foo()
Now for the 3rd paragraph:
When a user-defined method object is created by retrieving another method object from a class or instance, the behaviour is the same as for a function object, except that the func attribute of the new instance is not the original method object but its func attribute.
This means:
>>> class_method = f.cmethod
>>> class_method.__func__ is Foo.cmethod.__func__
True
>>> class_method.__self__ is Foo
True
Note that __self__ is a reference to the Foo class. Finally, the last paragraph:
When an instance method object is derived from a class method object, the “class instance” stored in self will actually be the class itself, so that calling either x.f(1) or C.f(1) is equivalent to calling f(C,1) where f is the underlying function.
This just says that all of the following are equivalent:
>>> f.cmethod(arg1, arg2)
>>> Foo.cmethod(arg1, arg2)
>>> f.cmethod.__func__(Foo, arg1, arg2)
>>> Foo.cmethod.__func__(Foo, arg1, arg2)
>>> f.cmethod.__func__(f.cmethod.__self__, arg1, arg2)
>>> Foo.cmethod.__func__(Foo.cmethod.__self__, arg1, arg2)
What is the difference between the following class methods?
Is it that one is static and the other is not?
class Test(object):
def method_one(self):
print "Called method_one"
def method_two():
print "Called method_two"
a_test = Test()
a_test.method_one()
a_test.method_two()
In Python, there is a distinction between bound and unbound methods.
Basically, a call to a member function (like method_one), a bound function
a_test.method_one()
is translated to
Test.method_one(a_test)
i.e. a call to an unbound method. Because of that, a call to your version of method_two will fail with a TypeError
>>> a_test = Test()
>>> a_test.method_two()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: method_two() takes no arguments (1 given)
You can change the behavior of a method using a decorator
class Test(object):
def method_one(self):
print "Called method_one"
#staticmethod
def method_two():
print "Called method two"
The decorator tells the built-in default metaclass type (the class of a class, cf. this question) to not create bound methods for method_two.
Now, you can invoke static method both on an instance or on the class directly:
>>> a_test = Test()
>>> a_test.method_one()
Called method_one
>>> a_test.method_two()
Called method_two
>>> Test.method_two()
Called method_two
Methods in Python are a very, very simple thing once you understood the basics of the descriptor system. Imagine the following class:
class C(object):
def foo(self):
pass
Now let's have a look at that class in the shell:
>>> C.foo
<unbound method C.foo>
>>> C.__dict__['foo']
<function foo at 0x17d05b0>
As you can see if you access the foo attribute on the class you get back an unbound method, however inside the class storage (the dict) there is a function. Why's that? The reason for this is that the class of your class implements a __getattribute__ that resolves descriptors. Sounds complex, but is not. C.foo is roughly equivalent to this code in that special case:
>>> C.__dict__['foo'].__get__(None, C)
<unbound method C.foo>
That's because functions have a __get__ method which makes them descriptors. If you have an instance of a class it's nearly the same, just that None is the class instance:
>>> c = C()
>>> C.__dict__['foo'].__get__(c, C)
<bound method C.foo of <__main__.C object at 0x17bd4d0>>
Now why does Python do that? Because the method object binds the first parameter of a function to the instance of the class. That's where self comes from. Now sometimes you don't want your class to make a function a method, that's where staticmethod comes into play:
class C(object):
#staticmethod
def foo():
pass
The staticmethod decorator wraps your class and implements a dummy __get__ that returns the wrapped function as function and not as a method:
>>> C.__dict__['foo'].__get__(None, C)
<function foo at 0x17d0c30>
Hope that explains it.
When you call a class member, Python automatically uses a reference to the object as the first parameter. The variable self actually means nothing, it's just a coding convention. You could call it gargaloo if you wanted. That said, the call to method_two would raise a TypeError, because Python is automatically trying to pass a parameter (the reference to its parent object) to a method that was defined as having no parameters.
To actually make it work, you could append this to your class definition:
method_two = staticmethod(method_two)
or you could use the #staticmethod function decorator.
>>> class Class(object):
... def __init__(self):
... self.i = 0
... def instance_method(self):
... self.i += 1
... print self.i
... c = 0
... #classmethod
... def class_method(cls):
... cls.c += 1
... print cls.c
... #staticmethod
... def static_method(s):
... s += 1
... print s
...
>>> a = Class()
>>> a.class_method()
1
>>> Class.class_method() # The class shares this value across instances
2
>>> a.instance_method()
1
>>> Class.instance_method() # The class cannot use an instance method
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unbound method instance_method() must be called with Class instance as first argument (got nothing instead)
>>> Class.instance_method(a)
2
>>> b = 0
>>> a.static_method(b)
1
>>> a.static_method(a.c) # Static method does not have direct access to
>>> # class or instance properties.
3
>>> Class.c # a.c above was passed by value and not by reference.
2
>>> a.c
2
>>> a.c = 5 # The connection between the instance
>>> Class.c # and its class is weak as seen here.
2
>>> Class.class_method()
3
>>> a.c
5
method_two won't work because you're defining a member function but not telling it what the function is a member of. If you execute the last line you'll get:
>>> a_test.method_two()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: method_two() takes no arguments (1 given)
If you're defining member functions for a class the first argument must always be 'self'.
Accurate explanation from Armin Ronacher above, expanding on his answers so that beginners like me understand it well:
Difference in the methods defined in a class, whether static or instance method(there is yet another type - class method - not discussed here so skipping it), lay in the fact whether they are somehow bound to the class instance or not. For example, say whether the method receives a reference to the class instance during runtime
class C:
a = []
def foo(self):
pass
C # this is the class object
C.a # is a list object (class property object)
C.foo # is a function object (class property object)
c = C()
c # this is the class instance
The __dict__ dictionary property of the class object holds the reference to all the properties and methods of a class object and thus
>>> C.__dict__['foo']
<function foo at 0x17d05b0>
the method foo is accessible as above. An important point to note here is that everything in python is an object and so references in the dictionary above are themselves pointing to other objects. Let me call them Class Property Objects - or as CPO within the scope of my answer for brevity.
If a CPO is a descriptor, then python interpretor calls the __get__() method of the CPO to access the value it contains.
In order to determine if a CPO is a descriptor, python interpretor checks if it implements the descriptor protocol. To implement descriptor protocol is to implement 3 methods
def __get__(self, instance, owner)
def __set__(self, instance, value)
def __delete__(self, instance)
for e.g.
>>> C.__dict__['foo'].__get__(c, C)
where
self is the CPO (it could be an instance of list, str, function etc) and is supplied by the runtime
instance is the instance of the class where this CPO is defined (the object 'c' above) and needs to be explicity supplied by us
owner is the class where this CPO is defined(the class object 'C' above) and needs to be supplied by us. However this is because we are calling it on the CPO. when we call it on the instance, we dont need to supply this since the runtime can supply the instance or its class(polymorphism)
value is the intended value for the CPO and needs to be supplied by us
Not all CPO are descriptors. For example
>>> C.__dict__['foo'].__get__(None, C)
<function C.foo at 0x10a72f510>
>>> C.__dict__['a'].__get__(None, C)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'list' object has no attribute '__get__'
This is because the list class doesnt implement the descriptor protocol.
Thus the argument self in c.foo(self) is required because its method signature is actually this C.__dict__['foo'].__get__(c, C) (as explained above, C is not needed as it can be found out or polymorphed)
And this is also why you get a TypeError if you dont pass that required instance argument.
If you notice the method is still referenced via the class Object C and the binding with the class instance is achieved via passing a context in the form of the instance object into this function.
This is pretty awesome since if you chose to keep no context or no binding to the instance, all that was needed was to write a class to wrap the descriptor CPO and override its __get__() method to require no context.
This new class is what we call a decorator and is applied via the keyword #staticmethod
class C(object):
#staticmethod
def foo():
pass
The absence of context in the new wrapped CPO foo doesnt throw an error and can be verified as follows:
>>> C.__dict__['foo'].__get__(None, C)
<function foo at 0x17d0c30>
Use case of a static method is more of a namespacing and code maintainability one(taking it out of a class and making it available throughout the module etc).
It maybe better to write static methods rather than instance methods whenever possible, unless ofcourse you need to contexualise the methods(like access instance variables, class variables etc). One reason is to ease garbage collection by not keeping unwanted reference to objects.
that is an error.
first of all, first line should be like this (be careful of capitals)
class Test(object):
Whenever you call a method of a class, it gets itself as the first argument (hence the name self) and method_two gives this error
>>> a.method_two()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: method_two() takes no arguments (1 given)
The second one won't work because when you call it like that python internally tries to call it with the a_test instance as the first argument, but your method_two doesn't accept any arguments, so it wont work, you'll get a runtime error.
If you want the equivalent of a static method you can use a class method.
There's much less need for class methods in Python than static methods in languages like Java or C#. Most often the best solution is to use a method in the module, outside a class definition, those work more efficiently than class methods.
The call to method_two will throw an exception for not accepting the self parameter the Python runtime will automatically pass it.
If you want to create a static method in a Python class, decorate it with the staticmethod decorator.
Class Test(Object):
#staticmethod
def method_two():
print "Called method_two"
Test.method_two()
Please read this docs from the Guido First Class everything Clearly explained how Unbound, Bound methods are born.
Bound method = instance method
Unbound method = static method.
The definition of method_two is invalid. When you call method_two, you'll get TypeError: method_two() takes 0 positional arguments but 1 was given from the interpreter.
An instance method is a bounded function when you call it like a_test.method_two(). It automatically accepts self, which points to an instance of Test, as its first parameter. Through the self parameter, an instance method can freely access attributes and modify them on the same object.
Unbound Methods
Unbound methods are methods that are not bound to any particular class instance yet.
Bound Methods
Bound methods are the ones which are bound to a specific instance of a class.
As its documented here, self can refer to different things depending on the function is bound, unbound or static.
Take a look at the following example:
class MyClass:
def some_method(self):
return self # For the sake of the example
>>> MyClass().some_method()
<__main__.MyClass object at 0x10e8e43a0># This can also be written as:>>> obj = MyClass()
>>> obj.some_method()
<__main__.MyClass object at 0x10ea12bb0>
# Bound method call:
>>> obj.some_method(10)
TypeError: some_method() takes 1 positional argument but 2 were given
# WHY IT DIDN'T WORK?
# obj.some_method(10) bound call translated as
# MyClass.some_method(obj, 10) unbound method and it takes 2
# arguments now instead of 1
# ----- USING THE UNBOUND METHOD ------
>>> MyClass.some_method(10)
10
Since we did not use the class instance — obj — on the last call, we can kinda say it looks like a static method.
If so, what is the difference between MyClass.some_method(10) call and a call to a static function decorated with a #staticmethod decorator?
By using the decorator, we explicitly make it clear that the method will be used without creating an instance for it first. Normally one would not expect the class member methods to be used without the instance and accesing them can cause possible errors depending on the structure of the method.
Also, by adding the #staticmethod decorator, we are making it possible to be reached through an object as well.
class MyClass:
def some_method(self):
return self
#staticmethod
def some_static_method(number):
return number
>>> MyClass.some_static_method(10) # without an instance
10
>>> MyClass().some_static_method(10) # Calling through an instance
10
You can’t do the above example with the instance methods. You may survive the first one (as we did before) but the second one will be translated into an unbound call MyClass.some_method(obj, 10) which will raise a TypeError since the instance method takes one argument and you unintentionally tried to pass two.
Then, you might say, “if I can call static methods through both an instance and a class, MyClass.some_static_method and MyClass().some_static_method should be the same methods.” Yes!