Related
I have this bit of code:
class ABC:
def test():
print('Hi there')
ABC.test()
Which outputs:
Hi there
Whereas this fails:
ABC().test()
TypeError: test() takes 0 positional arguments but 1 was given
# Makes sense
We know that when we invoke a classmethod like <classname>.<methodname>(), the class is implicitly passed as an argument to it, but test here takes no arguments.
Why don't we see a TypeError because of the unexpected argument? Isn't the class passed as an argument to test?
Nothing is passed to test() because functions do not bind to anything when accessed to a class. It remains unbound, you get the original function:
>>> class ABC:
... def test():
... print('Hi there')
...
>>> ABC.test
<function ABC.test at 0x1082706c0>
You can call functions directly, provided you pass in the right number of arguments. Here that is 0, so ABC.test() succeeds.
It is not a classmethod, that would require decorating it with #classmethod (or otherwise producing a classmethod object and storing it as an attribute of the class), at which point accessing the attribute binds the function to the class object, producing a bound method:
>>> class ABC:
... #classmethod
... def test():
... print('Hi there')
...
>>> ABC.test
<bound method ABC.test of <class '__main__.ABC'>>
Calling this results in an exception:
>>> ABC.test()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: test() takes 0 positional arguments but 1 was given
Functions can be bound too, but only when accessed on an instance of the class. Both function and classmethod objects implement the descriptor protocol, and accessing attributes on classes and on instances will trigger the __get__ method on descriptors. But only classmethod objects return a bound method in that case, functions just return themselves.
Specifically, the descriptor __get__ method is passed None for the first argument, the object to bind to, when the attribute access is on a class, and the instance when accessed on the instance. classmethod().__get__() ignores the instance argument and produces a method object that binds to the second argument (owner). Functions return self when the first argument is None, or a method object that binds to the instance otherwise:
>>> def foo(boundto='nothing'):
... print(f'Bound to {boundto}')
...
>>> foo.__get__('instance', 'owner')()
Bound to instance
>>> foo.__get__(None, 'owner')()
Bound to nothing
>>> classmethod(foo).__get__('instance', 'owner')()
Bound to owner
>>> classmethod(foo).__get__(None, 'owner')()
Bound to owner
What arguments do types.MethodType expect and what does it return?
https://docs.python.org/3.6/library/types.html doesn't say more about it:
types.MethodType
The type of methods of user-defined class instances.
For an example, from https://docs.python.org/3.6/howto/descriptor.html
To support method calls, functions include the __get__() method for
binding methods during attribute access. This means that all functions
are non-data descriptors which return bound or unbound methods
depending whether they are invoked from an object or a class. In pure
python, it works like this:
class Function(object):
. . .
def __get__(self, obj, objtype=None):
"Simulate func_descr_get() in Objects/funcobject.c"
if obj is None:
return self
return types.MethodType(self, obj)
Must the first argument self of types.MethodType be a callable object? In other words, must the class Function be a callable type, i.e. must Function have a method __call__?
If self is a callable object, does it take at least one argument?
Does types.MethodType(self, obj) mean giving obj as the first argument to the callable object self, i.e. currying self with obj?
How does types.MethodType(self, obj) create and return an instance of types.MethodType?
Thanks.
Usually you don't need to create instance of types.MethodType yourself. Instead, you'll get one automatically when you access a method on an instance of a class.
For example, if we make a class, create an instance of it, then access a method on the instance (without calling it), we'll get an instance of types.MethodType back:
import types
class Foo:
def bar(self):
pass
foo = Foo()
method = foo.bar
print(type(method) == types.MethodType) # prints True
The code you excerpt in your question is trying to show how this normally happens. It's not something you usually have to do yourself, though you can if you really want to. For instance, to create another instance of types.MethodType equivalent to method above, we could do:
method_manual = types.MethodType(Foo.bar, foo)
The first argument to MethodType is a callable object (normally a function, but it can be something else, like an instance of the Function class in the example you were reading). The second argument what we're binding the function to. When you call the method object (with e.g. method()), the bound object will be passed into the function as the first argument.
Usually the object the method gets bound to is an instance, though it can be something else. For instance, a classmethod decorated function will bind to the class it is called on, rather than an instance. Here's an example of that (both getting a method bound to a class automatically, and doing it manually ourselves):
class Foo2:
#classmethod
def baz(cls):
pass
foo2 = Foo2()
method2 = Foo2.baz
method2_via_an_instance = foo2.baz
method2_manual = types.MethodType(method2.__func__, Foo2)
All three of the method2-prefixed variables work exactly the same way (when you call them, they'll all call baz with Foo2 as the cls argument). The only wonky thing about the manual approach this time is that it's hard to get at the original baz function without getting a bound method instead, so I fished it out of one of the other bound method objects.
A final note: The name types.MethodType is an alias for the internal type used for bound methods, which doesn't otherwise have an accessible name. Unlike many classes, the repr of an instance is not an expression to recreate it (it will be something like "<bound method Foo.bar of <__main__.Foo object at 0x0000...>>"). Nor is the repr of the type a valid name to access the type by (the repr is "method").
Short Answer:
Must the first argument self of types.MethodType be a callable object?
In other words, must the class Function be a callable type, i.e. must
Function have a method __call__?
Yes
If self is a callable object, does it take at least one argument?
Depends
Does types.MethodType(self, obj) mean giving obj as the first argument
to the callable object self, i.e. currying self with obj?
Yes
How does types.MethodType(self, obj) create and return an instance of
types.MethodType?
It doesn't work like that.
Long Answer:
the code
class Function(object):
. . .
def __get__(self, obj, objtype=None):
"Simulate func_descr_get() in Objects/funcobject.c"
if obj is None:
return self
return types.MethodType(self, obj)
As Daniel explained is mainly to demonstrate for
To support method calls, functions include the __get__() method for
binding methods during attribute access. This means that all functions
are non-data descriptors which return bound or unbound methods
depending whether they are invoked from an object or a class. In pure
python, it works like this:
The types.MethodType() works when the Function has an object.
if obj is None would be False
Then it's a method of some object aka. bound method.
It explains how Python grammar work. As a function, it could be called in the
following two ways.
some_func_() or some_class.some_func()
The former part https://docs.python.org/3.6/howto/descriptor.html#invoking-descriptors explained.
For objects, the machinery is in object.__getattribute__() which
transforms b.x into type(b).__dict__['x'].__get__(b, type(b)). The
implementation works through a precedence chain that gives data
descriptors priority over instance variables, instance variables
priority over non-data descriptors, and assigns lowest priority to
__getattr__() if provided.
Here it's some demonstrate code
>>> import types
>>> types.MethodType
<type 'instancemethod'>
>>> def a(self):
... print(1)
...
>>> class B:
... pass
...
>>> types.MethodType(a,B)
<bound method ?.a of <class __main__.B at 0x7f4d3d5aa598>>
>>> B.t = types.MethodType(a,B)
>>> B.t()
1
>>> def s():
... print(3)
...
>>> B.r = types.MethodType(s,B)
>>> B.r
<bound method ?.s of <class __main__.B at 0x7f4d3d5aa598>>
>>> B.r()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: s() takes no arguments (1 given)
See also dynamically adding callable to class as instance "method"
Documentation doesn't say much, but you can always check its source code. The signature of MethodType constructor is:
def __init__(self, func: Callable[..., Any], obj: object) -> None: ...
It accepts a callable and object, and returns None.
MethodType can be used to add an instance method to an object, instead of a function; here's an example:
from types import MethodType
class MyClass:
language = 'Python'
# a function is bound to obj1
obj1 = MyClass()
obj1.say_hello = lambda: 'Hello World!'
print(type(obj1.say_hello)) # type is class 'function'
obj1.say_hello()
# a method is bound to obj2
obj2 = MyClass()
# this is used to bind a "method" to a specific object obj2, rather than a function
obj2.say_hello = MethodType(lambda self: f'Hello {self.language}!', obj2)
print(type(obj2.say_hello)) # type is class 'method'
obj2.say_hello()
It's not something you would ever call. Like most of the classes in the types module, it's more for comparing with existing objects (for example in isinstance).
As far as I have understood, a decorator is a function that takes another function as a parameter, performs some operation on it and then returns it.
Talking about the decorator #staticmethod, what exactly does that decorator do to eliminate the instance object that is being passed by default ?
The staticmethod decorator returns a staticmethod object. This object implements the descriptor protocol, same as functions do.
What this does is not that it gets rid of the instance, instead the staticmethod.__get__ ignores binding altogether and just returns the unaltered function object. For regular functions, function.__get__ would instead bind by returning a method object (which then tracks the instance and the function to combine them when called).
You can reproduce this by manually invoking the descriptor protocol:
>>> class Demo:
... def regular(self):
... pass
... #staticmethod
... def static():
... pass
...
>>> Demo.__dict__['regular'] # bypass __getattribute__
<function Demo.regular at 0x108515268>
>>> Demo.__dict__['static'] # bypass __getattribute__
<staticmethod object at 0x1084d4f60>
>>> Demo.__dict__['regular'].__get__(Demo()) # descriptor protocol, pass in an instance
<bound method Demo.regular of <__main__.Demo object at 0x1084e2668>>
>>> Demo.__dict__['static'].__get__(Demo()) # descriptor protocol, pass in an instance
<function Demo.static at 0x1085152f0>
By accessing the attributes of the Demo class through Demo.__dict__, we bypass the descriptor protocol normally enforced by the __getattribute__ method. As you can see, for a regular method a function object is returned, but for static a staticmethod object is found instead.
Calling .__get__(Demo()) on either to invoke the descriptor protocol then produces a method object, and the un-altered function object, respectively. This is exactly what direct access to the same names on an instance produces:
>>> Demo().regular
<bound method Demo.regular of <__main__.Demo object at 0x1084dde10>>
>>> Demo().static
<function Demo.static at 0x1085152f0>
Note that the same protocol is also the reason that classmethod objects are passed type(instance) instead of the instance as a first argument, and also why property objects call the underlying function on access.
New to Python and having done some reading, I'm making some methods in my custom class class methods rather than instance methods.
So I tested my code but I hadn't changed some of the method calls to call the method in the class rather than the instance, but they still worked:
class myClass:
#classmethod:
def foo(cls):
print 'Class method foo called with %s.'%(cls)
def bar(self):
print 'Instance method bar called with %s.'%(self)
myClass.foo()
thing = myClass()
thing.foo()
thing.bar()
This produces:
class method foo called with __main__.myClass.
class method foo called with __main__.myClass.
instance method bar called with <__main__.myClass instance at 0x389ba4>.
So what I'm wondering is why I can call a class method (foo) on an instance (thing.foo), (although it's the class that gets passed to the method)? It kind of makes sense, as 'thing' is a 'myClass', but I was expecting Python to give an error saying something along the lines of 'foo is a class method and can't be called on an instance'.
Is this just an expected consequence of inheritance with the 'thing' object inheriting the foo method from its superclass?
If I try to call the instance method via the class:
myClass.bar()
then I get:
TypeError: unbound method bar() must be called with myClass instance...
which makes perfect sense.
You can call it on an instance because #classmethod is a decorator (it takes a function as an argument and returns a new function).
Here is some relavent information from the Python documentation
It can be called either on the class (such as C.f()) or on an instance
(such as C().f()). The instance is ignored except for its class. If a
class method is called for a derived class, the derived class object
is passed as the implied first argument.
There's also quite a good SO discussion on #classmethod here.
Let's start with a quick overview of the descriptor protocol. If a class defines a __get__ method, an instance of that class is a descriptor. Accessing a descriptor as the attribute of another object produces the return value of the __get__ method, not the descriptor itself.
A function is a descriptor; this is how instance methods are implemented. Given
class myClass:
#classmethod:
def foo(cls):
print('Class method foo called with %s.' % (cls,))
def bar(self):
print 'Instance method bar called with %s.'%(self)
c = myClass()
the expression c.bar is equivalent to
myClass.__dict__['bar'].__get__(c, myClass)
while the expression myClass.bar is equivalent to the expression
myClass.__dict__['bar'].__get__(None, myClass)
Note the only difference is in the object passed as the first argument to __get__. The former returns a new method object, which when called passes c and its own arguments on to the function bar. This is why c.bar() and C.bar(c) are equivalent. The latter simply returns the function bar itself.
classmethod is a type that provides a different implementation of __get__. This means that c.foo() and myClass.foo() call __get__ as before:
# c.foo
myClass.__dict__['foo'].__get__(c, myClass)
# myClass.foo
myClass.__dict__['foo'].__get__(None, myClass)
Now, however, both calls return the same method object, and this method object, when called, passes myClass as the first argument to the original function object. That is, c.foo() is equivalent to myClass.foo(), which
is equivalent to x(myClass) (where x is the original function defined before the decoration bound the name foo to an instance of classmethod).
What is the difference between the following class methods?
Is it that one is static and the other is not?
class Test(object):
def method_one(self):
print "Called method_one"
def method_two():
print "Called method_two"
a_test = Test()
a_test.method_one()
a_test.method_two()
In Python, there is a distinction between bound and unbound methods.
Basically, a call to a member function (like method_one), a bound function
a_test.method_one()
is translated to
Test.method_one(a_test)
i.e. a call to an unbound method. Because of that, a call to your version of method_two will fail with a TypeError
>>> a_test = Test()
>>> a_test.method_two()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: method_two() takes no arguments (1 given)
You can change the behavior of a method using a decorator
class Test(object):
def method_one(self):
print "Called method_one"
#staticmethod
def method_two():
print "Called method two"
The decorator tells the built-in default metaclass type (the class of a class, cf. this question) to not create bound methods for method_two.
Now, you can invoke static method both on an instance or on the class directly:
>>> a_test = Test()
>>> a_test.method_one()
Called method_one
>>> a_test.method_two()
Called method_two
>>> Test.method_two()
Called method_two
Methods in Python are a very, very simple thing once you understood the basics of the descriptor system. Imagine the following class:
class C(object):
def foo(self):
pass
Now let's have a look at that class in the shell:
>>> C.foo
<unbound method C.foo>
>>> C.__dict__['foo']
<function foo at 0x17d05b0>
As you can see if you access the foo attribute on the class you get back an unbound method, however inside the class storage (the dict) there is a function. Why's that? The reason for this is that the class of your class implements a __getattribute__ that resolves descriptors. Sounds complex, but is not. C.foo is roughly equivalent to this code in that special case:
>>> C.__dict__['foo'].__get__(None, C)
<unbound method C.foo>
That's because functions have a __get__ method which makes them descriptors. If you have an instance of a class it's nearly the same, just that None is the class instance:
>>> c = C()
>>> C.__dict__['foo'].__get__(c, C)
<bound method C.foo of <__main__.C object at 0x17bd4d0>>
Now why does Python do that? Because the method object binds the first parameter of a function to the instance of the class. That's where self comes from. Now sometimes you don't want your class to make a function a method, that's where staticmethod comes into play:
class C(object):
#staticmethod
def foo():
pass
The staticmethod decorator wraps your class and implements a dummy __get__ that returns the wrapped function as function and not as a method:
>>> C.__dict__['foo'].__get__(None, C)
<function foo at 0x17d0c30>
Hope that explains it.
When you call a class member, Python automatically uses a reference to the object as the first parameter. The variable self actually means nothing, it's just a coding convention. You could call it gargaloo if you wanted. That said, the call to method_two would raise a TypeError, because Python is automatically trying to pass a parameter (the reference to its parent object) to a method that was defined as having no parameters.
To actually make it work, you could append this to your class definition:
method_two = staticmethod(method_two)
or you could use the #staticmethod function decorator.
>>> class Class(object):
... def __init__(self):
... self.i = 0
... def instance_method(self):
... self.i += 1
... print self.i
... c = 0
... #classmethod
... def class_method(cls):
... cls.c += 1
... print cls.c
... #staticmethod
... def static_method(s):
... s += 1
... print s
...
>>> a = Class()
>>> a.class_method()
1
>>> Class.class_method() # The class shares this value across instances
2
>>> a.instance_method()
1
>>> Class.instance_method() # The class cannot use an instance method
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unbound method instance_method() must be called with Class instance as first argument (got nothing instead)
>>> Class.instance_method(a)
2
>>> b = 0
>>> a.static_method(b)
1
>>> a.static_method(a.c) # Static method does not have direct access to
>>> # class or instance properties.
3
>>> Class.c # a.c above was passed by value and not by reference.
2
>>> a.c
2
>>> a.c = 5 # The connection between the instance
>>> Class.c # and its class is weak as seen here.
2
>>> Class.class_method()
3
>>> a.c
5
method_two won't work because you're defining a member function but not telling it what the function is a member of. If you execute the last line you'll get:
>>> a_test.method_two()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: method_two() takes no arguments (1 given)
If you're defining member functions for a class the first argument must always be 'self'.
Accurate explanation from Armin Ronacher above, expanding on his answers so that beginners like me understand it well:
Difference in the methods defined in a class, whether static or instance method(there is yet another type - class method - not discussed here so skipping it), lay in the fact whether they are somehow bound to the class instance or not. For example, say whether the method receives a reference to the class instance during runtime
class C:
a = []
def foo(self):
pass
C # this is the class object
C.a # is a list object (class property object)
C.foo # is a function object (class property object)
c = C()
c # this is the class instance
The __dict__ dictionary property of the class object holds the reference to all the properties and methods of a class object and thus
>>> C.__dict__['foo']
<function foo at 0x17d05b0>
the method foo is accessible as above. An important point to note here is that everything in python is an object and so references in the dictionary above are themselves pointing to other objects. Let me call them Class Property Objects - or as CPO within the scope of my answer for brevity.
If a CPO is a descriptor, then python interpretor calls the __get__() method of the CPO to access the value it contains.
In order to determine if a CPO is a descriptor, python interpretor checks if it implements the descriptor protocol. To implement descriptor protocol is to implement 3 methods
def __get__(self, instance, owner)
def __set__(self, instance, value)
def __delete__(self, instance)
for e.g.
>>> C.__dict__['foo'].__get__(c, C)
where
self is the CPO (it could be an instance of list, str, function etc) and is supplied by the runtime
instance is the instance of the class where this CPO is defined (the object 'c' above) and needs to be explicity supplied by us
owner is the class where this CPO is defined(the class object 'C' above) and needs to be supplied by us. However this is because we are calling it on the CPO. when we call it on the instance, we dont need to supply this since the runtime can supply the instance or its class(polymorphism)
value is the intended value for the CPO and needs to be supplied by us
Not all CPO are descriptors. For example
>>> C.__dict__['foo'].__get__(None, C)
<function C.foo at 0x10a72f510>
>>> C.__dict__['a'].__get__(None, C)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'list' object has no attribute '__get__'
This is because the list class doesnt implement the descriptor protocol.
Thus the argument self in c.foo(self) is required because its method signature is actually this C.__dict__['foo'].__get__(c, C) (as explained above, C is not needed as it can be found out or polymorphed)
And this is also why you get a TypeError if you dont pass that required instance argument.
If you notice the method is still referenced via the class Object C and the binding with the class instance is achieved via passing a context in the form of the instance object into this function.
This is pretty awesome since if you chose to keep no context or no binding to the instance, all that was needed was to write a class to wrap the descriptor CPO and override its __get__() method to require no context.
This new class is what we call a decorator and is applied via the keyword #staticmethod
class C(object):
#staticmethod
def foo():
pass
The absence of context in the new wrapped CPO foo doesnt throw an error and can be verified as follows:
>>> C.__dict__['foo'].__get__(None, C)
<function foo at 0x17d0c30>
Use case of a static method is more of a namespacing and code maintainability one(taking it out of a class and making it available throughout the module etc).
It maybe better to write static methods rather than instance methods whenever possible, unless ofcourse you need to contexualise the methods(like access instance variables, class variables etc). One reason is to ease garbage collection by not keeping unwanted reference to objects.
that is an error.
first of all, first line should be like this (be careful of capitals)
class Test(object):
Whenever you call a method of a class, it gets itself as the first argument (hence the name self) and method_two gives this error
>>> a.method_two()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: method_two() takes no arguments (1 given)
The second one won't work because when you call it like that python internally tries to call it with the a_test instance as the first argument, but your method_two doesn't accept any arguments, so it wont work, you'll get a runtime error.
If you want the equivalent of a static method you can use a class method.
There's much less need for class methods in Python than static methods in languages like Java or C#. Most often the best solution is to use a method in the module, outside a class definition, those work more efficiently than class methods.
The call to method_two will throw an exception for not accepting the self parameter the Python runtime will automatically pass it.
If you want to create a static method in a Python class, decorate it with the staticmethod decorator.
Class Test(Object):
#staticmethod
def method_two():
print "Called method_two"
Test.method_two()
Please read this docs from the Guido First Class everything Clearly explained how Unbound, Bound methods are born.
Bound method = instance method
Unbound method = static method.
The definition of method_two is invalid. When you call method_two, you'll get TypeError: method_two() takes 0 positional arguments but 1 was given from the interpreter.
An instance method is a bounded function when you call it like a_test.method_two(). It automatically accepts self, which points to an instance of Test, as its first parameter. Through the self parameter, an instance method can freely access attributes and modify them on the same object.
Unbound Methods
Unbound methods are methods that are not bound to any particular class instance yet.
Bound Methods
Bound methods are the ones which are bound to a specific instance of a class.
As its documented here, self can refer to different things depending on the function is bound, unbound or static.
Take a look at the following example:
class MyClass:
def some_method(self):
return self # For the sake of the example
>>> MyClass().some_method()
<__main__.MyClass object at 0x10e8e43a0># This can also be written as:>>> obj = MyClass()
>>> obj.some_method()
<__main__.MyClass object at 0x10ea12bb0>
# Bound method call:
>>> obj.some_method(10)
TypeError: some_method() takes 1 positional argument but 2 were given
# WHY IT DIDN'T WORK?
# obj.some_method(10) bound call translated as
# MyClass.some_method(obj, 10) unbound method and it takes 2
# arguments now instead of 1
# ----- USING THE UNBOUND METHOD ------
>>> MyClass.some_method(10)
10
Since we did not use the class instance — obj — on the last call, we can kinda say it looks like a static method.
If so, what is the difference between MyClass.some_method(10) call and a call to a static function decorated with a #staticmethod decorator?
By using the decorator, we explicitly make it clear that the method will be used without creating an instance for it first. Normally one would not expect the class member methods to be used without the instance and accesing them can cause possible errors depending on the structure of the method.
Also, by adding the #staticmethod decorator, we are making it possible to be reached through an object as well.
class MyClass:
def some_method(self):
return self
#staticmethod
def some_static_method(number):
return number
>>> MyClass.some_static_method(10) # without an instance
10
>>> MyClass().some_static_method(10) # Calling through an instance
10
You can’t do the above example with the instance methods. You may survive the first one (as we did before) but the second one will be translated into an unbound call MyClass.some_method(obj, 10) which will raise a TypeError since the instance method takes one argument and you unintentionally tried to pass two.
Then, you might say, “if I can call static methods through both an instance and a class, MyClass.some_static_method and MyClass().some_static_method should be the same methods.” Yes!