I am having trouble understanding why the following happens. I am having a decorator which does nothing except it checks whether a function is a method. I thought I have understood what method in Python is, but obviously, this is not the case:
import inspect
def deco(f):
def g(*args):
print inspect.ismethod(f)
return f(*args)
return g
class Adder:
#deco
def __call__(self, a):
return a + 1
class Adder2:
def __call__(self, a):
return a + 2
Adder2.__call__ = deco(Adder2.__call__)
Now, running the following:
>>> a = Adder()
>>> a(1)
False
2
>>> a2 = Adder2()
>>> a2(1)
True
3
I would expect for this code to print True two times.
So, decorating the function manually as in the Adder2 is not an exact equivalent to decorating via the #deco function?
Can someone be so glad and explain why this happens?
Methods are functions that are associated with a class. Methods are only created when you retrieve them from an already defined class; a method is a wrapper around a function, with a reference to the class as well (and optionally a reference to the instance).
What happens in the first case is: Python compiles your class definition Adder. It finds the decorator definition and a function. The decorator is passed the function, returning a new function. That function is added to the class definition (stored in the class __dict__). All this time you are dealing with a python function, not a method. That happens later.
When you then call a(1), a lookup reveals that the instance doesn't have a __call__ but the Adder class does, so it is retrieved using __getattribute__(). This finds a function (your deco decorator), which is a descriptor so it's __get__() method is called (so Adder.__call__.__get__(a, Adder)), returning a bound method, which is then called and passed in the 1 value. The method is bound because instance is not None when __get__() is called. Your decorator, which wrapped a function at class building time, prints False because it was passed an unwrapped function to start with.
In the second case, however, you retrieve a method (again via __getattribute__() calling __get__() on the undecorated Adder2.__call__ function), this time unbound (as there is no instance, only a class passed to __get__() (the full call is Adder2.__call__.__get__(None, Adder2)), and you then decorate that method. Now ismethod() prints True.
Note that in Python 3, the latter case changes. In Python 3 there no longer is a concept of an unbound method, only functions and bound methods. The term 'bound' is thus dropped altogether. Your second case would also print False as Adder2.__call__.__get__(None, Adder2) returns a function in that case.
Inside a class definition, __call__ is a function, not a method. The act of accessing the function through attribute lookup (e.g. using the dot syntax), such as with Adder2.__call__, returns an unbound method. And a2.__call__ returns a bound method (with self bound to a2).
Note that in Python3 the concept of unbound method has been dropped. There, Adder2.__call__ is a function as well.
Related
I am trying to create a class which gets given a function, which will then be run from that instance. However, when I tried to use staticmethod, I discovered that there is a difference between using the decorator and just passing staticmethod a function.
class WithDec():
def __init__(self):
pass
#staticmethod
def stat(val):
return val + 1
def OuterStat(val):
return val + 1
class WithoutDec():
def __init__(self, stat):
self.stat = staticmethod(stat)
With these two classes, the following occurs.
>>> WithDec().stat(2)
3
>>> WithoutDec(OuterStat).stat(2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'staticmethod' object is not callable
What is going on, and what can I do to stop it.
Static methods still work through the descriptor protocol, meaning that when it is a class attribute, accessing it via an instance still means that the __get__ method will be called to return an object that actually gets called. That is,
WithDec().stat(2)
is equivalent to
w = WithDec()
w.stat(2)
which is equivalent to
WithDec.stat.__get__(w, WithDec)(2)
However, the descriptor protocol is not invoked when the static method is an instance attribute, as is the case with WithoutDec. In that case
WithoutDec().stat(2)
tries to call the literal staticmethod instance stat, not the the function returned by stat.__get__.
What you wanted was to use staticmethod to create a class attribute, just not via decorator syntax:
class WithoutDec():
def stat(val):
return val + 1
stat = staticmethod(stat)
You first bind stat to a regular function (it's not really an instance method until you try to use it as an instance method), then replace the function with a staticmethod instance wrapping the original function.
The problem is that you are trying to use staticmethod() inside __init__, which is used to create an instance of the class, instead of at the class level directly, which defines the class, its methods and its static methods.
This code works:
def OuterStat(val):
return val + 1
class WithoutDec():
stat = staticmethod(OuterStat)
>>> WithoutDec.stat(2)
3
Note that trying to create an instance of WithoutDec with its own, different, version of stat, is contrary to the meaning of a method being static.
I found a very inspiring solution on this thread. Indeed your code is not very pythonic, and attributes a static method to an attribute of an instance of your class. The following code works:
class WithoutDec():
stat = None
#staticmethod
def OuterStat(val):
return val + 1
then you call:
my_without_dec = WithoutDec()
my_without_dec.stat = WithotuDec.OuterStat
my_without_dec.stat(2)
later if you want to create a new method, you call:
def new_func(val):
return val+1
WithoutDec.newStat = staticmethod(new_func)
my_without_dec.stat = WithoutDec.newStat
my_without_dec.stat(2)
Yes -
In this case, you just have to add the function as an attribute of the instance, it will work as expected, no need for any decorators:
def OuterStat(val):
return val + 1
class WithoutDec():
def __init__(self, stat):
self.stat = stat
The thing is: there is a difference if a function is an attribute of the class or an attribute of the instance. When it is set inside an instance method with self.func = X, it becomes an instance attribute - Python retrieves it the way it was stored, with no modifications, and it is simply another reference to the original function that can be called.
When a function is stored as a class attibute, instead, the default behavior is that it is used as an instance method: upon retrieving the function from an instance, Python arranges things so that self will be injected as the first argument to that function. In this case, the decorators #classmethod and #staticmethod exist to modify this behavior (injetct the class for classmethod or make no injection for staticmethod).
The thing is that staticmethod does not return a function - it returns a descriptor to be used as a class attribute, so that when the decorated function is retrieved from a class, it works as a plain function.
(Internal detail: all 3 behaviors: instance method, classmethod and staticmethod are implementing by having an appropriate __get__ method on the object that is used as an attribute to the class).
NB: There were some discussions in making "staticmethod" to become itself "callable", and simply call the wrapped function - I just checked it made it into Pythonn 3.10 beta 1. This means that your example code will work as is for Python 3.10 - nonetheless, the staticmethod call there is redundant, as stated in the beggining of this answer, and should not be used.
I have following class with a function:
class A:
def myfn():
print("In myfn method.")
Here, the function does not have self as argument. It also does not have #classmethod or #staticmethod as decorator. However, it works if called with class:
A.myfn()
Output:
In myfn method.
But give an error if called from any instance:
a = A()
a.myfn()
Error output:
Traceback (most recent call last):
File "testing.py", line 16, in <module>
a.myfn()
TypeError: myfn() takes 0 positional arguments but 1 was given
probably because self was also sent as an argument.
What kind of function will this be called? Will it be a static function? Is it advisable to use function like this in classes? What is the drawback?
Edit: This function works only when called with class and not with object/instance. My main question is what is such a function called?
Edit2: It seems from the answers that this type of function, despite being the simplest form, is not accepted as legal. However, as no serious drawback is mentioned in any of many answers, I find this can be a useful construct, especially to group my own static functions in a class that I can call as needed. I would not need to create any instance of this class. In the least, it saves me from typing #staticmethod every time and makes code look less complex. It also gets derived neatly for someone to extend my class. Although all such functions can be kept at top/global level, keeping them in class is more modular. However, I feel there should be a specific name for such a simple construct which works in this specific way and it should be recognized as legal. It may also help beginners understand why self argument is needed for usual functions in a Python class. This will only add to the simplicity of this great language.
The function type implements the descriptor protocol, which means when you access myfn via the class or an instance of the class, you don't get the actual function back; you get instead the result of that function's __get__ method. That is,
A.myfn == A.myfn.__get__(None, A)
Here, myfn is an instance method, though one that hasn't been defined properly to be used as such. When accessed via the class, though, the return value of __get__ is simply the function object itself, and the function can be called the same as a static method.
Access via an instance results in a different call to __get__. If a is an instance of A, then
a.myfn() == A.myfn.__get__(a, A)
Here , __get__ tries to return, essentially, a partial application of myfn to a, but because myfn doesn't take any arguments, that fails.
You might ask, what is a static method? staticmethod is a type that wraps a function and defines its own __get__ method. That method returns the underlying function whether or not the attribute is accessed via the class or an instance. Otherwise, there is very little difference between a static method and an ordinary function.
This is not a true method. Correctly declarated instance methods should have a self argument (the name is only a convention and can be changed if you want hard to read code), and classmethods and staticmethods should be introduced by their respective decorator.
But at a lower level, def in a class declaration just creates a function and assigns it to a class member. That is exactly what happens here: A.my_fn is a function and can successfully be called as A.my_fn().
But as it was not declared with #staticmethod, it is not a true static method and it cannot be applied on a A instance. Python sees a member of that name that happens to be a function which is neither a static nor a class method, so it prepends the current instance to the list of arguments and tries to execute it.
To answer your exact question, this is not a method but just a function that happens to be assigned to a class member.
Such a function isn't the same as what #staticmethod provides, but is indeed a static method of sorts.
With #staticmethod you can also call the static method on an instance of the class. If A is a class and A.a is a static method, you'll be able to do both A.a() and A().a(). Without this decorator, only the first example will work, because for the second one, as you correctly noticed, "self [will] also [be] sent as an argument":
class A:
#staticmethod
def a():
return 1
Running this:
>>> A.a() # `A` is the class itself
1
>>> A().a() # `A()` is an instance of the class `A`
1
On the other hand:
class B:
def b():
return 2
Now, the second version doesn't work:
>>> B.b()
2
>>> B().b()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: b() takes 0 positional arguments but 1 was given
further to #chepnet's answer, if you define a class whose objects implement the descriptor protocol like:
class Descr:
def __get__(self, obj, type=None):
print('get', obj, type)
def __set__(self, obj, value):
print('set', obj, value)
def __delete__(self, obj):
print('delete', obj)
you can embed an instance of this in a class and invoke various operations on it:
class Foo:
foo = Descr()
Foo.foo
obj = Foo()
obj.foo
which outputs:
get None <class '__main__.Foo'>
get <__main__.Foo object at 0x106d4f9b0> <class '__main__.Foo'>
as functions also implement the descriptor protocol, we can replay this by doing:
def bar():
pass
print(bar)
print(bar.__get__(None, Foo))
print(bar.__get__(obj, Foo))
which outputs:
<function bar at 0x1062da730>
<function bar at 0x1062da730>
<bound method bar of <__main__.Foo object at 0x106d4f9b0>>
hopefully that complements chepnet's answer which I found a little terse/opaque
I understand there are at least 3 kinds of methods in Python having different first arguments:
instance method - instance, i.e. self
class method - class, i.e. cls
static method - nothing
These classic methods are implemented in the Test class below including an usual method:
class Test():
def __init__(self):
pass
def instance_mthd(self):
print("Instance method.")
#classmethod
def class_mthd(cls):
print("Class method.")
#staticmethod
def static_mthd():
print("Static method.")
def unknown_mthd():
# No decoration --> instance method, but
# No self (or cls) --> static method, so ... (?)
print("Unknown method.")
In Python 3, the unknown_mthd can be called safely, yet it raises an error in Python 2:
>>> t = Test()
>>> # Python 3
>>> t.instance_mthd()
>>> Test.class_mthd()
>>> t.static_mthd()
>>> Test.unknown_mthd()
Instance method.
Class method.
Static method.
Unknown method.
>>> # Python 2
>>> Test.unknown_mthd()
TypeError: unbound method unknown_mthd() must be called with Test instance as first argument (got nothing instead)
This error suggests such a method was not intended in Python 2. Perhaps its allowance now is due to the elimination of unbound methods in Python 3 (REF 001). Moreover, unknown_mthd does not accept args, and it can be bound to called by a class like a staticmethod, Test.unknown_mthd(). However, it is not an explicit staticmethod (no decorator).
Questions
Was making a method this way (without args while not explicitly decorated as staticmethods) intentional in Python 3's design? UPDATED
Among the classic method types, what type of method is unknown_mthd?
Why can unknown_mthd be called by the class without passing an argument?
Some preliminary inspection yields inconclusive results:
>>> # Types
>>> print("i", type(t.instance_mthd))
>>> print("c", type(Test.class_mthd))
>>> print("s", type(t.static_mthd))
>>> print("u", type(Test.unknown_mthd))
>>> print()
>>> # __dict__ Types, REF 002
>>> print("i", type(t.__class__.__dict__["instance_mthd"]))
>>> print("c", type(t.__class__.__dict__["class_mthd"]))
>>> print("s", type(t.__class__.__dict__["static_mthd"]))
>>> print("u", type(t.__class__.__dict__["unknown_mthd"]))
>>> print()
i <class 'method'>
c <class 'method'>
s <class 'function'>
u <class 'function'>
i <class 'function'>
c <class 'classmethod'>
s <class 'staticmethod'>
u <class 'function'>
The first set of type inspections suggests unknown_mthd is something similar to a staticmethod. The second suggests it resembles an instance method. I'm not sure what this method is or why it should be used over the classic ones. I would appreciate some advice on how to inspect and understand it better. Thanks.
REF 001: What's New in Python 3: “unbound methods” has been removed
REF 002: How to distinguish an instance method, a class method, a static method or a function in Python 3?
REF 003: What's the point of #staticmethod in Python?
Some background: In Python 2, "regular" instance methods could give rise to two kinds of method objects, depending on whether you accessed them via an instance or the class. If you did inst.meth (where inst is an instance of the class), you got a bound method object, which keeps track of which instance it is attached to, and passes it as self. If you did Class.meth (where Class is the class), you got an unbound method object, which had no fixed value of self, but still did a check to make sure a self of the appropriate class was passed when you called it.
In Python 3, unbound methods were removed. Doing Class.meth now just gives you the "plain" function object, with no argument checking at all.
Was making a method this way intentional in Python 3's design?
If you mean, was removal of unbound methods intentional, the answer is yes. You can see discussion from Guido on the mailing list. Basically it was decided that unbound methods add complexity for little gain.
Among the classic method types, what type of method is unknown_mthd?
It is an instance method, but a broken one. When you access it, a bound method object is created, but since it accepts no arguments, it's unable to accept the self argument and can't be successfully called.
Why can unknown_mthd be called by the class without passing an argument?
In Python 3, unbound methods were removed, so Test.unkown_mthd is just a plain function. No wrapping takes place to handle the self argument, so you can call it as a plain function that accepts no arguments. In Python 2, Test.unknown_mthd is an unbound method object, which has a check that enforces passing a self argument of the appropriate class; since, again, the method accepts no arguments, this check fails.
#BrenBarn did a great job answering your question. This answer however, adds a plethora of details:
First of all, this change in bound and unbound method is version-specific, and it doesn't relate to new-style or classic classes:
2.X classic classes by default
>>> class A:
... def meth(self): pass
...
>>> A.meth
<unbound method A.meth>
>>> class A(object):
... def meth(self): pass
...
>>> A.meth
<unbound method A.meth>
3.X new-style classes by default
>>> class A:
... def meth(self): pass
...
>>> A.meth
<function A.meth at 0x7efd07ea0a60>
You've already mentioned this in your question, it doesn't hurt to mention it twice as a reminder.
>>> # Python 2
>>> Test.unknown_mthd()
TypeError: unbound method unknown_mthd() must be called with Test instance as first argument (got nothing instead)
Moreover, unknown_mthd does not accept args, and it can be bound to a class like a staticmethod, Test.unknown_mthd(). However, it is not an explicit staticmethod (no decorator)
unknown_meth doesn't accept args, normally because you've defined the function without so that it does not take any parameter. Be careful and cautious, static methods as well as your coded unknown_meth method will not be magically bound to a class when you reference them through the class name (e.g, Test.unknown_meth). Under Python 3.X Test.unknow_meth returns a simple function object in 3.X, not a method bound to a class.
1 - Was making a method this way (without args while not explicitly decorated as staticmethods) intentional in Python 3's design? UPDATED
I cannot speak for CPython developers nor do I claim to be their representative, but from my experience as a Python programmer, it seems like they wanted to get rid of a bad restriction, especially given the fact that Python is extremely dynamic, not a language of restrictions; why would you test the type of objects passed to class methods and hence restrict the method to specific instances of classes? Type testing eliminates polymorphism. It would be decent if you just return a simple function when a method is fetched through the class which functionally behaves like a static method, you can think of unknown_meth to be static method under 3.X so long as you're careful not to fetch it through an instance of Test you're good to go.
2- Among the classic method types, what type of method is unknown_mthd?
Under 3.X:
>>> from types import *
>>> class Test:
... def unknown_mthd(): pass
...
>>> type(Test.unknown_mthd) is FunctionType
True
It's simply a function in 3.X as you could see. Continuing the previous session under 2.X:
>>> type(Test.__dict__['unknown_mthd']) is FunctionType
True
>>> type(Test.unknown_mthd) is MethodType
True
unknown_mthd is a simple function that lives inside Test__dict__, really just a simple function which lives inside the namespace dictionary of Test. Then, when does it become an instance of MethodType? Well, it becomes an instance of MethodType when you fetch the method attribute either from the class itself which returns an unbound method or its instances which returns a bound method. In 3.X, Test.unknown_mthd is a simple function--instance of FunctionType, and Test().unknown_mthd is an instance of MethodType that retains the original instance of class Test and adds it as the first argument implicitly on function calls.
3- Why can unknown_mthd be called by the class without passing an argument?
Again, because Test.unknown_mthd is just a simple function under 3.X. Whereas in 2.X, unknown_mthd not a simple function and must be called be passed an instance of Test when called.
Are there more than three types of methods in Python?
Yes. There are the three built-in kinds that you mention (instance method, class method, static method), four if you count #property, and anyone can define new method types.
Once you understand the mechanism for doing this, it's easy to explain why unknown_mthd is callable from the class in Python 3.
A new kind of method
Suppose we wanted to create a new type of method, call it optionalselfmethod so that we could do something like this:
class Test(object):
#optionalselfmethod
def optionalself_mthd(self, *args):
print('Optional-Self Method:', self, *args)
The usage is like this:
In [3]: Test.optionalself_mthd(1, 2)
Optional-Self Method: None 1 2
In [4]: x = Test()
In [5]: x.optionalself_mthd(1, 2)
Optional-Self Method: <test.Test object at 0x7fe80049d748> 1 2
In [6]: Test.instance_mthd(1, 2)
Instance method: 1 2
optionalselfmethod works like a normal instance method when called on an instance, but when called on the class, it always receives None for the first parameter. If it were a normal instance method, you would always have to pass an explicit value for the self parameter in order for it to work.
So how does this work? How you can you create a new method type like this?
The Descriptor Protocol
When Python looks up a field of an instance, i.e. when you do x.whatever, it check in several places. It checks the instance's __dict__ of course, but it also checks the __dict__ of the object's class, and base classes thereof. In the instance dict, Python is just looking for the value, so if x.__dict__['whatever'] exists, that's the value. However, in the class dict, Python is looking for an object which implements the Descriptor Protocol.
The Descriptor Protocol is how all three built-in kinds of methods work, it's how #property works, and it's how our special optionalselfmethod will work.
Basically, if the class dict has a value with the correct name1, Python checks if it has an __get__ method, and calls it like type(x).whatever.__get__(x, type(x)) Then, the value returned from __get__ is used as the field value.
So for example, a trivial descriptor which always returns 3:
class GetExample:
def __get__(self, instance, cls):
print("__get__", instance, cls)
return 3
class Test:
get_test = GetExample()
Usage is like this:
In[22]: x = Test()
In[23]: x.get_test
__get__ <__main__.Test object at 0x7fe8003fc470> <class '__main__.Test'>
Out[23]: 3
Notice that the descriptor is called with both the instance and the class type. It can also be used on the class:
In [29]: Test.get_test
__get__ None <class '__main__.Test'>
Out[29]: 3
When a descriptor is used on a class rather than an instance, the __get__ method gets None for self, but still gets the class argument.
This allows a simple implementation of methods: functions simply implement the descriptor protocol. When you call __get__ on a function, it returns a bound method of instance. If the instance is None, it returns the original function. You can actually call __get__ yourself to see this:
In [30]: x = object()
In [31]: def test(self, *args):
...: print(f'Not really a method: self<{self}>, args: {args}')
...:
In [32]: test
Out[32]: <function __main__.test>
In [33]: test.__get__(None, object)
Out[33]: <function __main__.test>
In [34]: test.__get__(x, object)
Out[34]: <bound method test of <object object at 0x7fe7ff92d890>>
#classmethod and #staticmethod are similar. These decorators create proxy objects with __get__ methods which provide different binding. Class method's __get__ binds the method to the instance, and static method's __get__ doesn't bind to anything, even when called on an instance.
The Optional-Self Method Implementation
We can do something similar to create a new method which optionally binds to an instance.
import functools
class optionalselfmethod:
def __init__(self, function):
self.function = function
functools.update_wrapper(self, function)
def __get__(self, instance, cls):
return boundoptionalselfmethod(self.function, instance)
class boundoptionalselfmethod:
def __init__(self, function, instance):
self.function = function
self.instance = instance
functools.update_wrapper(self, function)
def __call__(self, *args, **kwargs):
return self.function(self.instance, *args, **kwargs)
def __repr__(self):
return f'<bound optionalselfmethod {self.__name__} of {self.instance}>'
When you decorate a function with optionalselfmethod, the function is replaced with our proxy. This proxy saves the original method and supplies a __get__ method which returns a boudnoptionalselfmethod. When we create a boundoptionalselfmethod, we tell it both the function to call and the value to pass as self. Finally, calling the boundoptionalselfmethod calls the original function, but with the instance or None inserted into the first parameter.
Specific Questions
Was making a method this way (without args while not explicitly
decorated as staticmethods) intentional in Python 3's design? UPDATED
I believe this was intentional; however the intent would have been to eliminate unbound methods. In both Python 2 and Python 3, def always creates a function (you can see this by checking a type's __dict__: even though Test.instance_mthd comes back as <unbound method Test.instance_mthd>, Test.__dict__['instance_mthd'] is still <function instance_mthd at 0x...>).
In Python 2, function's __get__ method always returns a instancemethod, even when accessed through the class. When accessed through an instance, the method is bound to that instance. When accessed through the class, the method is unbound, and includes a mechanism which checks that the first argument is an instance of the correct class.
In Python 3, function's __get__ method will return the original function unchanged when accessed through the class, and a method when accessed through the instance.
I don't know the exact rationale but I would guess that type-checking of the first argument to a class-level function was deemed unnecessary, maybe even harmful; Python allows duck-typing after all.
Among the classic method types, what type of method is unknown_mthd?
unknown_mthd is a plain function, just like any normal instance method. It only fails when called through the instance because when method.__call__ attempts to call the function unknown_mthd with the bound instance, it doesn't accept enough parameters to receive the instance argument.
Why can unknown_mthd be called by the class without passing an
argument?
Because it's just a plain function, same as any other function. I just doesn't take enough arguments to work correctly when used as an instance method.
You may note that both classmethod and staticmethod work the same whether they're called through an instance or a class, whereas unknown_mthd will only work correctly when when called through the class and fail when called through an instance.
1. If a particular name has both a value in the instance dict and a descriptor in the class dict, which one is used depends on what kind of descriptor it is. If the descriptor only defines __get__, the value in the instance dict is used. If the descriptor also defines __set__, then it's a data-descriptor and the descriptor always wins. This is why you can assign over top of a method but not a #property; method only define __get__, so you can put things in the same-named slot in the instance dict, while #properties define __set__, so even if they're read-only, you'll never get a value from the instance __dict__ even if you've previously bypassed property lookup and stuck a value in the dict with e.g. x.__dict__['whatever'] = 3.
For the error:
TypeError: takes exactly 1 argument (2 given)
With the following class method:
def extractAll(tag):
...
and calling it:
e.extractAll("th")
The error seems very odd when I'm giving it 1 argument, the method should take only 1 argument, but it's saying I'm not giving it 1 argument....I know the problem can be fixed by adding self into the method prototype but I wanted to know the reasoning behind the error.
Am I getting it because the act of calling it via e.extractAll("th") also passes in self as an argument? And if so, by removing the self in the call, would I be making it some kind of class method that can be called like Extractor.extractAll("th")?
The call
e.extractAll("th")
for a regular method extractAll() is indeed equivalent to
Extractor.extractAll(e, "th")
These two calls are treated the same in all regards, including the error messages you get.
If you don't need to pass the instance to a method, you can use a staticmethod:
#staticmethod
def extractAll(tag):
...
which can be called as e.extractAll("th"). But I wonder why this is a method on a class at all if you don't need to access any instance.
If a non-static method is member of a class, you have to define it like that:
def Method(self, atributes..)
So, I suppose your 'e' is instance of some class with implemented method that tries to execute and has too much arguments.
Am I getting it because the act of calling it via e.extractAll("th") also passes in self as an argument?
Yes, that's precisely it. If you like, the first parameter is the object name, e that you are calling it with.
And if so, by removing the self in the call, would I be making it some kind of class method that can be called like Extractor.extractAll("th")?
Not quite. A classmethod needs the #classmethod decorator, and that accepts the class as the first paramater (usually referenced as cls). The only sort of method that is given no automatic parameter at all is known as a staticmethod, and that again needs a decorator (unsurprisingly, it's #staticmethod). A classmethod is used when it's an operation that needs to refer to the class itself: perhaps instantiating objects of the class; a staticmethod is used when the code belongs in the class logically, but requires no access to class or instance.
But yes, both staticmethods and classmethods can be called by referencing the classname as you describe: Extractor.extractAll("th").
Yes, when you invoke e.extractAll(foo), Python munges that into extractAll(e, foo).
From http://docs.python.org/tutorial/classes.html
the special thing about methods is
that the object is passed as the first
argument of the function. In our
example, the call x.f() is exactly
equivalent to MyClass.f(x). In
general, calling a method with a list
of n arguments is equivalent to
calling the corresponding function
with an argument list that is created
by inserting the method’s object
before the first argument.
Emphasis added.
Summary (Some examples of how to define methods in classes in python)
#!/usr/bin/env python # (if running from bash)
class Class1(object):
def A(self, arg1):
print arg1
# this method requires an instance of Class1
# can access self.variable_name, and other methods in Class1
#classmethod
def B(cls, arg1):
cls.C(arg1)
# can access methods B and C in Class1
#staticmethod
def C(arg1):
print arg1
# can access methods B and C in Class1
# (i.e. via Class1.B(...) and Class1.C(...))
Example
my_obj=Class1()
my_obj.A("1")
# Class1.A("2") # TypeError: method A() must be called with Class1 instance
my_obj.B("3")
Class1.B("4")
my_obj.C("5")
Class1.C("6")`
try using:
def extractAll(self,tag):
attention to self
[Updated]: Answer inline below question
I have an inspecting program and one objective is for logic in a decorator to know whether the function it is decorating is a class method or regular function. This is failing in a strange way. Below is code run in Python 2.6:
def decorate(f):
print 'decorator thinks function is', f
return f
class Test(object):
#decorate
def test_call(self):
pass
if __name__ == '__main__':
Test().test_call()
print 'main thinks function is', Test().test_call
Then on execution:
decorator thinks function is <function test_call at 0x10041cd70>
main thinks function is <bound method Test.test_call of <__main__.Test object at 0x100425a90>>
Any clue on what's going wrong, and if it is possible for #decorate to correctly infer that test_call is a method?
[Answer]
carl's answer below is nearly perfect. I had a problem when using the decorator on a method that subclasses call. I adapted his code to include a im_func comparison on superclass members:
ismethod = False
for item in inspect.getmro(type(args[0])):
for x in inspect.getmembers(item):
if 'im_func' in dir(x[1]):
ismethod = x[1].im_func == newf
if ismethod:
break
else:
continue
break
As others have said, a function is decorated before it is bound, so you cannot directly determine whether it's a 'method' or 'function'.
A reasonable way to determine if a function is a method or not is to check whether 'self' is the first parameter. While not foolproof, most Python code adheres to this convention:
import inspect
ismethod = inspect.getargspec(method).args[0] == 'self'
Here's a convoluted way that seems to automatically figure out whether the method is a bound or not. Works for a few simple cases on CPython 2.6, but no promises. It decides a function is a method if the first argument to is an object with the decorated function bound to it.
import inspect
def decorate(f):
def detect(*args, **kwargs):
try:
members = inspect.getmembers(args[0])
members = (x[1].im_func for x in members if 'im_func' in dir(x[1]))
ismethod = detect in members
except:
ismethod = False
print ismethod
return f(*args, **kwargs)
return detect
#decorate
def foo():
pass
class bar(object):
#decorate
def baz(self):
pass
foo() # prints False
bar().baz() # prints True
No, this is not possible as you have requested, because there is no inherent difference between bound methods and functions. A method is simply a function wrapped up to get the calling instance as the first argument (using Python descriptors).
A call like:
Test.test_call
which returns an unbound method, translates to
Test.__dict__[ 'test_call' ].__get__( None, spam )
which is an unbound method, even though
Test.__dict__[ 'test_call' ]
is a function. This is because functions are descriptors whose __get__ methods return methods; when Python sees one of these in the lookup chain it calls the __get__ method instead of continuing up the chain.
In effect, the 'bound-methodiness' of a function is determined at runtime, not at define-time!
The decorator simply sees the function as it is defined, without looking it up in a __dict__, so cannot tell whether it is looking at a bound method.
It might be possible to do this with a class decorator that modifies __getattribute__, but that's a particularly nasty hack. Why must you have this functionality? Surely, since you have to place the decorator on the function yourself, you could pass it an argument that says whether said function is defined within a class?
class Test:
#decorate( method = True )
def test_call:
...
#decorate( method = False )
def test_call:
...
Your decorator is run before the function becomes a method. def keyword inside a class defines a function line in any other place, then the functions defined in the body of a class are added to the class as methods. Decorator operates on the function before it is processed by the class that is why your code 'fails'.
There is no way for the #decorate to see the function is actually a method. A workaround for that would be to decorate the function whatever it is (e.g. adding an attribute do_something_about_me_if_I_am_a_method ;-)) and then process it again after the class is computed (iterating over the class members and doing whatever you want with those decorated).
I tried a slightly different example, with one decorated method and one undecorated method.
def decorate(f):
print 'decorator thinks function is', f
return f
class Test(object):
#decorate
def test_call(self):
pass
def test_call_2(self):
pass
if __name__ == '__main__':
print 'main thinks function is', Test.test_call
print 'main thinks function 2 is', Test.test_call_2
Then the output is:
decorator thinks function is <function test_call at 0x100426b18>
main thinks function is <unbound method Test.test_call>
main thinks function 2 is <unbound method Test.test_call_2>
Thus, the decorator saw a different type than the main function did, but the decorator did not change the function's type, or it would be different from the undecorated function.