Say, I have the following class called Test with a method called start
>>> class Test:
... def __init__(self, *args, **kwargs):
... pass
... def start(self):
... pass
...
Now, I have a standalone independent function called func
>>> def func():
... print 'this is a func and not a method!!!'
...
>>>
[1]
Now, t.start is a method of an instance of __main__.Test which belong to 0xb769678c
>>> t = Test()
>>> t.start
<bound method Test.start of <__main__.Test instance at 0xb769678c>>
>>>
[2]
func is a function which belong to the location 0xb767ec6c
>>> func
<function func at 0xb767ec6c>
>>>
Now, we can extract the __module__ from t.start and func by using builtin __module__. Not surprisingly, func and t.start belong to the same module i.e. __main__
>>> func.__module__
'__main__'
>>> t.__module__
'__main__'
>>>
[3]
Now, lets store __module__ for t.start in a variable obj
>>> obj = __import__(t.start.__module__)
>>> obj
<module '__main__' (built-in)>
>>>
Now, I use getattr() to get the func handle <function func at 0xb767ec6c> for function func as follows and the output of getattr() is the identical to [2]
>>> print getattr(obj, 'func')
<function func at 0xb767ec6c>
>>>
>>> print getattr(__import__('__main__'), 'func')
<function func at 0xb767ec6c>
>>>
Question:
How do I use getattr() and the module name [3] to get the handle of Test.start [1] which should be <bound method Test.start of <__main__.Test instance at 0xb769678c>>
When I tried using getattr() on 't.start' I got the following Traceback
>>> print getattr(obj, 'Test.start')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'Test.start'
>>>
>>>
>>> print getattr(__import__('__main__'), 'Test.start')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'Test.start'
>>>
In other words, I have two data with me. They are
__import__('__main__')
The sting 'Test.start'
Now, how do I get the handle for t.start (note the instance here) which should be <bound method Test.start of <__main__.Test instance at 0xb769678c>>
I'm not sure if I understand your question(s), but I think this does what you want:
class Test:
def __init__(self, *args, **kwargs):
pass
def start(self):
pass
def func():
print('this is a func and not a method!!!')
t = Test()
module = __import__(t.start.__module__)
print(vars(module)['Test'].start)
print(vars(module)['func'])
print(vars(module)['t'].start)
(Python 3) output:
<function Test.start at 0x00E52460>
<function func at 0x00E524F0>
<bound method Test.start of <__main__.Test object at 0x008DF670>>
obj = __import__(t.start.__module__)
test_class = getattr(obj, 'Test')
print getattr(test_class, 'start')
I'm not sure if you need it straight from the module (or even if that is possible) :/
You could also use:
obj = __import__(t.start.__module__)
print obj.__dict__["Test"].__dict__["start"]
but you asked for getattr() so...
I wondered why you wrote obj = __import__(t.start.__module__)
I think it's to hold a name for the main module, in order to have the possibility to obtain the objects in the module's namespace as attributes of the module thanks to the getattr( ) function.
I inform you that you don't have to do this trick.
globals( ) is a dictionary that represents the global namespace of the main space.
Then you can write, for example, globals()["func"] to obtain the object func
I also inform you that there is another way than with getattr(N,"xyz") to obtain an object xyz in the namespace of the object N thanks to its name xyz ,
it's by the __dict__ method:
N.__dict__ gives access to the namespace of the object N
I wonder if your problem doesn't lies in a subtlety of Python that induced you in an error.
In your question, at a place you wrote t.__module__ and at another place you wrote t.start.__module__ .
For the two cases, the result is the same and equal to "__main__"
However,
1) t has no attribute of name __module__
2) t has even no attribute of name start !
If you print t.__dict__ you'll see the result is { } !
start isn't really belonging to the namespace of the instance t because it belongs in fact to the namespace of Test
The following code proves these affirmations:
class Test:
def __init__(self, *args, **kwargs):
pass
def start(self):
pass
y = Test()
y.ku = 102
print 'y.__dict__',y.__dict__
print 'Test.__dict__',Test.__dict__
result
y.__dict__ {'ku': 102}
Test.__dict__ {'start': <function start at 0x011E11B0>,
'__module__': '__main__',
'__doc__': None,
'__init__': <function __init__ at 0x011E1170>}
.
The reason why the expressions t.__module__ and t.start.__module__ give answers anyway is explained by this citation:
A class instance has a namespace implemented as a dictionary which is
the first place in which attribute references are searched.
When an
attribute is not found there, and the instance’s class has an
attribute by that name, the search continues with the class
attributes.
http://docs.python.org/2/reference/datamodel.html#index-55
Hence:
1) t has no attribute of name __module__ then Python searches for it in the class of t
2) t has no attribute of name start then Python searches for it in the class of t
.
Well, so, Python goes to the class of t to find the values of t.__module__ and t.start.__module__
This mechanism, described by the above citation, can be observed with the following code:
class Test:
def __init__(self, *args, **kwargs):
pass
def start(self):
pass
t = Test()
print 'getattr(t,"start")'
print getattr(t,"start")
# result is
getattr(t,"start")
<bound method Test.start of <__main__.Test instance at 0x011DF288>>
I mean that Python answers that the attribute of t whose name is "start" is Test.start , not t.start !
.
Thereby, from the fact that t.start.__module__ is "__main__", it seems to me that you may have believed that t.start is belonging to the module namespace, and hence you should be able to write something similar to getattr(obj,"func") to obtain the object start
But it is false.
You can't find the method start (I write the METHOD) in the global namespace = namespace of the module, since it is in the namespace of Test. You must attain it, as a method, through the instance or the class or through the class itself.
One more thing.
I precised : the METHOD, because I believe that the method t.start is based on a FUNCTION that is something different from it and that lies in the module's namespace. In reality, a method is a wrapper of pointers pointing to an instance and to this global namespace's function:
If you still don’t understand how methods work, a look at the
implementation can perhaps clarify matters. When an instance attribute
is referenced that isn’t a data attribute, its class is searched. If
the name denotes a valid class attribute that is a function object, a
method object is created by packing (pointers to) the instance object
and the function object just found together in an abstract object:
this is the method object. When the method object is called with an
argument list, a new argument list is constructed from the instance
object and the argument list, and the function object is called with
this new argument list.
http://docs.python.org/2/tutorial/classes.html#method-objects
Well what I want to underline is that it seems a function (not a method) exists somewhere, on which a method is based.
What I believe is that this function lies in the module namespace and that it's why Test.__dict__["start"]
gives
<function start at 0x011E40F0>
while
getattr(Test,"start")
and getattr(t,"start")
give
<unbound method Test.start>
and <bound method Test.start of <__main__.Test instance at 0x011DF530>>
without localizing the bound and unbound methods.
So, in my opinion, as far as I have correctly understood the doc (which is not explaining these points very well, I find, by the way) a method is a wrapper, and is based on a real function that lies in the module namespace, and that's why t.start.__module__ and Test.start.__module__ are "__main__"
It may seem a little weird that there exists a function in the module namespace that doesn't appear among the attributes of the module when we print them with globals() :
class Test:
def __init__(self, *args, **kwargs):
pass
def start(self):
pass
def func():
print 'this is a func and not a method!!!'
t = Test()
print '* globals()["func"]'
print globals()["func"]
print id(globals()["func"])
print "===================================="
print '* Test.__dict__["start"]'
print Test.__dict__["start"]
print id(Test.__dict__["start"])
print '----------------------------------------------'
print '* getattr(Test,"start")'
print getattr(Test,"start")
print id(getattr(Test,"start"))
print '----------------------------------------------'
print '* getattr(t,"start")'
print getattr(t,"start")
print id(getattr(t,"start"))
print "===================================="
print globals()
result
* globals()["func"]
<function func at 0x011C27B0>
18622384
====================================
* Test.__dict__["start"]
<function start at 0x011DEFB0>
18739120
----------------------------------------------
* getattr(Test,"start")
<unbound method Test.start>
18725304
----------------------------------------------
* getattr(t,"start")
<bound method Test.start of <__main__.Test instance at 0x011DF418>>
18725304
{'__builtins__': <module '__builtin__' (built-in)>,
'__package__': None,
't': <__main__.Test instance at 0x011DF418>,
'func': <function func at 0x011C27B0>,
'Test': <class __main__.Test at 0x011DC538>,
'__name__': '__main__',
'__doc__': None}
But the fact is that we indeed see a Test.__dict__["start"] function whose address 18739120 is different from the bound and unbound methods address 18725304.
Moreover, I don't see which other explanation could explain the totality of the facts exposed.
There's in fact nothing weird in the fact that a function that didn't receive any assigned name doesn't appear in the module namespace.
It is an internal function, necessary to Python, not at the disposal of the programmer, that's all.
Concerning func, using all the methods you employed:
class Test:
def __init__(self, *args, **kwargs):
pass
def start(self):
pass
def func():
print 'this is a func and not a method!!!'
t = Test()
print 'func --->',func
print id(func)
print '\nobj = __import__(t.start.__module__) done\n'
obj = __import__(t.start.__module__)
print '* globals()["func"]'
print globals()["func"]
print id(globals()["func"])
print '* obj.__dict__["func"]'
print obj.__dict__["func"]
print "* getattr(obj, 'func')"
print getattr(obj, 'func')
print "* getattr(__import__('__main__'), 'func')"
print getattr(__import__('__main__'), 'func')
result
func ---> <function func at 0x011C2470>
18621552
obj = __import__(t.start.__module__) done
* globals()["func"]
<function func at 0x011C2470> # <== address in hexadecimal
18621552 # <== address in decimal
* obj.__dict__["func"]
<function func at 0x011C2470>
* getattr(obj, 'func')
<function func at 0x011C2470>
* getattr(__import__('__main__'), 'func')
<function func at 0x011C2470>
.
Now taking apart your methods using obj :
class Test:
def __init__(self, *args, **kwargs):
pass
def start(self):
pass
print 'Test.start -->',Test.start
print id(Test.start)
print '----------------------------------------------'
print '* Test.__dict__["start"]'
print Test.__dict__["start"]
print id(Test.__dict__["start"])
print '* getattr(Test,"start")'
print getattr(Test,"start")
print id(getattr(Test,"start"))
print '\n'
print 't.start -->',t.start
print id(t.start)
print '----------------------------------------------'
print '* t.__dict__["start"]'
try:
print t.__dict__["start"]
print id(t.__dict__["start"])
except KeyError as e:
print 'KeyError :',e
print '* getattr(t,"start")'
print getattr(t,"start")
print id(getattr(t,"start"))
result
Test.start --> <unbound method Test.start>
18725264
----------------------------------------------
* Test.__dict__["start"]
<function start at 0x011E40F0>
18759920
* getattr(Test,"start")
<unbound method Test.start>
18725264
t.start --> <bound method Test.start of <__main__.Test instance at 0x011DB940>>
18725264
----------------------------------------------
* t.__dict__["start"]
KeyError : 'start'
* getattr(t,"start")
<bound method Test.start of <__main__.Test instance at 0x011DB940>>
18725264
Until now, I had seldom used getattr( ).
I realize on these results that getattr( ) gives results in a specific manner: the attribute start is described as a method, bound or unbound according if it is attained through an instance or through a class.
However, only __dict__ gives precise information on the real attributes of an object.
Thus, we see that start isn't really in the namespace of the instance, but only in the namespace of the class.
It is described as being a function when it is as element of the namespace Test.__dict__ and as being a method when given as an attribute through getattr( ). There are no addresses given in this last case.
The use of the vars( ) function gives the same results.
Related
In Python 2.7 I'd like to decorate an instance method test in class Foo with a decorator that is also a class called FooTestDecorator. From user Chirstop's question and the Python 2 docs' Descriptor HowTo guide I created this example.
There seems to be an issue however, when I print my decorated method object, it's (inspected?) name is wrong because it is noted as a question mark like Foo.?.
import types
class FooTestDecorator(object):
def __init__(self,func):
self.func=func
self.count=0
# tried self.func_name = func.func_name, but seemed to have no effect
def __get__(self,obj,objtype=None):
return types.MethodType(self,obj,objtype)
def __call__(self,*args,**kwargs):
self.count+=1
return self.func(*args,**kwargs)
class Foo:
#FooTestDecorator
def test(self,a):
print a
def bar(self,b):
print b
if you test it:
f=Foo()
print Foo.__dict__['test']
print Foo.test
print f.test
print Foo.__dict__['bar']
print Foo.bar
print f.bar
you get
<__main__.FooTestDecorator ...object...>
<unbound method Foo.?>
<bound method Foo.? of ...instance...>
<function bar at 0x...>
<unbound method Foo.bar>
<bound method Foo.bar of ...instance...>
You can see the replacement method is shown as Foo.?. This seems wrong.
How do I get my class-decorated instance method right?
note: My reason is that I want to use variables from the FooDecorator instance's self which I would set at init. I didn't put this in the example to keep it simpler.
Your decorator instance has no __name__ attribute, so Python has to do with a question mark instead.
Use functools.update_wrapper() to copy over the function name, plus a few other interesting special attributes (such as the docstring, the function module name and any custom attributes the function may have):
import types
from functools import update_wrapper
class FooTestDecorator(object):
def __init__(self,func):
self.func=func
self.count=0
update_wrapper(self, func)
def __get__(self,obj,objtype=None):
return types.MethodType(self,obj,objtype)
def __call__(self,*args,**kwargs):
self.count+=1
return self.func(*args,**kwargs)
Demo:
>>> f=Foo()
>>> print Foo.__dict__['test']
<__main__.FooTestDecorator object at 0x11077e210>
>>> print Foo.test
<unbound method Foo.test>
>>> print f.test
<bound method Foo.test of <__main__.Foo instance at 0x11077a830>>
In this piece of code:
def fa(a,b):
print a
print b
print " fa called"
class A:
#classmethod
def fa(a,b):
print a
print b
print " A.fa called"
class B:
def __init__(s,a):
s.a = a
obj1 = B(A.fa)
obj1.a("xxxx")
obj2 = B(fa)
obj2.a("xxxx")
Output:
__main__.A
xxxx
A.fa called
Traceback (most recent call last):
File "test.py", line 20, in <module>
obj2.a("xxxx")
TypeError: fa() takes exactly 2 arguments (1 given)
Why is the free method "fa" not receiving the "self" as a first parameter? The bound method A.fa behaves as expected.
The bound method A.fa receives A as the first parameter because it is a class method of A. No matter how you call this function it will always receive A as the first parameter.
The free method fa is not bound, so the only arguments it will receive are the ones that are passed in. No matter how you call this function, it will never receive parameters other than the ones that are passed in.
This behavior is different from a language like JavaScript, where how the method is called determines the context. In Python the implicit argument passing (similar to JavaScript context) is determined at the function definition time, and that binding or lack thereof will always be used for that function regardless of how it is called.
If you want to dynamically bind a free method you can do this using types.MethodType, for example:
def fa(x):
print x
class B: pass
>>> obj1 = B()
>>> obj1.a = MethodType(fa, obj1)
>>> obj1.a() # obj1.a behaves like an instance method bound to obj1
<__main__.B instance at 0x7f0589baf170>
>>> obj1.a2 = MethodType(fa, B)
>>> obj1.a2() # obj1.a2 acts like a class method bound to B
__main__.B
Because doing obj2.a = fa does not make a (fa) a method of obj2:
>>> class A(object):
... def meth(self, x, y):
... print x, y
...
>>>
>>> a = A()
>>>
>>> a.meth
<bound method A.meth of <__main__.A object at 0x10e281950>> # Method
>>>
>>> def fn(x, y):
... print x, y
...
>>>
>>> fn
<function fn at 0x10e287140>
>>> a.fn = fn
>>>
>>> a.fn
<function fn at 0x10e287140> # Not a method, still a function
Is it possible to access a method's attribute directly? I tried this and it fails:
class Test1:
def show_text(self):
self.my_text = 'hello'
Which results in:
>>> t = Test1()
>>> t.my_text
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: Test1 instance has no attribute 'my_text'
I found that using this made it work:
class Test1:
def __init__(self):
self.my_text = 'hello'
But I'm wondering if it's still possible to access attributes of methods directly? Or am I doing something Very Bad?
Instance variables are created once the object has been instantiated and only after they have been assigned to.
class Example(object):
def doSomething(self):
self.othervariable = 'instance variable'
>> foo = Example()
>> foo.othervariable
AttributeError: 'Example' object has no attribute 'othervariable'
Since othervariable is assigned inside doSomething - and we haven't called it yet -, it does not exist.
Once we call it, though:
>> foo.doSomething()
>> foo.othervariable
'instance variable'
__init__ is a special method that automatically gets invoked whenever class instantiation happens. Which is why when you assign your variable in there, it is accessible right after you create a new instance.
class Example(object):
def __init__(self):
self.othervariable = 'instance variable'
>> foo = Example()
>> foo.othervariable
'instance variable'
my_text attribute doesn't exist until you don't call show_text:
>>> class Test1:
... def show_text(self):
... self.my_text = 'hello'
...
>>> t = Test1()
>>> t.show_text()
>>> t.my_text
'hello'
If you want your attributes to be created during instance creation then place them in __init__ method.
Your first example didn't work: since you never use show_text() method, your object will never have attribute my_text (that will be "added" to your object only when you invoke that method).
Second example is good, because __init__ method is executed as soon as your object is instantiated.
Moreover, is a good practice to access object attribute through getter method on object itself so the best way you can modify your code is
class Test1:
def __init__(self,value):
self.my_text = value
def show_text(self):
return self.my_text
and then use in that way
t = Test1('hello')
t.show_text()
At last, will be also good to have a method like this
def set_text(self,new_text):
self.my_text = new_text
It is fairly easy to use the __getattr__ special method on Python classes to handle either missing properties or functions, but seemingly not both at the same time.
Consider this example which handles any property requested which is not defined explicitly elsewhere in the class...
class Props:
def __getattr__(self, attr):
return 'some_new_value'
>>> p = Props()
>>> p.prop # Property get handled
'some_new_value'
>>> p.func('an_arg', kw='keyword') # Function call NOT handled
Traceback (most recent call last):
File "<console>", line 1, in <module>
TypeError: 'str' object is not callable
Next, consider this example which handles any function call not defined explicitly elsewhere in the class...
class Funcs:
def __getattr__(self, attr):
def fn(*args, **kwargs):
# Do something with the function name and any passed arguments or keywords
print attr
print args
print kwargs
return
return fn
>>> f = Funcs()
>>> f.prop # Property get NOT handled
<function fn at 0x10df23b90>
>>> f.func('an_arg', kw='keyword') # Function call handled
func
('an_arg',)
{'kw': 'keyword'}
The question is how to handle both types of missing attributes in the same __getattr__? How to detect if the attribute requested was in property notation or in method notation with parentheses and return either a value or a function respectively? Essentially I want to handle SOME missing property attributes AND SOME missing function attributes and then resort to default behavior for all the other cases.
Advice?
How to detect if the attribute requested was in property notation or in method notation with parentheses and return either a value or a function respectively?
You can't. You also can't tell whether a requested method is an instance, class, or static method, etc. All you can tell is that someone is trying to retrieve an attribute for read access. Nothing else is passed into the getattribute machinery, so nothing else is available to your code.
So, you need some out-of-band way to know whether to create a function or some other kind of value. This is actually pretty common—you may actually be proxying for some other object that does have a value/function distinction (think of ctypes or PyObjC), or you may have a naming convention, etc.
However, you could always return an object that can be used either way. For example, if your "default behavior" is to return attributes are integers, or functions that return an integer, you can return something like this:
class Integerizer(object):
def __init__(self, value):
self.value = value
def __int__(self):
return self.value
def __call__(self, *args, **kw):
return self.value
There is no way to detect how the returned attribute was intended to be used. Everything on python objects are attributes, including the methods:
>>> class Foo(object):
... def bar(self): print 'bar called'
... spam='eggs'
...
>>> Foo.bar
<unbound method Foo.bar>
>>> Foo.spam
'eggs'
Python first looks up the attribute (bar or spam), and if you meant to call it (added parenthesis) then Python invokes the callable after lookup up the attribute:
>>> foo = Foo()
>>> fbar = foo.bar
>>> fbar()
'bar called'
In the above code I separated the lookup of bar from calling bar.
Since there is no distinction, you cannot detect in __getattr__ what the returned attribute will be used for.
__getattr__ is called whenever normal attribute access fails; in the following example monty is defined on the class, so __getattr__ is not called; it is only called for bar.eric and bar.john:
>>> class Bar(object):
... monty = 'python'
... def __getattr__(self, name):
... print 'Attribute access for {0}'.format(name)
... if name == 'eric':
... return 'idle'
... raise AttributeError(name)
...
>>> bar = Bar()
>>> bar.monty
'python'
>>> bar.eric
Attribute access for eric
'idle'
>>> bar.john
Attribute access for john
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 7, in __getattr__
AttributeError: john
Note that functions are not the only objects that you can invoke (call); any custom class that implements the __call__ method will do:
>>> class Baz(object):
... def __call__(self, name):
... print 'Baz sez: "Hello {0}!"'.format(name)
...
>>> baz = Baz()
>>> baz('John Cleese')
Baz sez: "Hello John Cleese!"
You could use that return objects from __getattr__ that can both be called and used as a value in different contexts.
Let's start with some code:
def func(*x):
print('func:', x)
class ABC:
def __init__(self, f):
self.f1 = f
def f2(*x):
print('f2:', x)
Now we do some tests:
>>> a = ABC(func)
>>> a.f1(10)
func: (10,)
>>> a.f2(10)
f2: (<__main__.ABC object at 0xb75381cc>, 10)
>>> a.f3 = func
>>> a.f3(10)
func: (10,)
>>> a.f1
<function func at 0xb74911ec>
>>> a.f2
<bound method ABC.f2 of <__main__.ABC object at 0xb75381cc>>
>>> a.f3
<function func at 0xb74911ec>
Note that func is a normal function and we are making it a method f1 of the class.
We can see that f2 is getting the class instance as the first argument, but f1 and f3 are not, even though all functions are called as class methods. We can also see that if we call a normal function as a method of a class, Python does not make a bound method from it.
So why is f1 or f3 NOT getting a class instance passed to it even when we are calling it as a method of a class? And also, how does Python know that we are calling an outer function as a method so that it should not pass an instance to it.
-- EDIT --
OK, so basically what I am doing wrong is that I am attaching the functions on the instance and NOT on the class object itself. These functions therefore simply become instance attributes. We can check this with:
>>> ABC.__dict__
... contents...
>>> a.__dict__
{'f1': <function func at 0xb74911ec>, 'f3': <function func at 0xb74911ec>}
Also note that this dict can not be assigned to:
>>> ABC.__dict__['f4'] = func
TypeError: 'dict_proxy' object does not support item assignment
You kind of partially answered your own question inspecting the object. In Python, objects behave like namespaces, so the first attribute points to a function and the second points to a method.
This is how you can add a method dynamically:
from types import MethodType
def func(*x):
print('func:', x)
class ABC:
def __init__(self, f):
self.f1 = MethodType(f, self, self.__class__)
def f2(*x):
print('f2:', x)
if __name__ == '__main__':
a = ABC(func)
print a.f1(10)
print a.f2(10)
a.f3 = MethodType(func, a, ABC)
print a.f3(10)
Note that it will bind the method to your instance, not to the base class. In order to monkeypatch the ABC class:
>>> ABC.f4 = MethodType(func, None, ABC)
>>> a.f4(1)
('func:', (<__main__.ABC instance at 0x02AA8AD0>, 1))
Monkeypatching is usually frowned upon in the Python circles, despite being popular in other dynamic languages (notably in Ruby when the language was younger).
If you ever resort to this powerful yet dangerous technique, my advice is:
never, ever override an existing class method. just don't.
That's because f1 and f3 are not class method they are just references to a global function defined in __main__:
In [5]: a.f1
Out[5]: <function __main__.func>
In [8]: a.f3
Out[8]: <function __main__.func>
In [9]: a.f2
Out[9]: <bound method ABC.f2 of <__main__.ABC instance at 0x8ac04ac>>
you can do something like this to make a global function a class method:
In [16]: class ABC:
def __init__(self,f):
ABC.f1=f
def f2(*x):
print('f2',x)
....:
In [17]: a=ABC(func)
In [18]: a.f1(10)
('func:', (<__main__.ABC instance at 0x8abb7ec>, 10))