i want the next code to work
class A(object):
def __getattr__(self, item):
print item
return self.item
def x(self):
print 4
a = A()
a.x()
and the output will ber
x
4
i know its not working becuase x is like a static variable and not an instance variable.
I saw this __getattr__ for static/class variables in python and it doesn't seem to work in my case
how can it be done?
thx
There are a couple of obvious problems with your code:
class A(object):
def __getattr__(self, item): # 1
print item
return self.item # 2
def x(self): # 1 again
print 4
__getattr__ will only be invoked if item cannot be found the normal way. For item == 'x', therefore, it is never invoked.
Which is probably just as well, since self.item looks for the attribute item, not the attribute corresponding to whatever is assigned to item. This doesn't exist, so would invoke __getattr__. If you try A().y() you'll get RuntimeError: maximum recursion depth exceeded while calling a Python object.
Instead, I think you want to use __getattribute__, which is always invoked. You need to be careful not to get the same runtime error, though; here I avoid it by calling the superclass implementation of __getattribute__, the naïve way of calling getattr(self, item) would fail:
class A(object):
def __getattribute__(self, item):
print item
return super(A, self).__getattribute__(item)
def x(self):
print 4
Which gives:
>>> A().x()
x
4
>>> A().y()
y
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in __getattribute__
AttributeError: 'A' object has no attribute 'y'
Note that both __getattr__ and __getattribute__ apply equally to attributes and methods (which are, more or less, just callable attributes).
Related
Refer to the second top answer to an existing question: Difference between __getattr__ vs __getattribute__, which including code suggested by someone:
class Count(object):
def __init__(self, mymin, mymax):
self.mymin = mymin
self.mymax = mymax
self.current = None
def __getattr__(self, item):
self.__dict__[item] = 0
return 0
def __getattribute__(self, item):
if item.startswith('cur'):
raise AttributeError
return super(Count, self).__getattribute__(item)
obj1 = Count(1, 10)
print(obj1.mymin)
print(obj1.mymax)
print(obj1.current)
My question is:
When I run the code, it did not run into an infinite recursion deep (by ending with maximum recursion depth exceeded). Why? And, if I change the code super(Count, self).__getattribute__(item) to super(object, self).__getattribute__(item), it did run into an infinite loop. Why again?
Please explain the reason with a detailed calling process.
I will try to make it simpler by breaking the self.__dict__[item] into 2 parts:
class Count(object):
def __getattr__(self, item):
print('__getattr__:', item)
d = self.__dict__
print('resolved __dict__')
d[item] = 0
return 0
def __getattribute__(self, item):
print('__getattribute__:', item)
if item.startswith('cur'):
raise AttributeError
return super(Count, self).__getattribute__(item)
obj1 = Count()
print(obj1.current)
The output is
__getattribute__: current
__getattr__: current
__getattribute__: __dict__
resolved __dict__
0
Now, if we replace super(Count, self) with the incorrect construct super(object, self) the message is not printed. It is because __getattribute__ will also mask the access to __dict__. However the super object will point to the base class of object which does not exist and hence our __getattribute__ function will always throw AttributeError.
Now, after __getattribute__ fails, __getattr__ is being tried for it ... well, instead of just resolving __dict__ to some value, it tries to get it as an attribute - and ends up calling__getattribute__ again. Hence we get.
....
__getattribute__: __dict__
__getattr__: __dict__
__getattribute__: __dict__
__getattr__: __dict__
__getattribute__: __dict__
__getattr__: __dict__
__getattribute__: __dict__
__getattr__: __dict__
__getattribute__: __dict__
__getattr__: __dict__
Traceback (most recent call last):
File "getattribute.py", line 15, in <module>
print(obj1.current)
File "getattribute.py", line 4, in __getattr__
d = self.__dict__
File "getattribute.py", line 4, in __getattr__
d = self.__dict__
File "getattribute.py", line 4, in __getattr__
d = self.__dict__
[Previous line repeated 328 more times]
File "getattribute.py", line 8, in __getattribute__
print('__getattribute__: ', item)
RecursionError: maximum recursion depth exceeded while calling a Python object
Had you used setattr(self, item, 0) instead of looking up self.__dict__ this could have been "avoided":
class Count(object):
def __getattr__(self, item):
setattr(self, item, 0)
return 0
def __getattribute__(self, item):
if item.startswith('cur'):
raise AttributeError
return super(object, self).__getattribute__(item)
obj1 = Count()
print(obj1.current)
of course such code would not have been correct - trying to access any other attribute would have failed nevertheless.
I'm trying to have a class act in every way like a tuple that's an attribute of the class, so len(instance) would be the same as len(instance.tup), instance[3] would return instance.tup[3], etc. Here's the class:
class mytup(object):
def __init__(self, a):
self.tup = tuple(a)
def __getattr__(self, nm):
f = types.MethodType(lambda self:getattr(self.tup, nm)(), self, type(self))
f.__func__.func_name = nm
setattr(self, nm, f)
return f
I can
mt = mytup(range(10))
But if I try to:
In [253]: len(mt)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-253-67688b907b8a> in <module>()
----> 1 len(mt)
TypeError: object of type 'mytup' has no len()
mt does in fact have a __len__ that I can call:
In [254]: mt.__len__
Out[254]: <bound method mytup.__len__ of <__main__.mytup object at 0x2e85150>>
In [255]: mt.__len__()
Out[255]: 10
(I even renamed it __len__). As near as I can tell, this should look just as if I did:
def __len__(self, *a):
return self.tup.__len__(*a)
But python won't let me len(mt) (or mt[2] or mt [1:5] for that matter).
New-style classes look-up "special methods"—those that start and end with two underscore characters—on an instance's class not the instance involved, so when len() is called it tries to call typeof(mt).__len__(). So the proper way to do what you want would be to use one of the Abstract Base Classes for Containers in the collections module (since Python 3.3)
import collections.abc
class MyTuple(collections.abc.Sequence):
def __init__(self, a):
self.tup = tuple(a)
def __len__(self):
return len(self.tup)
def __getitem__(self, index):
return self.tup[index]
mt = MyTuple(range(10))
print(len(mt)) # -> 10
print(mt[4]) # -> 4
The reason this isn't working as you have hoped is because doing:
setattr(self, nm, f)
Is not equivalent to
def __len__(self, *a):
return self.tup.__len__(*a)
In the latter case, your method is a property of the class because it is defined in class scope. It would be the equivlanet of setattr(cls, nm, f). If you check MyTup.__dict__ you will see it there. However, in the former case, __len__ is a property of the instance. So it will be in my_instance.__dict__. len checks the class for a __len__ method, and doesn't find one. Hence the error. Your __getattr__ is never actually called, and even if it were, it wouldn't allow you to use len. You can use an_instanec.__len__ diretly, though.
len does not use __getattr__ to get the __len__ function - it calls __len__ directly.
Calling x.__len__ is like calling getattr(x, '__len__') - which will return the x.__len__ method object.
len works behind the scene, so it can access this method directly, without invoking the __getattr__ helper.
Try to add a print statement in your __getattr__ to see what is printed when calling len (hint: nothing).
Here's a piece of code, which goes into an infinite recursion loop, which consists only of __repr__ function, seemingly calling itself. But I really can't see, how it calls itself. Moreover, I can't even understand, how it was called:
class MyList(list): #this is storage for MyDict objects
def __init__(self):
super(MyList, self).__init__()
class MyDict(dict):
def __init__(self, mylist):
self.mylist = mylist #mydict remembers mylist, to which it belongs
def __hash__(self):
return id(self)
def __eq__(self, other):
return self is other
def __repr__(self):
return str(self.mylist.index(self)) #!!!this is the crazy repr, going into recursion
def __str__(self):
return str(self.__repr__())
mylist = MyList()
mydict = MyDict(mylist)
mydict.update({1:2})
print str(mylist.index(mydict)) #here we die :(
Execution of this code results in:
Traceback (most recent call last):
File "test_analogue.py", line 20, in <module>
print str(mylist.index(mydict))
File "test_analogue.py", line 13, in __repr__
return str(self.mylist.index(self))
File "test_analogue.py", line 13, in __repr__
...
... (repetition of last 2 lines for ~666 times)
...
File "test_analogue.py", line 13, in __repr__
return str(self.mylist.index(self))
RuntimeError: maximum recursion depth exceeded while calling a Python object
Do you understand, how str(mylist.index(mydict)) managed to call __repr__? I'm completely puzzled. Thanks!
>> mylist.index('foo')
ValueError: 'foo' is not in list
You never actually added mydict to mylist, so the index method tries to raise this error. The error contains the repr of the dict. The repr of the dict, of course, tries to look up its index in the list that it isn't in, and this raises an exception, whose error message is calculated using the repr of the dict, which of course, tries to look up its index in the list that it isn't in, and...
It is fairly easy to use the __getattr__ special method on Python classes to handle either missing properties or functions, but seemingly not both at the same time.
Consider this example which handles any property requested which is not defined explicitly elsewhere in the class...
class Props:
def __getattr__(self, attr):
return 'some_new_value'
>>> p = Props()
>>> p.prop # Property get handled
'some_new_value'
>>> p.func('an_arg', kw='keyword') # Function call NOT handled
Traceback (most recent call last):
File "<console>", line 1, in <module>
TypeError: 'str' object is not callable
Next, consider this example which handles any function call not defined explicitly elsewhere in the class...
class Funcs:
def __getattr__(self, attr):
def fn(*args, **kwargs):
# Do something with the function name and any passed arguments or keywords
print attr
print args
print kwargs
return
return fn
>>> f = Funcs()
>>> f.prop # Property get NOT handled
<function fn at 0x10df23b90>
>>> f.func('an_arg', kw='keyword') # Function call handled
func
('an_arg',)
{'kw': 'keyword'}
The question is how to handle both types of missing attributes in the same __getattr__? How to detect if the attribute requested was in property notation or in method notation with parentheses and return either a value or a function respectively? Essentially I want to handle SOME missing property attributes AND SOME missing function attributes and then resort to default behavior for all the other cases.
Advice?
How to detect if the attribute requested was in property notation or in method notation with parentheses and return either a value or a function respectively?
You can't. You also can't tell whether a requested method is an instance, class, or static method, etc. All you can tell is that someone is trying to retrieve an attribute for read access. Nothing else is passed into the getattribute machinery, so nothing else is available to your code.
So, you need some out-of-band way to know whether to create a function or some other kind of value. This is actually pretty common—you may actually be proxying for some other object that does have a value/function distinction (think of ctypes or PyObjC), or you may have a naming convention, etc.
However, you could always return an object that can be used either way. For example, if your "default behavior" is to return attributes are integers, or functions that return an integer, you can return something like this:
class Integerizer(object):
def __init__(self, value):
self.value = value
def __int__(self):
return self.value
def __call__(self, *args, **kw):
return self.value
There is no way to detect how the returned attribute was intended to be used. Everything on python objects are attributes, including the methods:
>>> class Foo(object):
... def bar(self): print 'bar called'
... spam='eggs'
...
>>> Foo.bar
<unbound method Foo.bar>
>>> Foo.spam
'eggs'
Python first looks up the attribute (bar or spam), and if you meant to call it (added parenthesis) then Python invokes the callable after lookup up the attribute:
>>> foo = Foo()
>>> fbar = foo.bar
>>> fbar()
'bar called'
In the above code I separated the lookup of bar from calling bar.
Since there is no distinction, you cannot detect in __getattr__ what the returned attribute will be used for.
__getattr__ is called whenever normal attribute access fails; in the following example monty is defined on the class, so __getattr__ is not called; it is only called for bar.eric and bar.john:
>>> class Bar(object):
... monty = 'python'
... def __getattr__(self, name):
... print 'Attribute access for {0}'.format(name)
... if name == 'eric':
... return 'idle'
... raise AttributeError(name)
...
>>> bar = Bar()
>>> bar.monty
'python'
>>> bar.eric
Attribute access for eric
'idle'
>>> bar.john
Attribute access for john
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 7, in __getattr__
AttributeError: john
Note that functions are not the only objects that you can invoke (call); any custom class that implements the __call__ method will do:
>>> class Baz(object):
... def __call__(self, name):
... print 'Baz sez: "Hello {0}!"'.format(name)
...
>>> baz = Baz()
>>> baz('John Cleese')
Baz sez: "Hello John Cleese!"
You could use that return objects from __getattr__ that can both be called and used as a value in different contexts.
I want to override access to one variable in a class, but return all others normally. How do I accomplish this with __getattribute__?
I tried the following (which should also illustrate what I'm trying to do) but I get a recursion error:
class D(object):
def __init__(self):
self.test=20
self.test2=21
def __getattribute__(self,name):
if name=='test':
return 0.
else:
return self.__dict__[name]
>>> print D().test
0.0
>>> print D().test2
...
RuntimeError: maximum recursion depth exceeded in cmp
You get a recursion error because your attempt to access the self.__dict__ attribute inside __getattribute__ invokes your __getattribute__ again. If you use object's __getattribute__ instead, it works:
class D(object):
def __init__(self):
self.test=20
self.test2=21
def __getattribute__(self,name):
if name=='test':
return 0.
else:
return object.__getattribute__(self, name)
This works because object (in this example) is the base class. By calling the base version of __getattribute__ you avoid the recursive hell you were in before.
Ipython output with code in foo.py:
In [1]: from foo import *
In [2]: d = D()
In [3]: d.test
Out[3]: 0.0
In [4]: d.test2
Out[4]: 21
Update:
There's something in the section titled More attribute access for new-style classes in the current documentation, where they recommend doing exactly this to avoid the infinite recursion.
Actually, I believe you want to use the __getattr__ special method instead.
Quote from the Python docs:
__getattr__( self, name)
Called when an attribute lookup has not found the attribute in the usual places (i.e. it is not an instance attribute nor is it found in the class tree for self). name is the attribute name. This method should return the (computed) attribute value or raise an AttributeError exception.
Note that if the attribute is found through the normal mechanism, __getattr__() is not called. (This is an intentional asymmetry between __getattr__() and __setattr__().) This is done both for efficiency reasons and because otherwise __setattr__() would have no way to access other attributes of the instance. Note that at least for instance variables, you can fake total control by not inserting any values in the instance attribute dictionary (but instead inserting them in another object). See the __getattribute__() method below for a way to actually get total control in new-style classes.
Note: for this to work, the instance should not have a test attribute, so the line self.test=20 should be removed.
Python language reference:
In order to avoid infinite recursion
in this method, its implementation
should always call the base class
method with the same name to access
any attributes it needs, for example,
object.__getattribute__(self, name).
Meaning:
def __getattribute__(self,name):
...
return self.__dict__[name]
You're calling for an attribute called __dict__. Because it's an attribute, __getattribute__ gets called in search for __dict__ which calls __getattribute__ which calls ... yada yada yada
return object.__getattribute__(self, name)
Using the base classes __getattribute__ helps finding the real attribute.
How is the __getattribute__ method used?
It is called before the normal dotted lookup. If it raises AttributeError, then we call __getattr__.
Use of this method is rather rare. There are only two definitions in the standard library:
$ grep -Erl "def __getattribute__\(self" cpython/Lib | grep -v "/test/"
cpython/Lib/_threading_local.py
cpython/Lib/importlib/util.py
Best Practice
The proper way to programmatically control access to a single attribute is with property. Class D should be written as follows (with the setter and deleter optionally to replicate apparent intended behavior):
class D(object):
def __init__(self):
self.test2=21
#property
def test(self):
return 0.
#test.setter
def test(self, value):
'''dummy function to avoid AttributeError on setting property'''
#test.deleter
def test(self):
'''dummy function to avoid AttributeError on deleting property'''
And usage:
>>> o = D()
>>> o.test
0.0
>>> o.test = 'foo'
>>> o.test
0.0
>>> del o.test
>>> o.test
0.0
A property is a data descriptor, thus it is the first thing looked for in the normal dotted lookup algorithm.
Options for __getattribute__
You several options if you absolutely need to implement lookup for every attribute via __getattribute__.
raise AttributeError, causing __getattr__ to be called (if implemented)
return something from it by
using super to call the parent (probably object's) implementation
calling __getattr__
implementing your own dotted lookup algorithm somehow
For example:
class NoisyAttributes(object):
def __init__(self):
self.test=20
self.test2=21
def __getattribute__(self, name):
print('getting: ' + name)
try:
return super(NoisyAttributes, self).__getattribute__(name)
except AttributeError:
print('oh no, AttributeError caught and reraising')
raise
def __getattr__(self, name):
"""Called if __getattribute__ raises AttributeError"""
return 'close but no ' + name
>>> n = NoisyAttributes()
>>> nfoo = n.foo
getting: foo
oh no, AttributeError caught and reraising
>>> nfoo
'close but no foo'
>>> n.test
getting: test
20
What you originally wanted.
And this example shows how you might do what you originally wanted:
class D(object):
def __init__(self):
self.test=20
self.test2=21
def __getattribute__(self,name):
if name=='test':
return 0.
else:
return super(D, self).__getattribute__(name)
And will behave like this:
>>> o = D()
>>> o.test = 'foo'
>>> o.test
0.0
>>> del o.test
>>> o.test
0.0
>>> del o.test
Traceback (most recent call last):
File "<pyshell#216>", line 1, in <module>
del o.test
AttributeError: test
Code review
Your code with comments. You have a dotted lookup on self in __getattribute__.
This is why you get a recursion error. You could check if name is "__dict__" and use super to workaround, but that doesn't cover __slots__. I'll leave that as an exercise to the reader.
class D(object):
def __init__(self):
self.test=20
self.test2=21
def __getattribute__(self,name):
if name=='test':
return 0.
else: # v--- Dotted lookup on self in __getattribute__
return self.__dict__[name]
>>> print D().test
0.0
>>> print D().test2
...
RuntimeError: maximum recursion depth exceeded in cmp
Are you sure you want to use __getattribute__? What are you actually trying to achieve?
The easiest way to do what you ask is:
class D(object):
def __init__(self):
self.test = 20
self.test2 = 21
test = 0
or:
class D(object):
def __init__(self):
self.test = 20
self.test2 = 21
#property
def test(self):
return 0
Edit:
Note that an instance of D would have different values of test in each case. In the first case d.test would be 20, in the second it would be 0. I'll leave it to you to work out why.
Edit2:
Greg pointed out that example 2 will fail because the property is read only and the __init__ method tried to set it to 20. A more complete example for that would be:
class D(object):
def __init__(self):
self.test = 20
self.test2 = 21
_test = 0
def get_test(self):
return self._test
def set_test(self, value):
self._test = value
test = property(get_test, set_test)
Obviously, as a class this is almost entirely useless, but it gives you an idea to move on from.
Here is a more reliable version:
class D(object):
def __init__(self):
self.test = 20
self.test2 = 21
def __getattribute__(self, name):
if name == 'test':
return 0.
else:
return super(D, self).__getattribute__(name)
It calls __getattribute__ method from parent class, eventually falling back to object.__getattribute__ method if other ancestors don't override it.