I want to override access to one variable in a class, but return all others normally. How do I accomplish this with __getattribute__?
I tried the following (which should also illustrate what I'm trying to do) but I get a recursion error:
class D(object):
def __init__(self):
self.test=20
self.test2=21
def __getattribute__(self,name):
if name=='test':
return 0.
else:
return self.__dict__[name]
>>> print D().test
0.0
>>> print D().test2
...
RuntimeError: maximum recursion depth exceeded in cmp
You get a recursion error because your attempt to access the self.__dict__ attribute inside __getattribute__ invokes your __getattribute__ again. If you use object's __getattribute__ instead, it works:
class D(object):
def __init__(self):
self.test=20
self.test2=21
def __getattribute__(self,name):
if name=='test':
return 0.
else:
return object.__getattribute__(self, name)
This works because object (in this example) is the base class. By calling the base version of __getattribute__ you avoid the recursive hell you were in before.
Ipython output with code in foo.py:
In [1]: from foo import *
In [2]: d = D()
In [3]: d.test
Out[3]: 0.0
In [4]: d.test2
Out[4]: 21
Update:
There's something in the section titled More attribute access for new-style classes in the current documentation, where they recommend doing exactly this to avoid the infinite recursion.
Actually, I believe you want to use the __getattr__ special method instead.
Quote from the Python docs:
__getattr__( self, name)
Called when an attribute lookup has not found the attribute in the usual places (i.e. it is not an instance attribute nor is it found in the class tree for self). name is the attribute name. This method should return the (computed) attribute value or raise an AttributeError exception.
Note that if the attribute is found through the normal mechanism, __getattr__() is not called. (This is an intentional asymmetry between __getattr__() and __setattr__().) This is done both for efficiency reasons and because otherwise __setattr__() would have no way to access other attributes of the instance. Note that at least for instance variables, you can fake total control by not inserting any values in the instance attribute dictionary (but instead inserting them in another object). See the __getattribute__() method below for a way to actually get total control in new-style classes.
Note: for this to work, the instance should not have a test attribute, so the line self.test=20 should be removed.
Python language reference:
In order to avoid infinite recursion
in this method, its implementation
should always call the base class
method with the same name to access
any attributes it needs, for example,
object.__getattribute__(self, name).
Meaning:
def __getattribute__(self,name):
...
return self.__dict__[name]
You're calling for an attribute called __dict__. Because it's an attribute, __getattribute__ gets called in search for __dict__ which calls __getattribute__ which calls ... yada yada yada
return object.__getattribute__(self, name)
Using the base classes __getattribute__ helps finding the real attribute.
How is the __getattribute__ method used?
It is called before the normal dotted lookup. If it raises AttributeError, then we call __getattr__.
Use of this method is rather rare. There are only two definitions in the standard library:
$ grep -Erl "def __getattribute__\(self" cpython/Lib | grep -v "/test/"
cpython/Lib/_threading_local.py
cpython/Lib/importlib/util.py
Best Practice
The proper way to programmatically control access to a single attribute is with property. Class D should be written as follows (with the setter and deleter optionally to replicate apparent intended behavior):
class D(object):
def __init__(self):
self.test2=21
#property
def test(self):
return 0.
#test.setter
def test(self, value):
'''dummy function to avoid AttributeError on setting property'''
#test.deleter
def test(self):
'''dummy function to avoid AttributeError on deleting property'''
And usage:
>>> o = D()
>>> o.test
0.0
>>> o.test = 'foo'
>>> o.test
0.0
>>> del o.test
>>> o.test
0.0
A property is a data descriptor, thus it is the first thing looked for in the normal dotted lookup algorithm.
Options for __getattribute__
You several options if you absolutely need to implement lookup for every attribute via __getattribute__.
raise AttributeError, causing __getattr__ to be called (if implemented)
return something from it by
using super to call the parent (probably object's) implementation
calling __getattr__
implementing your own dotted lookup algorithm somehow
For example:
class NoisyAttributes(object):
def __init__(self):
self.test=20
self.test2=21
def __getattribute__(self, name):
print('getting: ' + name)
try:
return super(NoisyAttributes, self).__getattribute__(name)
except AttributeError:
print('oh no, AttributeError caught and reraising')
raise
def __getattr__(self, name):
"""Called if __getattribute__ raises AttributeError"""
return 'close but no ' + name
>>> n = NoisyAttributes()
>>> nfoo = n.foo
getting: foo
oh no, AttributeError caught and reraising
>>> nfoo
'close but no foo'
>>> n.test
getting: test
20
What you originally wanted.
And this example shows how you might do what you originally wanted:
class D(object):
def __init__(self):
self.test=20
self.test2=21
def __getattribute__(self,name):
if name=='test':
return 0.
else:
return super(D, self).__getattribute__(name)
And will behave like this:
>>> o = D()
>>> o.test = 'foo'
>>> o.test
0.0
>>> del o.test
>>> o.test
0.0
>>> del o.test
Traceback (most recent call last):
File "<pyshell#216>", line 1, in <module>
del o.test
AttributeError: test
Code review
Your code with comments. You have a dotted lookup on self in __getattribute__.
This is why you get a recursion error. You could check if name is "__dict__" and use super to workaround, but that doesn't cover __slots__. I'll leave that as an exercise to the reader.
class D(object):
def __init__(self):
self.test=20
self.test2=21
def __getattribute__(self,name):
if name=='test':
return 0.
else: # v--- Dotted lookup on self in __getattribute__
return self.__dict__[name]
>>> print D().test
0.0
>>> print D().test2
...
RuntimeError: maximum recursion depth exceeded in cmp
Are you sure you want to use __getattribute__? What are you actually trying to achieve?
The easiest way to do what you ask is:
class D(object):
def __init__(self):
self.test = 20
self.test2 = 21
test = 0
or:
class D(object):
def __init__(self):
self.test = 20
self.test2 = 21
#property
def test(self):
return 0
Edit:
Note that an instance of D would have different values of test in each case. In the first case d.test would be 20, in the second it would be 0. I'll leave it to you to work out why.
Edit2:
Greg pointed out that example 2 will fail because the property is read only and the __init__ method tried to set it to 20. A more complete example for that would be:
class D(object):
def __init__(self):
self.test = 20
self.test2 = 21
_test = 0
def get_test(self):
return self._test
def set_test(self, value):
self._test = value
test = property(get_test, set_test)
Obviously, as a class this is almost entirely useless, but it gives you an idea to move on from.
Here is a more reliable version:
class D(object):
def __init__(self):
self.test = 20
self.test2 = 21
def __getattribute__(self, name):
if name == 'test':
return 0.
else:
return super(D, self).__getattribute__(name)
It calls __getattribute__ method from parent class, eventually falling back to object.__getattribute__ method if other ancestors don't override it.
Related
I'm doing it like:
def set_property(property,value):
def get_property(property):
or
object.property = value
value = object.property
What's the pythonic way to use getters and setters?
Try this: Python Property
The sample code is:
class C(object):
def __init__(self):
self._x = None
#property
def x(self):
"""I'm the 'x' property."""
print("getter of x called")
return self._x
#x.setter
def x(self, value):
print("setter of x called")
self._x = value
#x.deleter
def x(self):
print("deleter of x called")
del self._x
c = C()
c.x = 'foo' # setter called
foo = c.x # getter called
del c.x # deleter called
What's the pythonic way to use getters and setters?
The "Pythonic" way is not to use "getters" and "setters", but to use plain attributes, like the question demonstrates, and del for deleting (but the names are changed to protect the innocent... builtins):
value = 'something'
obj.attribute = value
value = obj.attribute
del obj.attribute
If later, you want to modify the setting and getting, you can do so without having to alter user code, by using the property decorator:
class Obj:
"""property demo"""
#
#property # first decorate the getter method
def attribute(self): # This getter method name is *the* name
return self._attribute
#
#attribute.setter # the property decorates with `.setter` now
def attribute(self, value): # name, e.g. "attribute", is the same
self._attribute = value # the "value" name isn't special
#
#attribute.deleter # decorate with `.deleter`
def attribute(self): # again, the method name is the same
del self._attribute
(Each decorator usage copies and updates the prior property object, so note that you should use the same name for each set, get, and delete function/method.)
After defining the above, the original setting, getting, and deleting code is the same:
obj = Obj()
obj.attribute = value
the_value = obj.attribute
del obj.attribute
You should avoid this:
def set_property(property,value):
def get_property(property):
Firstly, the above doesn't work, because you don't provide an argument for the instance that the property would be set to (usually self), which would be:
class Obj:
def set_property(self, property, value): # don't do this
...
def get_property(self, property): # don't do this either
...
Secondly, this duplicates the purpose of two special methods, __setattr__ and __getattr__.
Thirdly, we also have the setattr and getattr builtin functions.
setattr(object, 'property_name', value)
getattr(object, 'property_name', default_value) # default is optional
The #property decorator is for creating getters and setters.
For example, we could modify the setting behavior to place restrictions the value being set:
class Protective(object):
#property
def protected_value(self):
return self._protected_value
#protected_value.setter
def protected_value(self, value):
if acceptable(value): # e.g. type or range check
self._protected_value = value
In general, we want to avoid using property and just use direct attributes.
This is what is expected by users of Python. Following the rule of least-surprise, you should try to give your users what they expect unless you have a very compelling reason to the contrary.
Demonstration
For example, say we needed our object's protected attribute to be an integer between 0 and 100 inclusive, and prevent its deletion, with appropriate messages to inform the user of its proper usage:
class Protective(object):
"""protected property demo"""
#
def __init__(self, start_protected_value=0):
self.protected_value = start_protected_value
#
#property
def protected_value(self):
return self._protected_value
#
#protected_value.setter
def protected_value(self, value):
if value != int(value):
raise TypeError("protected_value must be an integer")
if 0 <= value <= 100:
self._protected_value = int(value)
else:
raise ValueError("protected_value must be " +
"between 0 and 100 inclusive")
#
#protected_value.deleter
def protected_value(self):
raise AttributeError("do not delete, protected_value can be set to 0")
(Note that __init__ refers to self.protected_value but the property methods refer to self._protected_value. This is so that __init__ uses the property through the public API, ensuring it is "protected".)
And usage:
>>> p1 = Protective(3)
>>> p1.protected_value
3
>>> p1 = Protective(5.0)
>>> p1.protected_value
5
>>> p2 = Protective(-5)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in __init__
File "<stdin>", line 15, in protected_value
ValueError: protectected_value must be between 0 and 100 inclusive
>>> p1.protected_value = 7.3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 17, in protected_value
TypeError: protected_value must be an integer
>>> p1.protected_value = 101
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 15, in protected_value
ValueError: protectected_value must be between 0 and 100 inclusive
>>> del p1.protected_value
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 18, in protected_value
AttributeError: do not delete, protected_value can be set to 0
Do the names matter?
Yes they do. .setter and .deleter make copies of the original property. This allows subclasses to properly modify behavior without altering the behavior in the parent.
class Obj:
"""property demo"""
#
#property
def get_only(self):
return self._attribute
#
#get_only.setter
def get_or_set(self, value):
self._attribute = value
#
#get_or_set.deleter
def get_set_or_delete(self):
del self._attribute
Now for this to work, you have to use the respective names:
obj = Obj()
# obj.get_only = 'value' # would error
obj.get_or_set = 'value'
obj.get_set_or_delete = 'new value'
the_value = obj.get_only
del obj.get_set_or_delete
# del obj.get_or_set # would error
I'm not sure where this would be useful, but the use-case is if you want a get, set, and/or delete-only property. Probably best to stick to semantically same property having the same name.
Conclusion
Start with simple attributes.
If you later need functionality around the setting, getting, and deleting, you can add it with the property decorator.
Avoid functions named set_... and get_... - that's what properties are for.
In [1]: class test(object):
def __init__(self):
self.pants = 'pants'
#property
def p(self):
return self.pants
#p.setter
def p(self, value):
self.pants = value * 2
....:
In [2]: t = test()
In [3]: t.p
Out[3]: 'pants'
In [4]: t.p = 10
In [5]: t.p
Out[5]: 20
Using #property and #attribute.setter helps you to not only use the "pythonic" way but also to check the validity of attributes both while creating the object and when altering it.
class Person(object):
def __init__(self, p_name=None):
self.name = p_name
#property
def name(self):
return self._name
#name.setter
def name(self, new_name):
if type(new_name) == str: #type checking for name property
self._name = new_name
else:
raise Exception("Invalid value for name")
By this, you actually 'hide' _name attribute from client developers and also perform checks on name property type. Note that by following this approach even during the initiation the setter gets called. So:
p = Person(12)
Will lead to:
Exception: Invalid value for name
But:
>>>p = person('Mike')
>>>print(p.name)
Mike
>>>p.name = 'George'
>>>print(p.name)
George
>>>p.name = 2.3 # Causes an exception
This is an old question but the topic is very important and always current. In case anyone wants to go beyond simple getters/setters i have wrote an article about superpowered properties in python with support for slots, observability and reduced boilerplate code.
from objects import properties, self_properties
class Car:
with properties(locals(), 'meta') as meta:
#meta.prop(read_only=True)
def brand(self) -> str:
"""Brand"""
#meta.prop(read_only=True)
def max_speed(self) -> float:
"""Maximum car speed"""
#meta.prop(listener='_on_acceleration')
def speed(self) -> float:
"""Speed of the car"""
return 0 # Default stopped
#meta.prop(listener='_on_off_listener')
def on(self) -> bool:
"""Engine state"""
return False
def __init__(self, brand: str, max_speed: float = 200):
self_properties(self, locals())
def _on_off_listener(self, prop, old, on):
if on:
print(f"{self.brand} Turned on, Runnnnnn")
else:
self._speed = 0
print(f"{self.brand} Turned off.")
def _on_acceleration(self, prop, old, speed):
if self.on:
if speed > self.max_speed:
print(f"{self.brand} {speed}km/h Bang! Engine exploded!")
self.on = False
else:
print(f"{self.brand} New speed: {speed}km/h")
else:
print(f"{self.brand} Car is off, no speed change")
This class can be used like this:
mycar = Car('Ford')
# Car is turned off
for speed in range(0, 300, 50):
mycar.speed = speed
# Car is turned on
mycar.on = True
for speed in range(0, 350, 50):
mycar.speed = speed
This code will produce the following output:
Ford Car is off, no speed change
Ford Car is off, no speed change
Ford Car is off, no speed change
Ford Car is off, no speed change
Ford Car is off, no speed change
Ford Car is off, no speed change
Ford Turned on, Runnnnnn
Ford New speed: 0km/h
Ford New speed: 50km/h
Ford New speed: 100km/h
Ford New speed: 150km/h
Ford New speed: 200km/h
Ford 250km/h Bang! Engine exploded!
Ford Turned off.
Ford Car is off, no speed change
More info about how and why here: https://mnesarco.github.io/blog/2020/07/23/python-metaprogramming-properties-on-steroids
Properties are pretty useful since you can use them with assignment but then can include validation as well. You can see this code where you use the decorator #property and also #<property_name>.setter to create the methods:
# Python program displaying the use of #property
class AgeSet:
def __init__(self):
self._age = 0
# using property decorator a getter function
#property
def age(self):
print("getter method called")
return self._age
# a setter function
#age.setter
def age(self, a):
if(a < 18):
raise ValueError("Sorry your age is below eligibility criteria")
print("setter method called")
self._age = a
pkj = AgeSet()
pkj.age = int(input("set the age using setter: "))
print(pkj.age)
There are more details in this post I wrote about this as well: https://pythonhowtoprogram.com/how-to-create-getter-setter-class-properties-in-python-3/
You can use accessors/mutators (i.e. #attr.setter and #property) or not, but the most important thing is to be consistent!
If you're using #property to simply access an attribute, e.g.
class myClass:
def __init__(a):
self._a = a
#property
def a(self):
return self._a
use it to access every* attribute! It would be a bad practice to access some attributes using #property and leave some other properties public (i.e. name without an underscore) without an accessor, e.g. do not do
class myClass:
def __init__(a, b):
self.a = a
self.b = b
#property
def a(self):
return self.a
Note that self.b does not have an explicit accessor here even though it's public.
Similarly with setters (or mutators), feel free to use #attribute.setter but be consistent! When you do e.g.
class myClass:
def __init__(a, b):
self.a = a
self.b = b
#a.setter
def a(self, value):
return self.a = value
It's hard for me to guess your intention. On one hand you're saying that both a and b are public (no leading underscore in their names) so I should theoretically be allowed to access/mutate (get/set) both. But then you specify an explicit mutator only for a, which tells me that maybe I should not be able to set b. Since you've provided an explicit mutator I am not sure if the lack of explicit accessor (#property) means I should not be able to access either of those variables or you were simply being frugal in using #property.
*The exception is when you explicitly want to make some variables accessible or mutable but not both or you want to perform some additional logic when accessing or mutating an attribute. This is when I am personally using #property and #attribute.setter (otherwise no explicit acessors/mutators for public attributes).
Lastly, PEP8 and Google Style Guide suggestions:
PEP8, Designing for Inheritance says:
For simple public data attributes, it is best to expose just the attribute name, without complicated accessor/mutator methods. Keep in mind that Python provides an easy path to future enhancement, should you find that a simple data attribute needs to grow functional behavior. In that case, use properties to hide functional implementation behind simple data attribute access syntax.
On the other hand, according to Google Style Guide Python Language Rules/Properties the recommendation is to:
Use properties in new code to access or set data where you would normally have used simple, lightweight accessor or setter methods. Properties should be created with the #property decorator.
The pros of this approach:
Readability is increased by eliminating explicit get and set method calls for simple attribute access. Allows calculations to be lazy. Considered the Pythonic way to maintain the interface of a class. In terms of performance, allowing properties bypasses needing trivial accessor methods when a direct variable access is reasonable. This also allows accessor methods to be added in the future without breaking the interface.
and cons:
Must inherit from object in Python 2. Can hide side-effects much like operator overloading. Can be confusing for subclasses.
You can use the magic methods __getattribute__ and __setattr__.
class MyClass:
def __init__(self, attrvalue):
self.myattr = attrvalue
def __getattribute__(self, attr):
if attr == "myattr":
#Getter for myattr
def __setattr__(self, attr):
if attr == "myattr":
#Setter for myattr
Be aware that __getattr__ and __getattribute__ are not the same. __getattr__ is only invoked when the attribute is not found.
I'm doing it like:
def set_property(property,value):
def get_property(property):
or
object.property = value
value = object.property
What's the pythonic way to use getters and setters?
Try this: Python Property
The sample code is:
class C(object):
def __init__(self):
self._x = None
#property
def x(self):
"""I'm the 'x' property."""
print("getter of x called")
return self._x
#x.setter
def x(self, value):
print("setter of x called")
self._x = value
#x.deleter
def x(self):
print("deleter of x called")
del self._x
c = C()
c.x = 'foo' # setter called
foo = c.x # getter called
del c.x # deleter called
What's the pythonic way to use getters and setters?
The "Pythonic" way is not to use "getters" and "setters", but to use plain attributes, like the question demonstrates, and del for deleting (but the names are changed to protect the innocent... builtins):
value = 'something'
obj.attribute = value
value = obj.attribute
del obj.attribute
If later, you want to modify the setting and getting, you can do so without having to alter user code, by using the property decorator:
class Obj:
"""property demo"""
#
#property # first decorate the getter method
def attribute(self): # This getter method name is *the* name
return self._attribute
#
#attribute.setter # the property decorates with `.setter` now
def attribute(self, value): # name, e.g. "attribute", is the same
self._attribute = value # the "value" name isn't special
#
#attribute.deleter # decorate with `.deleter`
def attribute(self): # again, the method name is the same
del self._attribute
(Each decorator usage copies and updates the prior property object, so note that you should use the same name for each set, get, and delete function/method.)
After defining the above, the original setting, getting, and deleting code is the same:
obj = Obj()
obj.attribute = value
the_value = obj.attribute
del obj.attribute
You should avoid this:
def set_property(property,value):
def get_property(property):
Firstly, the above doesn't work, because you don't provide an argument for the instance that the property would be set to (usually self), which would be:
class Obj:
def set_property(self, property, value): # don't do this
...
def get_property(self, property): # don't do this either
...
Secondly, this duplicates the purpose of two special methods, __setattr__ and __getattr__.
Thirdly, we also have the setattr and getattr builtin functions.
setattr(object, 'property_name', value)
getattr(object, 'property_name', default_value) # default is optional
The #property decorator is for creating getters and setters.
For example, we could modify the setting behavior to place restrictions the value being set:
class Protective(object):
#property
def protected_value(self):
return self._protected_value
#protected_value.setter
def protected_value(self, value):
if acceptable(value): # e.g. type or range check
self._protected_value = value
In general, we want to avoid using property and just use direct attributes.
This is what is expected by users of Python. Following the rule of least-surprise, you should try to give your users what they expect unless you have a very compelling reason to the contrary.
Demonstration
For example, say we needed our object's protected attribute to be an integer between 0 and 100 inclusive, and prevent its deletion, with appropriate messages to inform the user of its proper usage:
class Protective(object):
"""protected property demo"""
#
def __init__(self, start_protected_value=0):
self.protected_value = start_protected_value
#
#property
def protected_value(self):
return self._protected_value
#
#protected_value.setter
def protected_value(self, value):
if value != int(value):
raise TypeError("protected_value must be an integer")
if 0 <= value <= 100:
self._protected_value = int(value)
else:
raise ValueError("protected_value must be " +
"between 0 and 100 inclusive")
#
#protected_value.deleter
def protected_value(self):
raise AttributeError("do not delete, protected_value can be set to 0")
(Note that __init__ refers to self.protected_value but the property methods refer to self._protected_value. This is so that __init__ uses the property through the public API, ensuring it is "protected".)
And usage:
>>> p1 = Protective(3)
>>> p1.protected_value
3
>>> p1 = Protective(5.0)
>>> p1.protected_value
5
>>> p2 = Protective(-5)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in __init__
File "<stdin>", line 15, in protected_value
ValueError: protectected_value must be between 0 and 100 inclusive
>>> p1.protected_value = 7.3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 17, in protected_value
TypeError: protected_value must be an integer
>>> p1.protected_value = 101
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 15, in protected_value
ValueError: protectected_value must be between 0 and 100 inclusive
>>> del p1.protected_value
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 18, in protected_value
AttributeError: do not delete, protected_value can be set to 0
Do the names matter?
Yes they do. .setter and .deleter make copies of the original property. This allows subclasses to properly modify behavior without altering the behavior in the parent.
class Obj:
"""property demo"""
#
#property
def get_only(self):
return self._attribute
#
#get_only.setter
def get_or_set(self, value):
self._attribute = value
#
#get_or_set.deleter
def get_set_or_delete(self):
del self._attribute
Now for this to work, you have to use the respective names:
obj = Obj()
# obj.get_only = 'value' # would error
obj.get_or_set = 'value'
obj.get_set_or_delete = 'new value'
the_value = obj.get_only
del obj.get_set_or_delete
# del obj.get_or_set # would error
I'm not sure where this would be useful, but the use-case is if you want a get, set, and/or delete-only property. Probably best to stick to semantically same property having the same name.
Conclusion
Start with simple attributes.
If you later need functionality around the setting, getting, and deleting, you can add it with the property decorator.
Avoid functions named set_... and get_... - that's what properties are for.
In [1]: class test(object):
def __init__(self):
self.pants = 'pants'
#property
def p(self):
return self.pants
#p.setter
def p(self, value):
self.pants = value * 2
....:
In [2]: t = test()
In [3]: t.p
Out[3]: 'pants'
In [4]: t.p = 10
In [5]: t.p
Out[5]: 20
Using #property and #attribute.setter helps you to not only use the "pythonic" way but also to check the validity of attributes both while creating the object and when altering it.
class Person(object):
def __init__(self, p_name=None):
self.name = p_name
#property
def name(self):
return self._name
#name.setter
def name(self, new_name):
if type(new_name) == str: #type checking for name property
self._name = new_name
else:
raise Exception("Invalid value for name")
By this, you actually 'hide' _name attribute from client developers and also perform checks on name property type. Note that by following this approach even during the initiation the setter gets called. So:
p = Person(12)
Will lead to:
Exception: Invalid value for name
But:
>>>p = person('Mike')
>>>print(p.name)
Mike
>>>p.name = 'George'
>>>print(p.name)
George
>>>p.name = 2.3 # Causes an exception
This is an old question but the topic is very important and always current. In case anyone wants to go beyond simple getters/setters i have wrote an article about superpowered properties in python with support for slots, observability and reduced boilerplate code.
from objects import properties, self_properties
class Car:
with properties(locals(), 'meta') as meta:
#meta.prop(read_only=True)
def brand(self) -> str:
"""Brand"""
#meta.prop(read_only=True)
def max_speed(self) -> float:
"""Maximum car speed"""
#meta.prop(listener='_on_acceleration')
def speed(self) -> float:
"""Speed of the car"""
return 0 # Default stopped
#meta.prop(listener='_on_off_listener')
def on(self) -> bool:
"""Engine state"""
return False
def __init__(self, brand: str, max_speed: float = 200):
self_properties(self, locals())
def _on_off_listener(self, prop, old, on):
if on:
print(f"{self.brand} Turned on, Runnnnnn")
else:
self._speed = 0
print(f"{self.brand} Turned off.")
def _on_acceleration(self, prop, old, speed):
if self.on:
if speed > self.max_speed:
print(f"{self.brand} {speed}km/h Bang! Engine exploded!")
self.on = False
else:
print(f"{self.brand} New speed: {speed}km/h")
else:
print(f"{self.brand} Car is off, no speed change")
This class can be used like this:
mycar = Car('Ford')
# Car is turned off
for speed in range(0, 300, 50):
mycar.speed = speed
# Car is turned on
mycar.on = True
for speed in range(0, 350, 50):
mycar.speed = speed
This code will produce the following output:
Ford Car is off, no speed change
Ford Car is off, no speed change
Ford Car is off, no speed change
Ford Car is off, no speed change
Ford Car is off, no speed change
Ford Car is off, no speed change
Ford Turned on, Runnnnnn
Ford New speed: 0km/h
Ford New speed: 50km/h
Ford New speed: 100km/h
Ford New speed: 150km/h
Ford New speed: 200km/h
Ford 250km/h Bang! Engine exploded!
Ford Turned off.
Ford Car is off, no speed change
More info about how and why here: https://mnesarco.github.io/blog/2020/07/23/python-metaprogramming-properties-on-steroids
Properties are pretty useful since you can use them with assignment but then can include validation as well. You can see this code where you use the decorator #property and also #<property_name>.setter to create the methods:
# Python program displaying the use of #property
class AgeSet:
def __init__(self):
self._age = 0
# using property decorator a getter function
#property
def age(self):
print("getter method called")
return self._age
# a setter function
#age.setter
def age(self, a):
if(a < 18):
raise ValueError("Sorry your age is below eligibility criteria")
print("setter method called")
self._age = a
pkj = AgeSet()
pkj.age = int(input("set the age using setter: "))
print(pkj.age)
There are more details in this post I wrote about this as well: https://pythonhowtoprogram.com/how-to-create-getter-setter-class-properties-in-python-3/
You can use accessors/mutators (i.e. #attr.setter and #property) or not, but the most important thing is to be consistent!
If you're using #property to simply access an attribute, e.g.
class myClass:
def __init__(a):
self._a = a
#property
def a(self):
return self._a
use it to access every* attribute! It would be a bad practice to access some attributes using #property and leave some other properties public (i.e. name without an underscore) without an accessor, e.g. do not do
class myClass:
def __init__(a, b):
self.a = a
self.b = b
#property
def a(self):
return self.a
Note that self.b does not have an explicit accessor here even though it's public.
Similarly with setters (or mutators), feel free to use #attribute.setter but be consistent! When you do e.g.
class myClass:
def __init__(a, b):
self.a = a
self.b = b
#a.setter
def a(self, value):
return self.a = value
It's hard for me to guess your intention. On one hand you're saying that both a and b are public (no leading underscore in their names) so I should theoretically be allowed to access/mutate (get/set) both. But then you specify an explicit mutator only for a, which tells me that maybe I should not be able to set b. Since you've provided an explicit mutator I am not sure if the lack of explicit accessor (#property) means I should not be able to access either of those variables or you were simply being frugal in using #property.
*The exception is when you explicitly want to make some variables accessible or mutable but not both or you want to perform some additional logic when accessing or mutating an attribute. This is when I am personally using #property and #attribute.setter (otherwise no explicit acessors/mutators for public attributes).
Lastly, PEP8 and Google Style Guide suggestions:
PEP8, Designing for Inheritance says:
For simple public data attributes, it is best to expose just the attribute name, without complicated accessor/mutator methods. Keep in mind that Python provides an easy path to future enhancement, should you find that a simple data attribute needs to grow functional behavior. In that case, use properties to hide functional implementation behind simple data attribute access syntax.
On the other hand, according to Google Style Guide Python Language Rules/Properties the recommendation is to:
Use properties in new code to access or set data where you would normally have used simple, lightweight accessor or setter methods. Properties should be created with the #property decorator.
The pros of this approach:
Readability is increased by eliminating explicit get and set method calls for simple attribute access. Allows calculations to be lazy. Considered the Pythonic way to maintain the interface of a class. In terms of performance, allowing properties bypasses needing trivial accessor methods when a direct variable access is reasonable. This also allows accessor methods to be added in the future without breaking the interface.
and cons:
Must inherit from object in Python 2. Can hide side-effects much like operator overloading. Can be confusing for subclasses.
You can use the magic methods __getattribute__ and __setattr__.
class MyClass:
def __init__(self, attrvalue):
self.myattr = attrvalue
def __getattribute__(self, attr):
if attr == "myattr":
#Getter for myattr
def __setattr__(self, attr):
if attr == "myattr":
#Setter for myattr
Be aware that __getattr__ and __getattribute__ are not the same. __getattr__ is only invoked when the attribute is not found.
class Liquid(object):
def foo(self):
pass
def bar(self):
pass
class Water(Liquid):
Say, I have the two classes above, Water inherits from Liquid. Is there any way I can restrict Water from inheriting one of the parent's methods, say bar()?
Sort of. But don't do it.
class Liquid(object):
def foo(self):
pass
def bar(self):
pass
class Water(Liquid):
def __getattribute__(self, name):
if name == 'bar':
raise AttributeError("'Water' object has no attribute 'bar'")
l = Liquid()
l.bar()
w = Water()
w.bar()
You can override the method to be a no-op, but you can't remove it. Doing so would violate one of the core principles of object-oriented design, namely that any object that inherits from some parent should be able to be used anywhere the parent is used. This is known as the Liskov Substitution Principle.
You can, as the other answers, say, break one of the inherited methods.
The alternative is to refactor out the "optional" methods, and inherit from a baseclass that doesn't have them:
class BaseLiquid(object):
def foo(self):
pass
class Barised(object):
def bar(self):
pass
class Liquid(BaseLiquid, Barised): pass
class Water(BaseLiquid):
def drip(self):
pass
This is probably not a good idea, but you could always use metaclasses to implement private attributes:
def private_attrs(name, bases, attrs):
def get_base_attrs(base):
result = {}
for deeper_base in base.mro()[1:]:
result.update( get_base_attrs(deeper_base) )
priv = []
if "__private__" in base.__dict__:
priv = base.__private__
for attr in base.__dict__:
if attr not in priv:
result.update( {attr: base.__dict__[attr]} )
return result
final_attrs = {}
for base in bases:
final_attrs.update( get_base_attrs(base) )
final_attrs.update(attrs)
return type(name, (), final_attrs)
class Liquid(object):
__metaclass__ = private_attrs
__private__ = ['bar']
def foo(self):
pass
def bar(self):
pass
class Water(Liquid):
__metaclass__ = private_attrs
print Water.foo
print Water.bar
Output is:
<unbound method Water.foo>
Traceback (most recent call last):
File "testing-inheritance.py", line 41, in <module>
print Water.bar
AttributeError: type object 'Water' has no attribute 'bar'
EDIT: This will mess up isinstance() because it doesn't modify bases of the class.
http://docs.python.org/release/2.5.2/ref/slots.html
I suspect you can do this, by using the slots attr.
It might be possible implementing a getattr method and throwing the appropriate exception if bar is called.
However, I agree, you don't want to do this in practice, since its a sign of bad design.
According to the docs, super(cls, obj) returns
a proxy object that delegates method calls to a parent or sibling
class of type cls
I understand why super() offers this functionality, but I need something slightly different: I need to create a proxy object that delegates methods calls (and attribute lookups) to class cls itself; and as in super, if cls doesn't implement the method/attribute, my proxy should continue looking in the MRO order (of the new not the original class). Is there any function I can write that achieves that?
Example:
class X:
def act():
#...
class Y:
def act():
#...
class A(X, Y):
def act():
#...
class B(X, Y):
def act():
#...
class C(A, B):
def act():
#...
c = C()
b = some_magic_function(B, c)
# `b` needs to delegate calls to `act` to B, and look up attribute `s` in B
# I will pass `b` somewhere else, and have no control over it
Of course, I could do b = super(A, c), but that relies on knowing the exact class hierarchy and the fact that B follows A in the MRO. It would silently break if any of these two assumptions change in the future. (Note that super doesn't make any such assumptions!)
If I just needed to call b.act(), I could use B.act(c). But I am passing b to someone else, and have no idea what they'll do with it. I need to make sure it doesn't betray me and start acting like an instance of class C at some point.
A separate question, the documentation for super() (in Python 3.2) only talks about its method delegation, and does not clarify that attribute lookups for the proxy are also performed the same way. Is it an accidental omission?
EDIT
The updated Delegate approach works in the following example as well:
class A:
def f(self):
print('A.f')
def h(self):
print('A.h')
self.f()
class B(A):
def g(self):
self.f()
print('B.g')
def f(self):
print('B.f')
def t(self):
super().h()
a_true = A()
# instance of A ends up executing A.f
a_true.h()
b = B()
a_proxy = Delegate(A, b)
# *unlike* super(), the updated `Delegate` implementation would call A.f, not B.f
a_proxy.h()
Note that the updated class Delegate is closer to what I want than super() for two reasons:
super() only does it proxying for the first call; subsequent calls will happen as normal, since by then the object is used, not its proxy.
super() does not allow attribute access.
Thus, my question as asked has a (nearly) perfect answer in Python.
It turns out that, at a higher level, I was trying to do something I shouldn't (see my comments here).
This class should cover the most common cases:
class Delegate:
def __init__(self, cls, obj):
self._delegate_cls = cls
self._delegate_obj = obj
def __getattr__(self, name):
x = getattr(self._delegate_cls, name)
if hasattr(x, "__get__"):
return x.__get__(self._delegate_obj)
return x
Use it like this:
b = Delegate(B, c)
(with the names from your example code.)
Restrictions:
You cannot retrieve some special attributes like __class__ etc. from the class you pass in the constructor via this proxy. (This restistions also applies to super.)
This might behave weired if the attribute you want to retrieve is some weired kind of descriptor.
Edit: If you want the code in the update to your question to work as desired, you can use the foloowing code:
class Delegate:
def __init__(self, cls):
self._delegate_cls = cls
def __getattr__(self, name):
x = getattr(self._delegate_cls, name)
if hasattr(x, "__get__"):
return x.__get__(self)
return x
This passes the proxy object as self parameter to any called method, and it doesn't need the original object at all, hence I deleted it from the constructor.
If you also want instance attributes to be accessible you can use this version:
class Delegate:
def __init__(self, cls, obj):
self._delegate_cls = cls
self._delegate_obj = obj
def __getattr__(self, name):
if name in vars(self._delegate_obj):
return getattr(self._delegate_obj, name)
x = getattr(self._delegate_cls, name)
if hasattr(x, "__get__"):
return x.__get__(self)
return x
A separate question, the documentation for super() (in Python 3.2)
only talks about its method delegation, and does not clarify that
attribute lookups for the proxy are also performed the same way. Is it
an accidental omission?
No, this is not accidental. super() does nothing for attribute lookups. The reason is that attributes on an instance are not associated with a particular class, they're just there. Consider the following:
class A:
def __init__(self):
self.foo = 'foo set from A'
class B(A):
def __init__(self):
super().__init__()
self.bar = 'bar set from B'
class C(B):
def method(self):
self.baz = 'baz set from C'
class D(C):
def __init__(self):
super().__init__()
self.foo = 'foo set from D'
self.baz = 'baz set from D'
instance = D()
instance.method()
instance.bar = 'not set from a class at all'
Which class "owns" foo, bar, and baz?
If I wanted to view instance as an instance of C, should it have a baz attribute before method is called? How about afterwards?
If I view instance as an instance of A, what value should foo have? Should bar be invisible because was only added in B, or visible because it was set to a value outside the class?
All of these questions are nonsense in Python. There's no possible way to design a system with the semantics of Python that could give sensible answers to them. __init__ isn't even special in terms of adding attributes to instances of the class; it's just a perfectly ordinary method that happens to be called as part of the instance creation protocol. Any method (or indeed code from another class altogether, or not from any class at all) can create attributes on any instance it has a reference to.
In fact, all of the attributes of instance are stored in the same place:
>>> instance.__dict__
{'baz': 'baz set from C', 'foo': 'foo set from D', 'bar': 'not set from a class at all'}
There's no way to tell which of them were originally set by which class, or were last set by which class, or whatever measure of ownership you want. There's certainly no way to get at "the A.foo being shadowed by D.foo", as you would expect from C++; they're the same attribute, and any writes to to it by one class (or from elsewhere) will clobber a value left in it by the other class.
The consequence of this is that super() does not perform attribute lookups the same way it does method lookups; it can't, and neither can any code you write.
In fact, from running some experiments, neither super nor Sven's Delegate actually support direct attribute retrieval at all!
class A:
def __init__(self):
self.spoon = 1
self.fork = 2
def foo(self):
print('A.foo')
class B(A):
def foo(self):
print('B.foo')
b = B()
d = Delegate(A, b)
s = super(B, b)
Then both work as expected for methods:
>>> d.foo()
A.foo
>>> s.foo()
A.foo
But:
>>> d.fork
Traceback (most recent call last):
File "<pyshell#43>", line 1, in <module>
d.fork
File "/tmp/foo.py", line 6, in __getattr__
x = getattr(self._delegate_cls, name)
AttributeError: type object 'A' has no attribute 'fork'
>>> s.spoon
Traceback (most recent call last):
File "<pyshell#45>", line 1, in <module>
s.spoon
AttributeError: 'super' object has no attribute 'spoon'
So they both only really work for calling some methods on, not for passing to arbitrary third party code to pretend to be an instance of the class you want to delegate to.
They don't behave the same way in the presence of multiple inheritance unfortunately. Given:
class Delegate:
def __init__(self, cls, obj):
self._delegate_cls = cls
self._delegate_obj = obj
def __getattr__(self, name):
x = getattr(self._delegate_cls, name)
if hasattr(x, "__get__"):
return x.__get__(self._delegate_obj)
return x
class A:
def foo(self):
print('A.foo')
class B:
pass
class C(B, A):
def foo(self):
print('C.foo')
c = C()
d = Delegate(B, c)
s = super(C, c)
Then:
>>> d.foo()
Traceback (most recent call last):
File "<pyshell#50>", line 1, in <module>
d.foo()
File "/tmp/foo.py", line 6, in __getattr__
x = getattr(self._delegate_cls, name)
AttributeError: type object 'B' has no attribute 'foo'
>>> s.foo()
A.foo
Because Delegate ignores the full MRO of whatever class _delegate_obj is an instance of, only using the MRO of _delegate_cls. Whereas super does what you asked in the question, but the behaviour seems quite strange: it's not wrapping an instance of C to pretend it's an instance of B, because direct instances of B don't have foo defined.
Here's my attempt:
class MROSkipper:
def __init__(self, cls, obj):
self.__cls = cls
self.__obj = obj
def __getattr__(self, name):
mro = self.__obj.__class__.__mro__
i = mro.index(self.__cls)
if i == 0:
# It's at the front anyway, just behave as getattr
return getattr(self.__obj, name)
else:
# Check __dict__ not getattr, otherwise we'd find methods
# on classes we're trying to skip
try:
return self.__obj.__dict__[name]
except KeyError:
return getattr(super(mro[i - 1], self.__obj), name)
I rely on the __mro__ attribute of classes to properly figure out where to start from, then I just use super. You could walk the MRO chain from that point yourself checking class __dict__s for methods instead if the weirdness of going back one step to use super is too much.
I've made no attempt to handle unusual attributes; those implemented with descriptors (including properties), or those magic methods looked up behind the scenes by Python, which often start at the class rather than the instance directly. But this behaves as you asked moderately well (with the caveat expounded on ad nauseum in the first part of my post; looking up attributes this way will not give you any different results than looking them up directly in the instance).
I have an object that holds lots of ids that are accessed statically. I want to split that up into another object which holds only those ids without the need of making modifications to the already existen code base. Take for example:
class _CarType(object):
DIESEL_CAR_ENGINE = 0
GAS_CAR_ENGINE = 1 # lots of these ids
class Car(object):
types = _CarType
I want to be able to access _CarType.DIESEL_CAR_ENGINE either by calling Car.types.DIESEL_CAR_ENGINE, either by Car.DIESEL_CAR_ENGINE for backwards compatibility with the existent code. It's clear that I cannot use __getattr__ so I am trying to find a way of making this work (maybe metaclasses ? )
Although this is not exactly what subclassing is made for, it accomplishes what you describe:
class _CarType(object):
DIESEL_CAR_ENGINE = 0
GAS_CAR_ENGINE = 1 # lots of these ids
class Car(_CarType):
types = _CarType
Something like:
class Car(object):
for attr, value in _CarType.__dict__.items():
it not attr.startswith('_'):
locals()[attr] = value
del attr, value
Or you can do it out of the class declaration:
class Car(object):
# snip
for attr, value in _CarType.__dict__.items():
it not attr.startswith('_'):
setattr(Car, attr, value)
del attr, value
This is how you could do this with a metaclass:
class _CarType(type):
DIESEL_CAR_ENGINE = 0
GAS_CAR_ENGINE = 1 # lots of these ids
def __init__(self,name,bases,dct):
for key in dir(_CarType):
if key.isupper():
setattr(self,key,getattr(_CarType,key))
class Car(object):
__metaclass__=_CarType
print(Car.DIESEL_CAR_ENGINE)
print(Car.GAS_CAR_ENGINE)
Your options fall into two substantial categories: you either copy the attributes from _CarType into Car, or set Car's metaclass to a custom one with a __getattr__ method that delegates to _CarType (so it isn't exactly true that you can't use __getattr__: you can, you just need to put in in Car's metaclass rather than in Car itself;-).
The second choice has implications that you might find peculiar (unless they are specifically desired): the attributes don't show up on dir(Car), and they can't be accessed on an instance of Car, only on Car itself. I.e.:
>>> class MetaGetattr(type):
... def __getattr__(cls, nm):
... return getattr(cls.types, nm)
...
>>> class Car:
... __metaclass__ = MetaGetattr
... types = _CarType
...
>>> Car.GAS_CAR_ENGINE
1
>>> Car().GAS_CAR_ENGINE
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'Car' object has no attribute 'GAS_CAR_ENGINE'
You could fix the "not from an instance" issue by also adding a __getattr__ to Car:
>>> class Car:
... __metaclass__ = MetaGetattr
... types = _CarType
... def __getattr__(self, nm):
... return getattr(self.types, nm)
...
to make both kinds of lookup work, as is probably expected:
>>> Car.GAS_CAR_ENGINE
1
>>> Car().GAS_CAR_ENGINE
1
However, defining two, essentially-equal __getattr__s, doesn't seem elegant.
So I suspect that the simpler approach, "copy all attributes", is preferable. In Python 2.6 or better, this is an obvious candidate for a class decorator:
def typesfrom(typesclass):
def decorate(cls):
cls.types = typesclass
for n in dir(typesclass):
if n[0] == '_': continue
v = getattr(typesclass, n)
setattr(cls, n, v)
return cls
return decorate
#typesfrom(_CarType)
class Car(object):
pass
In general, it's worth defining a decorator if you're using it more than once; if you only need to perform this task for one class ever, then expanding the code inline instead (after the class statement) may be better.
If you're stuck with Python 2.5 (or even 2.4), you can still define typesfrom the same way, you just apply it in a slightly less elegant matter, i.e., the Car definition becomes:
class Car(object):
pass
Car = typesfrom(_CarType)(Car)
Do remember decorator syntax (introduced in 2.2 for functions, in 2.6 for classes) is just a handy way to wrap these important and frequently recurring semantics.
class _CarType(object):
DIESEL_CAR_ENGINE = 0
GAS_CAR_ENGINE = 1 # lots of these ids
class Car:
types = _CarType
def __getattr__(self, name):
return getattr(self.types, name)
If an attribute of an object is not found, and it defines that magic method __getattr__, that gets called to try to find it.
Only works on a Car instance, not on the class.