How can I check if an attribute of an object has been set by the user or not? Currently, I have a class
class foo:
def __init__(self):
self.bar = 'baz'
Later I would like to check if the user set the value of bar or not, i.e. with something like
my_foo = foo()
my_foo.bar = 'mybaz'
so I would like to know if the second line above has been called or not (to throw a warning if it has not). I have two solutions, but I don't like either of them:
Check if my_foo.bar is equal to the default value. But it could be that the user sets my_foo.bar to the same value and then I don't want to throw a warning.
Don't set the default value in __init__, but only when it is used. Then it can be checked with getattr() and set with setattr().
I'm sure there is an elegant pythonic way to do it that I haven't thought of.
Use the #property decorator to construct getters and setters, and the make the setter tell you when a user changes the attribute, example below
class Foo:
def __init__(self):
self._x_was_modified = False
self._x = None
#property
def x(self):
return self._x
#x.setter
def x(self, value):
self._x_was_modified = True
self._x = value
foo = Foo()
print('x was modified by user: {}'.format(foo._x_was_modified))
foo.x = 42
print('x was modified by user: {}'.format(foo._x_was_modified))
This will output:
x was modified by user: False
x was modified by user: True
Another solution that again involves using a flag is done by playing around with the __setattr__ method:
class foo:
def __init__(self):
self._altered = -1
self.bar = 'baz'
def __setattr__(self, attrname, val):
if attrname is 'bar':
self._altered += 1
super.__setattr__(self, attrname, val)
# if you heard super is evil and you feel
# funny using it in your code instead use:
# object.__setattr__(self, attrname, val)
If somebody re-sets this variable the value of self._altered will be positive and as a result evaluate to True (remember, first access is made in __init__ during initialization and increments _altered once).
After this you can make a nice intuitive check of the form:
f = foo()
f.bar = "booz" # now f._altered == 1
if f._altered: raise MyAlcoholicException
and be done. Just another way of doing practically the same exact thing.
Related
Say I have a simple class Foo, which comes from an external library, thus I cannot change it directly:
class Foo(object):
def __init__(self, x):
self.x = x
I want to create a subclass Bar and prevent x from being change from an instance of Bar, but still use the x in Bar's methods.
Here's what I tried, and it will probably enlighten the basic idea, but unfortunately it doesn't work:
class Bar(Foo):
#property
def x(self):
return super().x
#x.setter
def x(self, value):
raise NotImplementedError('Do not change x directly, use "do_stuff()" instead')
def do_stuff(self, value):
if <something>:
super().x = value
So basically I've created some wrapper functions (do_stuff()) around an attribute, and now I want to prevent the attribute from being changed directly, as it might mess up some functionality of the wrapper functions. Is this possible in a reasonable way?
Edited with a better example of what I want. I'm not trying to prevent them from seeing the variable x, but instead changing it from outside of do_stuff()
This should be much simpler to accomplish if you are willing to avoid inheritance altogether:
def main():
bar = Bar(123)
bar.fizz()
bar.buzz()
bar.fizz()
bar.set_x(456)
print('bar.x =', bar.x)
try:
bar.x = 123
except AttributeError:
print('bar.x cannot be set directly')
else:
raise AssertionError('an AttributeError should have been raised')
bar.mutate_x(789)
bar.fizz()
bar.set_x(0)
bar.fizz()
bar.mutate_x(1)
bar.fizz()
bar.set_x('Hello World')
bar.fizz()
class Foo:
def __init__(self, x):
self.x = x
def fizz(self):
print(self.x)
def buzz(self):
self.x = None
class Bar:
def __init__(self, x):
self.__foo = foo = Foo(x)
self.__copy_methods(foo)
def __copy_methods(self, obj):
for name in dir(obj):
if name.startswith('__') or name.endswith('__'):
continue
attr = getattr(obj, name)
if callable(attr):
setattr(self, name, attr)
#property
def x(self):
return self.__foo.x
def set_x(self, value):
if isinstance(value, int) and value > 0:
self.__foo.x = value
mutate_x = set_x
if __name__ == '__main__':
main()
The short answer is: No, this is not possible in a reasonable way.
Python's guiding principle here, to use the phrasing from the style guide is that we are all responsible users. Meaning that code is trusted not to do silly things, and people should generally avoid messing with members of other people's classes without a good reason.
The first and best way to prevent people from accidentally changing a value is to mark it using the single underscore (_variable). This however may not offer you the protection you want against accidental modification of your variables.
The next step up in protection is to use a double underscore. Quoting from PEP-8:
To avoid name clashes with subclasses, use two leading underscores to invoke Python's name mangling rules.
Python mangles these names with the class name: if class Foo has an attribute named __a , it cannot be accessed by Foo.__a . (An insistent user could still gain access by calling Foo._Foo__a .) Generally, double leading underscores should be used only to avoid name conflicts with attributes in classes designed to be subclassed.
The mangling makes it more difficult to accidentally overwrite a value.
I added emphasis to that last sentence because it is important. Using this mechanism for preventing accidental access to a member is not really the something that should be done for a lot of members.
In your specific case, the way that I'd solve the problem would be to not subclass at all. Consider:
class Foo(object):
def __init__(self, x):
self.x = x
class Bar():
def __init__(self, x):
self._foo = Foo(x)
#property
def x(self):
return self._foo.x
def do_stuff(self, value):
# Validate the value, and the wrapped object's state
if valid:
self._foo.x = value
Of course this means that Bar has to wrap all of Foo's methods that you want to wrap. Yes, someone could still,
b = Bar(100)
b._foo.x = 127 # shame on them :)
or
b = Bar(100)
b._foo = EvilFoo(127)
but it's harder to unintentionally do.
You're on the right track, you want to make x a property instead of having it be an attribute in the subclass. Where you went wrong was trying to store the raw data for x on super. What you want to do is exploit the fact that the parent class can use the new property of the subclass transparently and does not need to know that it is now a property and not a attribute. Something like this should work for you:
class Foo(object):
def __init__(self, x):
self.x = x
class Bar(Foo):
_protected_x = None
#property
def x(self):
return self._protected_x
#x.setter
def x(self, value):
if self._protected_x is None:
self._protected_x = value
else:
raise ValueError("Use set_x to change x.")
def set_x(self, value):
self._protected_x = value
b = Bar(12)
print b.x
b.set_x(5)
print b.x
I am generally confused about the difference between a "property" and an "attribute", and can't find a great resource to concisely detail the differences.
Properties are a special kind of attribute. Basically, when Python encounters the following code:
spam = SomeObject()
print(spam.eggs)
it looks up eggs in spam, and then examines eggs to see if it has a __get__, __set__, or __delete__ method — if it does, it's a property. If it is a property, instead of just returning the eggs object (as it would for any other attribute) it will call the __get__ method (since we were doing lookup) and return whatever that method returns.
More information about Python's data model and descriptors.
With a property you have complete control on its getter, setter and deleter methods, which you don't have (if not using caveats) with an attribute.
class A(object):
_x = 0
'''A._x is an attribute'''
#property
def x(self):
'''
A.x is a property
This is the getter method
'''
return self._x
#x.setter
def x(self, value):
"""
This is the setter method
where I can check it's not assigned a value < 0
"""
if value < 0:
raise ValueError("Must be >= 0")
self._x = value
>>> a = A()
>>> a._x = -1
>>> a.x = -1
Traceback (most recent call last):
File "ex.py", line 15, in <module>
a.x = -1
File "ex.py", line 9, in x
raise ValueError("Must be >= 0")
ValueError: Must be >= 0
In general speaking terms a property and an attribute are the same thing. However, there is a property decorator in Python which provides getter/setter access to an attribute (or other data).
class MyObject(object):
# This is a normal attribute
foo = 1
#property
def bar(self):
return self.foo
#bar.setter
def bar(self, value):
self.foo = value
obj = MyObject()
assert obj.foo == 1
assert obj.bar == obj.foo
obj.bar = 2
assert obj.foo == 2
assert obj.bar == obj.foo
The property allows you to get and set values like you would normal attributes, but underneath there is a method being called translating it into a getter and setter for you. It's really just a convenience to cut down on the boilerplate of calling getters and setters.
Lets say for example, you had a class that held some x and y coordinates for something you needed. To set them you might want to do something like:
myObj.x = 5
myObj.y = 10
That is much easier to look at and think about than writing:
myObj.setX(5)
myObj.setY(10)
The problem is, what if one day your class changes such that you need to offset your x and y by some value? Now you would need to go in and change your class definition and all of the code that calls it, which could be really time consuming and error prone. The property allows you to use the former syntax while giving you the flexibility of change of the latter.
In Python, you can define getters, setters, and delete methods with the property function. If you just want the read property, there is also a #property decorator you can add above your method.
http://docs.python.org/library/functions.html#property
I learnt 2 differences from site of Bernd Klein, in summary:
1. A property is a more convenient way to achieve data encapsulation
For example, let's say you have a public attribute length. Later on, your project requires you to encapsulate it, i.e. to change it to private and provide a getter and setter => you have to change the the code you wrote before:
# Old code
obj1.length = obj1.length + obj2.length
# New code (using private attributes and getter and setter)
obj1.set_length(obj1.get_length() + obj2.get_length()) # => this is ugly
If you use #property and #length.setter => you don't need to change that old code.
2. A property can encapsulate multiple attributes
class Person:
def __init__(self, name, physic_health, mental_health):
self.name = name
self.__physic_health = physic_health
self.__mental_health = mental_health
#property
def condition(self):
health = self.__physic_health + self.__mental_health
if(health < 5.0):
return "I feel bad!"
elif health < 8.0:
return "I am ok!"
else:
return "Great!"
In this example, __physic_health and __mental_health are private and cannot be accessed directly from outside.
There is also one not obvious difference that i use to cache or refresh data , often we have a function connected to class attribute. For instance i need to read file once and keep content assigned to the attribute so the value is cached:
class Misc():
def __init__(self):
self.test = self.test_func()
def test_func(self):
print 'func running'
return 'func value'
cl = Misc()
print cl.test
print cl.test
Output:
func running
func value
func value
We accessed the attribute twice but our function was fired only once. Changing the above example to use property will cause attribute's value refresh each time you access it:
class Misc():
#property
def test(self):
print 'func running'
return 'func value'
cl = Misc()
print cl.test
print cl.test
Output:
func running
func value
func running
func value
I like to think that, if you want to set a restriction for an attribute, use a property.
Although all attributes are public, generally programmers differentiate public and private attributes with an underscore(_). Consider the following class,
class A:
def __init__(self):
self.b = 3 # To show public
self._c = 4 # To show private
Here, b attribute is intended to be accessed from outside class A. But, readers of this class might wonder, can b attribute be set from outside class A?
If we intend to not set b from outside, we can show this intention with #property.
class A:
def __init__(self):
self._c = 4 # To show private
#property
def b(self):
return 3
Now, b can't be set.
a = A()
print(a.b) # prints 3
a.b = 7 # Raises AttributeError
Or, if you wish to set only certain values,
class A:
#property
def b(self):
return self._b
#b.setter
def b(self, val):
if val < 0:
raise ValueError("b can't be negative")
self._b = val
a = A()
a.b = 6 # OK
a.b = -5 # Raises ValueError
Is it at all possible to monkey patch the value of a #property of an instance of a class that I do not control?
class Foo:
#property
def bar(self):
return here().be['dragons']
f = Foo()
print(f.bar) # baz
f.bar = 42 # MAGIC!
print(f.bar) # 42
Obviously the above would produce an error when trying to assign to f.bar. Is # MAGIC! possible in any way? The implementation details of the #property are a black box and not indirectly monkey-patchable. The entire method call needs to be replaced. It needs to affect a single instance only (class-level patching is okay if inevitable, but the changed behaviour must only selectively affect a given instance, not all instances of that class).
Subclass the base class (Foo) and change single instance's class to match the new subclass using __class__ attribute:
>>> class Foo:
... #property
... def bar(self):
... return 'Foo.bar'
...
>>> f = Foo()
>>> f.bar
'Foo.bar'
>>> class _SubFoo(Foo):
... bar = 0
...
>>> f.__class__ = _SubFoo
>>> f.bar
0
>>> f.bar = 42
>>> f.bar
42
from module import ClassToPatch
def get_foo(self):
return 'foo'
setattr(ClassToPatch, 'foo', property(get_foo))
To monkey patch a property, there is an even simpler way:
from module import ClassToPatch
def get_foo(self):
return 'foo'
ClassToPatch.foo = property(get_foo)
Idea: replace property descriptor to allow setting on certain objects. Unless a value is explicitly set this way, original property getter is called.
The problem is how to store the explicitly set values. We cannot use a dict keyed by patched objects, since 1) they are not necessarily comparable by identity; 2) this prevents patched objects from being garbage-collected. For 1) we could write a Handle that wraps objects and overrides comparison semantics by identity and for 2) we could use weakref.WeakKeyDictionary. However, I couldn't make these two work together.
Therefore we use a different approach of storing the explicitly set values on the object itself, using a "very unlikely attribute name". It is of course still possible that this name would collide with something, but that's pretty much inherent to languages such as Python.
This won't work on objects that lack a __dict__ slot. Similar problem would arise for weakrefs though.
class Foo:
#property
def bar (self):
return 'original'
class Handle:
def __init__(self, obj):
self._obj = obj
def __eq__(self, other):
return self._obj is other._obj
def __hash__(self):
return id (self._obj)
_monkey_patch_index = 0
_not_set = object ()
def monkey_patch (prop):
global _monkey_patch_index, _not_set
special_attr = '$_prop_monkey_patch_{}'.format (_monkey_patch_index)
_monkey_patch_index += 1
def getter (self):
value = getattr (self, special_attr, _not_set)
return prop.fget (self) if value is _not_set else value
def setter (self, value):
setattr (self, special_attr, value)
return property (getter, setter)
Foo.bar = monkey_patch (Foo.bar)
f = Foo()
print (Foo.bar.fset)
print(f.bar) # baz
f.bar = 42 # MAGIC!
print(f.bar) # 42
It looks like you need to move on from properties to the realms of data descriptors and non-data descriptors. Properties are just a specialised version of data descriptors. Functions are an example of non-data descriptors -- when you retrieve them from an instance they return a method rather than the function itself.
A non-data descriptor is just an instance of a class that has a __get__ method. The only difference with a data descriptor is that it has a __set__ method as well. Properties initially have a __set__ method that throws an error unless you provide a setter function.
You can achieve what you want really easily just by writing your own trivial non-data descriptor.
class nondatadescriptor:
"""generic nondata descriptor decorator to replace #property with"""
def __init__(self, func):
self.func = func
def __get__(self, obj, objclass):
if obj is not None:
# instance based access
return self.func(obj)
else:
# class based access
return self
class Foo:
#nondatadescriptor
def bar(self):
return "baz"
foo = Foo()
another_foo = Foo()
assert foo.bar == "baz"
foo.bar = 42
assert foo.bar == 42
assert another_foo.bar == "baz"
del foo.bar
assert foo.bar == "baz"
print(Foo.bar)
What makes all this work is that logic under the hood __getattribute__. I can't find the appropriate documentation at the moment, but order of retrieval is:
Data descriptors defined on the class are given the highest priority (objects with both __get__ and __set__), and their __get__ method is invoked.
Any attribute of the object itself.
Non-data descriptors defined on the class (objects with only a __get__ method).
All other attributes defined on the class.
Finally the __getattr__ method of the object is invoked as a last resort (if defined).
You can also patch property setters. Using #fralau 's answer:
from module import ClassToPatch
def foo(self, new_foo):
self._foo = new_foo
ClassToPatch.foo = ClassToPatch.foo.setter(foo)
reference
In case someone needs to patch a property while being able to call the original implementation, here is an example:
#property
def _cursor_args(self, __orig=mongoengine.queryset.base.BaseQuerySet._cursor_args):
# TODO: remove this hack when we upgrade MongoEngine
# https://github.com/MongoEngine/mongoengine/pull/2160
cursor_args = __orig.__get__(self)
if self._timeout:
cursor_args.pop("no_cursor_timeout", None)
return cursor_args
mongoengine.queryset.base.BaseQuerySet._cursor_args = _cursor_args
I am generally confused about the difference between a "property" and an "attribute", and can't find a great resource to concisely detail the differences.
Properties are a special kind of attribute. Basically, when Python encounters the following code:
spam = SomeObject()
print(spam.eggs)
it looks up eggs in spam, and then examines eggs to see if it has a __get__, __set__, or __delete__ method — if it does, it's a property. If it is a property, instead of just returning the eggs object (as it would for any other attribute) it will call the __get__ method (since we were doing lookup) and return whatever that method returns.
More information about Python's data model and descriptors.
With a property you have complete control on its getter, setter and deleter methods, which you don't have (if not using caveats) with an attribute.
class A(object):
_x = 0
'''A._x is an attribute'''
#property
def x(self):
'''
A.x is a property
This is the getter method
'''
return self._x
#x.setter
def x(self, value):
"""
This is the setter method
where I can check it's not assigned a value < 0
"""
if value < 0:
raise ValueError("Must be >= 0")
self._x = value
>>> a = A()
>>> a._x = -1
>>> a.x = -1
Traceback (most recent call last):
File "ex.py", line 15, in <module>
a.x = -1
File "ex.py", line 9, in x
raise ValueError("Must be >= 0")
ValueError: Must be >= 0
In general speaking terms a property and an attribute are the same thing. However, there is a property decorator in Python which provides getter/setter access to an attribute (or other data).
class MyObject(object):
# This is a normal attribute
foo = 1
#property
def bar(self):
return self.foo
#bar.setter
def bar(self, value):
self.foo = value
obj = MyObject()
assert obj.foo == 1
assert obj.bar == obj.foo
obj.bar = 2
assert obj.foo == 2
assert obj.bar == obj.foo
The property allows you to get and set values like you would normal attributes, but underneath there is a method being called translating it into a getter and setter for you. It's really just a convenience to cut down on the boilerplate of calling getters and setters.
Lets say for example, you had a class that held some x and y coordinates for something you needed. To set them you might want to do something like:
myObj.x = 5
myObj.y = 10
That is much easier to look at and think about than writing:
myObj.setX(5)
myObj.setY(10)
The problem is, what if one day your class changes such that you need to offset your x and y by some value? Now you would need to go in and change your class definition and all of the code that calls it, which could be really time consuming and error prone. The property allows you to use the former syntax while giving you the flexibility of change of the latter.
In Python, you can define getters, setters, and delete methods with the property function. If you just want the read property, there is also a #property decorator you can add above your method.
http://docs.python.org/library/functions.html#property
I learnt 2 differences from site of Bernd Klein, in summary:
1. A property is a more convenient way to achieve data encapsulation
For example, let's say you have a public attribute length. Later on, your project requires you to encapsulate it, i.e. to change it to private and provide a getter and setter => you have to change the the code you wrote before:
# Old code
obj1.length = obj1.length + obj2.length
# New code (using private attributes and getter and setter)
obj1.set_length(obj1.get_length() + obj2.get_length()) # => this is ugly
If you use #property and #length.setter => you don't need to change that old code.
2. A property can encapsulate multiple attributes
class Person:
def __init__(self, name, physic_health, mental_health):
self.name = name
self.__physic_health = physic_health
self.__mental_health = mental_health
#property
def condition(self):
health = self.__physic_health + self.__mental_health
if(health < 5.0):
return "I feel bad!"
elif health < 8.0:
return "I am ok!"
else:
return "Great!"
In this example, __physic_health and __mental_health are private and cannot be accessed directly from outside.
There is also one not obvious difference that i use to cache or refresh data , often we have a function connected to class attribute. For instance i need to read file once and keep content assigned to the attribute so the value is cached:
class Misc():
def __init__(self):
self.test = self.test_func()
def test_func(self):
print 'func running'
return 'func value'
cl = Misc()
print cl.test
print cl.test
Output:
func running
func value
func value
We accessed the attribute twice but our function was fired only once. Changing the above example to use property will cause attribute's value refresh each time you access it:
class Misc():
#property
def test(self):
print 'func running'
return 'func value'
cl = Misc()
print cl.test
print cl.test
Output:
func running
func value
func running
func value
I like to think that, if you want to set a restriction for an attribute, use a property.
Although all attributes are public, generally programmers differentiate public and private attributes with an underscore(_). Consider the following class,
class A:
def __init__(self):
self.b = 3 # To show public
self._c = 4 # To show private
Here, b attribute is intended to be accessed from outside class A. But, readers of this class might wonder, can b attribute be set from outside class A?
If we intend to not set b from outside, we can show this intention with #property.
class A:
def __init__(self):
self._c = 4 # To show private
#property
def b(self):
return 3
Now, b can't be set.
a = A()
print(a.b) # prints 3
a.b = 7 # Raises AttributeError
Or, if you wish to set only certain values,
class A:
#property
def b(self):
return self._b
#b.setter
def b(self, val):
if val < 0:
raise ValueError("b can't be negative")
self._b = val
a = A()
a.b = 6 # OK
a.b = -5 # Raises ValueError
Consider the following class :
class Token:
def __init__(self):
self.d_dict = {}
def __setattr__(self, s_name, value):
self.d_dict[s_name] = value
def __getattr__(self, s_name):
if s_name in self.d_dict.keys():
return self.d_dict[s_name]
else:
raise AttributeError('No attribute {0} found !'.format(s_name))
In my code Token have some other function (like get_all() wich return d_dict, has(s_name) which tell me if my token has a particular attribute).
Anyway, I think their is a flaw in my plan since it don't work : when I create a new instance, python try to call __setattr__('d_dict', '{}').
How can I achieve a similar behaviour (maybe in a more pythonic way ?) without having to write something like Token.set(name, value) and get(name) each I want to set or get an attribute for a token.
Critics about design flaw and/or stupidity welcome :)
Thank !
You need to special-case d_dict.
Although of course, in the above code, all you do is replicate what any object does with __dict__ already, so it's pretty pointless. Do I guess correctly if you intended to special case some attributes and actally use methods for those?
In that case, you can use properties.
class C(object):
def __init__(self):
self._x = None
#property
def x(self):
"""I'm the 'x' property."""
return self._x
#x.setter
def x(self, value):
self._x = value
#x.deleter
def x(self):
del self._x
The special-casing of __dict__ works like this:
def __init__(self):
self.__dict__['d_dict'] = {}
There is no need to use a new-style class for that.
A solution, not very pythonic but works. As Lennart Regebro pointed, you have to use a special case for d_dict.
class Token(object):
def __init__(self):
super(Token,self).__setattr__('d_dict', {})
def __getattr__(self,name):
return self.a[name]
def __setattr__(self,name,value):
self.a[name] = value
You need to use new style classes.
the problem seems to be in time of evaluation of your code in __init__ method.
You could define __new__ method and initialize d_dict variable there instead of __init__.
Thats a bit hackish but it works, remember though to comment it as after few months it'll be total magic.
>>> class Foo(object):
... def __new__(cls, *args):
... my_cls = super(Foo, cls).__new__(cls, *args)
... my_cls.d_dict = {}
... return my_cls
>>> f = Foo()
>>> id(f.d_dict)
3077948796L
>>> d = Foo()
>>> id(d.d_dict)
3078142804L
Word of explanation why I consider that hackish: call to __new__ returns new instance of class so then d_dict initialised in there is kind of static, but it's initialised with new instance of dictionary each time class is "created" so everything works as you need.
It's worth remembering that __getattr__ is only called if the attribute doesn't exist in the object, whereas __setattr__ is always called.
I think we'll be able to say something about the overall design of your class if you explain its purpose. For example,
# This is a class that serves as a dictionary but also has user-defined methods
class mydict(dict): pass
# This is a class that allows setting x.attr = value or getting x.attr:
class mysetget: pass
# This is a class that allows setting x.attr = value or getting x.attr:
class mygetsethas:
def has(self, key):
return key in self.__dict__
x = mygetsethas()
x.a = 5
print(x.has('a'), x.a)
I think the last class is closest to what you meant, and I also like to play with syntax and get lots of joy from it, but unfortunately this is not a good thing. Reasons why it's not advisable to use object attributes to re-implement dictionary: you can't use x.3, you conflict with x.has(), you have to put quotes in has('a') and many more.