While trying to create descriptors a few different ways I noticed some strange behavior that I'm trying to understand. Below are the three different ways I have gone about creating descriptors:
>>> class NumericValueOne():
... def __init__(self, name):
... self.name = name
... def __get__(self, obj, type=None) -> object:
... return obj.__dict__.get(self.name) or 0
... def __set__(self, obj, value) -> None:
... obj.__dict__[self.name] = value
>>> class NumericValueTwo():
... def __init__(self, name):
... self.name = name
... self.internal_name = '_' + self.name
... def __get__(self, obj, type=None) -> object:
... return getattr(obj, self.internal_name, 0)
... def __set__(self, obj, value) -> None:
... setattr(obj, self.internal_name, value)
>>> class NumericValueThree():
... def __init__(self, name):
... self.name = name
... def __get__(self, obj, type=None) -> object:
... return getattr(obj, self.name, 0)
... def __set__(self, obj, value) -> None:
... setattr(obj, self.name, value)
I then use them in the Foo classes, like below:
>>> class FooOne():
... number = NumericValueOne("number")
>>> class FooTwo():
... number = NumericValueTwo("number")
>>> class FooThree():
... number = NumericValueThree("number")
my_foo_object_one = FooOne()
my_foo_object_two = FooTwo()
my_foo_object_three = FooThree()
my_foo_object_one.number = 3
my_foo_object_two.number = 3
my_foo_object_three.number = 3
While FooOne and FooTwo work as expected when both setting & getting values. FooThree throws the following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 7, in __set__
File "<stdin>", line 7, in __set__
File "<stdin>", line 7, in __set__
[Previous line repeated 497 more times]
RecursionError: maximum recursion depth exceeded while calling a Python object
It looks like setattr() is calling the __set__() method? But why should it be doing that if setattr() is modifying the obj __dict__? And why does this work if we use internal_name?
Why is that we NEED to use a private variable in order to use the built-in getattr() and setattr() methods correctly? Also, how is this different from just directly modifying the obj __dict__ like in NumericValueOne?
But why should it be doing that if setattr() is modifying the obj __dict__?
setattr doesn't just modify the __dict__. It sets attributes, exactly like x.y = z would, and for the attribute you're trying to set, "set this attribute" means "call the setter you're already in". Hence, infinite recursion.
And why does this work if we use internal_name?
That name doesn't correspond to a property, so it just gets a __dict__ entry.
Related
I am new to python. I'm trying to create a configuration class with required validators. In below code snippet, accessing variable 'a' using python class and instance of class is returning a different value. Whether this is a proper design or should i initialise var 'a' only in the class constructor and do validation in the setter method.
class IntField:
def __get__(self, instance, owner):
if instance is None:
return self
return instance.__dict__[self.name]
def __set__(self, instance, value):
if not isinstance(value, int):
raise ValueError('expecting integer')
instance.__dict__[self.name] = value
def __set_name__(self, owner, name):
self.name = name
class DefaultConfig:
a = IntField()
def __init__(self):
self.a = 2
print(DefaultConfig.a)
print(DefaultConfig().a)
output:
<__main__.IntField object at 0x10c34b550>
2
I just want to know whether it's right way to do it
Rather asking for opinion only answer, by I will try to be as objective as I can.
Your code behaves as expected as long as instances attributes are processed:
>>> c = DefaultConfig()
>>> c.a = 'foo'
Traceback (most recent call last):
File "<pyshell#88>", line 1, in <module>
c.a = 'foo'
File "<pyshell#83>", line 10, in __set__
raise ValueError('expecting integer')
ValueError: expecting integer
>>> c.a = 4
>>> c.a
4
When inspecting DefaultConfig.a, the __get__ function is still used with instance=None. So you can choose one of 2 possible ways:
be transparent and show what the attribute actually is (what you currently do)
insist on the descriptor magic and return the default value (here 2).
For that latter way, code could become:
class IntField:
def __get__(self, instance, owner):
if instance is None:
return getattr(owner, '_default_' + self.name, self)
return instance.__dict__[self.name]
def __set__(self, instance, value):
if not isinstance(value, int):
raise ValueError('expecting integer')
instance.__dict__[self.name] = value
def __set_name__(self, owner, name):
self.name = name
class DefaultConfig:
a = IntField()
_default_a = 2
def __init__(self):
self.a = self._default_a
The trick here is that by convention, the default value for an attribute x is expected to be _default_x.
In that case, you will get:
print(DefaultConfig.a)
2
This question already has answers here:
Understanding __get__ and __set__ and Python descriptors
(8 answers)
Closed 5 years ago.
>>> class Const(object): # an overriding descriptor, see later
... def __init__(self, value):
... self.value = value
... def __set__(self, value):
... self.value = value
... def __get__(self, *_): # always return the constant value
... return self.value
...
>>>
>>> class X(object):
... c = Const(23)
...
>>> x=X()
>>> print(x.c) # prints: 23
23
>>> x.c = 42
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: __set__() takes 2 positional arguments but 3 were given
What does
TypeError: __set__() takes 2 positional arguments but 3 were given`
means?
Is __set__() a method belonging to the descriptor type Const?
What is __set__()'s signature?
Thanks.
Signature for __set__
The signature for __set__ is documented here:
object.__set__(self, instance, value) Called to set the attribute on
an instance instance of the owner class to a new value, value.
Meaning of the TypeError
The TypeError is tell you that the instance parameter is missing, it should be def __set__(self, instance, value): ....
Worked-out solution
Here's one approach to making the Constant class work correctly:
class Const(object):
def __init__(self, value):
self._value = value
def __set__(self, inst, value):
raise TypeError('Cannot assign to a constant')
def __get__(self, inst, cls=None):
return self._value
class X(object):
c = Const(23)
Trying it out in an interactive session gives:
>>> x = X()
>>> print(x.c)
23
>>> x.c = 42
Traceback (most recent call last):
File "<pyshell#3>", line 1, in <module>
x.c = 42
File "/Users/raymond/Documents/try_forth/tmp.py", line 5, in __set__
raise TypeError('Cannot assign to a constant')
TypeError: Cannot assign to a constant
I have a few classes each of which has a number of attributes. What all of the attributes have in common is that they should be numeric properties. This seems to be an ideal place to use python's decorators, but I can't seem to wrap my mind around what the correct implementation would be. Here is a simple example:
class Junk(object):
def __init__(self, var):
self._var = var
#property
def var(self):
"""A numeric variable"""
return self._var
#var.setter
def size(self, value):
# need to make sure var is an integer
if not isinstance(value, int):
raise ValueError("var must be an integer, var = {}".format(value))
self._var = value
#var.deleter
def size(self):
raise RuntimeError("You can't delete var")
It seems to me that it should be possible to write a decorator that does everything so that the above can be transformed into:
def numeric_property(*args, **kwargs):
...
class Junk(object):
def __init__(self, var):
self._var = var
#numeric_property
def var(self):
"""A numeric variable"""
return self._var
That way the new numeric_property decorator can be used in many classes.
A #property is just a special case of Python's descriptor protocol, so you can certainly build your own custom versions. For your case:
class NumericProperty:
"""A property that must be numeric.
Args:
attr (str): The name of the backing attribute.
"""
def __init__(self, attr):
self.attr = attr
def __get__(self, obj, type=None):
return getattr(obj, self.attr)
def __set__(self, obj, value):
if not isinstance(value, int):
raise ValueError("{} must be an integer, var = {!r}".format(self.attr, value))
setattr(obj, self.attr, value)
def __delete__(self, obj):
raise RuntimeError("You can't delete {}".format(self.attr))
class Junk:
var = NumericProperty('_var')
def __init__(self, var):
self.var = var
In use:
>>> j = Junk('hi')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/jonrsharpe/test.py", line 29, in __init__
self.var = var
File "/Users/jonrsharpe/test.py", line 17, in __set__
raise ValueError("{} must be an integer, var = {!r}".format(self.attr, value))
ValueError: _var must be an integer, var = 'hi'
>>> j = Junk(1)
>>> del j.var
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/jonrsharpe/test.py", line 21, in __delete__
raise RuntimeError("You can't delete {}".format(self.attr))
RuntimeError: You can't delete _var
>>> j.var = 'hello'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/jonrsharpe/test.py", line 17, in __set__
raise ValueError("{} must be an integer, var = {!r}".format(self.attr, value))
ValueError: _var must be an integer, var = 'hello'
>>> j.var = 2
>>> j.var
2
Option 1: inherit from property
property is a descriptor. See Descriptor HowTo on python.org.
So, can inherit from property and override the relevant methods.
For example, to enforce int on setter:
class numeric_property(property):
def __set__(self, obj, value):
assert isinstance(value, int), "numeric_property requires an int"
super(numeric_property, self).__set__(obj, value)
class A(object):
#numeric_property
def x(self):
return self._x
#x.setter
def x(self, value):
self._x = value
And now you have integers enforced:
>>> a = A()
>>> a.x = 'aaa'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in __set__
AssertionError: numeric_property requires an int
Option 2: Create a better descriptor
On the other hand, it may be even better to implement a brand new descriptor which does not inherit from property, which would enable you to define the property in one go.
It would be nicer to have this kind of interface:
class A(object):
x = numeric_property('_x')
For that you would implement a descriptor which takes the attribute name:
class numeric_property(object):
def __init__(self, private_attribute_name, default=0):
self.private_attribute_name = private_attribute_name
self.default = default
def __get__(self, obj, typ):
if not obj: return self
return getattr(obj, self.private_attribute_name, self.default)
def __set__(self, obj, value):
assert isinstance(value, int), "numeric_property requires an int"
setattr(obj, self.private_attribute_name, value)
Disclaimer :)
I would rather not enforce strict typing in Pyhon, because Python is much more powerful without it.
You may just create a function that does it for you . As simple as it can get, no need to create a custom descriptor:
def numprop(name, privname):
#property
def _numprop(self):
return getattr(self, privname)
#_numprop.setter
def _numprop(self, value):
if not isinstance(value, int):
raise ValueError("{name} must be an integer, {name} = {}".format(value, name=name))
setattr(self, privname, value)
#_numprop.deleter
def _numprop(self):
raise RuntimeError("You can't delete var")
return _numprop
class Junk(object):
def __init__(self, var):
self._var = var
var = numprop("var", "_var")
I have a property that has an assertion to check if a value is of type str.
To catch this assertionError I have created a decorator according to the examples I have found online.
Decorator:
def catch_assertionerror(function):
def handle_problems(*args, **kwargs):
try:
return function(*args, **kwargs)
except AssertionError:
# log_error(err.args[0])
print "error caught"
return handle_problems
Property:
#catch_assertionerror
#name.setter
def name(self, value):
assert isinstance(value, str), "This value should be a string"
self._name = name
Setting the name property:
self.name = self.parse_name_from_xml()
When I run this code, there is no error shown, so I guess it is caught, but on the other hand, the error message is not printed to the screen.
Then I tried a more simple example I found on Stachoverflow:
def handleError(function):
def handleProblems():
try:
function()
except Exception:
print "Oh noes"
return handleProblems
#handleError
def example():
raise Exception("Boom!")
This also handled the error but did not print the error message to the screen.
Could someone explain to me what I am missing here?
Your latter example works for me, but your main problem lies in that you're not wrapping a function with catch_assertionerror in
#catch_assertionerror
#name.setter
def name(self, value):
assert isinstance(value, str), "This value should be a string"
self._name = name
but a descriptor. To make matters worse, you return a function instead, not a new descriptor wrapping the original. Now when you assign to name attribute you just replace your wrapper function with the assigned value.
Step by step, using your original class definition:
class X(object):
#property
def name(self):
return self._name
#catch_assertionerror
#name.setter
def name(self, value):
assert isinstance(value, str), "This value should be a string"
self._name = value
>>> x = X()
>>> x.name
<unbound method X.handle_problems>
>>> x.__dict__
{}
>>> x.name = 2
>>> x.name
2
>>> x.__dict__
{'name': 2}
What you must do is wrap the method function instead and then pass it to the descriptor handling decorator:
class X(object):
#property
def name(self):
return self._name
#name.setter
#catch_assertionerror
def name(self, value):
assert isinstance(value, str), "This value should be a string"
self._name = value
and so:
>>> x = X()
>>> x.name
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in name
AttributeError: 'X' object has no attribute '_name'
>>> x.name = 2
error caught
>>> x.name = "asdf"
>>> x.name
'asdf'
In the future consider using functools.wraps and functools.update_wrapper. Without them your classes and functions are harder to inspect, because your wrappers will hide the original:
>>> #catch_assertionerror
... def this_name_should_show(): pass
...
>>> this_name_should_show
<function handle_problems at 0x7fd3d69e22a8>
Defining your decorator this way:
def catch_assertionerror(function):
#wraps(function)
def handle_problems(*args, **kwargs):
...
return handle_problems
will preserve the original function's information:
>>> #catch_assertionerror
... def this_name_should_show(): pass
...
>>> this_name_should_show
<function this_name_should_show at 0x7fd3d69e21b8>
It would have also indicated to you in your case that there's a problem:
# When trying to define the class
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 6, in X
File "<stdin>", line 2, in catch_assertionerror
File "/usr/lib/python2.7/functools.py", line 33, in update_wrapper
setattr(wrapper, attr, getattr(wrapped, attr))
AttributeError: 'property' object has no attribute '__module__'
When one defines a descriptor value retrieval etc. is overriden, making the instance of the descriptor effectively unaccessible.
I.e. one can't write instance_with_descriptor_attr.descriptor_attr.some_method_on_descriptor()... won't work. My question is basically how one can still access the descriptor's instance anway...
As noted by eryksun, Martijn's solution works for properties but not all descriptors:
class Desc(object):
def __init__(self, val=None):
self.val = val
def __get__(self, obj, cls):
return self.val
def __set__(self, obj, val):
self.val = val
class Test(object):
x = Desc(5)
>>> o = Test()
>>> print o.x
5
>>> print Test.x
5
The reason it works for property descriptors can be seen in the example property descriptor implementation in the docs:
http://docs.python.org/2/howto/descriptor.html#properties
the key is the __get__ function:
def __get__(self, obj, objtype=None):
if obj is None:
return self
if self.fget is None:
raise AttributeError, "unreadable attribute"
return self.fget(obj)
If obj is None it returns self, which is the instance of the descriptor itself. obj is the instance of the class accessing the descriptor. When you access the attribute from a class instance, obj is that instance, when you access it from a class object, then obj is None.
Changing the previous descriptor to:
class Desc(object):
def __init__(self, val=None):
self.val = val
def __get__(self, obj, cls):
if obj is None:
return self
return self.val
def __set__(self, obj, val):
self.val = val
class Test(object):
x = Desc(5)
yields (class must be re-defined if you're using a python shell)
o = Test()
>>> print o.x
5
>>> print Test.x
<__main__.Desc object at 0x23205d0>
You need to go up to the class itself:
type(instance_with_descriptor_attr).descriptor_attr
Demonstration:
>>> class Foo():
... #property
... def bar(self): return 'bar'
...
>>> foo = Foo()
>>> foo.bar
'bar'
>>> type(foo).bar
<property object at 0x109f24310>
if the __get__ method of the descriptor dont have a "return self" statement then the descriptor can only be accessed by the __dict__ attribute of the class:
class descriptor:
def __get__(self, instance, owner=None):
return 1
class A:
d = descriptor()
a = A()
a.d # --> 1
A.d # --> 1
A.__dict__['d'] # --> <__main__.descriptor object at ...>