We Keep Coding

Using setattr() and getattr() in Python descriptors

While trying to create descriptors a few different ways I noticed some strange behavior that I'm trying to understand. Below are the three different ways I have gone about creating descriptors:
>>> class NumericValueOne():
... def __init__(self, name):
... self.name = name
... def __get__(self, obj, type=None) -> object:
... return obj.__dict__.get(self.name) or 0
... def __set__(self, obj, value) -> None:
... obj.__dict__[self.name] = value
>>> class NumericValueTwo():
... def __init__(self, name):
... self.name = name
... self.internal_name = '_' + self.name
... def __get__(self, obj, type=None) -> object:
... return getattr(obj, self.internal_name, 0)
... def __set__(self, obj, value) -> None:
... setattr(obj, self.internal_name, value)
>>> class NumericValueThree():
... def __init__(self, name):
... self.name = name
... def __get__(self, obj, type=None) -> object:
... return getattr(obj, self.name, 0)
... def __set__(self, obj, value) -> None:
... setattr(obj, self.name, value)
I then use them in the Foo classes, like below:
>>> class FooOne():
... number = NumericValueOne("number")
>>> class FooTwo():
... number = NumericValueTwo("number")
>>> class FooThree():
... number = NumericValueThree("number")
my_foo_object_one = FooOne()
my_foo_object_two = FooTwo()
my_foo_object_three = FooThree()
my_foo_object_one.number = 3
my_foo_object_two.number = 3
my_foo_object_three.number = 3
While FooOne and FooTwo work as expected when both setting & getting values. FooThree throws the following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 7, in __set__
File "<stdin>", line 7, in __set__
File "<stdin>", line 7, in __set__
[Previous line repeated 497 more times]
RecursionError: maximum recursion depth exceeded while calling a Python object
It looks like setattr() is calling the __set__() method? But why should it be doing that if setattr() is modifying the obj __dict__? And why does this work if we use internal_name?
Why is that we NEED to use a private variable in order to use the built-in getattr() and setattr() methods correctly? Also, how is this different from just directly modifying the obj __dict__ like in NumericValueOne?

But why should it be doing that if setattr() is modifying the obj __dict__?
setattr doesn't just modify the __dict__. It sets attributes, exactly like x.y = z would, and for the attribute you're trying to set, "set this attribute" means "call the setter you're already in". Hence, infinite recursion.
And why does this work if we use internal_name?
That name doesn't correspond to a property, so it just gets a __dict__ entry.

Using python class variable vs instance variable

I am new to python. I'm trying to create a configuration class with required validators. In below code snippet, accessing variable 'a' using python class and instance of class is returning a different value. Whether this is a proper design or should i initialise var 'a' only in the class constructor and do validation in the setter method.
class IntField:
def __get__(self, instance, owner):
if instance is None:
return self
return instance.__dict__[self.name]
def __set__(self, instance, value):
if not isinstance(value, int):
raise ValueError('expecting integer')
instance.__dict__[self.name] = value
def __set_name__(self, owner, name):
self.name = name
class DefaultConfig:
a = IntField()
def __init__(self):
self.a = 2
print(DefaultConfig.a)
print(DefaultConfig().a)
output:
<__main__.IntField object at 0x10c34b550>
2

I just want to know whether it's right way to do it
Rather asking for opinion only answer, by I will try to be as objective as I can.
Your code behaves as expected as long as instances attributes are processed:
>>> c = DefaultConfig()
>>> c.a = 'foo'
Traceback (most recent call last):
File "<pyshell#88>", line 1, in <module>
c.a = 'foo'
File "<pyshell#83>", line 10, in __set__
raise ValueError('expecting integer')
ValueError: expecting integer
>>> c.a = 4
>>> c.a
4
When inspecting DefaultConfig.a, the __get__ function is still used with instance=None. So you can choose one of 2 possible ways:
be transparent and show what the attribute actually is (what you currently do)
insist on the descriptor magic and return the default value (here 2).
For that latter way, code could become:
class IntField:
def __get__(self, instance, owner):
if instance is None:
return getattr(owner, '_default_' + self.name, self)
return instance.__dict__[self.name]
def __set__(self, instance, value):
if not isinstance(value, int):
raise ValueError('expecting integer')
instance.__dict__[self.name] = value
def __set_name__(self, owner, name):
self.name = name
class DefaultConfig:
a = IntField()
_default_a = 2
def __init__(self):
self.a = self._default_a
The trick here is that by convention, the default value for an attribute x is expected to be _default_x.
In that case, you will get:
print(DefaultConfig.a)
2

Specialized #property decorators in python

I have a few classes each of which has a number of attributes. What all of the attributes have in common is that they should be numeric properties. This seems to be an ideal place to use python's decorators, but I can't seem to wrap my mind around what the correct implementation would be. Here is a simple example:
class Junk(object):
def __init__(self, var):
self._var = var
#property
def var(self):
"""A numeric variable"""
return self._var
#var.setter
def size(self, value):
# need to make sure var is an integer
if not isinstance(value, int):
raise ValueError("var must be an integer, var = {}".format(value))
self._var = value
#var.deleter
def size(self):
raise RuntimeError("You can't delete var")
It seems to me that it should be possible to write a decorator that does everything so that the above can be transformed into:
def numeric_property(*args, **kwargs):
...
class Junk(object):
def __init__(self, var):
self._var = var
#numeric_property
def var(self):
"""A numeric variable"""
return self._var
That way the new numeric_property decorator can be used in many classes.

A #property is just a special case of Python's descriptor protocol, so you can certainly build your own custom versions. For your case:
class NumericProperty:
"""A property that must be numeric.
Args:
attr (str): The name of the backing attribute.
"""
def __init__(self, attr):
self.attr = attr
def __get__(self, obj, type=None):
return getattr(obj, self.attr)
def __set__(self, obj, value):
if not isinstance(value, int):
raise ValueError("{} must be an integer, var = {!r}".format(self.attr, value))
setattr(obj, self.attr, value)
def __delete__(self, obj):
raise RuntimeError("You can't delete {}".format(self.attr))
class Junk:
var = NumericProperty('_var')
def __init__(self, var):
self.var = var
In use:
>>> j = Junk('hi')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/jonrsharpe/test.py", line 29, in __init__
self.var = var
File "/Users/jonrsharpe/test.py", line 17, in __set__
raise ValueError("{} must be an integer, var = {!r}".format(self.attr, value))
ValueError: _var must be an integer, var = 'hi'
>>> j = Junk(1)
>>> del j.var
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/jonrsharpe/test.py", line 21, in __delete__
raise RuntimeError("You can't delete {}".format(self.attr))
RuntimeError: You can't delete _var
>>> j.var = 'hello'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/jonrsharpe/test.py", line 17, in __set__
raise ValueError("{} must be an integer, var = {!r}".format(self.attr, value))
ValueError: _var must be an integer, var = 'hello'
>>> j.var = 2
>>> j.var
2

Option 1: inherit from property
property is a descriptor. See Descriptor HowTo on python.org.
So, can inherit from property and override the relevant methods.
For example, to enforce int on setter:
class numeric_property(property):
def __set__(self, obj, value):
assert isinstance(value, int), "numeric_property requires an int"
super(numeric_property, self).__set__(obj, value)
class A(object):
#numeric_property
def x(self):
return self._x
#x.setter
def x(self, value):
self._x = value
And now you have integers enforced:
>>> a = A()
>>> a.x = 'aaa'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in __set__
AssertionError: numeric_property requires an int
Option 2: Create a better descriptor
On the other hand, it may be even better to implement a brand new descriptor which does not inherit from property, which would enable you to define the property in one go.
It would be nicer to have this kind of interface:
class A(object):
x = numeric_property('_x')
For that you would implement a descriptor which takes the attribute name:
class numeric_property(object):
def __init__(self, private_attribute_name, default=0):
self.private_attribute_name = private_attribute_name
self.default = default
def __get__(self, obj, typ):
if not obj: return self
return getattr(obj, self.private_attribute_name, self.default)
def __set__(self, obj, value):
assert isinstance(value, int), "numeric_property requires an int"
setattr(obj, self.private_attribute_name, value)
Disclaimer :)
I would rather not enforce strict typing in Pyhon, because Python is much more powerful without it.

You may just create a function that does it for you . As simple as it can get, no need to create a custom descriptor:
def numprop(name, privname):
#property
def _numprop(self):
return getattr(self, privname)
#_numprop.setter
def _numprop(self, value):
if not isinstance(value, int):
raise ValueError("{name} must be an integer, {name} = {}".format(value, name=name))
setattr(self, privname, value)
#_numprop.deleter
def _numprop(self):
raise RuntimeError("You can't delete var")
return _numprop
class Junk(object):
def __init__(self, var):
self._var = var
var = numprop("var", "_var")

Handle exceptions with decorator on properties in Python

I have a property that has an assertion to check if a value is of type str.
To catch this assertionError I have created a decorator according to the examples I have found online.
Decorator:
def catch_assertionerror(function):
def handle_problems(*args, **kwargs):
try:
return function(*args, **kwargs)
except AssertionError:
# log_error(err.args[0])
print "error caught"
return handle_problems
Property:
#catch_assertionerror
#name.setter
def name(self, value):
assert isinstance(value, str), "This value should be a string"
self._name = name
Setting the name property:
self.name = self.parse_name_from_xml()
When I run this code, there is no error shown, so I guess it is caught, but on the other hand, the error message is not printed to the screen.
Then I tried a more simple example I found on Stachoverflow:
def handleError(function):
def handleProblems():
try:
function()
except Exception:
print "Oh noes"
return handleProblems
#handleError
def example():
raise Exception("Boom!")
This also handled the error but did not print the error message to the screen.
Could someone explain to me what I am missing here?

Your latter example works for me, but your main problem lies in that you're not wrapping a function with catch_assertionerror in
#catch_assertionerror
#name.setter
def name(self, value):
assert isinstance(value, str), "This value should be a string"
self._name = name
but a descriptor. To make matters worse, you return a function instead, not a new descriptor wrapping the original. Now when you assign to name attribute you just replace your wrapper function with the assigned value.
Step by step, using your original class definition:
class X(object):
#property
def name(self):
return self._name
#catch_assertionerror
#name.setter
def name(self, value):
assert isinstance(value, str), "This value should be a string"
self._name = value
>>> x = X()
>>> x.name
<unbound method X.handle_problems>
>>> x.__dict__
{}
>>> x.name = 2
>>> x.name
2
>>> x.__dict__
{'name': 2}
What you must do is wrap the method function instead and then pass it to the descriptor handling decorator:
class X(object):
#property
def name(self):
return self._name
#name.setter
#catch_assertionerror
def name(self, value):
assert isinstance(value, str), "This value should be a string"
self._name = value
and so:
>>> x = X()
>>> x.name
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in name
AttributeError: 'X' object has no attribute '_name'
>>> x.name = 2
error caught
>>> x.name = "asdf"
>>> x.name
'asdf'
In the future consider using functools.wraps and functools.update_wrapper. Without them your classes and functions are harder to inspect, because your wrappers will hide the original:
>>> #catch_assertionerror
... def this_name_should_show(): pass
...
>>> this_name_should_show
<function handle_problems at 0x7fd3d69e22a8>
Defining your decorator this way:
def catch_assertionerror(function):
#wraps(function)
def handle_problems(*args, **kwargs):
...
return handle_problems
will preserve the original function's information:
>>> #catch_assertionerror
... def this_name_should_show(): pass
...
>>> this_name_should_show
<function this_name_should_show at 0x7fd3d69e21b8>
It would have also indicated to you in your case that there's a problem:
# When trying to define the class
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 6, in X
File "<stdin>", line 2, in catch_assertionerror
File "/usr/lib/python2.7/functools.py", line 33, in update_wrapper
setattr(wrapper, attr, getattr(wrapped, attr))
AttributeError: 'property' object has no attribute '__module__'

How can I combine abc.abstractproperty with a classmethod to make an "abstract class property"?

I'd like to create a "class property" that is declared in an abstract base class, and then overridden in a concrete implementation class, while keeping the lovely assertion that the implementation must override the abstract base class' class property.
Although I took a look at this question my naive attempt to re-purpose the accepted answer didn't work:
>>> import abc
>>> class ClassProperty(abc.abstractproperty):
... def __get__(self, cls, owner):
... return self.fget.__get__(None, owner)()
...
>>> class Base(object):
... __metaclass__ = abc.ABCMeta
... #ClassProperty
... def foo(cls):
... raise NotImplementedError
...
>>> class Impl(Base):
... #ClassProperty
... def foo(cls):
... return 5
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/2rs2ts/src/myproj/env/lib/python2.7/abc.py", line 94, in __new__
value = getattr(cls, name, None)
File "<stdin>", line 3, in __get__
TypeError: Error when calling the metaclass bases
unbound method foo() must be called with Impl instance as first argument (got nothing instead)
I'm a little lost on what I should be doing. Any help?

You need to use this in addition to the #classmethod decorator.
class Impl(Base):
#ClassProperty
#classmethod
def foo(cls):
return 5
In [11]: Impl.foo
Out[11]: 5