We Keep Coding

Adding properties dynamically to a Python class that point to items in a dictionary

I am trying to attach properties dynamically to a class (Registry) for the sake of easy access to values in a dict. I am using defaultdict to define the dictionary, with the default value as an empty list.
But because of the way I am accessing the list values in the dictionary while defining the property, I end up with all properties pointing to the same list object.
Gist: https://gist.github.com/subhashb/adb75a3a05a611c3d9193da695d46dd4
from collections import defaultdict
from enum import Enum
class ElementTypes(Enum):
PHONE = "PHONE"
CAR = "CAR"
class Registry:
def __new__(cls, *args, **kwargs):
cls.setup_properties()
instance = super(Registry, cls).__new__(cls, *args, **kwargs)
return instance
def __init__(self):
self._elements = {}
def register(self, element_type, item):
if element_type.value not in self._elements:
self._elements[element_type.value] = []
self._elements[element_type.value].append(item)
def get(self, element_type):
return self._elements[element_type.value]
#classmethod
def setup_properties(cls):
for item in ElementTypes:
prop_name = item.value
prop = property(lambda self: getattr(self, "get")(item))
setattr(Registry, prop_name.lower(), prop)
registry = Registry()
registry.register(ElementTypes.PHONE, "phone1")
registry.register(ElementTypes.PHONE, "phone2")
registry.register(ElementTypes.CAR, "car1")
registry.register(ElementTypes.CAR, "car2")
assert dict(registry._elements) == {
"CAR": ["car1", "car2"],
"PHONE": ["phone1", "phone2"],
}
assert hasattr(registry, "phone")
assert hasattr(registry, "car")
assert registry.car == ["car1", "car2"]
assert registry.phone == ["phone1", "phone2"] # This fails
How do I define the code withing the property to be truly dynamic and get access to the individual list values in the dict?

First, don't setup properties in __new__, that gets called for every Registry object created! Instead, just assign the properties outside the class definition.
Secondly, this trips a lot of people up, but if you use a lambda inside a for-loop and you want to use the item variable, you need to make sure that you add an argument called item with the default value of item, otherwise all the properties will refer to the last item of the loop.
class Registry:
def __init__(self):
self._elements = defaultdict(list)
def register(self, element_type, item):
self._elements[element_type.value].append(item)
def get(self, element_type):
return self._elements[element_type.value]
for item in ElementTypes:
prop_name = item.value
prop = property(lambda self, item=item: self.get(item))
setattr(Registry, prop_name.lower(), prop)

Some doubts about #property in python 3

In order not to extend myself too much I will give a basic and hypothetical example of what I am trying to do.
Suppose the following class:
class foo():
def __init__(self):
self.keywords = []
## this method returns the entire list
def get_keywords(self):
return self.keywords
def set_keywords(self, value):
self.keywords.append(value)
But I want to code this in a pythonic way using the #property decorator.
My (wrong) attempt to do this:
class foo:
def __init__(self):
self.key = []
#property
def key(self):
return self.__key
#key.setter
def key(self, value):
self.__key.append(value)
So, whats is wrong in my attempt ?
ps: English is not my native language and I hope my doubt is understandable.

In your original code, self.set_keywords only appends to an existing list; it does not let you initialize the value of keywords to an arbitrary list. This restriction is preserved in your property-based code, which means you cannot assign directly to self.key; you have to initialize the underlying list in __init__ directly.
class foo:
def __init__(self):
# self.key = [] is equivalent to `self.__key.append([])`, but
# self.__key doesn't exist yet. (And would be wrong even if it did.)
self.__key = []
#property
def key(self):
return self.__key
#key.setter
def key(self, value):
self.__key.append(value)
However, this means an assignment like self.key = 3 doesn't actually perform what most people would expect of an assignment. It doesn't overwrite the old value, it adds to it instead. Use the setter to provide a fixed list, but a different method to add to an existing one.
class foo:
def __init__(self):
self.__keys = []
#property
def keys(self):
return self.__keys
#keys.setter
def keys(self, values):
self.__keys = values
def add_key(self, value):
self.__key.append(value)
And finally, it's not necessarily more Pythonic to use a property if you don't actually do any sort of extra work or validation in the getter or setter. If all you are doing is wrapping access to an underlying value, just let the value be used directly.
class foo:
def __init__(self):
self.keys = []
self.keys = [1,2,3]
print(self.keys)
self.keys.append(4)
# etc
The nice thing about properties is that if you start by allowing direct access to keys, then nothing about how you use keys changes if you later decide to replace it with a property.

You can give this a try:
class Foo:
def __init__(self):
self._key = []
#property
def key(self):
return self._key
#key.setter
def key(self, value):
self._key = value
Here are my two cents:
Rename the class foo to Foo
You can't initialize self.key, as this is the property, so initialize the correct variable in the constructor (i.e. __init__)
Private vars are prefixed with one _ scope and not two (two __ are Python internals)
I suppose you rather want my_instance.key = ['spam', 'eggs'] to replace the foo._key value than extend it. Because this is kind of a "setter" and that would result in a weird behaviour, or at least another developer won't expect that behaviour from that setter/function
However, and that's important: As long as you're only doing this, you won't need properties. You can simply initialize self.keys in the constructor and froget about the property and setter function. Later on, when you want to change the behaviour, you can still add the property and setter. That's one reason why we've properties in Python, so that you won't have to refactor your whole code in case "a bit more logic" comes into place.
Btw. if you're really depending everything on those dict functions, you might also want to inherit your class from the dict class. Depends what you're up to.

Extract (not known beforehand) attributes from objects in a list

I have a class whose attributes are not known beforehand:
class Event():
def __init__(self, **kwargs):
for key, value in kwargs.items():
setattr(self, key, value)
and another one which is basically a list of objects Event:
class Collection(list):
def __init__(self):
self.members = []
def add(self,new):
try:
self.members.extend(new)
except TypeError:
self.members.append(new)
Let's say now that I define 3 objects Event:
a = Event(name="a",value=1)
b = Event(name="b",value=2)
c = Event(name="c",other=True)
And I create a Collection from them:
col = Collection()
col.add([a,b,c])
What I want is to be able to print out all the values of the objects in the list for a given attribute (if the attribute does not exist for an object, it should return None or any other pre-defined value). For example:
print col.name #should return ["a","b","c"]
print col.value #should return [1,2,None]
I have read the following answer: Extract list of attributes from list of objects in python
But that doesn't work here since the name of my attribute is not known by advance, and some might not even be defined. How should I define my class Collection(), or maybe even re-think everything to achieve my goal ?

This is a variation of "I want to create dynamic variable names". The solution here is the same: use a dictionary.
class Event(object):
def __init__(self, **kwargs):
self.attributes = dict(kwargs)
Your Collection class will need a custom __getattr__ method, so that it can look up values in its Event list instead.
class Collection(object):
# assume self.events is a list of Event objects
def __getattr__(self, name):
return [event.attributes.get(name) for event in self.events]
You could stick with your current implementation of Event, and have Collection look at event.__dict__ instead of event.attributes. I don't recall, though, if __dict__ might contain anything else besides the attributes you explicitly set. I'd err on the side of caution.

You can just override the __getattr__ method of the Collection class, which is called when an attribute is accessed. In order to access to unknown set of attributes you can use event.__dict__. So, a possible solution is like this:
def __getattr__(self, name):
return [m.__dict__.get(name) for m in self.members]

How to add multiple similar properties in Python

I'm building a simulator, which will model various types of entities. So I've got a base class, ModelObject, and will use subclasses for all the different entities. Each entity will have a set of properties that I want to keep track of, so I've also got a class called RecordedDetail, that keeps tracks of changes (basically builds a list of (time_step, value) pairs) and each ModelObject has a dict to store these in. So I've got, effectively,
class ModelObject(object):
def __init__(self):
self.details = {}
self.time_step = 0
def get_detail(self, d_name):
""" get the current value of the specified RecordedDetail"""
return self.details[d_name].current_value()
def set_detail(self, d_name, value):
""" set the current value of the specified RecordedDetail"""
self.details[d_name].set_value(value, self.time_step)
class Widget(ModelObject):
def __init__(self):
super().__init__(self)
self.details["level"] = RecordedDetail()
self.details["angle"] = RecordedDetail()
#property
def level(self):
return self.get_detail("level")
#level.setter
def level(self, value):
self.set_detail("level", value)
#property
def angle(self):
return self.get_detail("angle")
#angle.setter
def angle(self):
self.set_detail("angle", value)
This gets terribly repetitious, and I can't help thinking there must be a way of automating it using a descriptor, but I can't work out how. I end up with
class RecordedProperty(object):
def __init__(self, p_name):
self.p_name = p_name
def __get__(self, instance, owner):
if instance is None:
return self
return instance.get_detail(self.p_name)
def __set__(self, instance, value):
instance.set_detail(self.p_name, value)
class Widget(ModelObject):
level = RecordedProperty("level")
angle = RecordedProperty("angle")
def __init__(self):
super().__init__(self)
self.details["level"] = RecordedDetail()
self.details["angle"] = RecordedDetail()
which is a bit of an improvement, but still a lot of typing.
So, a few questions.
Can I just add the descriptor stuff (__get__, __set__ etc) into the RecordedDetail class? Would there be any advantage to doing that?
Is there any way of typing the new property name (such as "level") fewer than three times, in two different places?
or
Am I barking up the wrong tree entirely?

The last bit of code is on the right track. You can make the process less nasty by using a metaclass to create a named RecordedProperty and a matching RecordedDetail for every item in a list. Here's a simple example:
class WidgetMeta(type):
def __new__(cls, name, parents, kwargs):
'''
Automate the creation of the class
'''
for item in kwargs['_ATTRIBS']:
kwargs[item] = RecordedProperty(item)
return super(WidgetMeta, cls).__new__(cls, name, parents, kwargs)
class Widget(ModelObject):
_ATTRIBS = ['level', 'angle']
__metaclass__ = WidgetMeta
def __init__(self, *args, **kwargs):
super().__init__(self)
self.Details = {}
for detail in self._ATTRIBS:
self.Details[detail] = RecordedDetail()
Subclasses would then just need to have different data in _ATTRIBS.
As an alternative (I think it's more complex) you could use the metaclass to customize the init in the same way you customize the new, creating the RecordedDetails out of the _ATTRIBS list.
A third option would be to create a RecordedDetail in every instance on first access. That would work fine as long as you don't have code that expects a RecordedDetail for every property even if the RecordedDetail has not been touched.
Caveat I'm not super familiar with Python3; I've used the above pattern often in 2.7x

How to implement property() with dynamic name (in python)

I am programming a simulations for single neurons. Therefore I have to handle a lot of Parameters. Now the Idea is that I have two classes, one for a SingleParameter and a Collection of parameters. I use property() to access the parameter value easy and to make the code more readable. This works perfect for a sinlge parameter but I don't know how to implement it for the collection as I want to name the property in Collection after the SingleParameter. Here an example:
class SingleParameter(object):
def __init__(self, name, default_value=0, unit='not specified'):
self.name = name
self.default_value = default_value
self.unit = unit
self.set(default_value)
def get(self):
return self._v
def set(self, value):
self._v = value
v = property(fget=get, fset=set, doc='value of parameter')
par1 = SingleParameter(name='par1', default_value=10, unit='mV')
par2 = SingleParameter(name='par2', default_value=20, unit='mA')
# par1 and par2 I can access perfectly via 'p1.v = ...'
# or get its value with 'p1.v'
class Collection(object):
def __init__(self):
self.dict = {}
def __getitem__(self, name):
return self.dict[name] # get the whole object
# to get the value instead:
# return self.dict[name].v
def add(self, parameter):
self.dict[parameter.name] = parameter
# now comes the part that I don't know how to implement with property():
# It shoule be something like
# self.__dict__[parameter.name] = property(...) ?
col = Collection()
col.add(par1)
col.add(par2)
col['par1'] # gives the whole object
# Now here is what I would like to get:
# col.par1 -> should result like col['par1'].v
# col.par1 = 5 -> should result like col['par1'].v = 5
Other questions that I put to understand property():
Why do managed attributes just work for class attributes and not for instance attributes in python?
How can I assign a new class attribute via __dict__ in python?

Look at built-in functions getattr and setattr. You'll probably be a lot happier.

Using the same get/set functions for both classes forces you into an ugly hack with the argument list. Very sketchy, this is how I would do it:
In class SingleParameter, define get and set as usual:
def get(self):
return self._s
def set(self, value):
self._s = value
In class Collection, you cannot know the information until you create the property, so you define the metaset/metaget function and particularize them only later with a lambda function:
def metaget(self, par):
return par.s
def metaset(self, value, par):
par.s = value
def add(self, par):
self[par.name] = par
setattr(Collection, par.name,
property(
fget=lambda x : Collection.metaget(x, par),
fset=lambda x, y : Collection.metaset(x,y, par))

Properties are meant to dynamically evaluate attributes or to make them read-only. What you need is customizing attribute access. __getattr__ and __setattr__ do that really fine, and there's also __getattribute__ if __getattr__ is not enough.
See Python docs on customizing attribute access for details.

Have you looked at the traits package? It seems that you are reinventing the wheel here with your parameter classes. Traits also have additional features that might be useful for your type of application (incidently I know a person that happily uses traits in neural simulations).

Now I implemented a solution with set-/getattr:
class Collection(object):
...
def __setattr__(self, name, value):
if 'dict' in self.__dict__:
if name in self.dict:
self[name].v = value
else:
self.__dict__[name] = value
def __getattr__(self, name):
return self[name].v
There is one thing I quite don't like that much: The attributes are not in the __dict__. And if I have them there as well I would have a copy of the value - which can be dangerous...

Finally I succeded to implement the classes with property(). Thanks a lot for the advice. It took me quite a bit to work it out - but I can promise you that this exercise helps you to understand better pythons OOP.
I implemented it also with __getattr__ and __setattr__ but still don't know the advantages and disadvantages to the property-solution. But this seems to be worth another question. The property-solutions seems to be quit clean.
So here is the code:
class SingleParameter(object):
def __init__(self, name, default_value=0, unit='not specified'):
self.name = name
self.default_value = default_value
self.unit = unit
self.set(default_value)
def get(*args):
self = args[0]
print "get(): "
print args
return self._v
def set(*args):
print "set(): "
print args
self = args[0]
value = args[-1]
self._v = value
v = property(fget=get, fset=set, doc='value of parameter')
class Collection(dict):
# inheriting from dict saves the methods: __getitem__ and __init__
def add(self, par):
self[par.name] = par
# Now here comes the tricky part.
# (Note: this property call the get() and set() methods with one
# more argument than the property of SingleParameter)
setattr(Collection, par.name,
property(fget=par.get, fset=par.set))
# Applying the classes:
par1 = SingleParameter(name='par1', default_value=10, unit='mV')
par2 = SingleParameter(name='par2', default_value=20, unit='mA')
col = Collection()
col.add(par1)
col.add(par2)
# Setting parameter values:
par1.v = 13
col.par1 = 14
# Getting parameter values:
par1.v
col.par1
# checking identity:
par1.v is col.par1
# to access the whole object:
col['par1']
As I am new I am not sure how to move on:
how to treat follow up questions (like this itself):
get() is seems to be called twice - why?
oop-design: property vs. "__getattr__ & __setattr__" - when should I use what?
is it rude to check the own answer to the own question as accepted?
is it recommended to rename the title in order to put correlated questions or questions elaborated with the same example into the same context?
Other questions that I put to understand property():
Why do managed attributes just work for class attributes and not for instance attributes in python?
How can I assign a new class attribute via __dict__ in python?

I have a class that does something similar, but I did the following in the collection object:
setattr(self, par.name, par.v)