We Keep Coding

Extract (not known beforehand) attributes from objects in a list

I have a class whose attributes are not known beforehand:
class Event():
def __init__(self, **kwargs):
for key, value in kwargs.items():
setattr(self, key, value)
and another one which is basically a list of objects Event:
class Collection(list):
def __init__(self):
self.members = []
def add(self,new):
try:
self.members.extend(new)
except TypeError:
self.members.append(new)
Let's say now that I define 3 objects Event:
a = Event(name="a",value=1)
b = Event(name="b",value=2)
c = Event(name="c",other=True)
And I create a Collection from them:
col = Collection()
col.add([a,b,c])
What I want is to be able to print out all the values of the objects in the list for a given attribute (if the attribute does not exist for an object, it should return None or any other pre-defined value). For example:
print col.name #should return ["a","b","c"]
print col.value #should return [1,2,None]
I have read the following answer: Extract list of attributes from list of objects in python
But that doesn't work here since the name of my attribute is not known by advance, and some might not even be defined. How should I define my class Collection(), or maybe even re-think everything to achieve my goal ?

This is a variation of "I want to create dynamic variable names". The solution here is the same: use a dictionary.
class Event(object):
def __init__(self, **kwargs):
self.attributes = dict(kwargs)
Your Collection class will need a custom __getattr__ method, so that it can look up values in its Event list instead.
class Collection(object):
# assume self.events is a list of Event objects
def __getattr__(self, name):
return [event.attributes.get(name) for event in self.events]
You could stick with your current implementation of Event, and have Collection look at event.__dict__ instead of event.attributes. I don't recall, though, if __dict__ might contain anything else besides the attributes you explicitly set. I'd err on the side of caution.

You can just override the __getattr__ method of the Collection class, which is called when an attribute is accessed. In order to access to unknown set of attributes you can use event.__dict__. So, a possible solution is like this:
def __getattr__(self, name):
return [m.__dict__.get(name) for m in self.members]

Problems when converting a dictionary to object

I am using a technique discussed here before, to turn a dictionary into an object, so that I can access the elements of the dictionary with the dot (.) notion, as instance variables.
This is what I am doing:
# Initial dictionary
myData = {'apple':'1', 'banana':'2', 'house':'3', 'car':'4', 'hippopotamus':'5'}
# Create the container class
class Struct:
def __init__(self, **entries):
self.__dict__.update(entries)
# Finally create the instance and bind the dictionary to it
k = Struct(**myData)
So now, I can do:
print k.apple
and the result is:
1
This works, however the issues start if I try to add some other methods to the "Struct" class. For example lets say that I am adding a simple method that just creates an variable:
class Struct:
def __init__(self, **entries):
self.__dict__.update(entries)
def testMe(self):
self.myVariable = 67
If I do:
k.testMe()
My dictionary object is broken, "myVariable" is inserted as a key with the value "67". So If I do:
print k.__dict__
I am getting:
{'apple': '1', 'house': '3', 'myVariable': 67, 'car': '4', 'banana': '2', 'hippopotamus': '5'}
Is there a way to fix this? I kind of understand what is happening, but not sure If I need to entirely change my approach and build a class with internal methods to handle the dictionary object or is there a simpler way to fix this problem?
Here is the original link:
Convert Python dict to object?
Thanks.

For your needs, don't store you variables in __dict__. Use your own dictionary instead, and override .__getattr__ (for print k.apple) and __setattr__ (for k.apple=2):
# Initial dictionary
myData = {'apple':'1', 'banana':'2', 'house':'3', 'car':'4', 'hippopotamus':'5'}
# Create the container class
class Struct:
_dict = {}
def __init__(self, **entries):
self._dict = entries
def __getattr__(self, name):
try:
return self._dict[name]
except KeyError:
raise AttributeError(
"'{}' object has no attribute or key '{}'".format(
self.__class__.__name__, name))
def __setattr__(self, name, value):
if name in self._dict:
self._dict[name] = value
else:
self.__dict__[name] = value
def testMe(self):
self.myVariable = 67
def FormattedDump(self):
return str(self._dict)
# Finally create the instance and bind the dictionary to it
k = Struct(**myData)
print k.apple
print k.FormattedDump()
k.testMe()
k.apple = '2'
print k.FormattedDump()
In the alternative, if your FormattedDump() routine is bothering you, you could just fix it:
# Initial dictionary
myData = {'apple':'1', 'banana':'2', 'house':'3', 'car':'4', 'hippopotamus':'5'}
# Create the container class
class Struct:
def __init__(self, **entries):
self.__dict__.update(entries)
self.public_names = entries.keys()
def testMe(self):
self.myVariable = 67
def GetPublicDict(self):
return {key:getattr(self, key) for key in self.public_names}
def FormattedDump(self):
return str(self.GetPublicDict())
# Finally create the instance and bind the dictionary to it
k = Struct(**myData)
print k.apple
print k.FormattedDump()
k.testMe()
k.apple = '2'
print k.FormattedDump()

Python: renaming method names on-the-fly

I have many files using classes with the following syntax:
o = module.CreateObject()
a = o.get_Field
and now the implementation has changed from 'get_XXX' and 'set_XXX' to just 'XXX':
o = module.CreateObject()
a = o.Field
This implementation is an external package, which I don't want to change. Is it possible to write a wrapper which will on-the-fly intercept all calls to 'get_XXX' and replace then with calls to the new name 'XXX'?
o = MyRenamer(module.CreateObject())
a = o.get_Field # works as before, o.Field is called
a = o.DoIt() # works as before, o.DoIt is called
It needs to intercept all calls, not just to a finite-set of fields, decide based on the method name if to modify it and cause a method with a modified name to be called.

If you want to continue to use get_Field and set_Field on an object that has switched to using properties (where you simply access or assign to Field), it's possible to use an wrapper object:
class NoPropertyAdaptor(object):
def __init__(self, obj):
self.obj = obj
def __getattr__(self, name):
if name.startswith("get_"):
return lambda: getattr(self.obj, name[4:])
elif name.startswith("set_"):
return lambda value: setattr(self.obj, name[4:], value)
else:
return getattr(self.obj, name)
This will have problems if you are using extra syntax, like indexing or iteration on the object, or if you need to recognize the type of the object using isinstance.
A more sophisticated solution would be to create a subclass that does the name rewriting and force the object to use it. This isn't exactly a wrapping, since outside code will still deal with the object directly (and so magic methods and isinstance) will work as expected. This approach will work for most objects, but it might fail for types that have fancy metaclass magic going on and for some builtin types:
def no_property_adaptor(obj):
class wrapper(obj.__class__):
def __getattr__(self, name):
if name.startswith("get_"):
return lambda: getattr(self, name[4:])
elif name.startswith("set_"):
return lambda value: setattr(self, name[4:], value)
else:
return super(wrapper, self).__getattr__(name)
obj.__class__ = wrapper
return obj

You can 'monkey patch' any python class; import the class directly and add a property:
import original_module
#property
def get_Field(self):
return self.Field
original_module.OriginalClass.get_Field = get_Field
You'd need to enumerate what fields you wanted to access this way:
def addField(fieldname, class):
#property
def get_Field(self):
return getattr(self, fieldname)
setattr(original_module.OriginalClass, 'get_{}'.format(fieldname), get_Field)
for fieldname in ('Foo', 'Bar', 'Baz'):
addField(fieldname, original_module.OriginalClass)

Mapping obj.method({argument:value}) to obj.argument(value)

I don't know if this will make sense, but...
I'm trying to dynamically assign methods to an object.
#translate this
object.key(value)
#into this
object.method({key:value})
To be more specific in my example, I have an object (which I didn't write), lets call it motor, which has some generic methods set, status and a few others. Some take a dictionary as an argument and some take a list. To change the motor's speed, and see the result, I use:
motor.set({'move_at':10})
print motor.status('velocity')
The motor object, then formats this request into a JSON-RPC string, and sends it to an IO daemon. The python motor object doesn't care what the arguments are, it just handles JSON formatting and sockets. The strings move_at and velocity are just two of what might be hundreds of valid arguments.
What I'd like to do is the following instead:
motor.move_at(10)
print motor.velocity()
I'd like to do it in a generic way since I have so many different arguments I can pass. What I don't want to do is this:
# create a new function for every possible argument
def move_at(self,x)
return self.set({'move_at':x})
def velocity(self)
return self.status('velocity')
#and a hundred more...
I did some searching on this which suggested the solution lies with lambdas and meta programming, two subjects I haven't been able to get my head around.
UPDATE:
Based on the code from user470379 I've come up with the following...
# This is what I have now....
class Motor(object):
def set(self,a_dict):
print "Setting a value", a_dict
def status(self,a_list):
print "requesting the status of", a_list
return 10
# Now to extend it....
class MyMotor(Motor):
def __getattr__(self,name):
def special_fn(*value):
# What we return depends on how many arguments there are.
if len(value) == 0: return self.status((name))
if len(value) == 1: return self.set({name:value[0]})
return special_fn
def __setattr__(self,attr,value): # This is based on some other answers
self.set({attr:value})
x = MyMotor()
x.move_at = 20 # Uses __setattr__
x.move_at(10) # May remove this style from __getattr__ to simplify code.
print x.velocity()
output:
Setting a value {'move_at': 20}
Setting a value {'move_at': 10}
10
Thank you to everyone who helped!

What about creating your own __getattr__ for the class that returns a function created on the fly? IIRC, there's some tricky cases to watch out for between __getattr__ and __getattribute__ that I don't recall off the top of my head, I'm sure someone will post a comment to remind me:
def __getattr__(self, name):
def set_fn(self, value):
return self.set({name:value})
return set_fn
Then what should happen is that calling an attribute that doesn't exist (ie: move_at) will call the __getattr__ function and create a new function that will be returned (set_fn above). The name variable of that function will be bound to the name parameter passed into __getattr__ ("move_at" in this case). Then that new function will be called with the arguments you passed (10 in this case).
Edit
A more concise version using lambdas (untested):
def __getattr__(self, name):
return lambda value: self.set({name:value})

There are a lot of different potential answers to this, but many of them will probably involve subclassing the object and/or writing or overriding the __getattr__ function.
Essentially, the __getattr__ function is called whenever python can't find an attribute in the usual way.
Assuming you can subclass your object, here's a simple example of what you might do (it's a bit clumsy but it's a start):
class foo(object):
def __init__(self):
print "initting " + repr(self)
self.a = 5
def meth(self):
print self.a
class newfoo(foo):
def __init__(self):
super(newfoo, self).__init__()
def meth2(): # Or, use a lambda: ...
print "meth2: " + str(self.a) # but you don't have to
self.methdict = { "meth2":meth2 }
def __getattr__(self, name):
return self.methdict[name]
f = foo()
g = newfoo()
f.meth()
g.meth()
g.meth2()
Output:
initting <__main__.foo object at 0xb7701e4c>
initting <__main__.newfoo object at 0xb7701e8c>
5
5
meth2: 5

You seem to have certain "properties" of your object that can be set by
obj.set({"name": value})
and queried by
obj.status("name")
A common way to go in Python is to map this behaviour to what looks like simple attribute access. So we write
obj.name = value
to set the property, and we simply use
obj.name
to query it. This can easily be implemented using the __getattr__() and __setattr__() special methods:
class MyMotor(Motor):
def __init__(self, *args, **kw):
self._init_flag = True
Motor.__init__(self, *args, **kw)
self._init_flag = False
def __getattr__(self, name):
return self.status(name)
def __setattr__(self, name, value):
if self._init_flag or hasattr(self, name):
return Motor.__setattr__(self, name, value)
return self.set({name: value})
Note that this code disallows the dynamic creation of new "real" attributes of Motor instances after the initialisation. If this is needed, corresponding exceptions could be added to the __setattr__() implementation.

Instead of setting with function-call syntax, consider using assignment (with =). Similarly, just use attribute syntax to get a value, instead of function-call syntax. Then you can use __getattr__ and __setattr__:
class OtherType(object): # this is the one you didn't write
# dummy implementations for the example:
def set(self, D):
print "setting", D
def status(self, key):
return "<value of %s>" % key
class Blah(object):
def __init__(self, parent):
object.__setattr__(self, "_parent", parent)
def __getattr__(self, attr):
return self._parent.status(attr)
def __setattr__(self, attr, value):
self._parent.set({attr: value})
obj = Blah(OtherType())
obj.velocity = 42 # prints setting {'velocity': 42}
print obj.velocity # prints <value of velocity>

How to implement property() with dynamic name (in python)

I am programming a simulations for single neurons. Therefore I have to handle a lot of Parameters. Now the Idea is that I have two classes, one for a SingleParameter and a Collection of parameters. I use property() to access the parameter value easy and to make the code more readable. This works perfect for a sinlge parameter but I don't know how to implement it for the collection as I want to name the property in Collection after the SingleParameter. Here an example:
class SingleParameter(object):
def __init__(self, name, default_value=0, unit='not specified'):
self.name = name
self.default_value = default_value
self.unit = unit
self.set(default_value)
def get(self):
return self._v
def set(self, value):
self._v = value
v = property(fget=get, fset=set, doc='value of parameter')
par1 = SingleParameter(name='par1', default_value=10, unit='mV')
par2 = SingleParameter(name='par2', default_value=20, unit='mA')
# par1 and par2 I can access perfectly via 'p1.v = ...'
# or get its value with 'p1.v'
class Collection(object):
def __init__(self):
self.dict = {}
def __getitem__(self, name):
return self.dict[name] # get the whole object
# to get the value instead:
# return self.dict[name].v
def add(self, parameter):
self.dict[parameter.name] = parameter
# now comes the part that I don't know how to implement with property():
# It shoule be something like
# self.__dict__[parameter.name] = property(...) ?
col = Collection()
col.add(par1)
col.add(par2)
col['par1'] # gives the whole object
# Now here is what I would like to get:
# col.par1 -> should result like col['par1'].v
# col.par1 = 5 -> should result like col['par1'].v = 5
Other questions that I put to understand property():
Why do managed attributes just work for class attributes and not for instance attributes in python?
How can I assign a new class attribute via __dict__ in python?

Look at built-in functions getattr and setattr. You'll probably be a lot happier.

Using the same get/set functions for both classes forces you into an ugly hack with the argument list. Very sketchy, this is how I would do it:
In class SingleParameter, define get and set as usual:
def get(self):
return self._s
def set(self, value):
self._s = value
In class Collection, you cannot know the information until you create the property, so you define the metaset/metaget function and particularize them only later with a lambda function:
def metaget(self, par):
return par.s
def metaset(self, value, par):
par.s = value
def add(self, par):
self[par.name] = par
setattr(Collection, par.name,
property(
fget=lambda x : Collection.metaget(x, par),
fset=lambda x, y : Collection.metaset(x,y, par))

Properties are meant to dynamically evaluate attributes or to make them read-only. What you need is customizing attribute access. __getattr__ and __setattr__ do that really fine, and there's also __getattribute__ if __getattr__ is not enough.
See Python docs on customizing attribute access for details.

Have you looked at the traits package? It seems that you are reinventing the wheel here with your parameter classes. Traits also have additional features that might be useful for your type of application (incidently I know a person that happily uses traits in neural simulations).

Now I implemented a solution with set-/getattr:
class Collection(object):
...
def __setattr__(self, name, value):
if 'dict' in self.__dict__:
if name in self.dict:
self[name].v = value
else:
self.__dict__[name] = value
def __getattr__(self, name):
return self[name].v
There is one thing I quite don't like that much: The attributes are not in the __dict__. And if I have them there as well I would have a copy of the value - which can be dangerous...

Finally I succeded to implement the classes with property(). Thanks a lot for the advice. It took me quite a bit to work it out - but I can promise you that this exercise helps you to understand better pythons OOP.
I implemented it also with __getattr__ and __setattr__ but still don't know the advantages and disadvantages to the property-solution. But this seems to be worth another question. The property-solutions seems to be quit clean.
So here is the code:
class SingleParameter(object):
def __init__(self, name, default_value=0, unit='not specified'):
self.name = name
self.default_value = default_value
self.unit = unit
self.set(default_value)
def get(*args):
self = args[0]
print "get(): "
print args
return self._v
def set(*args):
print "set(): "
print args
self = args[0]
value = args[-1]
self._v = value
v = property(fget=get, fset=set, doc='value of parameter')
class Collection(dict):
# inheriting from dict saves the methods: __getitem__ and __init__
def add(self, par):
self[par.name] = par
# Now here comes the tricky part.
# (Note: this property call the get() and set() methods with one
# more argument than the property of SingleParameter)
setattr(Collection, par.name,
property(fget=par.get, fset=par.set))
# Applying the classes:
par1 = SingleParameter(name='par1', default_value=10, unit='mV')
par2 = SingleParameter(name='par2', default_value=20, unit='mA')
col = Collection()
col.add(par1)
col.add(par2)
# Setting parameter values:
par1.v = 13
col.par1 = 14
# Getting parameter values:
par1.v
col.par1
# checking identity:
par1.v is col.par1
# to access the whole object:
col['par1']
As I am new I am not sure how to move on:
how to treat follow up questions (like this itself):
get() is seems to be called twice - why?
oop-design: property vs. "__getattr__ & __setattr__" - when should I use what?
is it rude to check the own answer to the own question as accepted?
is it recommended to rename the title in order to put correlated questions or questions elaborated with the same example into the same context?
Other questions that I put to understand property():
Why do managed attributes just work for class attributes and not for instance attributes in python?
How can I assign a new class attribute via __dict__ in python?

I have a class that does something similar, but I did the following in the collection object:
setattr(self, par.name, par.v)