I am using Python descriptors to create complex interfaces on host objects.
I don't get the behaviour I would intuitively expect when I run code such as this:
class Accessor(object):
def __get__(self,inst,instype):
self._owner = inst
return self
def set(self,value):
self._owner._val = value
def get(self):
if hasattr(self._owner,'_val'):
return self._owner._val
else: return None
class TestClass(object):
acc = Accessor()
source = TestClass()
destination = TestClass()
source.acc.set('banana')
destination.acc.set('mango')
destination.acc.set(source.acc.get())
print destination.acc.get()
# Result: mango
I would expect in this case for destination.acc.get() to return 'banana', not 'mango'.
However, the intention (to copy _val from 'source' to 'destination') works if the code is refactored like this:
val = source.acc.get()
destination.acc.set(val)
print destination.acc.get()
# Result: banana
What is is that breaks down the 'client' reference passed through get if descriptors are used in a single line versus broken into separate lines? Is there a way to get the behaviour I would intuitively expect?
Many thanks in advance.
K
Your implementation ALMOST works. The problem with it comes up with destination.acc.set(source.acc.get()). What happens is that it first looks up destination.acc, which will set _owner to destination, but before it can call set(), it has to resolve the parameter, source.acc.get(), which will end up setting acc's _owner to source.
Since destination.acc and source.acc are the same object (descriptors are stored on the class, not the instance), you're calling set() on it after its _owner is set to source. That means you're setting source._val, not destination._val.
The way to get the behavior you would intuitively expect is to get rid or your get() and set() and replace them with __get__() and __set__(), so that your descriptor can be used for the reason a descriptor is used.
class Accessor(object):
def __get__(self, instance, owner): # you should use the conventional parameter names
if instance is None:
return self
else:
return instance._val
def __set__(self, instance, value):
instance._val = value
Then you could rewrite your client code as
source = TestClass()
destination = TestClass()
source.acc = 'banana'
destination.acc = 'mango'
destination.acc = source.acc
print destination.acc
The point of descriptors is to remove explicit getter and setter calls with implicit ones that look like simple attribute use. If you still want to be using your getters and setters on Accessor, then don't make it a descriptor. Do this instead:
class Accessor(object):
def get(self):
if hasattr(self, '_val'):
return self._val
else:
return None
def set(self, val):
self._val = val
Then rewrite TestClass to look more like this:
class TestClass(object):
def __init__(self):
self.acc = Accessor()
After that, your original client code would work.
I already said why it's not working in my other post. So, here's a way to use a descriptor while still retaining your get() and set() methods.
class Accessor(object):
def __get__(self, instance, owner):
if instance is None:
return self
elif not hasattr(instance, '_val'):
setattr(instance, '_val', Acc())
return getattr(instance, '_val')
class Acc(object):
def get(self):
if hasattr(self, '_val'):
return self._val
else:
return None
def set(self, val):
self._val = val
class TestClass(object):
acc = Accessor()
The trick is to move the get() and set() methods to a new class that is returned instead of returning self from the descriptor.
Related
I understand what I am asking here is probably not the best code design, but the reason for me asking is strictly academic. I am trying to understand how to make this concept work.
Typically, I will return self from a class method so that the following methods can be chained together. My understanding is by returning self, I am simply returning an instance of the class, for the following methods to work on.
But in this case, I am trying to figure out how to return both self and another value from the method. The idea is if I do not want to chain, or I do not call any class attributes, I want to retrieve the data from the method being called.
Consider this example:
class Test(object):
def __init__(self):
self.hold = None
def methoda(self):
self.hold = 'lol'
return self, 'lol'
def newmethod(self):
self.hold = self.hold * 2
return self, 2
t = Test()
t.methoda().newmethod()
print(t.hold)
In this case, I will get an AttributeError: 'tuple' object has no attribute 'newmethod' which is to be expected because the methoda method is returning a tuple which does not have any methods or attributes called newmethod.
My question is not about unpacking multiple returns, but more about how can I continue to chain methods when the preceding methods are returning multiple values. I also understand that I can control the methods return with an argument to it, but that is not what I am trying to do.
As mentioned previously, I do realize this is probably a bad question, and I am happy to delete the post if the question doesnt make any sense.
Following the suggestion by #JohnColeman, you can return a special tuple with attribute lookup delegated to your object if it is not a normal tuple attribute. That way it acts like a normal tuple except when you are chaining methods.
You can implement this as follows:
class ChainResult(tuple):
def __new__(cls, *args):
return super(ChainResult, cls).__new__(cls, args)
def __getattribute__(self, name):
try:
return getattr(super(), name)
except AttributeError:
return getattr(super().__getitem__(0), name)
class Test(object):
def __init__(self):
self.hold = None
def methoda(self):
self.hold = 'lol'
return ChainResult(self, 'lol')
def newmethod(self):
self.hold = self.hold * 2
return ChainResult(self, 2)
Testing:
>>> t = Test()
>>> t.methoda().newmethod()
>>> print(t.hold)
lollol
The returned result does indeed act as a tuple:
>>> t, res = t.methoda().newmethod()
>>> print(res)
2
>>> print(isinstance(t.methoda().newmethod(), tuple))
True
You could imagine all sorts of semantics with this, such as forwarding the returned values to the next method in the chain using closure:
class ChainResult(tuple):
def __new__(cls, *args):
return super(ChainResult, cls).__new__(cls, args)
def __getattribute__(self, name):
try:
return getattr(super(), name)
except AttributeError:
attr = getattr(super().__getitem__(0), name)
if callable(attr):
chain_results = super().__getitem__(slice(1, None))
return lambda *args, **kw: attr(*(chain_results+args), **kw)
else:
return attr
For example,
class Test:
...
def methodb(self, *args):
print(*args)
would produce
>>> t = Test()
>>> t.methoda().methodb('catz')
lol catz
It would be nice if you could make ChainResults invisible. You can almost do it by initializing the tuple base class with the normal results and saving your object in a separate attribute used only for chaining. Then use a class decorator that wraps every method with ChainResults(self, self.method(*args, **kw)). It will work okay for methods that return a tuple but a single value return will act like a length 1 tuple, so you will need something like obj.method()[0] or result, = obj.method() to work with it. I played a bit with delegating to tuple for a multiple return or to the value itself for a single return; maybe it could be made to work but it introduces so many ambiguities that I doubt it could work well.
I want to call a function when an object attribute is set:
class MyClass():
myattrib = None
def __setattr__(self, prop, val):
self.myattrib = val
print("setting myattrib")
x = MyClass()
x.myattrib = "something"
The problem is, that this creates an infinite recursion.
The idea is to call a function when a member variable is set (and actually set that member variable), but at the same time run extra code (showcased with the print() statement in the example above).
The other obvious way to do this, is using set_attrib() functions, as is common in Java, but I'd like to do it the pythonic way, and set the attributes directly, but at the same time running extra code.
Call the base version via super():
class MyClass(object):
myattrib = None
def __setattr__(self, prop, val):
super().__setattr__('myattrib', val)
print("setting myattrib")
You probably do not want to ignore the prop argument here, it is not necessarily 'myattrib' that is being set.
However, consider using a property instead of intercepting all attribute setting:
class MyClass(object):
_myattrib = None
#property:
def myattrib(self):
return self._myattrib
#myattrib.setter
def myattrib(self, val):
self._myattrib = val
print("setting myattrib")
I added object as a base-class; this is the default in Python 3, but a requirement for super() and property objects to work in Python 2.
If you want to do a specific thing when any attribute is set, use __setattr__ and call the inherited version using super for the actual assignment:
def __setattr__(self, prop, val):
super().__setattr__(prop, val)
print("setting {} to {!r}".format(prop, val)
If you only want to do something special for one attribute (not all attributes), you should probably use a property instead:
class MyClass():
#property
def some_attribute(self):
return self._val
#some_attribute.setter
def some_attribute(self, value):
self._val = value
print("set some_attribute to {!r}".format(value))
I would like to declare properties in the constructor of my class.
The class MaterialOne shows how I have it currently. Every property has to be defined separately. However, I will have groups of similar properties, which I would like to give the same fset/fget/fdel.
Since it requires a lot of code to write all the properties explicit, I would like to define the properties in a more concise way. I therefore thought of letting the constructor handle this. Class MateralTwo shows how I have that in mind.
Unfortunately it doesn't work, as I get TypeErrors:
TypeError: get_property() takes exactly 1 argument (2 given)
I can understand why it complains, but I can't think of any solution. I don't necessarily want to define the properties from a list in the constructor. What I am looking for is a more concise and clean method of defining them.
class MaterialOne(object):
def __init__(self):
pass;
def del_property(attr):
"""Abstract deller"""
def del_attr(self):
setattr(self, attr, None);
return del_attr
def set_property(attr):
"""Abstract setter."""
def set_attr(self, x):
setattr(self, attr, x);
return set_attr
def get_property(attr):
"""Abstract getter"""
def get_attr(self):
if getattr(self, attr) is not None:
return getattr(self, attr);
else:
return 'Some calculated value..'
return get_attr
_young = None;
_shear = None;
_poisson = None;
young = property(fget=get_property('_young'), fset=set_property('_young'), fdel=del_property('_young'));
shear = property(fget=get_property('_shear'), fset=set_property('_shear'), fdel=del_property('_shear'));
poisson = property(fget=get_property('_poisson'), fset=set_property('_poisson'), fdel=del_property('_poisson'));
class MaterialTwo(object):
def __init__(self):
properties = ['young', 'shear', 'poisson'];
self.create_properties(properties)
def del_property(attr):
"""Abstract deller"""
def del_attr(self):
setattr(self, attr, None);
return del_attr
def set_property(attr):
"""Abstract setter."""
def set_attr(self, x):
setattr(self, attr, x);
return set_attr
def get_property(attr):
"""Abstract getter"""
def get_attr(self):
if getattr(self, attr) is not None:
return getattr(self, attr);
else:
return 'Some calculated value..'
return get_attr
def create_properties(self, items):
for item in items:
storage = '_'+item;
setattr(self, storage, None);
setattr(self, item, property(fget=self.get_property(storage), fset=self.set_property(storage), fdel=self.del_property(storage)));
steel = MaterialOne();
steel.young = 2e11;
steel.poisson = 0.3;
print steel.poisson
print steel.shear
carbon = MaterialTwo();
carbon.young = 2e11;
carbon.poisson = 0.3;
print carbon.poisson
print carbon.shear
To clarify some more on the code. What I would like to write are Classes for materials, Solid, Liquid, Gas, each of them a subclass of Material. Many material properties will just be assigned. Some can be calculated based on which have been defined. Given two elastic moduli a third can be calculated for instance.
This I have implemented now using something quite similar as MaterialOne. However, as I am getting more material properties, and will also include more of these kind of calculations, I would like to make it cleaner, more organized. Writing it as I did in MaterialTwo is a possibility to me.
setattr(self, item, property(...
This is ultimately your problem. Since properties are descriptors, they must be bound to the class, not the instance. You will need to override __getattr__(), __setattr__(), and __delattr__() and implement it there.
All definitions should have self as the first argument.
This is how these functions should be:
def del_property(self, attr):
def sel_property(self, attr):
def get_property(self, attr):
When methods are called on an object in Python, the object is silently passed as the first argument, which is why you're getting an error saying you're passing in too many arguments. As other people said, you need to include self as the first parameter to your functions.
I don't know if this will make sense, but...
I'm trying to dynamically assign methods to an object.
#translate this
object.key(value)
#into this
object.method({key:value})
To be more specific in my example, I have an object (which I didn't write), lets call it motor, which has some generic methods set, status and a few others. Some take a dictionary as an argument and some take a list. To change the motor's speed, and see the result, I use:
motor.set({'move_at':10})
print motor.status('velocity')
The motor object, then formats this request into a JSON-RPC string, and sends it to an IO daemon. The python motor object doesn't care what the arguments are, it just handles JSON formatting and sockets. The strings move_at and velocity are just two of what might be hundreds of valid arguments.
What I'd like to do is the following instead:
motor.move_at(10)
print motor.velocity()
I'd like to do it in a generic way since I have so many different arguments I can pass. What I don't want to do is this:
# create a new function for every possible argument
def move_at(self,x)
return self.set({'move_at':x})
def velocity(self)
return self.status('velocity')
#and a hundred more...
I did some searching on this which suggested the solution lies with lambdas and meta programming, two subjects I haven't been able to get my head around.
UPDATE:
Based on the code from user470379 I've come up with the following...
# This is what I have now....
class Motor(object):
def set(self,a_dict):
print "Setting a value", a_dict
def status(self,a_list):
print "requesting the status of", a_list
return 10
# Now to extend it....
class MyMotor(Motor):
def __getattr__(self,name):
def special_fn(*value):
# What we return depends on how many arguments there are.
if len(value) == 0: return self.status((name))
if len(value) == 1: return self.set({name:value[0]})
return special_fn
def __setattr__(self,attr,value): # This is based on some other answers
self.set({attr:value})
x = MyMotor()
x.move_at = 20 # Uses __setattr__
x.move_at(10) # May remove this style from __getattr__ to simplify code.
print x.velocity()
output:
Setting a value {'move_at': 20}
Setting a value {'move_at': 10}
10
Thank you to everyone who helped!
What about creating your own __getattr__ for the class that returns a function created on the fly? IIRC, there's some tricky cases to watch out for between __getattr__ and __getattribute__ that I don't recall off the top of my head, I'm sure someone will post a comment to remind me:
def __getattr__(self, name):
def set_fn(self, value):
return self.set({name:value})
return set_fn
Then what should happen is that calling an attribute that doesn't exist (ie: move_at) will call the __getattr__ function and create a new function that will be returned (set_fn above). The name variable of that function will be bound to the name parameter passed into __getattr__ ("move_at" in this case). Then that new function will be called with the arguments you passed (10 in this case).
Edit
A more concise version using lambdas (untested):
def __getattr__(self, name):
return lambda value: self.set({name:value})
There are a lot of different potential answers to this, but many of them will probably involve subclassing the object and/or writing or overriding the __getattr__ function.
Essentially, the __getattr__ function is called whenever python can't find an attribute in the usual way.
Assuming you can subclass your object, here's a simple example of what you might do (it's a bit clumsy but it's a start):
class foo(object):
def __init__(self):
print "initting " + repr(self)
self.a = 5
def meth(self):
print self.a
class newfoo(foo):
def __init__(self):
super(newfoo, self).__init__()
def meth2(): # Or, use a lambda: ...
print "meth2: " + str(self.a) # but you don't have to
self.methdict = { "meth2":meth2 }
def __getattr__(self, name):
return self.methdict[name]
f = foo()
g = newfoo()
f.meth()
g.meth()
g.meth2()
Output:
initting <__main__.foo object at 0xb7701e4c>
initting <__main__.newfoo object at 0xb7701e8c>
5
5
meth2: 5
You seem to have certain "properties" of your object that can be set by
obj.set({"name": value})
and queried by
obj.status("name")
A common way to go in Python is to map this behaviour to what looks like simple attribute access. So we write
obj.name = value
to set the property, and we simply use
obj.name
to query it. This can easily be implemented using the __getattr__() and __setattr__() special methods:
class MyMotor(Motor):
def __init__(self, *args, **kw):
self._init_flag = True
Motor.__init__(self, *args, **kw)
self._init_flag = False
def __getattr__(self, name):
return self.status(name)
def __setattr__(self, name, value):
if self._init_flag or hasattr(self, name):
return Motor.__setattr__(self, name, value)
return self.set({name: value})
Note that this code disallows the dynamic creation of new "real" attributes of Motor instances after the initialisation. If this is needed, corresponding exceptions could be added to the __setattr__() implementation.
Instead of setting with function-call syntax, consider using assignment (with =). Similarly, just use attribute syntax to get a value, instead of function-call syntax. Then you can use __getattr__ and __setattr__:
class OtherType(object): # this is the one you didn't write
# dummy implementations for the example:
def set(self, D):
print "setting", D
def status(self, key):
return "<value of %s>" % key
class Blah(object):
def __init__(self, parent):
object.__setattr__(self, "_parent", parent)
def __getattr__(self, attr):
return self._parent.status(attr)
def __setattr__(self, attr, value):
self._parent.set({attr: value})
obj = Blah(OtherType())
obj.velocity = 42 # prints setting {'velocity': 42}
print obj.velocity # prints <value of velocity>
I am programming a simulations for single neurons. Therefore I have to handle a lot of Parameters. Now the Idea is that I have two classes, one for a SingleParameter and a Collection of parameters. I use property() to access the parameter value easy and to make the code more readable. This works perfect for a sinlge parameter but I don't know how to implement it for the collection as I want to name the property in Collection after the SingleParameter. Here an example:
class SingleParameter(object):
def __init__(self, name, default_value=0, unit='not specified'):
self.name = name
self.default_value = default_value
self.unit = unit
self.set(default_value)
def get(self):
return self._v
def set(self, value):
self._v = value
v = property(fget=get, fset=set, doc='value of parameter')
par1 = SingleParameter(name='par1', default_value=10, unit='mV')
par2 = SingleParameter(name='par2', default_value=20, unit='mA')
# par1 and par2 I can access perfectly via 'p1.v = ...'
# or get its value with 'p1.v'
class Collection(object):
def __init__(self):
self.dict = {}
def __getitem__(self, name):
return self.dict[name] # get the whole object
# to get the value instead:
# return self.dict[name].v
def add(self, parameter):
self.dict[parameter.name] = parameter
# now comes the part that I don't know how to implement with property():
# It shoule be something like
# self.__dict__[parameter.name] = property(...) ?
col = Collection()
col.add(par1)
col.add(par2)
col['par1'] # gives the whole object
# Now here is what I would like to get:
# col.par1 -> should result like col['par1'].v
# col.par1 = 5 -> should result like col['par1'].v = 5
Other questions that I put to understand property():
Why do managed attributes just work for class attributes and not for instance attributes in python?
How can I assign a new class attribute via __dict__ in python?
Look at built-in functions getattr and setattr. You'll probably be a lot happier.
Using the same get/set functions for both classes forces you into an ugly hack with the argument list. Very sketchy, this is how I would do it:
In class SingleParameter, define get and set as usual:
def get(self):
return self._s
def set(self, value):
self._s = value
In class Collection, you cannot know the information until you create the property, so you define the metaset/metaget function and particularize them only later with a lambda function:
def metaget(self, par):
return par.s
def metaset(self, value, par):
par.s = value
def add(self, par):
self[par.name] = par
setattr(Collection, par.name,
property(
fget=lambda x : Collection.metaget(x, par),
fset=lambda x, y : Collection.metaset(x,y, par))
Properties are meant to dynamically evaluate attributes or to make them read-only. What you need is customizing attribute access. __getattr__ and __setattr__ do that really fine, and there's also __getattribute__ if __getattr__ is not enough.
See Python docs on customizing attribute access for details.
Have you looked at the traits package? It seems that you are reinventing the wheel here with your parameter classes. Traits also have additional features that might be useful for your type of application (incidently I know a person that happily uses traits in neural simulations).
Now I implemented a solution with set-/getattr:
class Collection(object):
...
def __setattr__(self, name, value):
if 'dict' in self.__dict__:
if name in self.dict:
self[name].v = value
else:
self.__dict__[name] = value
def __getattr__(self, name):
return self[name].v
There is one thing I quite don't like that much: The attributes are not in the __dict__. And if I have them there as well I would have a copy of the value - which can be dangerous...
Finally I succeded to implement the classes with property(). Thanks a lot for the advice. It took me quite a bit to work it out - but I can promise you that this exercise helps you to understand better pythons OOP.
I implemented it also with __getattr__ and __setattr__ but still don't know the advantages and disadvantages to the property-solution. But this seems to be worth another question. The property-solutions seems to be quit clean.
So here is the code:
class SingleParameter(object):
def __init__(self, name, default_value=0, unit='not specified'):
self.name = name
self.default_value = default_value
self.unit = unit
self.set(default_value)
def get(*args):
self = args[0]
print "get(): "
print args
return self._v
def set(*args):
print "set(): "
print args
self = args[0]
value = args[-1]
self._v = value
v = property(fget=get, fset=set, doc='value of parameter')
class Collection(dict):
# inheriting from dict saves the methods: __getitem__ and __init__
def add(self, par):
self[par.name] = par
# Now here comes the tricky part.
# (Note: this property call the get() and set() methods with one
# more argument than the property of SingleParameter)
setattr(Collection, par.name,
property(fget=par.get, fset=par.set))
# Applying the classes:
par1 = SingleParameter(name='par1', default_value=10, unit='mV')
par2 = SingleParameter(name='par2', default_value=20, unit='mA')
col = Collection()
col.add(par1)
col.add(par2)
# Setting parameter values:
par1.v = 13
col.par1 = 14
# Getting parameter values:
par1.v
col.par1
# checking identity:
par1.v is col.par1
# to access the whole object:
col['par1']
As I am new I am not sure how to move on:
how to treat follow up questions (like this itself):
get() is seems to be called twice - why?
oop-design: property vs. "__getattr__ & __setattr__" - when should I use what?
is it rude to check the own answer to the own question as accepted?
is it recommended to rename the title in order to put correlated questions or questions elaborated with the same example into the same context?
Other questions that I put to understand property():
Why do managed attributes just work for class attributes and not for instance attributes in python?
How can I assign a new class attribute via __dict__ in python?
I have a class that does something similar, but I did the following in the collection object:
setattr(self, par.name, par.v)