We Keep Coding

How would I 'listen' to/decorate a setter from an imported class

I'm not sure whether this is a great approach to be using, but I'm not hugely experienced with Python so please accept my apologies. I've tried to do some research on this but other related questions have been given alternative problem-specific solutions - none of which apply to my specific case.
I have a class that handles the training/querying of my specific machine learning model. This algorithm is running on a remote sensor, various values are fed into the object which returns None if the algorithm isn't trained. Once trained, it returns either True or False depending on the classification assigned to new inputs. Occasionally, the class updates a couple of threshold parameters and I need to know when this occurs.
I am using sockets to pass messages from the remote sensor to my main server. I didn't want to complicate the ML algorithm class by filling it up with message passing code and so instead I've been handling this in a Main class that imports the "algorithm" class. I want the Main class to be able to determine when the threshold parameters are updated and report this back to the server.
class MyAlgorithmClass:
def feed_value(self):
....
class Main:
def __init__(self):
self._algorithm_data = MyAlgorithmClass()
self._sensor_data_queue = Queue()
def process_data(self):
while True:
sensor_value = self._sensor_data_queue.get()
result, value = self._algorithm_data.feed_value(sensor_value)
if result is None:
# value represents % training complete
self._socket.emit('training', value)
elif result is True:
# value represents % chance that input is categoryA
self._socket.emit('categoryA', value)
elif result is False:
...
My initial idea was to add a property to MyAlgorithmClass with a setter. I could then decorate this in my Main class so that every time the setter is called, I can use the value... for example:
class MyAlgorithmClass:
#property
def param1(self):
return self._param1
#param1.setter
def param1(self, value):
self._param1 = value
class Main:
def __init__(self):
self._algorithm_data = MyAlgorithmClass()
self._sensor_data_queue = Queue()
def watch_param1(func):
def inner(*args):
self._socket.emit('param1_updated', *args)
func(*args)
My problem now, is how do I decorate the self._algorithm_data.param1 setter with watch_param1? If I simply set self._algorithm_data.param1 = watch_param1 then I will just end up setting self._algorithm_data._param1 equal to my function which isn't what I want to do.
I could use getter/setter methods instead of a property, but this isn't very pythonic and as multiple people are modifying this code, I don't want the methods to be replaced/changed for properties by somebody else later on.
What is the best approach here? This is a small example but I will have slightly more complex examples of this later on and I don't want something that will cause overcomplication of the algorithm class. Obviously, another option is the Observer pattern but I'm not sure how appropriate it is here where I only have a single variable to monitor in some cases.
I'm really struggling to get a good solution put together so any advice would be much appreciated.
Thanks in advance,
Tom

Use descriptors. They let you customize attribute lookup, storage, and deletion in Python.
A simplified toy version of your code with descriptors looks something like:
class WatchedParam:
def __init__(self, name):
self.name = name
def __get__(self, instance, insttype=None):
print(f"{self.name} : value accessed")
return getattr(instance, '_' + self.name)
def __set__(self, instance, new_val):
print(f"{self.name} : value set")
setattr(instance, '_' + self.name, new_val)
class MyAlgorithmClass:
param1 = WatchedParam("param1")
param2 = WatchedParam("param2")
def __init__(self, param1, param2, param3):
self.param1 = param1
self.param2 = param2
self.param3 = param3
class Main:
def __init__(self):
self._data = MyAlgorithmClass(10, 20, 50)
m = Main()
m._data.param1 # calls WatchedParam.__get__
m._data.param2 = 100 # calls WatchedParam.__set__
The WatchedParam class is a descriptor and can be used in MyAlgorithmClass to specify the parameters that need to be monitored.

The solution I went for is as follows, using a 'Proxy' subclass which overrides the properties. Eventually, once I have a better understanding of the watched parameters, I won't need to watch them anymore. At this point I will be able to swap out the Proxy for the base class and continue using the code as normal.
class MyAlgorithmClassProxy(MyAlgorithmClass):
#property
def watch_param1(self):
return MyAlgorithmClass.watch_param1.fget(self)
#watch_param1.setter
def watch_param1(self, value):
self._socket.emit('param1_updated', *args)
MyAlgorithmClass.watch_param1.fset(self, value)

Setter for a field of a field

Given the code:
class Point:
def __init__(self, x, y):
self.x = x
self.y = y
class Block:
size = 40
def __init__(self, x=1, y=1):
self.pixel_position = Point(x * Block.size, y * Block.size)
self.__block_position = Point(x, y)
#property
def block_position(self):
return self.__block_position
#block_position.setter
def block_position(self, point):
#I want for pixel_position to be updated whenever block position changes
self.pixel_position.x += point.x * size
self.pixel_position.y += point.y * size
self.__block_position = point
Now for such code simple assignment works well
block = Block()
block.block_position = Point(2, 1)
but if I want to increment x of block position... well, code doesn't go into setter.
block.block_position.x -= 1
# now block_position = (1, 1) but pixel_position = (80, 40) not (40, 40)
How may I change it?
I know I can resolve this problem with adding property for __pixel_position that will calculate Block.size * __block_position before returning itself, but that approach doesn't satisfy me - well I want to know how in python one can set a property for a field of a field.
My question is not about finding any solution, but to find a solution where changing field block_position.x will redirect me to my setter/getter.

The problem there is that you are mixing some concepts trying to make use of Python property and things get confusing -
The basic thing is that if pixel_position is to be to be calculated - it should itself a property.
What your are trying to do is to put some gates on the setting of values on "block_position" and derive - when block_position is changed the new value for pixel_position. That is not working because you have not "gatted" always one could possibly modify the values inside your block_position.
What happens when you make:
block.block_position.x += 1
Is that the property getter for block_position is activated - the Point object in there then have its x attribute changed - but this change never goes through the outter block object,as x is an ordinay attribute, not a property.
Now, it would be possible to instrument your Point class so that actions could be triggered whenever x ou y are changed - but that could become really complicated, and fast.
A better approache there is to have pixel_position itself be a property, instead of an ordinary attribute, and have its values lazily calculated - generated whenever they are needed - and not depend on the value to be setted eagerly whenever block_position changes. This is a pattern from "reactive programing".
Actually, you will find out that "block_poisition" itself can be an ordinary instance attribute, and not be a property - unless you want its checker to ensure the assigned object is an instance of Point.
class Point:
def __init__(self, x, y):
self.x = x
self.y = y
def __mul__(self, scale):
return Point(self.x * scale, self.y * scale)
class Block:
size = 40
def __init__(self, x=1,y=1):
self._block_position = Point(x,y)
#property
def block_position(self):
return self._block_position
#block_position.setter
def block_position(self, point):
if not isinstance(point, Point):
# raise TypeError() # or, as an option, accept sequences of 2 coordinates:
point = Point(point[0], point[1])
self._block_position = point
#property
#pixel_position(self):
return self._block_position * self.size
So, now things are the other way around - and pixel_position can't be setted and is guaranteed to be always updated.
Three things in there:
I've added the __mul__ method to your Point so now one can just use the * operator to multiply it by a scalar.
There is no need or sense in actually prepending __ to Python "hidden" attributes. Early tutorials or unofficial documentation to the language could misleadingly say it is a way of having "private attributes". That is an error - there is no private attributes in Python what __ does is name mangling in order to allow a class to have attributes that are not messed up by subclasses. In almost 20 years of Python programming I've never actually needed that feature. On the other hand, it can give you strange errors if you have subclasses of Block. Just don't. The accepted convention is that one single "_" indicates an attribute that should not be changed or accessed directly by users of the class and this have no side effects.
Without a setter, pixel_position is mostly "unchangeable" - and if one does change the attributes inside it after retrieving block.pixel_position , he will be changing a detached Point instance.
If you really need round-trip changing between pixel_position and block_position (that is, make in such a way the class user can change either attribute and have the change reflected in the other), rather than trying to instrument a change notification inside Point , I suggest you make Point an immutable class instead. Anyone wanting to change coordinates would have to create a new point instance - as a result block.block_position.x += 1 would not work anymore - one would have to do: block.block_position += Point(1, 0) (and then you implement __add__ in Point, just as I did __mul__). Then you could write your setter for pixel_position to force a new value to block_position if it gets changed.
On the upside, if you make Point immutable, you can add __hash__ to it and have it working in sets and as dictionary keys: a lot of other uses open up.
Check the class V2 implementation on this project to have an idea of a nice implementation (Disclaimer: the link project is something I am working now as a hobby, and I have actually implemented this class over the past week)

Since you asked for it, I'm providing an example implementation where properties of an object notify their owner when being set, for it to synchronize with another object. I only consider 1D points (x) in this example since the implementation for y is the very same:
class NotificationProperty(property):
def __init__(self, fget=None, fset=None, fdel=None, doc=None, notify=None):
super().__init__(fget, fset, fdel, doc)
self.notify = notify
def __set_name__(self, owner, name):
self.name = name
def __set__(self, instance, val):
super().__set__(instance, val)
self.notify(instance, self.name, val)
# Should define similar methods for other attributes
# (see https://docs.python.org/3/howto/descriptor.html#properties).
def setter(self, fset):
return type(self)(self.fget, fset, self.fdel, self.__doc__, self.notify)
def notification_property(func):
from functools import partial
return partial(NotificationProperty, notify=func)
class SyncPoint:
def __init__(self, x, sync_with=None):
self.sync_with = sync_with
self.x = x
def sync(self, which, value):
if self.sync_with is not None:
obj, scale = self.sync_with
value = int(scale * value)
if getattr(obj, which) != value: # Check if already synced -> avoid RecursionError.
setattr(obj, which, value)
#notification_property(sync)
def x(self):
return self._x
#x.setter
def x(self, val):
self._x = val
class Block:
size = 40
def __init__(self, x=1):
self.pixel_position = SyncPoint(self.size * x)
self.block_position = SyncPoint(x, sync_with=(self.pixel_position, self.size))
self.pixel_position.sync_with = (self.block_position, 1/self.size)
block = Block(3)
print('block_pos: ', block.block_position.x) # block_pos: 3
print('pixel_pos: ', block.pixel_position.x) # pixel_pos: 120
block.block_position.x -= 1
print('block_pos: ', block.block_position.x) # block_pos: 2
print('pixel_pos: ', block.pixel_position.x) # pixel_pos: 80
block.pixel_position.x -= Block.size
print('block_pos: ', block.block_position.x) # block_pos: 1
print('pixel_pos: ', block.pixel_position.x) # pixel_pos: 40
Variation: specify the notify function via x.setter(func)
The following is a variation of the above code which let's you specify the function to be called for notifications during definition of x.setter. This might feel more intuitive since the notification happens on __set__ but in the end it's a matter of taste:
from functools import partial
class notification_property(property):
def __init__(self, fget=None, fset=None, fdel=None, doc=None, notify=None):
super().__init__(fget, fset, fdel, doc)
self.notify = notify
def __set_name__(self, owner, name):
self.name = name
def __set__(self, instance, val):
super().__set__(instance, val)
self.notify(instance, self.name, val)
# Should define similar methods for other attributes
# (see https://docs.python.org/3/howto/descriptor.html#properties).
def setter(self, func=None):
return partial(type(self), self.fget, fdel=self.fdel, doc=self.__doc__, notify=(func or self.notify))
class SyncPoint:
def __init__(self, x, sync_with=None):
self.sync_with = sync_with
self.x = x
def sync(self, which, value):
if self.sync_with is not None:
obj, scale = self.sync_with
value = int(scale * value)
if getattr(obj, which) != value: # Check if already synced -> avoid RecursionError.
setattr(obj, which, value)
#notification_property
def x(self):
return self._x
#x.setter(sync)
def x(self, val):
self._x = val

You can resolve this by making pixel_position a property since this attribute seems dependent on the other, block_position. That way block_position doesn't even have to be a property:
class Point:
def __init__(self, x, y):
self.x = x
self.y = y
def __repr__(self):
return f'{type(self).__name__}({self.x}, {self.y})'
class Block:
size = 40
def __init__(self, x=1, y=1):
self.block_position = Point(x,y)
#property
def pixel_position(self):
return Point(self.block_position.x * self.size, self.block_position.y * self.size)
block = Block()
block.block_position = Point(2,1)
block.block_position.x -= 1
print(block.block_position) # Point(1, 1)
print(block.pixel_position) # Point(40, 40)
In general dependent attributes are suitable for properties as they can be computed "on-the-fly" from other, independent ones.

Recreate class object changing one constructor argument

I'd like to have implement an update method in my class that recreates the class but changes just one constructor argument.
My attempt:
class Updateable:
def update(self, var, var_str, **kwargs):
kwargs.update(self._vars)
kwargs[var_str] = var
self.__init__(**kwargs)
class Rectangle(Updateable):
def __init__(self, length, perimeter):
self._vars = locals()
self.length = length
self.width = 0.5*(perimeter - 2.*length)
r = Rectangle(10, 20)
r.update('perimeter', 16)
The problem is, the whole locals() thing, I think, is pretty dodgy, and it means that any class that is Updateable needs to assign self._vars.
What would be the correct way to achieve this functionality? Decorators, metaclasses? Something simpler?

Please correct me if I misunderstood your question, or if you don't want high level advise and your just question solved.
What your __init__ currently does is recalculate the properties of a geometric shape if a (possibly relevant) variable is changed. Step 1 is to take this out of the __init__, and into a seperate def which is called by init. The main thing here is that you do not pass variables to this function, but use the class variables which have been set in either __init__ or one the superclass updating methods.
Step 2 is to change your update function.
Python has a form of getters and setters called properties allowing you to hook tasks to updating your variables. On the other hand a more generalized way is more similar to your own update, and is listed as option 2 below
Example alternative
class Updateable:
# Option 1
#property
def perimeter(self):
return self.__perimeter
#perimeter.setter
def perimeter(self, perimeter):
self.__perimeter = perimeter
self.recalculate_everything() # or self.calculate_width() or something
# Option 2
def update(self, **kwargs):
for key, value in kwargs.items():
setattr(self, key, value)
self.recalculate_everything
class Rectable(Updateable):
def __init__(self, length, perimeter):
self.__length = length
self.__perimeter = perimeter
recalculate_everything()
def recalculate_everything():
self.calculate_width()
...
def calculate_width():
self.__width = 0.5*(self.__perimeter - 2.*self.__length)

Laurens Koppenol suggested using properties (Python's generic support for computed attribute) which is a good idea but his example code is both broken in many ways and more complicated than it has to be, so here's a simpler, working and pythonic example (no Updatable class nor any other extraneous stuff required):
class Rectangle(object):
def __init__(self, length, perimeter):
self.length = length
self.perimeter = perimeter
#property
def width(self):
return 0.5*(self.perimeter - 2.*self.length)
If you want to cache the width value (to avoid useless computations) but still make sure it's updated when length or perimeter change, you'll need to make them all properties:
class Rectangle(object):
def __init__(self, length, perimeter):
self.length = length
self.perimeter = perimeter
#property
def length(self):
return self._length
#length.setter
def length(self, value):
self._length = value
self._width = None
#property
def perimeter(self):
return self._perimeter
#length.setter
def perimiter(self, value):
self._perimeter = value
self._width = None
#property
def width(self):
if self._width is None:
self._width = 0.5*(self.perimeter - 2.*self.length)
return self._width
or (if you have a lot of such stuff) use some "cached_property with invalidation" implementation as this one: Storing calculated values in an object
edit: wrt/ your question, the call to locals is indeed ugly (and can easily break - you may have local variables that are not supposed to be parts of _vars), as well as the need to explicitely set self._vars in child classes. Also the update() API is itself quite ugly IMHO. Now you don't need anything fancy to make the whole thing more pythonic - here's a solution whose only boilerplate is the need to call Updateable.__init__ with named arguments (won't work with positional ones):
class Updateable(object):
def __init__(self, **kwargs):
self._vars = kwargs
def update(self, **kwargs):
vars = self._vars.copy()
vars.update(**kwargs)
self.__init__(**vars)
class Rectangle(Updateable):
def __init__(self, length, perimeter):
super(Rectangle, self).__init__(length=length, perimeter=perimeter)
self.length = length
self.width = 0.5*(perimeter - 2.*length)
r = Rectangle(10, 20)
r.update(perimeter=40)
As a side note, I personnaly find quite disturbing that your Rectangle class takes a perimeter argument but stores a width instead... Maybe you should consider a perimeter property ? (even if read-only to avoid recomputing etc)

As others have observed, computations like width should be moved to a property or method; they don't belong in an initialiser.
If you really want to return a new instance, this would work for the simplest cases, where the instance's attributes are immutable objects like strings or integers:
import copy
class Copyable:
"""Mixin to create copies with a changed attribute."""
def copy_and_modify(self, var, var_str, **kwargs):
new = copy.copy(self)
setattr(new, var, var_str)
return new
class Rectangle(Copyable):
def __init__(self, length, perimeter):
self.perimeter = perimeter
self.length = length
#property
def width(self):
return 0.5 * (self.perimeter - 2.0 * self.length)
however if your objects contain nested mutable structures such as dictionaries or lists you would need to change copy_and_modify to use copy.deepcopy (but note deep copying is slow)
class Copyable:
def copy_and_modify(self, var, var_str, **kwargs):
new = copy.deepcopy(self)
setattr(new, var, var_str)
return new
You can define __copy__ and __deepcopy__ methods in your subclasses as described in the docs to fine-tune control over the copying process.

problem with setters and getters

Hi I'm Trying to work with setters and getters in python for one class and I want to use the setter with other instruction, so when setting the property value, the setter do more than only assign the value to the property. Maybe I'm confused on how the setter and getter work. Here is an example:
class test:
self.x = None
#property
def var(self):
return self.x
#var.setter
def var(self, value):
print("Assigning Value")
self.x = value
v = test()
v.var = "hello"
So what I'm trying to do in this example is to print "Assigning Value" when x value is assigned, but so far Isn't working.
My question is. Do I'm doing something wrong?,Do I miss something?, or that's not the way setters and getters work?
Thanks

Change
class test:
To:
class test(object):
In Python 2.x you can only use the descriptors if inheriting from object. (Its a backwards compatibility thing). Fortunately, this has been resolved in Python 3.x

At this point, there is no self:
class test:
self.x = None
# ^^^^^ BAD
change it to either set in the initializer:
class test:
def __init__(self):
self.x = None
or use a class attribute normally.
class test:
x = None
If you aren't sure which, go with the first; it'll confuse you less to use that pattern until you are more confident about the difference.

How to implement property() with dynamic name (in python)

I am programming a simulations for single neurons. Therefore I have to handle a lot of Parameters. Now the Idea is that I have two classes, one for a SingleParameter and a Collection of parameters. I use property() to access the parameter value easy and to make the code more readable. This works perfect for a sinlge parameter but I don't know how to implement it for the collection as I want to name the property in Collection after the SingleParameter. Here an example:
class SingleParameter(object):
def __init__(self, name, default_value=0, unit='not specified'):
self.name = name
self.default_value = default_value
self.unit = unit
self.set(default_value)
def get(self):
return self._v
def set(self, value):
self._v = value
v = property(fget=get, fset=set, doc='value of parameter')
par1 = SingleParameter(name='par1', default_value=10, unit='mV')
par2 = SingleParameter(name='par2', default_value=20, unit='mA')
# par1 and par2 I can access perfectly via 'p1.v = ...'
# or get its value with 'p1.v'
class Collection(object):
def __init__(self):
self.dict = {}
def __getitem__(self, name):
return self.dict[name] # get the whole object
# to get the value instead:
# return self.dict[name].v
def add(self, parameter):
self.dict[parameter.name] = parameter
# now comes the part that I don't know how to implement with property():
# It shoule be something like
# self.__dict__[parameter.name] = property(...) ?
col = Collection()
col.add(par1)
col.add(par2)
col['par1'] # gives the whole object
# Now here is what I would like to get:
# col.par1 -> should result like col['par1'].v
# col.par1 = 5 -> should result like col['par1'].v = 5
Other questions that I put to understand property():
Why do managed attributes just work for class attributes and not for instance attributes in python?
How can I assign a new class attribute via __dict__ in python?

Look at built-in functions getattr and setattr. You'll probably be a lot happier.

Using the same get/set functions for both classes forces you into an ugly hack with the argument list. Very sketchy, this is how I would do it:
In class SingleParameter, define get and set as usual:
def get(self):
return self._s
def set(self, value):
self._s = value
In class Collection, you cannot know the information until you create the property, so you define the metaset/metaget function and particularize them only later with a lambda function:
def metaget(self, par):
return par.s
def metaset(self, value, par):
par.s = value
def add(self, par):
self[par.name] = par
setattr(Collection, par.name,
property(
fget=lambda x : Collection.metaget(x, par),
fset=lambda x, y : Collection.metaset(x,y, par))

Properties are meant to dynamically evaluate attributes or to make them read-only. What you need is customizing attribute access. __getattr__ and __setattr__ do that really fine, and there's also __getattribute__ if __getattr__ is not enough.
See Python docs on customizing attribute access for details.

Have you looked at the traits package? It seems that you are reinventing the wheel here with your parameter classes. Traits also have additional features that might be useful for your type of application (incidently I know a person that happily uses traits in neural simulations).

Now I implemented a solution with set-/getattr:
class Collection(object):
...
def __setattr__(self, name, value):
if 'dict' in self.__dict__:
if name in self.dict:
self[name].v = value
else:
self.__dict__[name] = value
def __getattr__(self, name):
return self[name].v
There is one thing I quite don't like that much: The attributes are not in the __dict__. And if I have them there as well I would have a copy of the value - which can be dangerous...

Finally I succeded to implement the classes with property(). Thanks a lot for the advice. It took me quite a bit to work it out - but I can promise you that this exercise helps you to understand better pythons OOP.
I implemented it also with __getattr__ and __setattr__ but still don't know the advantages and disadvantages to the property-solution. But this seems to be worth another question. The property-solutions seems to be quit clean.
So here is the code:
class SingleParameter(object):
def __init__(self, name, default_value=0, unit='not specified'):
self.name = name
self.default_value = default_value
self.unit = unit
self.set(default_value)
def get(*args):
self = args[0]
print "get(): "
print args
return self._v
def set(*args):
print "set(): "
print args
self = args[0]
value = args[-1]
self._v = value
v = property(fget=get, fset=set, doc='value of parameter')
class Collection(dict):
# inheriting from dict saves the methods: __getitem__ and __init__
def add(self, par):
self[par.name] = par
# Now here comes the tricky part.
# (Note: this property call the get() and set() methods with one
# more argument than the property of SingleParameter)
setattr(Collection, par.name,
property(fget=par.get, fset=par.set))
# Applying the classes:
par1 = SingleParameter(name='par1', default_value=10, unit='mV')
par2 = SingleParameter(name='par2', default_value=20, unit='mA')
col = Collection()
col.add(par1)
col.add(par2)
# Setting parameter values:
par1.v = 13
col.par1 = 14
# Getting parameter values:
par1.v
col.par1
# checking identity:
par1.v is col.par1
# to access the whole object:
col['par1']
As I am new I am not sure how to move on:
how to treat follow up questions (like this itself):
get() is seems to be called twice - why?
oop-design: property vs. "__getattr__ & __setattr__" - when should I use what?
is it rude to check the own answer to the own question as accepted?
is it recommended to rename the title in order to put correlated questions or questions elaborated with the same example into the same context?
Other questions that I put to understand property():
Why do managed attributes just work for class attributes and not for instance attributes in python?
How can I assign a new class attribute via __dict__ in python?

I have a class that does something similar, but I did the following in the collection object:
setattr(self, par.name, par.v)