We Keep Coding

How would I 'listen' to/decorate a setter from an imported class

I'm not sure whether this is a great approach to be using, but I'm not hugely experienced with Python so please accept my apologies. I've tried to do some research on this but other related questions have been given alternative problem-specific solutions - none of which apply to my specific case.
I have a class that handles the training/querying of my specific machine learning model. This algorithm is running on a remote sensor, various values are fed into the object which returns None if the algorithm isn't trained. Once trained, it returns either True or False depending on the classification assigned to new inputs. Occasionally, the class updates a couple of threshold parameters and I need to know when this occurs.
I am using sockets to pass messages from the remote sensor to my main server. I didn't want to complicate the ML algorithm class by filling it up with message passing code and so instead I've been handling this in a Main class that imports the "algorithm" class. I want the Main class to be able to determine when the threshold parameters are updated and report this back to the server.
class MyAlgorithmClass:
def feed_value(self):
....
class Main:
def __init__(self):
self._algorithm_data = MyAlgorithmClass()
self._sensor_data_queue = Queue()
def process_data(self):
while True:
sensor_value = self._sensor_data_queue.get()
result, value = self._algorithm_data.feed_value(sensor_value)
if result is None:
# value represents % training complete
self._socket.emit('training', value)
elif result is True:
# value represents % chance that input is categoryA
self._socket.emit('categoryA', value)
elif result is False:
...
My initial idea was to add a property to MyAlgorithmClass with a setter. I could then decorate this in my Main class so that every time the setter is called, I can use the value... for example:
class MyAlgorithmClass:
#property
def param1(self):
return self._param1
#param1.setter
def param1(self, value):
self._param1 = value
class Main:
def __init__(self):
self._algorithm_data = MyAlgorithmClass()
self._sensor_data_queue = Queue()
def watch_param1(func):
def inner(*args):
self._socket.emit('param1_updated', *args)
func(*args)
My problem now, is how do I decorate the self._algorithm_data.param1 setter with watch_param1? If I simply set self._algorithm_data.param1 = watch_param1 then I will just end up setting self._algorithm_data._param1 equal to my function which isn't what I want to do.
I could use getter/setter methods instead of a property, but this isn't very pythonic and as multiple people are modifying this code, I don't want the methods to be replaced/changed for properties by somebody else later on.
What is the best approach here? This is a small example but I will have slightly more complex examples of this later on and I don't want something that will cause overcomplication of the algorithm class. Obviously, another option is the Observer pattern but I'm not sure how appropriate it is here where I only have a single variable to monitor in some cases.
I'm really struggling to get a good solution put together so any advice would be much appreciated.
Thanks in advance,
Tom

Use descriptors. They let you customize attribute lookup, storage, and deletion in Python.
A simplified toy version of your code with descriptors looks something like:
class WatchedParam:
def __init__(self, name):
self.name = name
def __get__(self, instance, insttype=None):
print(f"{self.name} : value accessed")
return getattr(instance, '_' + self.name)
def __set__(self, instance, new_val):
print(f"{self.name} : value set")
setattr(instance, '_' + self.name, new_val)
class MyAlgorithmClass:
param1 = WatchedParam("param1")
param2 = WatchedParam("param2")
def __init__(self, param1, param2, param3):
self.param1 = param1
self.param2 = param2
self.param3 = param3
class Main:
def __init__(self):
self._data = MyAlgorithmClass(10, 20, 50)
m = Main()
m._data.param1 # calls WatchedParam.__get__
m._data.param2 = 100 # calls WatchedParam.__set__
The WatchedParam class is a descriptor and can be used in MyAlgorithmClass to specify the parameters that need to be monitored.

The solution I went for is as follows, using a 'Proxy' subclass which overrides the properties. Eventually, once I have a better understanding of the watched parameters, I won't need to watch them anymore. At this point I will be able to swap out the Proxy for the base class and continue using the code as normal.
class MyAlgorithmClassProxy(MyAlgorithmClass):
#property
def watch_param1(self):
return MyAlgorithmClass.watch_param1.fget(self)
#watch_param1.setter
def watch_param1(self, value):
self._socket.emit('param1_updated', *args)
MyAlgorithmClass.watch_param1.fset(self, value)

Some doubts about #property in python 3

In order not to extend myself too much I will give a basic and hypothetical example of what I am trying to do.
Suppose the following class:
class foo():
def __init__(self):
self.keywords = []
## this method returns the entire list
def get_keywords(self):
return self.keywords
def set_keywords(self, value):
self.keywords.append(value)
But I want to code this in a pythonic way using the #property decorator.
My (wrong) attempt to do this:
class foo:
def __init__(self):
self.key = []
#property
def key(self):
return self.__key
#key.setter
def key(self, value):
self.__key.append(value)
So, whats is wrong in my attempt ?
ps: English is not my native language and I hope my doubt is understandable.

In your original code, self.set_keywords only appends to an existing list; it does not let you initialize the value of keywords to an arbitrary list. This restriction is preserved in your property-based code, which means you cannot assign directly to self.key; you have to initialize the underlying list in __init__ directly.
class foo:
def __init__(self):
# self.key = [] is equivalent to `self.__key.append([])`, but
# self.__key doesn't exist yet. (And would be wrong even if it did.)
self.__key = []
#property
def key(self):
return self.__key
#key.setter
def key(self, value):
self.__key.append(value)
However, this means an assignment like self.key = 3 doesn't actually perform what most people would expect of an assignment. It doesn't overwrite the old value, it adds to it instead. Use the setter to provide a fixed list, but a different method to add to an existing one.
class foo:
def __init__(self):
self.__keys = []
#property
def keys(self):
return self.__keys
#keys.setter
def keys(self, values):
self.__keys = values
def add_key(self, value):
self.__key.append(value)
And finally, it's not necessarily more Pythonic to use a property if you don't actually do any sort of extra work or validation in the getter or setter. If all you are doing is wrapping access to an underlying value, just let the value be used directly.
class foo:
def __init__(self):
self.keys = []
self.keys = [1,2,3]
print(self.keys)
self.keys.append(4)
# etc
The nice thing about properties is that if you start by allowing direct access to keys, then nothing about how you use keys changes if you later decide to replace it with a property.

You can give this a try:
class Foo:
def __init__(self):
self._key = []
#property
def key(self):
return self._key
#key.setter
def key(self, value):
self._key = value
Here are my two cents:
Rename the class foo to Foo
You can't initialize self.key, as this is the property, so initialize the correct variable in the constructor (i.e. __init__)
Private vars are prefixed with one _ scope and not two (two __ are Python internals)
I suppose you rather want my_instance.key = ['spam', 'eggs'] to replace the foo._key value than extend it. Because this is kind of a "setter" and that would result in a weird behaviour, or at least another developer won't expect that behaviour from that setter/function
However, and that's important: As long as you're only doing this, you won't need properties. You can simply initialize self.keys in the constructor and froget about the property and setter function. Later on, when you want to change the behaviour, you can still add the property and setter. That's one reason why we've properties in Python, so that you won't have to refactor your whole code in case "a bit more logic" comes into place.
Btw. if you're really depending everything on those dict functions, you might also want to inherit your class from the dict class. Depends what you're up to.

Create multiple classes or multiple objects in Python?

I have the following problem and I need advice on how to solve it the best technically in Python. As I am new to programming I would like to have some advice.
So I will have the following object and they should store something. Here is an example:
object 1: cash dividends (they will have the following properties)
exdate (will store a list of dates)
recorddate (will store a list of dates)
paydate (will store a list of dates)
ISIN (will store a list of text)
object 2: stocksplits (they will have the following prpoerties)
stockplitratio (will be some ration)
exdate(list of dates)
...
I have tried to solve it like this:
class cashDividends(object):
def __init__(self, _gross,_net,_ISIN, _paydate, _exdate, _recorddate, _frequency, _type, _announceddate, _currency):
self.gross = _gross
self.net = _net
self.ISIN = _ISIN
self.paydate = _paydate
self.exdate = _exdate
self.recorddate = _recorddate
self.frequency = _frequency
self.type = _type
self.announceddate = _announceddate
self.currency = _currency
So if I have this I would have to create another class named stockplits and then define an __init__ function again.
However is there a way where I can have one class like "Corporate Actions" and then have stock splits and cashdividends in there ?

Sure you can! In python you can pass classes to other classes.
Here a simple example:
class A():
def __init__(self):
self.x = 0
class B():
def __init__(self):
self.x = 1
class Container():
def __init__(self, objects):
self.x = [obj.x for obj in objects]
a = A()
b = B()
c = Container([a,b])
c.x
[0,1]

If I understood correctly what you want is an object that has other objects from a class you created as property?
class CorporateActions(object):
def __init__(self, aCashDividend, aStockSplit):
self.cashDividend = aCashDividend
self.stockSplit = aStockSplit
myCashDividends = CashDividends(...) #corresponding parameters here
myStockSplit = StockSplit(...)
myCorporateActions = CorporateActions(myCashDividends, myStockSplit)

Strictly speaking this answer isn't an answer for the final question. However, it is a way to make your life slightly easier.
Consider creating a sort-of template class (I'm using this term loosely; there's no such thing in Python) that does the __init__ work for you. Like this:
class KwargAttrs():
def __init__(self, **kwargs):
for k,v in kwargs.items():
setattr(self, k, v)
def _update(self, **kwargs):
args_dict = {k:(kwargs[k] if k in kwargs else self.__dict__[k]) for k in self.__dict__}
self.__dict__.update(args_dict)
This class uses every supplied keyword argument as an object attribute. Use it this way:
class CashDividends(KwargAttrs):
def __init__(self, gross, net, ISIN, paydate, exdate, recorddate, frequency, type, announceddate, currency):
# save the namespace before it gets polluted
super().__init__(**locals())
# work that might pollute local namespace goes here
# OPTIONAL: update the argument values in case they were modified:
super()._update(**locals())
Using a method like this, you don't have to go through the argument list and assign every single object attribute; it happens automatically.
We bookend everything you need to accomplish in the __init__ method with method calls to the parent-class via super(). We do this because locals() returns a dict every variable in the function's current namespace, so you need to 1.) capture that namespace before any other work pollutes it and 2.) update the namespace in case any work changes the argument values.
The call to update is optional, but the values of the supplied arguments will not be updated if something is done to them after the call to super().__init__() (that is, unless you change the values using setattr(self, 'argname, value)`, which is not a bad idea).
You can continue using this class like so:
class StockSplits(KwargAttrs):
def __init__(self, stocksplitratio, gross, net, ISIN, paydate, exdate, recorddate, frequency, type, announceddate, currency):
super().__init__(**locals())
As mentioned in the other answers you can create a container for our other classes, but you can even do that using this same template class:
class CorporateActions(KwargAttrs):
def __init__(self, stock_splits , cash_dividends):
super().__init__(**locals())
ca = CorporateActions(stock_splits = StockSplits(<arguments>), cash_dividends = CashDividends(<arguments>) )

Using a class instance as a class attribute, descriptors, and properties

I have recently stated trying to use the newer style of classes in Python (those derived from object). As an excersise to familiarise myself with them I am trying to define a class which has a number of class instances as attributes, with each of these class instances describing a different type of data, e.g. 1d lists, 2d arrays, scalars etc. Essentially I wish to be able to write
some_class.data_type.some_variable
where data_type is a class instance describing a collection of variables. Below is my first attempt at implementing this, using just a profiles_1d instance and rather generic names:
class profiles_1d(object):
def __init__(self, x, y1=None, y2=None, y3=None):
self.x = x
self.y1 = y1
self.y2 = y2
self.y3 = y3
class collection(object):
def __init__(self):
self._profiles_1d = None
def get_profiles(self):
return self._profiles_1d
def set_profiles(self, x, *args, **kwargs):
self._profiles_1d = profiles_1d(x, *args, **kwargs)
def del_profiles(self):
self._profiles_1d = None
profiles1d = property(fget=get_profiles, fset=set_profiles, fdel=del_profiles,
doc="One dimensional profiles")
Is the above code roughly an appropriate way of tackling this problem. The examples I have seen of using property just set the value of some variable. Here I require my set method to initialise an instance of some class. If not, any other suggestions of better ways to implement this would be greatly appreciated.
In addition, is the way I am defining my set method ok? Generally the set method, as far as I understand, defines what to do when the user types, in this example,
collection.profiles1d = ...
The only way I can correctly set the attributes of the profiles_1d instance with the above code is to type collection.set_profiles([...], y1=[...], ...), but I think that I shouldn't be directly calling this method. Ideally I would want to type collection.profiles = ([...], y1=[...], ...): is this correct/possible?
Finally, I have seen a decorators mentioned alot with repect to the new style of classes, but this is something I know very little about. Is the use of decorators appropriate here? Is this something I should know more about for this problem?

First, it's good you're learning new-style classes. They've got lots of advantages.
The modern way to make properties in Python is:
class Collection(object):
def __init__(self):
self._profiles_1d = None
#property
def profiles(self):
"""One dimensional profiles"""
return self._profiles_1d
#profiles.setter
def profiles(self, argtuple):
args, kwargs = argtuple
self._profiles_1d = profiles_1d(*args, **kwargs)
#profiles.deleter
def profiles(self):
self._profiles_1d = None
then set profiles by doing
collection = Collection()
collection.profiles = (arg1, arg2, arg3), {'kwarg1':val1, 'kwarg2':val2}
Notice all three methods having the same name.
This is not normally done; either have them pass the attributes to collections constructor or have them create the profiles_1d themselves and then do collections.profiles = myprofiles1d or pass it to the constructor.
When you want the attribute to manage access to itself instead of the class managing access to the attribute, make the attribute a class with a descriptor. Do this if, unlike in the property example above, you actually want the data stored inside the attribute (instead of another, faux-private instance variable). Also, it's good for if you're going to use the same property over and over again -- make it a descriptor and you don't need to write the code multiple times or use a base class.
I actually like the page by #S.Lott -- Building Skills in Python's Attributes, Properties and Descriptors.

When creating propertys (or other descriptors) that need to call other instance methods the naming convention is to prepend an _ to those methods; so your names above would be _get_profiles, _set_profiles, and _del_profiles.
In Python 2.6+ each property is also a decorator, so you don't have to create the (otherwise useless) _name methods:
#property
def test(self):
return self._test
#test.setter
def test(self, newvalue):
# validate newvalue if necessary
self._test = newvalue
#test.deleter
def test(self):
del self._test
It looks like your code is trying to set profiles on the class instead of instances -- if this is so, properties on the class won't work as collections.profiles would be overridden with a profiles_1d object, clobbering the property... if this is really what you want, you'll have to make a metaclass and put the property there instead.
Hopefully you are talking about instances, so the class would look like:
class Collection(object): # notice the capital C in Collection
def __init__(self):
self._profiles_1d = None
#property
def profiles1d(self):
"One dimensional profiles"
return self._profiles_1d
#profiles1d.setter
def profiles1d(self, value):
self._profiles_1d = profiles_1d(*value)
#profiles1d.deleter
def profiles1d(self):
del self._profiles_1d
and then you would do something like:
collection = Collection()
collection.profiles1d = x, y1, y2, y3
A couple things to note: the setter method gets called with only two items: self, and the new value (which is why you were having to call set_profiles1d manually); when doing an assignment, keyword naming is not an option (that only works in function calls, which an assignment is not). If it makes sense for you, you can get fancy and do something like:
collection.profiles1d = (x, dict(y1=y1, y2=y2, y3=y3))
and then change the setter to:
#profiles1d.setter
def profiles1d(self, value):
x, y = value
self._profiles_1d = profiles_1d(x, **y)
which is still fairly readable (although I prefer the x, y1, y2, y3 version myself).

How to implement property() with dynamic name (in python)

I am programming a simulations for single neurons. Therefore I have to handle a lot of Parameters. Now the Idea is that I have two classes, one for a SingleParameter and a Collection of parameters. I use property() to access the parameter value easy and to make the code more readable. This works perfect for a sinlge parameter but I don't know how to implement it for the collection as I want to name the property in Collection after the SingleParameter. Here an example:
class SingleParameter(object):
def __init__(self, name, default_value=0, unit='not specified'):
self.name = name
self.default_value = default_value
self.unit = unit
self.set(default_value)
def get(self):
return self._v
def set(self, value):
self._v = value
v = property(fget=get, fset=set, doc='value of parameter')
par1 = SingleParameter(name='par1', default_value=10, unit='mV')
par2 = SingleParameter(name='par2', default_value=20, unit='mA')
# par1 and par2 I can access perfectly via 'p1.v = ...'
# or get its value with 'p1.v'
class Collection(object):
def __init__(self):
self.dict = {}
def __getitem__(self, name):
return self.dict[name] # get the whole object
# to get the value instead:
# return self.dict[name].v
def add(self, parameter):
self.dict[parameter.name] = parameter
# now comes the part that I don't know how to implement with property():
# It shoule be something like
# self.__dict__[parameter.name] = property(...) ?
col = Collection()
col.add(par1)
col.add(par2)
col['par1'] # gives the whole object
# Now here is what I would like to get:
# col.par1 -> should result like col['par1'].v
# col.par1 = 5 -> should result like col['par1'].v = 5
Other questions that I put to understand property():
Why do managed attributes just work for class attributes and not for instance attributes in python?
How can I assign a new class attribute via __dict__ in python?

Look at built-in functions getattr and setattr. You'll probably be a lot happier.

Using the same get/set functions for both classes forces you into an ugly hack with the argument list. Very sketchy, this is how I would do it:
In class SingleParameter, define get and set as usual:
def get(self):
return self._s
def set(self, value):
self._s = value
In class Collection, you cannot know the information until you create the property, so you define the metaset/metaget function and particularize them only later with a lambda function:
def metaget(self, par):
return par.s
def metaset(self, value, par):
par.s = value
def add(self, par):
self[par.name] = par
setattr(Collection, par.name,
property(
fget=lambda x : Collection.metaget(x, par),
fset=lambda x, y : Collection.metaset(x,y, par))

Properties are meant to dynamically evaluate attributes or to make them read-only. What you need is customizing attribute access. __getattr__ and __setattr__ do that really fine, and there's also __getattribute__ if __getattr__ is not enough.
See Python docs on customizing attribute access for details.

Have you looked at the traits package? It seems that you are reinventing the wheel here with your parameter classes. Traits also have additional features that might be useful for your type of application (incidently I know a person that happily uses traits in neural simulations).

Now I implemented a solution with set-/getattr:
class Collection(object):
...
def __setattr__(self, name, value):
if 'dict' in self.__dict__:
if name in self.dict:
self[name].v = value
else:
self.__dict__[name] = value
def __getattr__(self, name):
return self[name].v
There is one thing I quite don't like that much: The attributes are not in the __dict__. And if I have them there as well I would have a copy of the value - which can be dangerous...

Finally I succeded to implement the classes with property(). Thanks a lot for the advice. It took me quite a bit to work it out - but I can promise you that this exercise helps you to understand better pythons OOP.
I implemented it also with __getattr__ and __setattr__ but still don't know the advantages and disadvantages to the property-solution. But this seems to be worth another question. The property-solutions seems to be quit clean.
So here is the code:
class SingleParameter(object):
def __init__(self, name, default_value=0, unit='not specified'):
self.name = name
self.default_value = default_value
self.unit = unit
self.set(default_value)
def get(*args):
self = args[0]
print "get(): "
print args
return self._v
def set(*args):
print "set(): "
print args
self = args[0]
value = args[-1]
self._v = value
v = property(fget=get, fset=set, doc='value of parameter')
class Collection(dict):
# inheriting from dict saves the methods: __getitem__ and __init__
def add(self, par):
self[par.name] = par
# Now here comes the tricky part.
# (Note: this property call the get() and set() methods with one
# more argument than the property of SingleParameter)
setattr(Collection, par.name,
property(fget=par.get, fset=par.set))
# Applying the classes:
par1 = SingleParameter(name='par1', default_value=10, unit='mV')
par2 = SingleParameter(name='par2', default_value=20, unit='mA')
col = Collection()
col.add(par1)
col.add(par2)
# Setting parameter values:
par1.v = 13
col.par1 = 14
# Getting parameter values:
par1.v
col.par1
# checking identity:
par1.v is col.par1
# to access the whole object:
col['par1']
As I am new I am not sure how to move on:
how to treat follow up questions (like this itself):
get() is seems to be called twice - why?
oop-design: property vs. "__getattr__ & __setattr__" - when should I use what?
is it rude to check the own answer to the own question as accepted?
is it recommended to rename the title in order to put correlated questions or questions elaborated with the same example into the same context?
Other questions that I put to understand property():
Why do managed attributes just work for class attributes and not for instance attributes in python?
How can I assign a new class attribute via __dict__ in python?

I have a class that does something similar, but I did the following in the collection object:
setattr(self, par.name, par.v)