We Keep Coding

Can I change the __call__ of a Python Object without rewriting the code?

Consider the following scenario in pseudo-python-code:
class Component_i: # same for i = 1,2,3,...
__call__(self, input):
return input
class MyClass:
def __init__(self):
self.comp_1 = Component_1()
self.comp_2 = Component_2()
self.comp_3 = Component_3()
def __call__(self, input):
return self.comp_3(self.comp_2(self.comp_1(input)))
A = MyClass()
Now i would like to do the following:
class Component_2B:
__call__(self, input, option = True):
if option:
return input
else:
return do_stuff_with_input(input)
A.comp_2 = Component_2B()
And then I would like to change the __call__ of A to also accept the optional input option, and this optional input to be passed to A.comp_2 whenever it is called. But to make things complicated, I would like to achieve this without rewriting the __call__.
Does anyone have an Idea?
Here is a MWE:
class MultiplyWithParameter:
def __init__(self, parameter: float):
self.p = parameter
def __call__(self, input: float):
return self.p * input
class ProcessDataClass:
def __init__(self, parameter: float):
self.mul = MultiplyWithParameter(parameter)
def __call__(self, input: float):
input = input * 5 + 4
input = self.mul(input)
return input - 3
DataProcessor = ProcessDataClass(2.5)
class OptionalMultiplyWithParameter:
def __init__(self, parameter: float):
self.p = parameter
def __call__(self, input: float, multiply: Bool = True):
if multiply:
return self.p * input
else:
return input
now i want to set
DataProcessor.mul = OptionalMultiplyWithParameter(DataProcessor.mul.p)
and use DataProcessor(4.23, False). This shall pass the False to DataProcessor.mul during the call. I want to do this without rewriting the __call__ of ProcessDataClass (otherwise would be simple).
Here is some Motivation:
Assume there is a package called coollibrary in the internet from which I would like to use function coolclassfactory. This function returns an Instance of CoolClass which has a range of attributes and a __call__. Some of its attributes might be of class ProcessWithParameters. When initialized, CoolClass loads parameters from the Internet. Now I would like to give my CoolClass Instance which i built with coolclassfactory as an attribute to MyClass. But I would then like to replace all attributes that my CoolClass Instance has and that are of class ProcessWithParameters with another class OptionalProcessWithParameters, which essentially allows at runtime to choose between two possible ways of how to process input with parameters. Since this runtime option is new, i have to add it during the call (i assume? - here is important this should be safe to be used in multiprocessing!). Now unfortunately because I use coolclassfactory and this returns one of many possible subclasses of CoolClass each with an individual __call__, i can not simply rewrite the __call__, because that changes depending on how coolclassfactory is called. Essentially I would have to rewrite all the calls of subclasses of CoolClass, which I am trying to avoid...

you can always subclass what ever class and rewrite/expand/adjust any part of the original to your liking and with the use of super() access the original functionality
from your example, something like this for example:
class ProcessDataClassOptional(ProcessDataClass):
def __init__(self, parameter: float):
super().__init__(parameter) #use the original init to do its thing
self._mul = OptionalMultiplyWithParameter(parameter) #our modification
def __call__(self, input: float, multiply: Bool = True):
self.mul = lambda x: self._mul(x, multiply) # now the original will use our modify mul
return super().__call__(input)
and now you can instead use the adjusted version
DataProcessor = ProcessDataClassOptional(2.5)
DataProcessor(4.23)
DataProcessor(4.23, False)
when used as the original there shouldn't be any change, but now you can use the new behavior with it too

How would I 'listen' to/decorate a setter from an imported class

I'm not sure whether this is a great approach to be using, but I'm not hugely experienced with Python so please accept my apologies. I've tried to do some research on this but other related questions have been given alternative problem-specific solutions - none of which apply to my specific case.
I have a class that handles the training/querying of my specific machine learning model. This algorithm is running on a remote sensor, various values are fed into the object which returns None if the algorithm isn't trained. Once trained, it returns either True or False depending on the classification assigned to new inputs. Occasionally, the class updates a couple of threshold parameters and I need to know when this occurs.
I am using sockets to pass messages from the remote sensor to my main server. I didn't want to complicate the ML algorithm class by filling it up with message passing code and so instead I've been handling this in a Main class that imports the "algorithm" class. I want the Main class to be able to determine when the threshold parameters are updated and report this back to the server.
class MyAlgorithmClass:
def feed_value(self):
....
class Main:
def __init__(self):
self._algorithm_data = MyAlgorithmClass()
self._sensor_data_queue = Queue()
def process_data(self):
while True:
sensor_value = self._sensor_data_queue.get()
result, value = self._algorithm_data.feed_value(sensor_value)
if result is None:
# value represents % training complete
self._socket.emit('training', value)
elif result is True:
# value represents % chance that input is categoryA
self._socket.emit('categoryA', value)
elif result is False:
...
My initial idea was to add a property to MyAlgorithmClass with a setter. I could then decorate this in my Main class so that every time the setter is called, I can use the value... for example:
class MyAlgorithmClass:
#property
def param1(self):
return self._param1
#param1.setter
def param1(self, value):
self._param1 = value
class Main:
def __init__(self):
self._algorithm_data = MyAlgorithmClass()
self._sensor_data_queue = Queue()
def watch_param1(func):
def inner(*args):
self._socket.emit('param1_updated', *args)
func(*args)
My problem now, is how do I decorate the self._algorithm_data.param1 setter with watch_param1? If I simply set self._algorithm_data.param1 = watch_param1 then I will just end up setting self._algorithm_data._param1 equal to my function which isn't what I want to do.
I could use getter/setter methods instead of a property, but this isn't very pythonic and as multiple people are modifying this code, I don't want the methods to be replaced/changed for properties by somebody else later on.
What is the best approach here? This is a small example but I will have slightly more complex examples of this later on and I don't want something that will cause overcomplication of the algorithm class. Obviously, another option is the Observer pattern but I'm not sure how appropriate it is here where I only have a single variable to monitor in some cases.
I'm really struggling to get a good solution put together so any advice would be much appreciated.
Thanks in advance,
Tom

Use descriptors. They let you customize attribute lookup, storage, and deletion in Python.
A simplified toy version of your code with descriptors looks something like:
class WatchedParam:
def __init__(self, name):
self.name = name
def __get__(self, instance, insttype=None):
print(f"{self.name} : value accessed")
return getattr(instance, '_' + self.name)
def __set__(self, instance, new_val):
print(f"{self.name} : value set")
setattr(instance, '_' + self.name, new_val)
class MyAlgorithmClass:
param1 = WatchedParam("param1")
param2 = WatchedParam("param2")
def __init__(self, param1, param2, param3):
self.param1 = param1
self.param2 = param2
self.param3 = param3
class Main:
def __init__(self):
self._data = MyAlgorithmClass(10, 20, 50)
m = Main()
m._data.param1 # calls WatchedParam.__get__
m._data.param2 = 100 # calls WatchedParam.__set__
The WatchedParam class is a descriptor and can be used in MyAlgorithmClass to specify the parameters that need to be monitored.

The solution I went for is as follows, using a 'Proxy' subclass which overrides the properties. Eventually, once I have a better understanding of the watched parameters, I won't need to watch them anymore. At this point I will be able to swap out the Proxy for the base class and continue using the code as normal.
class MyAlgorithmClassProxy(MyAlgorithmClass):
#property
def watch_param1(self):
return MyAlgorithmClass.watch_param1.fget(self)
#watch_param1.setter
def watch_param1(self, value):
self._socket.emit('param1_updated', *args)
MyAlgorithmClass.watch_param1.fset(self, value)

More efficient way of setting default method argument to instance attribute [duplicate]

I want to pass a default argument to an instance method using the value of an attribute of the instance:
class C:
def __init__(self, format):
self.format = format
def process(self, formatting=self.format):
print(formatting)
When trying that, I get the following error message:
NameError: name 'self' is not defined
I want the method to behave like this:
C("abc").process() # prints "abc"
C("abc").process("xyz") # prints "xyz"
What is the problem here, why does this not work? And how could I make this work?

You can't really define this as the default value, since the default value is evaluated when the method is defined which is before any instances exist. The usual pattern is to do something like this instead:
class C:
def __init__(self, format):
self.format = format
def process(self, formatting=None):
if formatting is None:
formatting = self.format
print(formatting)
self.format will only be used if formatting is None.
To demonstrate the point of how default values work, see this example:
def mk_default():
print("mk_default has been called!")
def myfun(foo=mk_default()):
print("myfun has been called.")
print("about to test functions")
myfun("testing")
myfun("testing again")
And the output here:
mk_default has been called!
about to test functions
myfun has been called.
myfun has been called.
Notice how mk_default was called only once, and that happened before the function was ever called!

In Python, the name self is not special. It's just a convention for the parameter name, which is why there is a self parameter in __init__. (Actually, __init__ is not very special either, and in particular it does not actually create the object... that's a longer story)
C("abc").process() creates a C instance, looks up the process method in the C class, and calls that method with the C instance as the first parameter. So it will end up in the self parameter if you provided it.
Even if you had that parameter, though, you would not be allowed to write something like def process(self, formatting = self.formatting), because self is not in scope yet at the point where you set the default value. In Python, the default value for a parameter is calculated when the function is compiled, and "stuck" to the function. (This is the same reason why, if you use a default like [], that list will remember changes between calls to the function.)
How could I make this work?
The traditional way is to use None as a default, and check for that value and replace it inside the function. You may find it is a little safer to make a special value for the purpose (an object instance is all you need, as long as you hide it so that the calling code does not use the same instance) instead of None. Either way, you should check for this value with is, not ==.

Since you want to use self.format as a default argument this implies that the method needs to be instance specific (i.e. there is no way to define this at class level). Instead you can define the specific method during the class' __init__ for example. This is where you have access to instance specific attributes.
One approach is to use functools.partial in order to obtain an updated (specific) version of the method:
from functools import partial
class C:
def __init__(self, format):
self.format = format
self.process = partial(self.process, formatting=self.format)
def process(self, formatting):
print(formatting)
c = C('default')
c.process()
# c.process('custom') # Doesn't work!
c.process(formatting='custom')
Note that with this approach you can only pass the corresponding argument by keyword, since if you provided it by position, this would create a conflict in partial.
Another approach is to define and set the method in __init__:
from types import MethodType
class C:
def __init__(self, format):
self.format = format
def process(self, formatting=self.format):
print(formatting)
self.process = MethodType(process, self)
c = C('test')
c.process()
c.process('custom')
c.process(formatting='custom')
This allows also passing the argument by position, however the method resolution order becomes less apparent (which can affect the IDE inspection for example, but I suppose there are IDE specific workarounds for that).
Another approach would be to create a custom type for these kind of "instance attribute defaults" together with a special decorator that performs the corresponding getattr argument filling:
import inspect
class Attribute:
def __init__(self, name):
self.name = name
def decorator(method):
signature = inspect.signature(method)
def wrapper(self, *args, **kwargs):
bound = signature.bind(*((self,) + args), **kwargs)
bound.apply_defaults()
bound.arguments.update({k: getattr(self, v.name) for k, v in bound.arguments.items()
if isinstance(v, Attribute)})
return method(*bound.args, **bound.kwargs)
return wrapper
class C:
def __init__(self, format):
self.format = format
#decorator
def process(self, formatting=Attribute('format')):
print(formatting)
c = C('test')
c.process()
c.process('custom')
c.process(formatting='custom')

You can't access self in the method definition. My workaround is this -
class Test:
def __init__(self):
self.default_v = 20
def test(self, v=None):
v = v or self.default_v
print(v)
Test().test()
> 20
Test().test(10)
> 10

"self" need to be pass as the first argument to any class functions if you want them to behave as non-static methods.
it refers to the object itself. You could not pass "self" as default argument as it's position is fix as first argument.
In your case instead of "formatting=self.format" use "formatting=None" and then assign value from code as below:
[EDIT]
class c:
def __init__(self, cformat):
self.cformat = cformat
def process(self, formatting=None):
print "Formating---",formatting
if formatting == None:
formatting = self.cformat
print formatting
return formatting
else:
print formatting
return formatting
c("abc").process() # prints "abc"
c("abc").process("xyz") # prints "xyz"
Note : do not use "format" as variable name, 'cause it is built-in function in python

Instead of creating a list of if-thens that span your default arguements, one can make use of a 'defaults' dictionary and create new instances of a class by using eval():
class foo():
def __init__(self,arg):
self.arg = arg
class bar():
def __init__(self,*args,**kwargs):
#default values are given in a dictionary
defaults = {'foo1':'foo()','foo2':'foo()'}
for key in defaults.keys():
#if key is passed through kwargs, use that value of that key
if key in kwargs: setattr(self,key,kwargs[key])
#if no key is not passed through kwargs
#create a new instance of the default value
else: setattr(self,key, eval(defaults[key]))
I throw this at the beginning of every class that instantiates another class as a default argument. It avoids python evaluating the default at compile... I would love a cleaner pythonic approach, but lo'.

How To Call A Class Function From A Variable

I need a class router (for lack of a better word). The router needs to instantiate a class & call a function of that class instance based on variables passed to it.
How do I properly define the class function?
How do I properly call the class function?
Example code:
class ClassWorker1:
def function_1(self):
print('1a')
def function_2(self):
print('2a')
def function_3(self):
print('3a')
class ClassWorker2:
def function_1(self):
print('1b')
def function_2(self):
print('2b')
def function_3(self):
print('3b')
class ClassRouter(object):
def __init__(self, class_name, class_function):
self.class_instance = class_name()
self.class_function = class_function
self.main()
def main(self):
# how should I call the class function here?
self.class_instance.class_function()
return
a = 1
b = 1
if a == 1:
class_name = ClassWorker1
else:
class_name = ClassWorker1
if b == 1:
# Strings won't work as class function definition
# I won't know the class at this point. I will only know
# the shared function name at this point.
# how could this class function be defined directly?
class_function = 'function_1'
elif b == 2:
class_function = 'function_2'
else:
class_function = 'function_3'
ClassRouter(class_name, class_function)

I need a class router (for lack of a better word).
Are you sure you need a class for this ?
The router needs to instantiate a class & call a function of that class instance
When it belongs to a class or instance, a function is usually named a "method". Not really important but it makes things clearer. Also, an "instance" is obviously always, by definition, an instance of a class ;)
How do I properly define the class function?
How do I properly call the class function?
Does the router really have to be a class ? But anyway...
There are a couple distinct issues here (I of course assume you need something that's generic enough).
The first one is that your class (the one that will be instanciated by the "router") constructor may need some args - position or named or both. If it's the router's duty to instanciate the class (but should it be ?), you'll have to pass those args (both position and named) to the router. And since your router has to be generic (else it's useless) you cannot explicitely name these args in your router's constructor.
Hopefully, Python has a way to "unpack" tuples (for position args) and dicts (for named args) when calling a function, using respectively the * and ** operators at call time, ie:
def somefunc(arg1, arg2, arg3="foo", arg4=None):
print arg1, arg2, arg3, arg4
args = ("one", "two", "three")
kwargs = {"arg4": "four"}
somefunc(*args, **kwargs)
This let you pass arguments to a function in a generic way.
So if you want your router to be in charge of instanciating the "target" class, you'll have to support this:
class Router(object):
def __init__(self, cls, clsargs=None, clskwargs=None):
if clsargs is None:
clsargs = ()
if clskwargs is None:
clskwargs = {}
self._obj = cls(*clsargs, **clskwargs)
class Worker(object):
def __init__(self, name):
self.name = name
print self.name
r = Router(Worker, clsargs=("foo",))
# or
r = Router(Worker, clskwargs={"name":"foo"})
Now note that at this point you really don't gain anything (except for more code) from having the router instanciating the Worker - since you need to have the Worker class and it's constructor's args to instanciate the router, you could as well just instanciate the Worker yourself and pass the Worker instance to the router:
Since you must have a reference to the class passed to the router (else you can't pass it ), you could as well
class Router(object):
def __init__(self, obj):
self._obj = obj
class Worker(object):
def __init__(self, name):
self.name = name
print self.name
r = Router(Worker("foo"))
# or
r = Router(Worker(name="foo"))
The cases where it would make sense to have the router instanciate the worker are:
1/ if the Worker's constructor arguments are not known when the router is instanciated and are to be passed later (which requires a distinct router method to pass those args)
2/ if the Worker's instanciation is very costly and you're not even sure you'll really need it, in which case you want to wait until the router's "main" method is called to instanciate the worker.
The second issue is "how do I get the worker's method by name". This one has already been answered by Lukas: you use getattr(obj, attrname).
The third issue is "if my worker method needs arguments, how do I pass them". This is the same problem as with the worker's constructor arguments, so the solution is obviously the same. Depending on the concrete use case, you'll have to pass those args either when instanciating the router or when calling it's "main" method.
wrt/ this "main" method, remember that you can define your own callable types by implementing the __call__ method, ie
class NotAFunc(object):
def __init__(self, wot):
self.wot = wot
def __call__(self, count):
print self.wot * count
notafunc = NotAFunc("wot ? ")
notafunc(42)
So it might make sense to use this as your router's "main" method
Now do you really need a router class at all ? Python functions are object on their own (so a function can take a function and/or return a function), and moreover act as closures (a closure is a function that "captures" part of the environment where it's been defined):
def route(instance, methodname, methargs=None, methkwargs=None):
method = getattr(instance, methodname)
if methargs is None:
methargs = ()
if methkwargs is None:
methkwargs = {}
def func():
return method(*methargs, **methkwargs)
return func
class Worker(object):
def __init__(self, name):
self.name = name
def work(self, count):
return [self.name for i in range(count)]
r = route(Worker("foo"), "work", (42,))
print r()
Note that while I kept your "router" term, most of what I described above are known patterns. You may want to search for "proxy", "proxy method", and (for the last exemple) "partial evaluation".

You are looking for dynamic attribute lookup.
class C:
def c1(self, x):
return 2*x
instance = C()
method_name = 'c1'
method = getattr(instance, method_name)
print(method(1)) # call method and print result

You'll need to override the __new__ method of your (new-style!) class.
class ClassRouter(object):
def __new__(self, class_name, *args):
if arg=="Class1":
new_instance = ClassWorker1(*args)
new_instance.method()
return new_instance
elif arg=="Class2":
return ClassWorker2(*args)

Python metaprogramming: automatically generate member functions

How do I write a function that adds a method to a class? I have:
class A:
def method(self):
def add_member(name):
self.new_method = def name...?
add_member("f1")
add_member("f2")
In order to answer what I'm trying to do. I'm trying to factor out some pyqt slots. I want to be able to call a function create_slider that will create a QSlider and a QLabel and create the slider handling code, and make the slider-handler update the text in the QLabel. Here's the slot that needs to be factored out:
def on_sample_slider(self, value):
self.samples = pow(4, value)
self.sample_label.setText('%d' % self.samples)
here's a method that generates some UI, but it would be nice to also have it generate the on_sample_slider method every time it is called:
def insert_labeled_slider(hbox, name, slider_target):
# name
hbox.addWidget(QLabel(name))
# label
label = QLabel()
label.setMinimumSize(40, 0)
hbox.addWidget(self.sample_label)
#slider
slider = QSlider(Qt.Horizontal)
slider.setRange(0, 6)
slider.setTracking(True)
slider.setPageStep(1)
hbox.addWidget(slider)
self.connect(self.sample_slider, SIGNAL('valueChanged(int)'),
self.on_sample_slider)
self.sample_slider.setValue(0)
return (label, slider)
Final code:
def attach_on_slider(obj, name, variable, label, base):
def on_slider(self, value):
variable = base**value
label.setText('%d' % variable)
# This next line creates a method from the function
# The first arg is the function and the second arg is the object
# upon which you want it to be a method.
method = types.MethodType(on_slider, obj)
obj.__dict__["on_slider_" + name] = method
return method
class A:
def insert_labeled_slider(hbox, name, label_name, variable):
# name
hbox.addWidget(QLabel(label_name))
# label
label = QLabel()
label.setMinimumSize(40, 0)
hbox.addWidget(label)
#slider
slider = QSlider(Qt.Horizontal)
slider.setRange(0, 6)
slider.setTracking(True)
slider.setPageStep(1)
hbox.addWidget(slider)
on_slider_method = attach_on_slider(self, name, variable, label, 4)
self.connect(slider, SIGNAL('valueChanged(int)'),
on_slider_method)
slider.setValue(0)
return (label, slider)

Here's an real example from your newly posted code:
import types
def attach_on_sample_slider(obj, base):
def on_sample_slider(self, value):
self.samples = base**value
self.sample_label.setText('%d' % self.samples)
# This next line creates a method from the function
# The first arg is the function and the second arg is the object
# upon which you want it to be a method.
obj.on_sample_slider = types.MethodType(on_sample_slider, obj)
You can now call it like
def some_method(self, foo):
attach_on_sample_slider(self, 4)
original post
Since you say the the member functions are identical, I would do it something like this
def make_method(name):
def method(self, whatever, args, go, here):
#whatever code goes here
method.__name__ = name
return method
class A(object):
method1 = make_method('method1')
method2 = make_method('method2')
Strictly speaking, passing in the name and setting the __name__ attribute on the new function isn't necessary but it can help with debugging. It's a little bit of duplication and can pay for itself. If you are going to skip that though, you might as well do
class A(object):
def method1(self, arg1, arg2):
#code goes here
method2 = method1
method3 = method1
This creates identical methods. Calling either of them will yield the same method.
The first form is more powerful because you can pass other arguments besides the name into make_method and have the different versions of the returned method access those parameters in closure so they work differently. Here's a stupid example with functions (works the same with methods):
def make_opener(filename):
def opener():
return open(filename)
return opener
open_config = make_opener('config.cfg')
open_log = make_opener('log.log')
Here, they're all essentially the same function but do slightly different things because they have access to the value of filename that they were created with. Closures are definitely something to look into if you're going to be doing a lot of this sort of thing.
There can be a lot more to this so if you have particular questions that this doesn't address, you should update your question.

Add bound methods to instantiated objects in Python tells this issue.