I'm writing a class that wraps around an object of another class. The intend is to change some of its method behaviors while able to extend all of its other interfaces. I'm not using inheritance because the inner class object can die and outer class needs to be able to replace it with a live one without destroying itself.
so I have:
class Inner():
def foo(): pass
def goo(): pass
class Outer():
self.inner = InnerFactory(innerType)
def foo():
try:
self.inner.foo()
except:
del self.inner
self.inner = InnerFactory(innerType)
self.inner.foo()
The question is how to extend goo w/o explicitly rewrite as I may have tons of other such methods I'm not aware of.
Actually after reading some of the feedbacks below, I realized I'm not using the great function getattr. However, I don't quite follow why the suggestions below all seem to use such a complicated version. Why can't it be as simple as:
def __getattr__( self, name ):
if self.inner:
return getattr( self.inner, name )
else:
raise Exception( 'attribute %s not found' % name )
Something like the code below does what you want, but: 1) it is ugly; 2) it is not thread safe; 3) it falls into a loop until some method from Inner raises an exception (this is not due to the implementation but due to the initial idea posted); 4) some more reasons to avoid using it :)
class Inner:
def foo(self):
print "foo"
def bar(self):
print "bar"
class Outer:
def __init__(self):
self.inner = Inner()
def __getattr__(self, name):
ret = getattr(self.inner, name)
def wrapper(*args):
try:
ret(*args)
except:
del self.inner
self.inner = Inner()
updated = self.__getattr__(name)
updated(*args)
return wrapper
def blah(self):
print "Blah"
outer = Outer()
outer.foo()
outer.bar()
outer.blah()
outer.nosuchattr()
My solution is similar to #khachik plus some method caching.
Be careful it's easy to get into infinite loop with __ getattr__.
Also you may want to add threading locks if needed
Code is untested treat it as pseudo code.
class Outer(object):
def __init__(self):
self.method_cache = {}
self.methods_to_override = ['foo', 'goo']
def __getattr__(self, method_name):
if method_name in self.methods_to_override:
if method_name in self.method_cache:
return self.method_cache[method_name]
else:
def wrapper(*args, **kw):
wrapped = getattr(self.inner, method_name)
try:
return wrapped(*args, **kw)
except InnerDiedError:
self.inner = self.InnerFactory(innerType)
wrapped = getattr(self.inner, method_name)
return wrapped(*args, **kw)
self.method_cache[method_name] = wrapper
return wrapper
Related
I have a class Stuff that has several methods, some of which have some argument, let's call it argument_x. For example:
class Stuff:
def method_1(self, argument_x, **other_args):
pass
def method_2(self, argument_x, **other_args):
pass
def method_3(self, I_dont_have_argument_x):
pass
Now I want to subclass this class wrapping all methods that have argument_x in the same way. For example if I were to proceed by hand I would do:
class StuffWithConstantX(Stuff):
def __init__(self, argument_x_value):
super().__init__()
self._argument_x_value = argument_x_value
def method_1(self, **other_args):
super().method_1(argument_x=self._argument_x_value, **other_args)
def method_2(self, **other_args):
super().method_2(argument_x=self._argument_x_value, **other_args)
As method_3 does not have argument_x I leave it unchanged.
Is it possible to automate this? How?
Here's how you might define this as a wrapper, rather than a subclass:
class Stuff:
def method_1(self, argument_x, **other_args):
print("method 1:", argument_x)
def method_2(self, argument_x, **other_args):
print("method 2:", argument_x)
def method_3(self, i_dont_have_argument_x):
print("method 3:", i_dont_have_argument_x)
class StuffWithConstantX:
def __init__(self, argument_x_value) -> None:
self._stuff = Stuff()
self._argument_x = argument_x_value
def __getattr__(self, __name: str):
attr = getattr(self._stuff, __name)
if not callable(attr):
return attr
def wrapped(*args, **kwargs):
try:
return attr(argument_x=self._argument_x, *args, **kwargs)
except TypeError:
# Beware -- if there's a TypeError raised from attr itself,
# it will get run twice before the caller sees the exception.
# You can potentially work around this by closely inspecting
# either the exception or the attr object itself.
return attr(*args, **kwargs)
return wrapped
stuff = StuffWithConstantX("foo")
stuff.method_1()
stuff.method_2()
stuff.method_3("bar")
method 1: foo
method 2: foo
method 3: bar
As noted in the comments, this code is more or less impossible to statically typecheck, and I would not recommend actually using this pattern unless you have a really good reason.
Here's another way you could do it.
import inspect
import functools
class StuffWithConstantX(Stuff):
def __init__(self, argument_x_value):
super().__init__()
self._argument_x_value = argument_x_value
for func_name, func in inspect.getmembers(Stuff, inspect.isfunction):
arg_names = inspect.getfullargspec(func).args
if 'argument_x' in arg_names:
setattr(self, func_name, functools.partial(func, self=self, argument_x=self._argument_x_value))
Can one write something like:
class Test(object):
def _decorator(self, foo):
foo()
#self._decorator
def bar(self):
pass
This fails: self in #self is unknown
I also tried:
#Test._decorator(self)
which also fails: Test unknown
I would like to temporarily change some instance variables
in the decorator and then run the decorated method, before
changing them back.
Would something like this do what you need?
class Test(object):
def _decorator(foo):
def magic( self ) :
print "start magic"
foo( self )
print "end magic"
return magic
#_decorator
def bar( self ) :
print "normal call"
test = Test()
test.bar()
This avoids the call to self to access the decorator and leaves it hidden in the class namespace as a regular method.
>>> import stackoverflow
>>> test = stackoverflow.Test()
>>> test.bar()
start magic
normal call
end magic
>>>
edited to answer question in comments:
How to use the hidden decorator in another class
class Test(object):
def _decorator(foo):
def magic( self ) :
print "start magic"
foo( self )
print "end magic"
return magic
#_decorator
def bar( self ) :
print "normal call"
_decorator = staticmethod( _decorator )
class TestB( Test ):
#Test._decorator
def bar( self ):
print "override bar in"
super( TestB, self ).bar()
print "override bar out"
print "Normal:"
test = Test()
test.bar()
print
print "Inherited:"
b = TestB()
b.bar()
print
Output:
Normal:
start magic
normal call
end magic
Inherited:
start magic
override bar in
start magic
normal call
end magic
override bar out
end magic
What you're wanting to do isn't possible. Take, for instance, whether or not the code below looks valid:
class Test(object):
def _decorator(self, foo):
foo()
def bar(self):
pass
bar = self._decorator(bar)
It, of course, isn't valid since self isn't defined at that point. The same goes for Test as it won't be defined until the class itself is defined (which its in the process of). I'm showing you this code snippet because this is what your decorator snippet transforms into.
So, as you can see, accessing the instance in a decorator like that isn't really possible since decorators are applied during the definition of whatever function/method they are attached to and not during instantiation.
If you need class-level access, try this:
class Test(object):
#classmethod
def _decorator(cls, foo):
foo()
def bar(self):
pass
Test.bar = Test._decorator(Test.bar)
import functools
class Example:
def wrapper(func):
#functools.wraps(func)
def wrap(self, *args, **kwargs):
print("inside wrap")
return func(self, *args, **kwargs)
return wrap
#wrapper
def method(self):
print("METHOD")
wrapper = staticmethod(wrapper)
e = Example()
e.method()
This is one way to access(and have used) self from inside a decorator defined inside the same class:
class Thing(object):
def __init__(self, name):
self.name = name
def debug_name(function):
def debug_wrapper(*args):
self = args[0]
print 'self.name = ' + self.name
print 'running function {}()'.format(function.__name__)
function(*args)
print 'self.name = ' + self.name
return debug_wrapper
#debug_name
def set_name(self, new_name):
self.name = new_name
Output (tested on Python 2.7.10):
>>> a = Thing('A')
>>> a.name
'A'
>>> a.set_name('B')
self.name = A
running function set_name()
self.name = B
>>> a.name
'B'
The example above is silly, but it works.
Here's an expansion on Michael Speer's answer to take it a few steps further:
An instance method decorator which takes arguments and acts on a function with arguments and a return value.
class Test(object):
"Prints if x == y. Throws an error otherwise."
def __init__(self, x):
self.x = x
def _outer_decorator(y):
def _decorator(foo):
def magic(self, *args, **kwargs) :
print("start magic")
if self.x == y:
return foo(self, *args, **kwargs)
else:
raise ValueError("x ({}) != y ({})".format(self.x, y))
print("end magic")
return magic
return _decorator
#_outer_decorator(y=3)
def bar(self, *args, **kwargs) :
print("normal call")
print("args: {}".format(args))
print("kwargs: {}".format(kwargs))
return 27
And then
In [2]:
test = Test(3)
test.bar(
13,
'Test',
q=9,
lollipop=[1,2,3]
)
start magic
normal call
args: (13, 'Test')
kwargs: {'q': 9, 'lollipop': [1, 2, 3]}
Out[2]:
27
In [3]:
test = Test(4)
test.bar(
13,
'Test',
q=9,
lollipop=[1,2,3]
)
start magic
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-3-576146b3d37e> in <module>()
4 'Test',
5 q=9,
----> 6 lollipop=[1,2,3]
7 )
<ipython-input-1-428f22ac6c9b> in magic(self, *args, **kwargs)
11 return foo(self, *args, **kwargs)
12 else:
---> 13 raise ValueError("x ({}) != y ({})".format(self.x, y))
14 print("end magic")
15 return magic
ValueError: x (4) != y (3)
I found this question while researching a very similar problem. My solution is to split the problem into two parts. First, you need to capture the data that you want to associate with the class methods. In this case, handler_for will associate a Unix command with handler for that command's output.
class OutputAnalysis(object):
"analyze the output of diagnostic commands"
def handler_for(name):
"decorator to associate a function with a command"
def wrapper(func):
func.handler_for = name
return func
return wrapper
# associate mount_p with 'mount_-p.txt'
#handler_for('mount -p')
def mount_p(self, slurped):
pass
Now that we've associated some data with each class method, we need to gather that data and store it in a class attribute.
OutputAnalysis.cmd_handler = {}
for value in OutputAnalysis.__dict__.itervalues():
try:
OutputAnalysis.cmd_handler[value.handler_for] = value
except AttributeError:
pass
I use this type of decorator in some debugging situations, it allows overriding class properties by decorating, without having to find the calling function.
class myclass(object):
def __init__(self):
self.property = "HELLO"
#adecorator(property="GOODBYE")
def method(self):
print self.property
Here is the decorator code
class adecorator (object):
def __init__ (self, *args, **kwargs):
# store arguments passed to the decorator
self.args = args
self.kwargs = kwargs
def __call__(self, func):
def newf(*args, **kwargs):
#the 'self' for a method function is passed as args[0]
slf = args[0]
# replace and store the attributes
saved = {}
for k,v in self.kwargs.items():
if hasattr(slf, k):
saved[k] = getattr(slf,k)
setattr(slf, k, v)
# call the method
ret = func(*args, **kwargs)
#put things back
for k,v in saved.items():
setattr(slf, k, v)
return ret
newf.__doc__ = func.__doc__
return newf
Note: because I've used a class decorator you'll need to use #adecorator() with the brackets on to decorate functions, even if you don't pass any arguments to the decorator class constructor.
The simple way to do it.
All you need is to put the decorator method outside the class.
You can still use it inside.
def my_decorator(func):
#this is the key line. There's the aditional self parameter
def wrap(self, *args, **kwargs):
# you can use self here as if you were inside the class
return func(self, *args, **kwargs)
return wrap
class Test(object):
#my_decorator
def bar(self):
pass
Declare in inner class.
This solution is pretty solid and recommended.
class Test(object):
class Decorators(object):
#staticmethod
def decorator(foo):
def magic(self, *args, **kwargs) :
print("start magic")
foo(self, *args, **kwargs)
print("end magic")
return magic
#Decorators.decorator
def bar( self ) :
print("normal call")
test = Test()
test.bar()
The result:
>>> test = Test()
>>> test.bar()
start magic
normal call
end magic
>>>
Decorators seem better suited to modify the functionality of an entire object (including function objects) versus the functionality of an object method which in general will depend on instance attributes. For example:
def mod_bar(cls):
# returns modified class
def decorate(fcn):
# returns decorated function
def new_fcn(self):
print self.start_str
print fcn(self)
print self.end_str
return new_fcn
cls.bar = decorate(cls.bar)
return cls
#mod_bar
class Test(object):
def __init__(self):
self.start_str = "starting dec"
self.end_str = "ending dec"
def bar(self):
return "bar"
The output is:
>>> import Test
>>> a = Test()
>>> a.bar()
starting dec
bar
ending dec
I have a Implementation of Decorators that Might Help
import functools
import datetime
class Decorator(object):
def __init__(self):
pass
def execution_time(func):
#functools.wraps(func)
def wrap(self, *args, **kwargs):
""" Wrapper Function """
start = datetime.datetime.now()
Tem = func(self, *args, **kwargs)
end = datetime.datetime.now()
print("Exection Time:{}".format(end-start))
return Tem
return wrap
class Test(Decorator):
def __init__(self):
self._MethodName = Test.funca.__name__
#Decorator.execution_time
def funca(self):
print("Running Function : {}".format(self._MethodName))
return True
if __name__ == "__main__":
obj = Test()
data = obj.funca()
print(data)
You can decorate the decorator:
import decorator
class Test(object):
#decorator.decorator
def _decorator(foo, self):
foo(self)
#_decorator
def bar(self):
pass
This question already has answers here:
Attaching a decorator to all functions within a class
(11 answers)
Closed 5 years ago.
I have several classes and they have same implements name but difference realization. I want to decorate all methods in some classes but others not. I have thought about inheritance, but some classes have some methods do not need to be decorated. The problem is that I don't want to decorate methods one by one, some classes they need to be decorated by a same decorator, Is there any solution to fix it?
Your can start all method that required to be decorated with some prefix and then use something like this:
class Xobject(object):
def __init__(self, decorator):
for method_name in dir(self):
if method_name.startswith("dec_"):
attr = getattr(self, method_name)
wrapped = decorator(attr)
setattr(self, method_name, wrapped)
def dec_me_1(self):
print("In dec_me1")
return 0
def dec_me_2(self):
print("In dec_me2")
return 1
def decorator(func):
def wrapped(*args):
print("TEST")
return func(*args)
return wrapped
x = Xobject(decorator)
x.dec_me_1()
x.dec_me_2()
UPDATE:
You can decorate class by mean of function below. When using Python you should know that class in Python is also object so you could change it and pass it to the other function.
def decorator(func):
def wrapped(*args):
print("TEST")
return func(*args)
return wrapped
def decorate_object(p_object, decorator):
for method_name in dir(p_object):
if method_name.startswith("dec_"):
attr = getattr(p_object, method_name)
wrapped = decorator(attr)
setattr(p_object, method_name, wrapped)
decorate_object(Xobject, decorator)
x = Xobject()
x.dec_me_1()
x.dec_me_2()
Also your can decorate already instantiated object same way:
x = Xobject()
x.dec_me_1()
x.dec_me_2()
decorate_object(x, decorator)
x.dec_me_1()
x.dec_me_2()
I'm sure there are a few approaches to this, but the main leading options are:
Create a custom metaclass, where the __new__ method iterates across the attribute dictionary, identifies methods, and decorates them. See http://eli.thegreenplace.net/2011/08/14/python-metaclasses-by-example/ for an example of Python metaclass programming. Disadvantages: that may be more complex than we'd want to get into here.
Do the same in a regular class's __init__ method. Disadvantages: that only decorates instance methods and not class or static methods, and it's slower because it runs every time you create a new instance.
Do it outside the class:
class Foo(object):
def bar(self):
print 'bar'
for name, ref in vars(Foo):
if callable(ref): ...
Disadvantages: You only get one chance to do it right: at import time. Subclasses don't get modified.
Do it in a class-level decorator. Same disadvantages as doing it outside the class (I think).
At some point you have to be explicit about what gets wrapped and what doesn't.
If I've understood you correctly, I think you could do something like this:
def wrapper(func):
def inner(*args, **kwargs):
print "%s was called" func.__name__
return func(*args, **kwargs)
return inner
class A(object):
def foo(self):
print "foo called"
def bar(self):
print "BAR CALLED"
class B(A):
#wrapper
def foo(self):
super(B, self).foo()
class C(A):
#wrapper
def bar(self):
super(C, self).bar()
Stick = A()
Dave = B()
Jupiter = C()
Jupiter.foo() #prints "foo called"
Jupiter.bar() #prints "bar wrapped" and "BAR CALLED"
I have a subclass that adds graphics capabilities to a superclass that implements the algorithms. So, in addition to a few extra initialization functions, this subclass will only need to refresh the graphics after the execution of each algorithm-computing function in the superclass.
Classes:
class graph(algorithms):
... #initialization and refresh decorators
#refreshgraph
def algorithm1(self, *args, **kwargs):
return algorithms.algorithm1(self, *args, **kwargs)
#refreshgraph
def algorithm2(self, *args, **kwargs):
return algorithms.algorithm2(self, *args, **kwargs)
... #and so on
Is there an pythonic way to automatically decorate all the non-private methods defined in the superclass, such that if I add a new algorithm there I don't need to explicitly mention it in my subclass? I would also like to be able to explicitly exclude some of the superclass' methods.
The subclass always gets all the methods from the parent class(es) by default. If you wish to make emulate the behavior other languages use for privacy (eg the 'private' or 'protected' modifiers in C#) you have two options:
1) Python convention (and it's just a convention) is that methods with a single leading underscore in their names are not designed for access from outside the defining class.
2) Names with a double leading underscore are mangled in the bytecode so they aren't visible to other code under their own names. ParentClass.__private is visible inside ParentClass, but can only be accessed from outside ParentClass as ParentClass._ParentClass__private. (Great explanations here). Nothing in Python is truly private ;)
To override an inherited method just define the same name in a derived class. To call the parent class method inside the derived class you can do it as you did in your example, or using super:
def algorithm2(self, *args, **kwargs):
super(graph, self).algorithm2(self, *args, **kwargs)
# do other derived stuff here....
self.refresh()
This is ugly, but I think it does what you want, but without inheritance:
class DoAfter(object):
def __init__(self, obj, func):
self.obj = obj
self.func = func
def __getattribute__(self, attr, *a, **kw):
obj = object.__getattribute__(self, 'obj')
if attr in dir(obj):
x = getattr(obj, attr)
if callable(x):
def b(*a, **kw):
retval = x(*a, **kw)
self.func()
return retval
return b
else:
return x
else:
return object.__getattribute__(self, attr)
Use it like this:
>>> class A(object):
... def __init__(self):
... self.a = 1
...
... def boo(self, c):
... self.a += c
... return self.a
>>> def do_something():
... print 'a'
>>> a = A()
>>> print a.boo(1)
2
>>> print a.boo(2)
4
>>> b = DoAfter(a, do_something)
>>> print b.boo(1)
a
5
>>> print b.boo(2)
a
7
A increments a counter each time A.boo is called. DoAfter wraps A, so that any method in the instance a can be called as if it were a member of b. Note that every method is wrapped this way, so do_something() is called whenever a method is accessed.
This is barely tested, not recommended, and probably a bad idea. But, I think it does what you asked for.
EDIT: to do this with inheritance:
class graph(algorithms):
def refreshgraph(self):
print 'refreshgraph'
def __getattribute__(self, attr):
if attr in dir(algorithms):
x = algorithms.__getattribute__(self, attr)
if callable(x):
def wrapped(*a, **kw):
retval = x(*a, **kw)
self.refreshgraph()
return retval
return wrapped
else:
return x
else:
return object.__getattribute__(self, attr)
How can I modify a self variable with a decorator?
Ex.
class Foo:
def __init__(self,a):
self.a = a
self.li = []
def afunction(self):
pass
I want to add the function object afunction to the list self.li so I can call it in a list. Ex. Have a list of functions defined by the class. How would I do that?
Thanks
I don't think you need a decorator. Functions are first-class objects in Python:
class Foo:
def __init__(self,a):
self.a = a
self.li = [self.afunction]
def afunction(self):
pass
If your intention is to mark certain functions of a class as a special type so that you can identify them later for some other purpose, you could use a decorator, or you could just use a naming convention.
def marked(function):
function.marked = 1
return function
class MarkAware(object):
def run_marked(self, *args, **kwargs):
for name in dir(self):
meth = getattr(self, name)
if hasattr(meth, 'marked'):
meth(*args, **kwargs)
def foo(self):
pass
#marked
def bar(self):
pass
Alternative:
class NameConvention(object):
def run_batchable(self, *args, **kwargs):
for name in dir(self):
if name.startswith('batchable_'):
getattr(self, name)(*args, **kwargs)
def foo(self):
pass
def batchable_bar(self):
pass
As Lattyware explains in a comment to unutbu's answer, you can't directly do what you're asking, because any decorator on afunction will be run while the class itself is being created, not when each instance is created.
If all you really want is "a list of functions defined by the class", you don't need anything fancy at all for that. Just create that list in __init__:
def __init__(self, a):
self.a = a
self.li = [f for f in dir(self) if inspect.ismethod(f)]
If you want a list of certain specific functions, the easiest way is the way unutbu suggests, which still doesn't require a decorator.
If you want the decorator just to mark "this method should go into li", see sr2222's answer.
None of these are what you asked for, but they are probably what you want. There are a few ways to actually use a decorator to add the function to self.li, but they're all pretty horrible, and you probably don't want them. For example:
class Foo:
def __init__(self,a):
self.a = a
self.li = []
def mydecorator(f):
self.li.append(f)
return f
#mydecorator
def afunction(self):
print('a')
self.afunction = new.instancemethod(afunction, self, Foo)