We Keep Coding

Class decorators for methods in classes

How do class decorators for methods in classes work? Here is a sample of what I've done through some experimenting:
from functools import wraps
class PrintLog(object):
def __call__(self, func):
#wraps(func)
def wrapped(*args):
print('I am a log')
return func(*args)
return wrapped
class foo(object):
def __init__(self, rs: str) -> None:
self.ter = rs
#PrintLog()
def baz(self) -> None:
print('inside baz')
bar = foo('2')
print('running bar.baz()')
bar.baz()
And this works perfectly fine. However, I was under the impression that decorators do not need to be called with (), but when I remove the brackets from #PrintLog(), I get this error:
def baz(self) -> None:
TypeError: PrintLog() takes no arguments
Is there something I am missing/do not understand? I've also tried passing in a throwaway arg with __init__(), and it works.
class PrintLog(object):
def __init__(self, useless):
print(useless)
def __call__(self, func):
#wraps(func)
def wrapped(*args):
print('I am a log')
return func(*args)
return wrapped
class foo(object):
def __init__(self, rs: str) -> None:
self.ter = rs
#PrintLog("useless arg that I'm passing to __init__")
def baz(self) -> None:
print('inside baz')
Again, this works, but I don't want to pass any argument to the decorator.
tl;dr: This question in python 3.x.
Help appreciated!

Class decorators accept the function as a subject within the __init__ method (hence the log message), so your decorator code should look like:
class PrintLog(object):
def __init__(self, function):
self.function = function
def __call__(self):
#wraps(self.function)
def wrapped(*args):
print('I am a log')
return self.function(*args)
return wrapped
Sorry if this doesn’t work, I’m answering on my mobile device.
EDIT:
Okay so this is probably not what you want, but this is the way to do it:
from functools import update_wrapper, partial, wraps
class PrintLog(object):
def __init__(self, func):
update_wrapper(self, func)
self.func = func
def __get__(self, obj, objtype):
"""Support instance methods."""
return partial(self.__call__, obj)
def __call__(self, obj, *args, **kwargs):
#wraps(self.func)
def wrapped(*args):
print('I am a log')
return self.func(*args)
return wrapped(obj, *args)
class foo(object):
def __init__(self, rs: str) -> None:
self.ter = rs
#PrintLog
def baz(self) -> None:
print('inside baz')
bar = foo('2')
print('running bar.baz()')
bar.baz()
The decorator has to have the __get__ method defined because you're applying the decorator to an instance method. How would a descriptor have the context of the foo instance?
Ref: Decorating Python class methods - how do I pass the instance to the decorator?

There is a big picture you're missing.
#decorator
def foo(...):
function_definition
is almost identical (except for some internal mangling) to
temp = foo
foo = decorator(temp)
It doesn't matter what the decorator is, as long as it can act like a function.
Your example is equivalent to:
baz = PrintLog("useless thing")(<saved defn of baz>)
Since PrintLog is a class, PrintLog(...) creates an instance of PrintLog. That instance has a __call__ method, so it can act like a function.
Some decorators are designed to take arguments. Some decorators are designed not to take arguments. Some, like #lru_cache, are pieces of Python magic which look to see if the "argument" is a function (so the decorator is being used directly) or a number/None, so that it returns a function that then becomes the decorator.

Registering methods with decorator at class level

I'd like to be able to register/return methods at a class level. The closest answer I could find was here: Auto-register class methods using decorator, except it was centered on a global register and I'm looking for something specific to the class per below.
Code:
class ExampleClass:
def get_reports(self):
# return list of all method names with #report decorator
pass
def report(self):
# decorator here
pass
#report
def report_x(self):
return
#report
def report_y(self):
return
def method_z(self):
pass
where I'd like ExampleClass.get_reports() to return ['report_x', 'report_y'].
Not all reports will be preceded with report_, so there is likely no way to just look at method names. I'm trying to figure out how to do this to apply to a variety of situations, so just looking for 'report_' does not work in this context.

You can declare a Reporter class like this and use an instance as a class property. I used the __call__ override to shorten the decorator, but you can name the function report and decorate methods as #report.report
class Reporter:
def __init__(self):
# Maintain a set of reported methods
self._reported = set()
def __call__(self, fn, *args, **kwargs):
# Register method
self._reported.add(fn.__name__)
def decorate(*args, **kwargs):
return fn(*args, **kwargs)
return decorate
class ExampleClass:
report = Reporter()
def get_reports(self):
# return list of all method names with #report decorator
return list(self.report._reported)
#report
def report_x(self):
return
#report
def report_y(self):
return
def method_z(self):
pass

This turns out to be similar to Mach_Zero's answer. So key differences, this returns the methods, not the method names, and the implementation of get_reports() is somewhat simpler with the use of __iter__.
Code:
class Reports:
def __init__(self):
self.reports = []
def __call__(self, func):
self.reports.append(func)
return func
def __iter__(self):
return iter(self.reports)
class ExampleClass:
report = Reports()
#classmethod
def get_reports(cls):
# return list of all method names with #report decorator
return list(cls.report)
#report
def report_x(self):
return
#report
def report_y(self):
return
def method_z(self):
pass
Test Code:
print(ExampleClass.get_reports())
Results:
[
<function ExampleClass.report_x at 0x000000000AF7B2F0>,
<function ExampleClass.report_y at 0x000000000AF7B378>
]

Using decorators to implement Observer Pattern in Python3

This question is not in general about the observer pattern. It is focused on the use of decorators in that pattern. The question is based on the answer of a similar question.
#!/usr/bin/env python3
class Observable:
"""
The object that need to be observed. Alternative names are 'Subject'.
In the most cases it is a data object.
"""
def __init__(self):
self._observers = []
def register_observer(self, callback):
self._observers.append(callback)
return callback
def _broadcast_observers(self, *args, **kwargs):
for callback in self._observers:
callback(*args, **kwargs)
class TheData(Observable):
"""
Example of a data class just for demonstration.
"""
def __init__(self, data):
Observable.__init__(self)
self._data = data
#property
def data(self):
return self._data
#data.setter
def data(self, data):
self._data = data
self._broadcast_observers()
class TheGUIElement:
"""
Example of a gui class (Widget) just for demonstration.
e. g. it could be a text field in GUI.
"""
def __init__(self, data):
self._data = data
#data.register_observer(self._data_updated)
self._redraw()
def _redraw(self):
print('in _redraw(): ' + data.data)
#Observable.register_observer
def _data_updated(self, **kwargs):
"""
This is the callback that is called by the Observable if the
data changed.
"""
print('in _data_updated() - kwargs: {}'.format(kwargs))
self._redraw()
if __name__ == '__main__':
data = TheData('DATA')
gui = TheGUIElement(data)
data.data = 'SECOND DATA'
This code doesn't work because of this error.
Traceback (most recent call last):
File "./o.py", line 42, in <module>
class TheGUIElement:
File "./o.py", line 55, in TheGUIElement
#Observable.register_observer
TypeError: register_observer() missing 1 required positional argument: 'callback'
It is unclear to me how to use a decorator for to register the observers (e.g. TheGUIElement).

To register the callback, you need to have an actual object. In your code, how is #Observable.register_observer supposed to find which instance is should register on?
Please drop that Observable thing that's a javaism, cumbersome in python.
Look at this.
#!/usr/bin/env python
class SomeData(object):
def __init__(self, value):
self.callbacks = []
self.foo = value
def register(self, callback):
self.callbacks.append(callback)
return callback
def notify(self, *args, **kwargs):
for callback in self.callbacks:
callback(self, *args, **kwargs)
class SomeGUI(object):
def redraw(self, obj, key, newvalue):
print('redrawing %s with value %s' % (self, newvalue))
if __name__ == '__main__':
my_data = SomeData(42)
# Register some function using decorator syntax
#my_data.register
def print_it(obj, key, value):
print('Key %s changed to %s' % (key, value))
# Register the SomeGUI element
my_gui = SomeGUI()
my_data.register(my_gui.redraw)
# Try changing it. Note my_data is dumb for now, notify manually.
my_data.foo = 10
my_data.notify("foo", 10)
I intentionally removed automatic notifications to illustrate registration by itself.
Let's add it back. But there is no point using that Observable class. Let's make it lighter, simply defining an event class.
#!/usr/bin/env python3
class Event(object):
def __init__(self):
self.callbacks = []
def notify(self, *args, **kwargs):
for callback in self.callbacks:
callback(*args, **kwargs)
def register(self, callback):
self.callbacks.append(callback)
return callback
class SomeData(object):
def __init__(self, foo):
self.changed = Event()
self._foo = foo
#property
def foo(self):
return self._foo
#foo.setter
def foo(self, value):
self._foo = value
self.changed.notify(self, 'foo', value)
class SomeGUI(object):
def redraw(self, obj, key, newvalue):
print('redrawing %s with value %s' % (self, newvalue))
if __name__ == '__main__':
my_data = SomeData(42)
# Register some function using decorator syntax
#my_data.changed.register
def print_it(obj, key, value):
print('Key %s changed to %s' % (key, value))
# Register the SomeGUI element
my_gui = SomeGUI()
my_data.changed.register(my_gui.redraw)
# Try changing it.
my_data.foo = 10
As you probably noted now, the decorator syntax is useful in those circumstances:
You have a single registry. Either a singleton or the class itself class are first-order objects, and most are singletons.
You dynamically define the function and register it as you go.
Now, those manual getters/setters you have are cumbersome as well, if you have many why not factor them out?
#!/usr/bin/env python3
class Event(object):
def __init__(self):
self.callbacks = []
def notify(self, *args, **kwargs):
for callback in self.callbacks:
callback(*args, **kwargs)
def register(self, callback):
self.callbacks.append(callback)
return callback
#classmethod
def watched_property(cls, event_name, key):
actual_key = '_%s' % key
def getter(obj):
return getattr(obj, actual_key)
def setter(obj, value):
event = getattr(obj, event_name)
setattr(obj, actual_key, value)
event.notify(obj, key, value)
return property(fget=getter, fset=setter)
class SomeData(object):
foo = Event.watched_property('changed', 'foo')
def __init__(self, foo):
self.changed = Event()
self.foo = foo
class SomeGUI(object):
def redraw(self, obj, key, newvalue):
print('redrawing %s with value %s' % (self, newvalue))
if __name__ == '__main__':
my_data = SomeData(42)
# Register some function using decorator syntax
#my_data.changed.register
def print_it(obj, key, value):
print('Key %s changed to %s' % (key, value))
# Register the SomeGUI element
my_gui = SomeGUI()
my_data.changed.register(my_gui.redraw)
# Try changing it.
my_data.foo = 10
For reference, all three programs output the exact same thing:
$ python3 test.py
Key foo changed to 10
redrawing <__main__.SomeGUI object at 0x7f9a90d55fd0> with value 10

Even though the thread is kinda old (probably the problem is already solved), I would like to share a solution of mine to the "Decorated Observer Pattern" problem:
https://pypi.org/project/notifyr/
I created a package that implements decorators which add the observer-observed methods/attributes to python classes. I managed to use the package in a Django project too, but with a few adaptations (the .observers attribute is not persisted in the database, so I had to load the list of observers into it every time I expected to notify them).
Here is an implementation example:
Original Code:
class Dog(object):
def __init__(self, name):
self.name = name
def bark(self):
print('Woof')
def sleep(self):
print(self.name, 'is now asleep: ZZzzzzZzzZ...')
class Person(object):
def __init__(self, name):
self.name = name
def educate_dog(self, dog):
print(self.name + ':','Sleep,', dog.name)
dog.sleep()
Suppose we want a person to educate a dog every time the animal barks:
from notifyr.agents import observed, observer
from notifyr.functions import target
#observed
class Dog(object):
def __init__(self, name):
self.name = name
#target
def bark(self):
print('Woof')
def sleep(self):
print(self.name, 'is now asleep: ZZzzzzZzzZ...')
#observer('educate_dog')
class Person(object):
def __init__(self, name):
self.name = name
def educate_dog(self, dog):
print(self.name + ':','Sleep,', dog.name)
dog.sleep()
Given the decorated classes, it is possible to achieve the following result:
d = Dog('Tobby')
p = Person('Victor')
d.attach(p) # Victor is now observing Tobby
d.bark()
# Woof
# Victor: Sleep, Tobby
# Tobby is now asleep: ZZzzzzZzzZ...
The package is still very primitive, but it presents a working solution to this type of situation.

I was recently looking for something similar and here's what I came up with. It works by intercepting the __setattr__ method -- a useful stunt I plan on keeping in my pocket for later.
def watchableClass(cls):
"""
Class Decorator!
* If the class has a "dirty" member variable, then it will be
automatically set whenever any class value changes
* If the class has an "onChanged()" method, it will be called
automatically whenever any class value changes
* All this only takes place if the value is different from what it was
that is, if myObject.x is already 10 and you set myObject.x=10
nothing happens
* DOES NOT work with getter/setter functions. But then, you are
already in a function, so do what you want!
EXAMPLE:
#watchableClass
class MyClass:
def __init__(self):
self.dirty=False
def onChanged(self):
print('class has changed')
"""
if hasattr(cls,'__setattr__'):
cls.__setattr_unwatched__=cls.__setattr__
cls.__setattr__=_setObjValueWatchedCascade
else:
cls.__setattr__=_setObjValueWatched
return cls
def _setObjValueWatched(ob,k,v):
"""
called when an object value is set
"""
different=not k in ob.__dict__ or ob.__dict__[k]!=v
if different:
ob.__dict__[k]=v
if k not in ('dirty'):
if hasattr(ob,'dirty'):
ob.dirty=True
if hasattr(ob,'onChanged'):
ob.onChanged()
def _setObjValueWatchedCascade(ob,k,v):
"""
called when an object value is set
IF the class had its own __setattr__ member defined!
"""
different=not k in ob.__dict__ or ob.__dict__[k]!=v
ob.__setattr_unwatched__(k,v)
if different:
if k not in ('dirty'):
if hasattr(ob,'dirty'):
ob.dirty=True
if hasattr(ob,'onChanged'):
ob.onChanged()

Storing state between decorators in python

I'll start by saying, I have a suspicion this is a solution that could be solved with a functional programming approach, but I don't know nearly enough of the concepts (but have been trying).
I've based my current solution on:
https://pythonconquerstheuniverse.wordpress.com/2012/04/29/python-decorators/
http://www.brianholdefehr.com/decorators-and-functional-python
https://github.com/mitsuhiko/click/blob/master/click/decorators.py
In the interest of learning (that's all this is!) how to build decorators, I decided to make a simple cache decorator.
But I get stuck in a loop where, I try to encapsulate the function I'm wrapping in a class, but every time I call the function I've wrapped, I call __call__ in wrapping class and so on ad infinitum.
I think I could have a nest of closures between the chain decorators, but I don't know how to collect all my variables in one scope.
I appreciate I could put all my arguments in a single decorator call, but my intention here is to learn how to chain decorators and store state between them.
Can anyone suggest a way (or ammend my way) to store state between chained decorators?
My intended design was:
# main.py
import http.client
from cache import cache
#cache.keys('domain', 'url')
#cache.lifetime(3600)
def make_http_request(domain,url='/'):
conn = httplib.HTTPConnection(domain)
conn.request("GET",url)
return conn.getresponse()
if __name__ == '__main__':
print(make_http_request('http://example.com/'))
with cache.py looking like
import hashlib
import os
import inspect
__author__ = 'drews'
def expand(path):
return os.path.abspath(os.path.expanduser(path))
class CacheManager():
"""Decorator to take the result and store it in a a file. If the result is needed again, then the file result is returned"""
def __init__(self, function, function_arg_name):
self.lifetime = 3600
self.cache_keys = None
self.cache_path = '~/.decorator_cache/'
self.f = function
self.arg_names = function_arg_name
def __call__(self, *args, **kwargs):
if len(args) > 0:
arg_names = self.arg_names
if 'self' in arg_names:
arg_names.remove('self')
key_args = dict(zip(arg_names, args))
key_args.update(kwargs)
else:
key_args = kwargs
self._initialise(cache_path=expand(self.cache_path))
key = self._build_key(key_args)
if self.key_exists(key):
result = self.get_key(key)
else:
result = self.f()
self.set_key(key, result)
return result
def _build_key(self, key_elements):
m = hashlib.md5()
for key in self.cache_keys:
m.update(key_elements[key].encode('utf-8'))
return m.hexdigest()
def _initialise(self, cache_path):
def initialise_path(path):
if not os.path.isdir(path):
(head, tail) = os.path.split(path)
if not os.path.isdir(head):
initialise_path(head)
os.mkdir(path)
initialise_path(cache_path)
def key_exists(self, key):
path = os.path.join(expand(self.cache_path), key)
return os.path.exists(path)
class CacheDefinitionDecorator(object):
def __init__(self, *args, **kwargs):
self.d_args = args
class CacheKeyDefinitionDecorator(CacheDefinitionDecorator):
def __call__(self, func, *args, **kwargs):
if not isinstance(func, CacheManager):
func = CacheManager(func,inspect.getargspec(func)[0])
func.cache_keys = self.d_args
return func
class CacheLifetimeDefintionDecorator(CacheDefinitionDecorator):
def __call__(self, func, *args, **kwargs):
if not isinstance(func, CacheManager):
func = CacheManager(func,inspect.getargspec(func)[0])
func.lifetime = self.d_args[0]
return func
class CacheStruct(object):
def __init__(self, **kwargs):
for item in kwargs:
setattr(self, item, kwargs[item])
cache = CacheStruct(
keys=CacheKeyDefinitionDecorator,
lifetime=CacheLifetimeDefintionDecorator
)

How to write container/wrapper class that adapts to member class interface?

I'm writing a class that wraps around an object of another class. The intend is to change some of its method behaviors while able to extend all of its other interfaces. I'm not using inheritance because the inner class object can die and outer class needs to be able to replace it with a live one without destroying itself.
so I have:
class Inner():
def foo(): pass
def goo(): pass
class Outer():
self.inner = InnerFactory(innerType)
def foo():
try:
self.inner.foo()
except:
del self.inner
self.inner = InnerFactory(innerType)
self.inner.foo()
The question is how to extend goo w/o explicitly rewrite as I may have tons of other such methods I'm not aware of.
Actually after reading some of the feedbacks below, I realized I'm not using the great function getattr. However, I don't quite follow why the suggestions below all seem to use such a complicated version. Why can't it be as simple as:
def __getattr__( self, name ):
if self.inner:
return getattr( self.inner, name )
else:
raise Exception( 'attribute %s not found' % name )

Something like the code below does what you want, but: 1) it is ugly; 2) it is not thread safe; 3) it falls into a loop until some method from Inner raises an exception (this is not due to the implementation but due to the initial idea posted); 4) some more reasons to avoid using it :)
class Inner:
def foo(self):
print "foo"
def bar(self):
print "bar"
class Outer:
def __init__(self):
self.inner = Inner()
def __getattr__(self, name):
ret = getattr(self.inner, name)
def wrapper(*args):
try:
ret(*args)
except:
del self.inner
self.inner = Inner()
updated = self.__getattr__(name)
updated(*args)
return wrapper
def blah(self):
print "Blah"
outer = Outer()
outer.foo()
outer.bar()
outer.blah()
outer.nosuchattr()

My solution is similar to #khachik plus some method caching.
Be careful it's easy to get into infinite loop with __ getattr__.
Also you may want to add threading locks if needed
Code is untested treat it as pseudo code.
class Outer(object):
def __init__(self):
self.method_cache = {}
self.methods_to_override = ['foo', 'goo']
def __getattr__(self, method_name):
if method_name in self.methods_to_override:
if method_name in self.method_cache:
return self.method_cache[method_name]
else:
def wrapper(*args, **kw):
wrapped = getattr(self.inner, method_name)
try:
return wrapped(*args, **kw)
except InnerDiedError:
self.inner = self.InnerFactory(innerType)
wrapped = getattr(self.inner, method_name)
return wrapped(*args, **kw)
self.method_cache[method_name] = wrapper
return wrapper