I want to construct classes for use as decorators with the following principles intact:
It should be possible to stack multiple such class decorators on top off 1 function.
The resulting function name pointer should be indistinguishable from the same function without a decorator, save maybe for just which type/class it is.
Ordering off the decorators should not be relevant unless actually mandated by the decorators. Ie. independent decorators could be applied in any order.
This is for a Django project, and the specific case I am working on now the method needs 2 decorators, and to appear as a normal python function:
#AccessCheck
#AutoTemplate
def view(request, item_id) {}
#AutoTemplate changes the function so that instead of returning a HttpResponse, it just returns a dictionary for use in the context. A RequestContext is used, and the template name is inferred from the method name and module.
#AccessCheck adds additional checks on the user based on the item_id.
I am guessing it's just to get the constructor right and copy the appropriate attributes, but which attributes are these?
The following decorator won't work as I describe:
class NullDecl (object):
def __init__ (self, func):
self.func = func
def __call__ (self, * args):
return self.func (*args)
As demonstrated by the following code:
#NullDecl
#NullDecl
def decorated():
pass
def pure():
pass
# results in set(['func_closure', 'func_dict', '__get__', 'func_name',
# 'func_defaults', '__name__', 'func_code', 'func_doc', 'func_globals'])
print set(dir(pure)) - set(dir(decorated));
Additionally, try and add "print func.name" in the NullDecl constructor, and it will work for the first decorator, but not the second - as name will be missing.
Refined eduffy's answer a bit, and it seems to work pretty well:
class NullDecl (object):
def __init__ (self, func):
self.func = func
for n in set(dir(func)) - set(dir(self)):
setattr(self, n, getattr(func, n))
def __call__ (self, * args):
return self.func (*args)
def __repr__(self):
return self.func
A do-nothing decorator class would look like this:
class NullDecl (object):
def __init__ (self, func):
self.func = func
for name in set(dir(func)) - set(dir(self)):
setattr(self, name, getattr(func, name))
def __call__ (self, *args):
return self.func (*args)
And then you can apply it normally:
#NullDecl
def myFunc (x,y,z):
return (x+y)/z
The decorator module helps you writing signature-preserving decorators.
And the PythonDecoratorLibrary might provide useful examples for decorators.
To create a decorator that wraps functions in a matter that make them indistinguishable from the original function, use functools.wraps.
Example:
def mydecorator(func):
#functools.wraps(func):
def _mydecorator(*args, **kwargs):
do_something()
try:
return func(*args, **kwargs)
finally:
clean_up()
return _mydecorator
# ... and with parameters
def mydecorator(param1, param2):
def _mydecorator(func):
#functools.wraps(func)
def __mydecorator(*args, **kwargs):
do_something(param1, param2)
try:
return func(*args, **kwargs)
finally:
clean_up()
return __mydecorator
return _mydecorator
(my personal preference is to create decorators using functions, not classes)
The ordering of decorators is as follows:
#d1
#d2
def func():
pass
# is equivalent to
def func():
pass
func = d1(d2(func))
Related
I am trying to apply a class decorator to each of its methods. But staticmethod and classmethod are not working for me. I got this:
import functools
import re
def decor(cls):
def decorator(f):
if isinstance(f, type):
for attr in f.__dict__:
if callable(getattr(f, attr)) and not re.match(r"__\w*__", attr):
setattr(f, attr, decorator(getattr(f, attr)))
return f
#functools.wraps(f)
def wrapper(*args, **kwargs):
return f(*args, **kwargs)
return wrapper
return decorator(cls)
#decor
class MyClass:
def printer1(self, string):
print(string)
#classmethod
def printer2(cls, string):
print(string)
#staticmethod
def printer3(string):
print(string)
MyClass().printer1("foo") # this works
MyClass().printer2("foo") # this does not work
MyClass.printer2("foo") # this works
MyClass().printer3("foo") # this does not work
MyClass.printer3("foo") # this works
# the error is the following:
TypeError: printer() takes 1 positional argument but 2 were given
Basically every time I instantiate the class, I get the error when calling a staticmethod and a classmethod.
You don't seem to understand the working principle of staticmethod and classmethod. This is a link about the working principle of classmethod. The working principle of the staticmethod is similar to that of the classmethod. The difference is that when accessing the staticmethod, the class will not be bound to the function, but simply return the function.
There are two problems:
The decorator also wraps the class and returns a function instead of a class, which prevents staticmethod and classmethod from working properly.
For staticmethod and classmethod, getattr will get the result returned after the descriptor triggers the __get__ method, not necessarily the function itself. And here you only use wrapper to wrap, and do not restore them to the corresponding special methods.
Here is a working example:
def decorator(cls_or_func):
if isinstance(cls_or_func, type):
cls = cls_or_func
for name, attr in cls.__dict__.items():
if isinstance(attr, (staticmethod, classmethod)):
setattr(cls, name, type(attr)(decorator(attr.__func__)))
elif callable(attr) and not re.match(r"__\w*__", name):
setattr(cls, name, decorator(attr))
return cls
func = cls_or_func
#functools.wraps(func)
def wrapper(*args, **kwargs):
return func(*args, **kwargs)
return wrapper
I am reading a book whose name is python cookbook. In chapter 9.9, I don't understand the below code, especially in __init__(). It uses wraps(func)(self) in __init__(), but it doesn't assign wraps(func)(self) to self. Why self.__wrapped__ can get the original function in __call__()?
import types
from functools import wraps
class Profiled:
def __init__(self, func):
wraps(func)(self)
self.ncalls = 0
def __call__(self, *args, **kwargs):
self.ncalls += 1
return self.__wrapped__(*args, **kwargs)
def __get__(self, instance, cls):
if instance is None:
return self
else:
return types.MethodType(self, instance)
#Profiled
def add(x, y):
return x + y
print(add(1, 2))
print(add(1, 3))
print(add.ncalls)
print(add)
I changed it to self = wraps(func)(self), found that it also can work? Who can explain that? I also couldn't understand self in self = wraps(func)(self). What's "self"?
wraps is usually used as a decorator:
#wraps(some_other_func)
def func(...):
...
So that the decorated function has some of its metadata altered to make it look like it was defined with
def some_other_func(...)
Decorator syntax is just equivalent to
def func(...):
...
func = wraps(some_other_func)(func)
but when the value assigned to func is the same object that func originally referred to, the assignment isn't strictly necessary. That's not true for decorators in general, so the assignment is always made. But since you are calling the function returned by wraps(func) explicitly, you can omit the unnecessary assignment back to self.
How do class decorators for methods in classes work? Here is a sample of what I've done through some experimenting:
from functools import wraps
class PrintLog(object):
def __call__(self, func):
#wraps(func)
def wrapped(*args):
print('I am a log')
return func(*args)
return wrapped
class foo(object):
def __init__(self, rs: str) -> None:
self.ter = rs
#PrintLog()
def baz(self) -> None:
print('inside baz')
bar = foo('2')
print('running bar.baz()')
bar.baz()
And this works perfectly fine. However, I was under the impression that decorators do not need to be called with (), but when I remove the brackets from #PrintLog(), I get this error:
def baz(self) -> None:
TypeError: PrintLog() takes no arguments
Is there something I am missing/do not understand? I've also tried passing in a throwaway arg with __init__(), and it works.
class PrintLog(object):
def __init__(self, useless):
print(useless)
def __call__(self, func):
#wraps(func)
def wrapped(*args):
print('I am a log')
return func(*args)
return wrapped
class foo(object):
def __init__(self, rs: str) -> None:
self.ter = rs
#PrintLog("useless arg that I'm passing to __init__")
def baz(self) -> None:
print('inside baz')
Again, this works, but I don't want to pass any argument to the decorator.
tl;dr: This question in python 3.x.
Help appreciated!
Class decorators accept the function as a subject within the __init__ method (hence the log message), so your decorator code should look like:
class PrintLog(object):
def __init__(self, function):
self.function = function
def __call__(self):
#wraps(self.function)
def wrapped(*args):
print('I am a log')
return self.function(*args)
return wrapped
Sorry if this doesn’t work, I’m answering on my mobile device.
EDIT:
Okay so this is probably not what you want, but this is the way to do it:
from functools import update_wrapper, partial, wraps
class PrintLog(object):
def __init__(self, func):
update_wrapper(self, func)
self.func = func
def __get__(self, obj, objtype):
"""Support instance methods."""
return partial(self.__call__, obj)
def __call__(self, obj, *args, **kwargs):
#wraps(self.func)
def wrapped(*args):
print('I am a log')
return self.func(*args)
return wrapped(obj, *args)
class foo(object):
def __init__(self, rs: str) -> None:
self.ter = rs
#PrintLog
def baz(self) -> None:
print('inside baz')
bar = foo('2')
print('running bar.baz()')
bar.baz()
The decorator has to have the __get__ method defined because you're applying the decorator to an instance method. How would a descriptor have the context of the foo instance?
Ref: Decorating Python class methods - how do I pass the instance to the decorator?
There is a big picture you're missing.
#decorator
def foo(...):
function_definition
is almost identical (except for some internal mangling) to
temp = foo
foo = decorator(temp)
It doesn't matter what the decorator is, as long as it can act like a function.
Your example is equivalent to:
baz = PrintLog("useless thing")(<saved defn of baz>)
Since PrintLog is a class, PrintLog(...) creates an instance of PrintLog. That instance has a __call__ method, so it can act like a function.
Some decorators are designed to take arguments. Some decorators are designed not to take arguments. Some, like #lru_cache, are pieces of Python magic which look to see if the "argument" is a function (so the decorator is being used directly) or a number/None, so that it returns a function that then becomes the decorator.
I want to be able to call a method according to some standard format:
outputs = obj.meth(in_0, in_1, ...)
, where outputs is a tuple of arrays, and each input is an array.
However, in most instances, I only return one array, and don't want to be forced to return a tuple of length 1 just for the sake of the standard format. (My actual formatting problem is more complicated but lets stick with this explanation for now.)
I want to be able to define a class like:
class _SomeClass(object):
def __init__(self):
self._amount_to_add = 1
#single_return_format
def add_one(self, x):
return x+self._amount_to_add
And then be able to call it as follows:
obj = _SomeClass()
assert obj.add_one(3) == 4
assert obj.add_one.standard_format(3)==(4, )
Question is: how do I define the decorator to allow this behaviour?
I tried:
def single_return_format(fcn):
fcn.standard_format = lambda *args: (fcn(*args), )
return fcn
, but it fails on the line with the second assert with:
TypeError: add_one() takes exactly 2 arguments (1 given)
Because the add_one requires "self" as an argument, and the the object has not even been created yet at the time the decorator modifies the function.
So Stack, how can I do this?
Notes:
1) I know I could do this with base-classes and inheritance instead, but that becomes a problem when you have more than one method in the class that you want to decorate this way.
2) The actual problem comes from using theano - the standard format is outputs, updates = fcn(*inputs), but most functions don't return any updates, so you want to be able to define those functions in a natural way, but still have the option of calling them according to this standard interface.
That's indeed a problem, because the way the "bound" method is retrieved from the function doesn't consider this way.
I see two ways:
You could just wrap the function:
def single_return_format(fcn):
# TODO Do some functools.wraps here...
return lambda *args, **kwargs: (fcn(*args, **kwargs), )
No fooling around with .standard_format, but a mere replacement of the function. So the function can define itself as returning one value, but can only be called as returning the tuple.
If this is not what you want, you can define a class for decorating methods which overrides __get__ and does the wrapping in a "live fashion". Of course, it can as well redefine __call__ so that it is usable for (standalone, non-method) functions as well.
To get exactly what you want you'd have to write a non-data descriptor and a set of wrapper classes for your functions. The reason for this is that the process of getting functions from objects as methods is highly optimised and it's not possible to hijack this mechanism. Instead you have to write your own classes that simulate this mechanism -- which will slow down your code if you are making lots of small method calls.
The very best way I can think to get the desired functionality is not to use any of the methods that you describe, but rather write a wrapper function that you use when needed to call a normal function in the standard format. eg.
def vectorise(method, *args, **kwargs):
return tuple(method(arg, **kwargs) for arg in args)
obj = _SomeClass()
result = vectorise(obj.add_one, 1, 2, 3)
Indeed, this is how numpy takes functions that operate on one argument and turns them into a function that works on arrays.
import numpy
def add_one(x):
return x + 1
arr = numpy.vectorize(add_one)([1, 2, 3])
If you really, really want to use non-data descriptors then following will work. Be warned these method calls are considerably slower. On my computer a normal method call takes 188 nanoseconds versus 1.53 microseconds for a "simple" method call -- a ten-fold difference. And vectorise call takes half the time a standard_form call does. The vast majority of that time is the lookup of the methods. The actual method calls are quite fast.
class simple_form:
"""Allows a simple function to be called in a standard way."""
def __init__(self, func):
self.func = func
def __get__(self, instance, owner):
if instance is None:
return self.func
return SimpleFormMethod(self.func, instance)
class MethodBase:
"""Provides support for getting the string representation of methods."""
def __init__(self, func, instance):
self.func = func
self.instance = instance
def _format(self):
return "<bound {method_class} {obj_class}.{func} of {obj}>".format(
method_class=self.__class__.__name__,
obj_class=self.instance.__class__.__name__,
func=self.func.__name__,
obj=self.instance)
def __str__(self):
return self._format()
def __repr__(self):
return self._format()
class SimpleFormMethod(MethodBase):
def __call__(self, *args, **kwargs):
return self.func(self.instance, *args, **kwargs)
#property
def standard_form(self):
return StandardFormMethod(self.func, self.instance)
class StandardFormMethod(MethodBase):
def __call__(self, *args, **kwargs):
return tuple(self.func(self.instance, arg, **kwargs) for arg in args)
class Number(object):
def __init__(self, value):
self.value = value
def add_to(self, *values):
return tuple(val + self.value for val in values)
#simple_form
def divide_into(self, value):
return value / self.value
num = Number(2)
print("normal method access:", num.add_to, sep="\n")
print("simple form method access:", num.divide_into, sep="\n")
print("standard form method access:", num.divide_into.standard_form, sep="\n")
print("access to underlying function:", Number.divide_into, sep="\n")
print("simple example usage:", num.divide_into(3))
print("standard example usage:", num.divide_into.standard_form(*range(3)))
Dunes gave the correct answer. I've stripped it down to bare bones so that it solves the problem in the question. The stripped-down code is here:
class single_return_format(object):
def __init__(self, func):
self._func = func
def __get__(self, instance, owner):
return SimpleFormMethod(instance, self._func)
class SimpleFormMethod(object):
def __init__(self, instance, func):
self._instance = instance
self._func = func
def __call__(self, *args, **kwargs):
return self._func(self._instance, *args, **kwargs)
#property
def standard_format(self):
return lambda *args, **kwargs: (self._func(self._instance, *args, **kwargs), )
class _SomeClass(object):
def __init__(self):
self._amount_to_add = 1
#single_return_format
def add_one(self, x):
return x+self._amount_to_add
obj = _SomeClass()
assert obj.add_one(3) == 4
assert obj.add_one.standard_format(3) == (4, )
Assume I have some simple class
class TestClass:
def doSomething(self):
print 'Did something'
I would like to decorate the doSomething method, for example to count the number of calls
class SimpleDecorator(object):
def __init__(self,func):
self.func=func
self.count=0
def __get__(self,obj,objtype=None):
return MethodType(self,obj,objtype)
def __call__(self,*args,**kwargs):
self.count+=1
return self.func(*args,**kwargs)
Now this counts the number of calls to the decorated method, however I would like to have per-instance counter, such that after
foo1=TestClass()
foo1.doSomething()
foo2=TestClass()
foo1.doSomething.count is 1 and foo2.doSomething.count is 0. From what I understand, this is not possible using decorators. Is there some way to achieve such behaviour?
Utilize the fact that self (i.e. the object which the method is invoked on) is passed as a parameter to the method:
import functools
def counted(method):
#functools.wraps(method)
def wrapped(obj, *args, **kwargs):
if hasattr(obj, 'count'):
obj.count += 1
else:
obj.count = 1
return method(obj, *args, **kwargs)
return wrapped
In above code, we intercept the object as obj parameter of the decorated version of method. Usage of the decorator is pretty straightforward:
class Foo(object):
#counted
def do_something(self): pass
Wouldn't the first element of *args be the object the method is being invoked on? Can't you just store the count there?