We Keep Coding

Python3 impossible to pass #property as decorator argument

I've implemented decorator that can receive extra arguments and want to use it with class methods. I want to pass #property as decorator argument, but instead of #property result I got this:
<property object at 0x7f50f5195230>
This is my decorator:
class Decorator(object):
def __init__(self, some_arg):
self.func = None
self.some_arg = some_arg
def __get__(self, instance, owner):
import functools
return functools.partial(self.__call__, instance)
def __call__(self, func):
self.func = func
def wrapper(*args, **kwargs):
return self._process_sync(*args, **kwargs)
return wrapper
def _process_sync(self, *args, **kwargs):
try:
print(self.some_arg)
return self.func(*args, **kwargs)
except Exception as e:
print(e)
return None
My test class:
class Test(object):
#property
def some_data(self):
return {'key': 'value'}
#Decorator(some_data)
def some_method(self):
print('method output')
return None
Usage:
test = Test()
test.some_method()
Two questions:
How to correctly pass property to receive #property result instead of <property object at 0x7f50f5195230>
Does it possible to pass class properties/methods to the decorator if they are below in code?

A property object is a descriptor. To get a value out of it, you need to call its __get__ method with an appropriate instance. Figuring out when to do that in your current code is not easy, since your Decorator object has a bunch of different roles. It's both a decorator factory (getting initialized with an argument in the #Decorator(x) line), and the decorator itself (getting called with the function to be decorated). You've given it a __get__ method, but I don't expect that to ever get used, since the instance of Decorator never gets assigned to a class variable (only the wrapper function that gets returned from __call__).
Anyway, here's a modified version where the Decorator handles almost all parts of the descriptor protocol itself:
class Decorator:
def __init__(self, arg):
self.arg = arg # this might be a descriptor, like a property or unbound method
def __call__(self, func):
self.func = func
return self # we still want to be the descriptor in the class
def __get__(self, instance, owner):
try:
arg = self.arg.__get__(instance, owner) # try to bind the arg to the instance
except AttributeError: # if it doesn't work, self.arg is not a descriptor, that's OK
arg = self.arg
def wrapper(*args, **kwargs): # this is our version of a bound method object
print(arg) # do something with the bound arg here
return self.func.__get__(instance, owner)(*args, **kwargs)
return wrapper

How to use a class method to decorate other class methods?

When designing a class, I find that in the class methods, there are repeated steps are called each time when invoke the class method. For example:
class Queue(object):
def __init__(self):
self.connection = Connection()
def create(self, name):
self.probe = self.connection.plug()
self.probe.create(name)
self.probe.unplug()
def delete(self, name):
self.probe = self.connection.plug()
self.probe.delete(name)
self.probe.unplug()
And there are many methods require the similar steps to 'plug' and 'unplug' the 'probe'. In this design we need to 'plug' and 'unplug' the 'probe' each time we perform the actions.
Thus I am thinking about the wrap those functions by decorator to make the code looking less repeated.
class Queue(object):
def __init__(self):
self.connection = Connection()
def _with_plug(self, fn):
def wrapper(*args, **kwargs):
self.probe = self.connection.plug()
fn(*args, **kwargs)
self.probe.unplug()
#_with_plug
def create(self, name):
self.probe.create(name)
#_with_plug
def delete(self, name):
self.probe.delete(name)
But this strategy is not working. How could I use a method in the class to decorate other methods to perform such actions before and after calling a method?

Seems like a bit of muddled arguments to me:
file deco.py, say
def _with_plug(fn): # decorator takes exactly one argument, the function to wrap
print("wrapping", fn.__name__)
def wrapper(self, *args, **kwds):
print("wrapper called")
self.probe = [self.connection, ".plug()"]
fn(self, *args, **kwds)
self.probe.append(".unplug()")
return wrapper # decorator must return the wrapped function
class Queue(object):
def __init__(self):
self.connection = "Connection()"
#_with_plug
def create(self, name):
self.probe.append("create(name)")
#_with_plug
def delete(self, name):
self.probe.append("delete(name)")
Check:
>>> import deco
wrapping create
wrapping delete
>>> q = deco.Queue()
>>> q.create("name")
wrapper called
>>> q.probe
['Connection()', '.plug()', 'create(name)', '.unplug()']
Observe that the decorator function is called at definition time of the to-be-wrapped function, i.e. before the class definition is completed and long before the first instance is created. Therefore you can't reference self in the way you tried.

You should define your decorator function outside of the class body and your decorator function should return the wrapped function in order for it to work. Something like:
def _with_plug(fn):
def wrapper(self, *args, **kwargs):
self.probe = self.connection.plug()
fn(self, *args, **kwargs)
self.probe.unplug()
return wrapper
class Queue(object):
def __init__(self):
self.connection = Connection()
#_with_plug
def create(self, name):
self.probe.create(name)
#_with_plug
def delete(self, name):
self.probe.delete(name)

How to pass self into a decorator?

How do I pass self.key below into the decorator?
class CacheMix(object):
def __init__(self, *args, **kwargs):
super(CacheMix, self).__init__(*args, **kwargs)
key_func = Constructor(
memoize_for_request=True,
params={'updated_at': self.key}
)
#cache_response(key_func=key_func)
def list(self, *args, **kwargs):
pass
class ListView(CacheMix, generics.ListCreateAPIView):
key = 'test_key'
I get the error:
'self' is not defined

Here's an example of doing it with a class decorator as I tried to describe to you in the comments. I filled-in a few undefined references in your question and used a super-simplified version of your cache_response function decorator, but hopefully this will convey the idea concretely enough for you to be able adapt it to your real code.
import inspect
import types
class Constructor(object):
def __init__(self, memoize_for_request=True, params=None):
self.memoize_for_request = memoize_for_request
self.params = params
def __call__(self):
def key_func():
print('key_func called with params:')
for k, v in self.params.items():
print(' {}: {!r}'.format(k, v))
key_func()
def cache_response(key_func):
def decorator(fn):
def decorated(*args, **kwargs):
key_func()
fn(*args, **kwargs)
return decorated
return decorator
def example_class_decorator(cls):
key_func = Constructor( # define key_func here using cls.key
memoize_for_request=True,
params={'updated_at': cls.key} # use decorated class's attribute
)
# create and apply cache_response decorator to marked methods
# (in Python 3 use types.FunctionType instead of types.UnboundMethodType)
decorator = cache_response(key_func)
for name, fn in inspect.getmembers(cls):
if isinstance(fn, types.UnboundMethodType) and hasattr(fn, 'marked'):
setattr(cls, name, decorator(fn))
return cls
def decorate_me(fn):
setattr(fn, 'marked', 1)
return fn
class CacheMix(object):
def __init__(self, *args, **kwargs):
super(CacheMix, self).__init__(*args, **kwargs)
#decorate_me
def list(self, *args, **kwargs):
classname = self.__class__.__name__
print('list() method of {} object called'.format(classname))
#example_class_decorator
class ListView(CacheMix):
key = 'test_key'
listview = ListView()
listview.list()
Output:
key_func called with params:
updated_at: 'test_key'
list() method of ListView object called

I just found out that if you write the decorator function like so:
def decorator(the_func):
#wraps(the_func)
def wrapper(*args, **kwargs):
the_func(*args, **kwargs)
return wrapper
and decorate any method which takes self as an argument, self will appear in args. Therefore you can do this:
from functools import wraps
class myClass:
def __init__(self):
self.myValue = "Hello"
def decorator(the_func):
#wraps(the_func)
def wrapper(*args, **kwargs):
print(args[0].myValue)
the_func(*args, **kwargs)
return wrapper
#decorator
def myFunction(self):
print("World")
Call it like you normally would
foo = myClass()
foo.myFunction()
and you should get
Hello
World

Python #property.setter

The basic way of creating decorators is
def my_decorator(f):
def _f(*args, **kwargs):
# do something using f
pass
return _f
#my_decorator
def f(...):
...
But that way you cannot define decorators like #property.setter, because the name of the property (and thus the name of the decorator) is different every time.
How is it #property.setter defined then? Is it possible to do something similar in Python, or is it built-in feature available only from C (implementation) level?

What you are looking for is something called a descriptor:
class Descriptor(object):
def __get__(self, instance, _type=None):
pass
def __set__(self, obj, value):
pass
You are free to implement things as you see fit. The property decorator is just an instance of a descriptor (more or less) and you can see how they'd implement it in the documents describing this item.
Here's an example:
class _Wrapper(object):
def __init__(self, caller, instance):
self.caller = caller
self.instance = instance
def __call__(self, *args, **kwargs):
print "I've been wrapped!"
return self.caller(self.instance, *args, **kwargs)
class Accouncer(object):
def __init__(self, method):
self.method = method
def __get__(self, instance, _type=None):
return _Wrapper(self.method, instance)
def vocal(func):
return Accouncer(func)
class Ha(object):
#vocal
def stuff(self):
return 1

Using the same decorator (with arguments) with functions and methods

I have been trying to create a decorator that can be used with both functions and methods in python. This on it's own is not that hard, but when creating a decorator that takes arguments, it seems to be.
class methods(object):
def __init__(self, *_methods):
self.methods = _methods
def __call__(self, func):
def inner(request, *args, **kwargs):
print request
return func(request, *args, **kwargs)
return inner
def __get__(self, obj, type=None):
if obj is None:
return self
new_func = self.func.__get__(obj, type)
return self.__class__(new_func)
The above code wraps the function/method correctly, but in the case of a method, the request argument is the instance it is operating on, not the first non-self argument.
Is there a way to tell if the decorator is being applied to a function instead of a method, and deal accordingly?

To expand on the __get__ approach. This can be generalized into a decorator decorator.
class _MethodDecoratorAdaptor(object):
def __init__(self, decorator, func):
self.decorator = decorator
self.func = func
def __call__(self, *args, **kwargs):
return self.decorator(self.func)(*args, **kwargs)
def __get__(self, instance, owner):
return self.decorator(self.func.__get__(instance, owner))
def auto_adapt_to_methods(decorator):
"""Allows you to use the same decorator on methods and functions,
hiding the self argument from the decorator."""
def adapt(func):
return _MethodDecoratorAdaptor(decorator, func)
return adapt
In this way you can just make your decorator automatically adapt to the conditions it is used in.
def allowed(*allowed_methods):
#auto_adapt_to_methods
def wrapper(func):
def wrapped(request):
if request not in allowed_methods:
raise ValueError("Invalid method %s" % request)
return func(request)
return wrapped
return wrapper
Notice that the wrapper function is called on all function calls, so don't do anything expensive there.
Usage of the decorator:
class Foo(object):
#allowed('GET', 'POST')
def do(self, request):
print "Request %s on %s" % (request, self)
#allowed('GET')
def do(request):
print "Plain request %s" % request
Foo().do('GET') # Works
Foo().do('POST') # Raises

The decorator is always applied to a function object -- have the decorator print the type of its argument and you'll be able to confirm that; and it should generally return a function object, too (which is already a decorator with the proper __get__!-) although there are exceptions to the latter.
I.e, in the code:
class X(object):
#deco
def f(self): pass
deco(f) is called within the class body, and, while you're still there, f is a function, not an instance of a method type. (The method is manufactured and returned in f's __get__ when later f is accessed as an attribute of X or an instance thereof).
Maybe you can better explain one toy use you'd want for your decorator, so we can be of more help...?
Edit: this goes for decorators with arguments, too, i.e.
class X(object):
#deco(23)
def f(self): pass
then it's deco(23)(f) that's called in the class body, f is still a function object when passed as the argument to whatever callable deco(23) returns, and that callable should still return a function object (generally -- with exceptions;-).

Since you're already defining a __get__ to use your decorator on the Bound Method, you could pass a flag telling it if it's being used on a method or function.
class methods(object):
def __init__(self, *_methods, called_on_method=False):
self.methods = _methods
self.called_on_method
def __call__(self, func):
if self.called_on_method:
def inner(self, request, *args, **kwargs):
print request
return func(request, *args, **kwargs)
else:
def inner(request, *args, **kwargs):
print request
return func(request, *args, **kwargs)
return inner
def __get__(self, obj, type=None):
if obj is None:
return self
new_func = self.func.__get__(obj, type)
return self.__class__(new_func, called_on_method=True)

Here is a general way I found to detect whether a decorated callable is a function or method:
import functools
class decorator(object):
def __init__(self, func):
self._func = func
self._obj = None
self._wrapped = None
def __call__(self, *args, **kwargs):
if not self._wrapped:
if self._obj:
self._wrapped = self._wrap_method(self._func)
self._wrapped = functools.partial(self._wrapped, self._obj)
else:
self._wrapped = self._wrap_function(self._func)
return self._wrapped(*args, **kwargs)
def __get__(self, obj, type=None):
self._obj = obj
return self
def _wrap_method(self, method):
#functools.wraps(method)
def inner(self, *args, **kwargs):
print('Method called on {}:'.format(type(self).__name__))
return method(self, *args, **kwargs)
return inner
def _wrap_function(self, function):
#functools.wraps(function)
def inner(*args, **kwargs):
print('Function called:')
return function(*args, **kwargs)
return inner
Example usage:
class Foo(object):
#decorator
def foo(self, foo, bar):
print(foo, bar)
#decorator
def foo(foo, bar):
print(foo, bar)
foo(12, bar=42) # Function called: 12 42
foo(12, 42) # Function called: 12 42
obj = Foo()
obj.foo(12, bar=42) # Method called on Foo: 12 42
obj.foo(12, 42) # Method called on Foo: 12 42

A partial (specific) solution I have come up with relies on exception handling. I am attempting to create a decorator to only allow certain HttpRequest methods, but make it work with both functions that are views, and methods that are views.
So, this class will do what I want:
class methods(object):
def __init__(self, *_methods):
self.methods = _methods
def __call__(self, func):
#wraps(func)
def inner(*args, **kwargs):
try:
if args[0].method in self.methods:
return func(*args, **kwargs)
except AttributeError:
if args[1].method in self.methods:
return func(*args, **kwargs)
return HttpResponseMethodNotAllowed(self.methods)
return inner
Here are the two use cases: decorating a function:
#methods("GET")
def view_func(request, *args, **kwargs):
pass
and decorating methods of a class:
class ViewContainer(object):
# ...
#methods("GET", "PUT")
def object(self, request, pk, *args, **kwargs):
# stuff that needs a reference to self...
pass
Is there a better solution than to use exception handling?