I want to make a Python decorator that can take an argument (the name of an argument in the function it is decorating) and use that argument to find the value of the argument in the function. I know that sounds super confusing, ha! But what I want to do would look like this (it's for a Flask web application):
This is what I have right now:
#app.route('/admin/courses/<int:course_id>')
def view_course(course_id):
... view code here
And I want to do something like this:
#app.route('/admin/courses/<int:course_id>')
#access_course_permission(course_id)
def view_course(course_id):
... view code here
Right now I know I can do something like:
def access_course_permission(f):
#wraps(f)
def decorated_function(*args, **kwargs):
# check args[0] right here to find the course ID
return f(*args, **kwargs)
return decorated_function
which works great as long as course_id is always the first parameter. However, it isn't. That's why I want to be able to specify the name.
If this is not possible, I guess I could just pass the index of the parameter to the decorator, but that isn't very nice...
You can use the inspect.getfullargspec() function to access the names used in a function:
try:
# Python 3
from inspect import getfullargspec
except ImportError:
# Python 2, use inspect.getargspec instead
# this is the same function really, without support for annotations
# and keyword-only arguments
from inspect import getargspec as getfullargspec
from functools import wraps
def access_course_permission(argument_name):
def decorator(f):
argspec = getfullargspec(f)
argument_index = argspec.args.index(argument_name)
#wraps(f)
def wrapper(*args, **kwargs):
try:
value = args[argument_index]
except IndexError:
value = kwargs[argument_name]
# do something with value
return f(*args, **kwargs)
return wrapper
return decorator
The above finds out at what index your specific argument is positioned; this covers both positional and keyword arguments (because in Python, you can pass in a value for a keyword argument by position too).
Note however that for your specific example, Flask will call view_course with course_id as a keyword argument, so using kwargs[argument_name] would suffice.
You'll have to pass in a string to name that argument:
#app.route('/admin/courses/<int:course_id>')
#access_course_permission('course_id')
def view_course(course_id):
# ... view code here
Note however, that in Flask you could just access request.view_args, without the need to parse this information out of the function arguments:
course_id = requests.view_args[argument_name]
Related
I'm trying to code a method from a class that uses a decorator from another class. The problem is that I need information stored in the Class that contains the decorator (ClassWithDecorator.decorator_param). To achieve that I'm using partial, injecting self as the first argument, but when I do that the self, from the class that uses the decorator " gets lost" somehow and I end up getting an error. Note that this does not happen if I remove partial() from my_decorator() and "self" will be correctly stored inside *args.
See the code sample:
from functools import partial
class ClassWithDecorator:
def __init__(self):
self.decorator_param = "PARAM"
def my_decorator(self, decorated_func):
def my_callable(ClassWithDecorator_instance, *args, **kwargs):
# Do something with decorator_param
print(ClassWithDecorator_instance.decorator_param)
return decorated_func(*args, **kwargs)
return partial(my_callable, self)
decorator_instance = ClassWithDecorator()
class WillCallDecorator:
def __init__(self):
self.other_param = "WillCallDecorator variable"
#decorator_instance.my_decorator
def decorated_method(self):
pass
WillCallDecorator().decorated_method()
I get
PARAM
Traceback (most recent call last):
File "****/decorator.py", line 32, in <module>
WillCallDecorator().decorated_method()
File "****/decorator.py", line 12, in my_callable
return decorated_func(*args, **kwargs)
TypeError: decorated_method() missing 1 required positional argument: 'self'
How can I pass the self corresponding to WillCallDecorator() into decorated_method() but at the same time pass information from its own class to my_callable() ?
It seems that you may want to use partialmethod instead of partial:
From the docs:
class functools.partialmethod(func, /, *args, **keywords)
When func is a non-descriptor callable, an appropriate bound method is created dynamically. This behaves like a normal Python function when used as a method: the self argument will be inserted as the first positional argument, even before the args and keywords supplied to the partialmethod constructor.
So much simpler just to use the self variable you already have. There is absolutely no reason to be using partial or partialmethod here at all:
from functools import partial
class ClassWithDecorator:
def __init__(self):
self.decorator_param = "PARAM"
def my_decorator(self, decorated_func):
def my_callable(*args, **kwargs):
# Do something with decorator_param
print(self.decorator_param)
return decorated_func(*args, **kwargs)
return my_callable
decorator_instance = ClassWithDecorator()
class WillCallDecorator:
def __init__(self):
self.other_param = "WillCallDecorator variable"
#decorator_instance.my_decorator
def decorated_method(self):
pass
WillCallDecorator().decorated_method()
Also, to answer your question about why your code didn't work, when you access something.decorated_method() the code checks whether decorated_method is a function and if so turns it internally into a call WillCallDecorator.decorated_method(something). But the value returned from partial is a functools.partial object, not a function. So the class lookup binding won't happen here.
In more detail, something.method(arg) is equivalent to SomethingClass.method.__get__(something, arg) when something doesn't have an attribute method and its type SomethingClass does have the attribute and the attribute has a method __get__ but the full set of steps for attribute lookup is quite complicated.
I want use a same param in decorater and function like:
def outer(deco_param):
def decorator(func):
print("param:%s" % deco_param)
return func
return decorator
#outer(deco_param=s) # problems here: 's' is unreferenced
def test(s):
print("test:%s" % s)
I got this idea when using Flask but I didn't know how did they make it, which supports using a param of view function in its decorater like:
app.route(rule="/user/<int:uid>")
def access_user(uid): # if I use any other name except from 'uid', IDE would raise a tip
...
What's more, Flask seems to make a static check. If I miss uid argument or use any other name, IDE(pycharm) would raise a tip of "Function 'access_user` doesn't have a parameter 'int:uid'". That's the effect I want.
Can anyone please give me some advice? Thanks
You don't need to pass the same parameter to both the outer function and the inner function. To call the parameters via decorator, you should add a wrapper function to pass the parameters (*args) from inner one to outer one.
You can write like this:
def outer():
def decorator(func):
def wrapper(*args):
print("param:%s" % func.__param__)
return func(*args)
return wrapper
return decorator
#outer
def test(s):
print("test:%s" % s)
I am using retry function from the package retrying. I want to pass the arguments of the retry decorator from a function and I am not sure how to achieve that.
#retry # (wait_exponential_multiplier=x,wait_exponential_max=y)
def post(url, json, exponential_multiplier, exponential_max):
...
return(abc)
I want to pass the arguments of retry when calling the post(). I know when the function is compiled, the resulting function object is passed to the decorator so I am not sure if this is possible- or if I should may be approach it differently.
If you just want to use the library as is, then you cannot really use the decorator like this. It's arguments are constant from when it is invoked (excepting messing about with mutable arguments). Instead, you could always invoke the decorator before calling the function each time. This allows you to change the retrying arguments as and when you need to.
eg.
def post(url, json):
...
rety(post, wait_exponential_multiplier=...)(url=..., json=...)
But at that point, you might as well just skip the decorator altogether, and use what the decorator is using.
from retrying import Retrying
def post(url, json):
...
Retrying(wait_exponential_multiplier=...).call(post, url=..., json=...)
Either of these ways allow you to keep the post function pure and abstracted away from the concept of retrying (making it easier to call post when you don't want retrying behaviour).
You could also write a convenience function that wrapper that fills in defaults for your program. eg.
def retrier(wait_exponential_multiplier=2, **kwargs):
return Retrying(wait_exponential_multiplier=wait_exponential_multiplier, **kwargs)
retrier(wait_exponential_max=10).call(post, url=..., json=...)
retrier(wait_exponential_multiplier=3, wait_exponential_max=10).call(post, url=..., json=...)
Generally speaking there no good ways to achieve this. You surely can write code like this:
def post(url, json):
...
return(abc)
...
decorated_func = retry(wait_exponential_max=1)(post)
a = decorated_func(url, json)
and it will work. But it looks rather ugly and will construct decorated object for every call ("regular" decorators are executed once in import time).
If decorator itself is not very complex - you can use this approach in some more user-friendly manner:
def _post(url, json):
return(abc)
def post(url, json, wait_exponential_max=None, **kwargs):
return retry(wait_exponential_max=wait_exponential_max, **kwargs)(_post)(url, json)
You have to create a new decorator which pass its own arguments down to the decorated function and transforms the function using the retry decorator:
def retry_that_pass_down_arguments(**decorator_arguments):
def internal_decorator(f):
def decorated_function(*args, **kwargs):
# Add the decorator key-word arguments to key-word arguments of the decorated function
kwargs.update(decorator_arguments)
return retry(**decorator_arguments)(f)(*args, **kwargs)
return decorated_function
return internal_decorator
Then you can just do:
#retry_that_pass_down_arguments(wait_exponential_multiplier=x, wait_exponential_max=y)
def post(url, json, exponential_multiplier=None, exponential_max=None):
...
return(abc)
This is a complement to Jundiaius's answers to show that you can even use the inspect module to correctly handle the signature of the decorated function:
def deco_and_pass(deco, **kwparams):
"""Decorates a function with a decorator and parameter.
The parameters are passed to the decorator and forwarded to the function
The function must be prepared to receive those parameters, but they will
be removed from the signature of the decorated function."""
def outer(f):
sig = inspect.signature(f) # remove parameters from the function signature
params = collections.OrderedDict(sig.parameters)
for k in kwparams:
del params[k]
def inner(*args, **kwargs): # define the decorated function
kwargs.update(kwparams) # add the parameters
# and call the function through the parameterized decorator
return deco(**kwparams)(f)(*args, **kwargs)
inner.__signature__ = inspect.signature(f).replace(
parameters = params.values())
inner.__doc__ = f.__doc__ # update doc and signature
return inner
return outer
Example usage:
#deco_and_pass(retry,wait_exponential_multiplier=x,wait_exponential_max=y)
def post(url, json, exponential_multiplier, exponential_max):
...
return(abc)
...
post(url, json)
The signature of the decorated function is only def post(url, json)
Limits: the above code only accepts and passes keyword arguments for the decorator
So I have this set of code, testing decorator base_permissions_check:
def authenticated(self,*arg,**kwargs):
function_call = inspect.stack()[1][4][0].strip()
matched = re.match('^self\.', function_call)
if not matched:
raise Exception("function is private")
return self.user.is_authenticated()
def base_permissions_check(func):
def wrap(self,**kwargs):
if not self.authenticated(kwargs):
return self.permissions
# func(kwargs)
return func(kwargs)
return wrap
#public
#base_permissions_check
def has_video_permission(self,**kwargs):
The error says that authenticated() takes 1 positional argument but 2 were given when I call has_video_permission. I really don't know what's wrong? while I pass only one kwarg into it
The wrapped func() function is not bound to the instance as a method, so you need to pass in self explicitly. You also need to apply the kwargs dictionary as separate keyword arguments using the **kwargs call syntax:
return func(self, **kwargs)
Your authenticated method is not actually using the kwargs dictionary you pass in, so you could just call self.authenticated(), but if you need to have access to those keyword arguments, you probably want to use the **kwargs call syntax there too:
if not self.authenticated(**kwargs):
Since you see the error when you call the decorated function, you are probably passing in a positional argument, but your wrapper only accepts keyword arguments (beyond self). Perhaps you want to add *args to handle those positional arguments:
def base_permissions_check(func):
def wrap(self, *args, **kwargs):
if not self.authenticated(**kwargs):
return self.permissions
# func(kwargs)
return func(self, *args, **kwargs)
return wrap
#public
#base_permissions_check
def has_video_permission(self, *args, **kwargs):
# ...
I have a decorator like this:
def region_required(view_func):
def new_view(request, ctx = {}, *args, **kw):
import pdb; pdb.set_trace()
ctx['regions'] = Region.objects.all()
return view_func(request, context=ctx, *args, **kw)
return new_view
I uses it to decorate view functions.
Today I noticed something really strange, the ctx (context) argument sometimes have data that belongs to last HTTP request. Then I narrowed it down to this decorator and found that somehow ctx has value when it should be {} here.
So I set a break point, and goes one level up, then I found its caller does NOT pass anything to it at all.
How can an argument has value that is neither passed in by caller nor default value?
You're using a mutable argument as a default parameter. Every time it's called without that argument, it will include the one you provided, including prior modifications. Use None as the default value instead, check for it, and if it is None, assign the empty dict in the body of the function.
def region_required(view_func):
def new_view(request, ctx=None, *args, **kw):
if ctx is None:
ctx = {}
import pdb; pdb.set_trace()
ctx['regions'] = Region.objects.all()
return view_func(request, context=ctx, *args, **kw)
return new_view
ctx is likely getting assigned the first arg of the wrapped function. You can't have a keyword arg followed by *args. As far as the caller is concerned, ctx is just another arg.
For example:
my_func(request, 1, 2, 3)
will result in ctx being assigned 1. ctx needs to be considered part of kwargs.
ctx = kwargs.get('ctx', {})