# This version only accepts one argument
# def shout(fn):
# def wrapper(name):
# return fn(name).upper()
# return wrapper
# This version works with any number of args
def shout(fn):
def wrapper(*args, **kwargs):
return fn(*args, **kwargs).upper()
return wrapper
#shout
def greet(name):
return f"Hi, I'm {name}."
#shout
def order(main, side):
return f"Hi, I'd like the {main}, with a side of {side}, please."
#shout
def lol():
return "lol"
print(greet("todd"))
print(order(side="burger", main="fries"))
print(lol())
In the above code,
def wrapper(*args, **kwargs):
print(f"abc is {fn.__name__}")
return fn(*args, **kwargs).upper()
return wrapper
When wrapper functions executed, how does it knows the value of arguments which are to be assigned to , * args and **kwargs. We have not defined the values of arguments here, but instead func is given the parameters.
side="burger", main="fries".
**kwargs open up the dictionary but when did we defined such dictionary?
How does side="burger", main="fries" are set as arguments of wrapper function and Why are they being assigned to args and kwargs ?
Why are arguments given to fync being assigned to parameters of wrapper function?
They are set when you call the function. When you call the wrapper, all positional arguments are packed together into args and all keyword arguments are put into kwargs, which are then unpacked and passed to the wrapped function.
When you call:
order(side="burger", main="fries")
You are actually calling wrapper() with those parameters.
This is the time at which **kwargs is assigned.
wrapper() then goes on to call your actual function order() with *args and **kwargs.
Related
In Python 3.4+, functools.wraps preserves the signature of the function it wraps. Unfortunately, if you create decorators that are meant to be stacked on top of each other, the second (or later) decorator in the sequence will be seeing the generic *args and **kwargs signature of the wrapper and not preserving the signature of the original function all the way at the bottom of the sequence of decorators. Here's an example.
from functools import wraps
def validate_x(func):
#wraps(func)
def wrapper(*args, **kwargs):
assert kwargs['x'] <= 2
return func(*args, **kwargs)
return wrapper
def validate_y(func):
#wraps(func)
def wrapper(*args, **kwargs):
assert kwargs['y'] >= 2
return func(*args, **kwargs)
return wrapper
#validate_x
#validate_y
def foo(x=1, y=3):
print(x + y)
# call the double wrapped function.
foo()
This gives
-------------------------------------------------------------------------
KeyError Traceback (most recent call last)
<ipython-input-5-69c17467332d> in <module>
22
23
---> 24 foo()
<ipython-input-5-69c17467332d> in wrapper(*args, **kwargs)
4 #wraps(func)
5 def wrapper(*args, **kwargs):
----> 6 assert kwargs['x'] <= 2
7 return func(*args, **kwargs)
8 return wrapper
KeyError: 'x'
and if you switch the order of the decorators, you get the same key error for 'y'.
I tried replacing wraps(func) with wraps(func.__wrapped__) in the second decorator, but this still doesn't work (not to mention it requires the programmer to explicitly know where in the stack of decorators they are working for given wrapper functionality).
I also took a look at inspect.signature(foo) and this seems to give the right thing, but I found that this is because inspect.signature has a follow_wrapped parameter that defaults to True so it somehow knows to follow the sequence of wrapped functions, but apparently the regular method call framework for invoking foo() will not follow this same protocol for resolve args and kwargs of the outer decorated wrapper.
How can I just have wraps faithfully passthrough the signature so that wraps(wraps(wraps(wraps(f)))) (so to speak) always faithfully replicated the signature of f?
You are not actually passing any arguments to you function foo so *args and **kwargs are empty for both decorators. If you pass arguments the decorators will work just fine
foo(x=2, y = 3) # prints 5
You can try to get default function arguments using inspect
You can't really get the default values without using inspect and you also need to account for positional args (*args) vs keyword args (**kwargs). So normalize the data if it's there if it's missing then inspect the function
import inspect
from functools import wraps
def get_default_args(func):
signature = inspect.signature(func)
return {
k: v.default
for k, v in signature.parameters.items()
if v.default is not inspect.Parameter.empty
}
def validate_x(func):
#wraps(func)
def wrapper(*args, **kwargs):
if args and not kwargs and len(args) == 2:
kwargs['x'] = args[0]
kwargs['y'] = args[1]
args = []
if not args and not kwargs:
kwargs = get_default_args(func)
assert kwargs['x'] <= 2
return func(*args, **kwargs)
return wrapper
def validate_y(func):
#wraps(func)
def wrapper(*args, **kwargs):
if args and not kwargs and len(args) == 2:
kwargs['x'] = args[0]
kwargs['y'] = args[1]
args = []
if not args and not kwargs:
kwargs = get_default_args(func)
assert kwargs['y'] >= 2
return func(*args, **kwargs)
return wrapper
#validate_x
#validate_y
def foo(x=1, y=3):
print(x + y)
# call the double wrapped function.
foo()
# call with positional args
foo(1, 4)
# call with keyword args
foo(x=2, y=10)
This prints
4
5
12
Your diagnosis is incorrect; in fact, functools.wraps preserves the signature of the double-decorated function:
>>> import inspect
>>> inspect.signature(foo)
<Signature (x=1, y=3)>
We can also observe that it is not a problem with calling the function with the wrong signature, since that would raise a TypeError, not a KeyError.
You seem to be under the impression that when just one decorator is used, kwargs will be populated with the argument default values. This doesn't happen at all:
def test_decorator(func):
#wraps(func)
def wrapper(*args, **kwargs):
print('args:', args)
print('kwargs:', kwargs)
return func(*args, **kwargs)
return wrapper
#test_decorator
def foo(x=1):
print('x:', x)
The output is:
>>> foo()
args: ()
kwargs: {}
x: 1
So as you can see, neither args nor kwargs receives the argument's default value, even when just one decorator is used. They are both empty, because foo() calls the wrapper function with no positional arguments and no keyword arguments.
The actual problem is that your code has a logical error. The decorators validate_x and validate_y expect the arguments to be passed as keyword arguments, but in fact they might be passed as positional arguments or not at all (so the default values would apply), in which case 'x' and/or 'y' won't be present in kwargs.
There is no easy way to make your decorators work with an argument which could be passed as either keyword or positional; if you make the arguments keyword-only, then you can test whether 'x' or 'y' are in kwargs before validating them.
def validate_x(func):
#wraps(func)
def wrapper(*args, **kwargs):
if 'x' in kwargs and kwargs['x'] > 2:
raise ValueError('Invalid x, should be <= 2, was ' + str(x))
return func(*args, **kwargs)
return wrapper
#validate_x
def bar(*, x=1): # keyword-only arg, prevent passing as positional arg
...
It's usually better to explicitly raise an error, instead of using assert, because your program can be run with assert disabled.
Beware also that it's possible to declare a function like #validate_x def baz(*, x=5): ... where the default x is invalid. This won't raise any error because the default argument value isn't checked by the decorator.
I wanted an easy-to-use decorator class that can be used to force certain parameters to be pre-defined types lest an error be raised. The issue is that I have to specify each variable twice, one for each args and one for kwargs. This is true even with default parameters, provided there is no single splat parameter in the original function.
Simplified:
class ParamConstraint:
def __init__(self, *args, error=TypeError, **kwargs):
self.args = args
self.kwargs = kwargs
self.error = error
def __call__(self, function):
def wrap(*args, **kwargs):
for (arg, constraint) in zip(args, self.args):
if not isinstance(arg, constraint):
raise self.error(
"Value '{}' is not of type '{}'.".format(arg, constraint.__name__)
)
for kwarg in self.kwargs:
if kwarg in kwargs:
assert isinstance(kwargs[kwarg], self.kwargs[kwarg])
return function(*args, **kwargs)
return __import__("functools").wraps(function)(wrap)
This is easy enough to use. Say I wanted a function that would multiply a number by 2, but I only want integers, not floats. I can use:
#ParamConstraint(int)
def foo(x):
return x*2
print(foo(5)) # => prints 10
print(foo(x=5.0)) # => prints 10.0
print(foo(5.0)) # => raises TypeError
Now, I can prevent this by passing both positional and non-positional arguments into a function, but it gets very tedious after a while. For example, I'd have to write out 6 constraints for 3 parameters:
#ParamConstraint((int, float), int, FunctionType, value=(int, float), count=int, func=FunctionType):
def foo(value, count, func):
for _ in range(count):
value = func(value)
return value
Ideally, I should be able to just pass it by KWarg and not provide any positional arguments. However, this does not work because I have no way of finding out what the positional arguments of the original function are. I have used the inspect module minorly to no avail.
How can I remediate this?
I am new to python and decorators and am stumped in writing a decorator which reports not only passed args and kwargs but ALSO the unchanged default kwargs.
This is what I have so far.
def document_call(fn):
def wrapper(*args, **kwargs):
print 'function %s called with positional args %s and keyword args %s' % (fn.__name__, args, kwargs)
return fn(*args, **kwargs)
return wrapper
#document_call
def square(n, trial=True, output=False):
# kwargs are a bit of nonsense to test function
if not output:
print 'no output'
if trial:
print n*n
square(6) # with this call syntax, the default kwargs are not reported
# function square called with positional args (6,) and keyword args {}
# no output
36
square(7,output=True) # only if a kwarg is changed from default is it reported
# function square called with positional args (7,) and keyword args {'output': True}
49
The 'problem' is that this decorator reports the args that are passed in the call to square but does not report the default kwargs defined in the square definition. The only way kwargs are reported is if they're changed from their default i.e. passed to the square call.
Any recommendations for how I get the kwargs in the square definition reported too?
Edit after following up on the inspect suggestions, which helped me to the solution below. I changed the output of positional params to include their names because I thought it made the output easier to understand.
import inspect
def document_call(fn):
def wrapper(*args, **kwargs):
argspec = inspect.getargspec(fn)
n_postnl_args = len(argspec.args) - len(argspec.defaults)
# get kwargs passed positionally
passed = {k:v for k,v in zip(argspec.args[n_postnl_args:], args[n_postnl_args:])}
# update with kwargs
passed.update({k:v for k,v in kwargs.iteritems()})
print 'function %s called with \n positional args %s\n passed kwargs %s\n default kwargs %s' % (
fn.__name__, {k:v for k,v in zip(argspec.args, args[:n_postnl_args])},
passed,
{k:v for k,v in zip(argspec.args[n_postnl_args:], argspec.defaults) if k not in passed})
return fn(*args, **kwargs)
return wrapper
That was a good learning experience. It's neat to see three different solutions to the same problem. Thanks to the Answerers!
You'll have to introspect the function that you wrapped, to read the defaults. You can do this with the inspect.getargspec() function.
The function returns a tuple with, among others, a sequence of all argument names, and a sequence of default values. The last of the argument names pair up with the defaults to form name-default pairs; you can use this to create a dictionary and extract unused defaults from there:
import inspect
argspec = inspect.getargspec(fn)
positional_count = len(argspec.args) - len(argspec.defaults)
defaults = dict(zip(argspec.args[positional_count:], argspec.defaults))
You'll need to take into account that positional arguments can specify default arguments too, so the dance to figure out keyword arguments is a little more involved but looks like this:
def document_call(fn):
argspec = inspect.getargspec(fn)
positional_count = len(argspec.args) - len(argspec.defaults)
defaults = dict(zip(argspec.args[positional_count:], argspec.defaults))
def wrapper(*args, **kwargs):
used_kwargs = kwargs.copy()
used_kwargs.update(zip(argspec.args[positional_count:], args[positional_count:]))
print 'function %s called with positional args %s and keyword args %s' % (
fn.__name__, args[:positional_count],
{k: used_kwargs.get(k, d) for k, d in defaults.items()})
return fn(*args, **kwargs)
return wrapper
This determines what keyword paramaters were actually used from both the positional arguments passed in, and the keyword arguments, then pulls out default values for those not used.
Demo:
>>> square(39)
function square called with positional args (39,) and keyword args {'trial': True, 'output': False}
no output
1521
>>> square(39, False)
function square called with positional args (39,) and keyword args {'trial': False, 'output': False}
no output
>>> square(39, False, True)
function square called with positional args (39,) and keyword args {'trial': False, 'output': True}
>>> square(39, False, output=True)
function square called with positional args (39,) and keyword args {'trial': False, 'output': True}
Starting with Python 3.5 you can use BoundArguments.apply_defaults to fill in missing arguments with their default value:
import inspect
def document_call(fn):
def wrapper(*args, **kwargs):
bound = inspect.signature(fn).bind(*args, **kwargs)
bound.apply_defaults()
print(f'{fn.__name__} called with {bound}')
return fn(*args, **kwargs)
return wrapper
Since the decorator function wrapper takes any argument and just passes everything on, of course it does not know anything about the parameters of the wrapped function and its default values.
So without actually looking at the decorated function, you will not get this information. Fortunately, you can use the inspect module to figure out the default arguments of the wrapped function.
You can use the inspect.getargspec function to get the information about the default argument values in the function signature. You just need to match them up properly with the parameter names:
def document_call(fn):
argspec = inspect.getargspec(fn)
defaultArguments = list(reversed(zip(reversed(argspec.args), reversed(argspec.defaults))))
def wrapper(*args, **kwargs):
all_kwargs = kwargs.copy()
for arg, value in defaultArguments:
if arg not in kwargs:
all_kwargs[arg] = value
print 'function %s called with positional args %s and keyword args %s' % (fn.__name__, args, all_kwargs)
# still make the call using kwargs, to let the function handle its default values
return fn(*args, **kwargs)
return wrapper
Note that you could still improve this as right now you are handling positional and named arguments separately. For example, in your square function, you could also set trial by passing it as a positional argument after n. This will make it not appear in the kwargs. So you’d have to match the positional arguments with your kwargs to get the full information. You can get all the information about the positions from the argspec.
In python 3.6 I did it using inspect.getfullargspec:
def document_call(func):
#wraps(func)
def decorator(*args, **kwargs):
fullargspec = getfullargspec(func)
default_kwargs = fullargspec.kwonlydefaults
print('Default kwargs', default_kwargs)
print('Passed kwargs', kwargs)
return func(*args, **kwargs)
return decorator
Be aware about using the * separator when defining the decorated function for this to work
#document_call
def square(n, *, trial=True, output=False):
# kwargs are a bit of nonsense to test function
if not output:
print 'no output'
if trial:
print n*n
Here is the code modified to work with python3
import inspect
import decorator
#decorator.decorator
def log_call(fn,*args, **kwargs):
sign = inspect.signature(fn)
arg_names = list(sign.parameters.keys())
passed = {k:v for k,v in zip(arg_names[:len(args)], args)}
passed.update({k:v for k,v in kwargs.items()})
params_str = ", ".join([f"{k}={passed.get(k, '??')}" for k in arg_names])
print (f"{fn.__name__}({params_str})")
return fn(*args, **kwargs)
Note I'm using additional library "decorator" as it preserves the function signature.
I have a decorator that validates some parameters and passes an validated key to various functions:
from functools import wraps
ref validate(f):
#wraps(f) # This is to ensure docstrings are passed through the decorated function
def redirect_if_invalid(request, *args, **kwargs):
if request.valid == False:
return HttpResponseRedirect('/login')
else:
newkwargs = { 'key': request.key }
return f(request, *args, **newkwargs)
return redirect_if_invalid
This is used by some other functions:
#validate
def genericHandler(request, key)
pass
I'd call the function like this:
genericHandler(request)
And the decorator generates the 'key' kwarg. However, I'd like to optionally pass in the key at some other point, ie call:
genericHandler(request, 'keydata')
Currently this gives me an error:
TypeError: genericHandler() got multiple values for keyword argument 'key'
How can I get around this? To reiterate, the main goal is to be able to call genericHandler() with or without the optional parameter, and have the decorator generate the parameter only if it's missing.
So far inside the decorator, I can't figure out how to determine whether the 'key' parameter was passed in or not because functools.wraps() seems to hide it.
There's not any reasonable way to do this if you want your wrapper's signature to still be (request, *args, **kwargs). On the other hand, it looks like your decorator already assumes that the wrapped function takes a key parameter, so why not rewrite the wrapper to take one as well? In that case it becomes trivial to check if it's been passed or not.
def validate(f):
#wraps(f)
def redirect_if_invalid(request, key=None):
# do the validation
if key is None:
key = request.key
return f(request, key)
return redirect_if_invalid
You can add the *args and **kwargs parameters back if you like, of course.
So the best way for me to do this was to explicitly pass kwargs as kwargs. So decorated functions should actually be:
#validate
def genericHandler(request, **kwargs)
key = kwargs.get('key')
pass
This way, I can call the function either with or without args:
genericHandler(request)
or
genericHandler(request, **{ 'key' : key })
And the actual decorated looks like:
def validate(f):
#wraps(f) # This is to ensure docstrings are passed through the decorated function
def redirect_if_invalid(request, *args, **kwargs):
key = kwargs.get('key')
if not key:
kwargs.set('key', request.key)
return f(request, *args, **kwargs)
return redirect_if_invalid
Say you have the following code:
def addTags(functionHere):
def wrapper():
return "NumberTag" + functionHere() + "NumberTagOver"
return wrapper
#addTags
def numbers(firstValue, secondValue):
return firstValue + secondValue
Then putting numbers(4, 5) into the interpreter spits back a trace saying that wrapper takes no arguments. Okay fine, then putting numbers() into the interpreter spits back that numbers wants two arguments! Confused.
Your wrapper function replaces the wrapped function, and needs to match the number of arguments it takes. Your wrapped function takes two arguments (firstValue, secondValue), but your wrapper takes none at all.
You could add these two to your decorator wrapper too:
def addTags(functionHere):
def wrapper(firstValue, secondValue):
return "NumberTag" + functionHere(firstValue, secondValue) + "NumberTagOver"
return wrapper
but that ties your decorator to that specific function.
There is a trick you can use however:
def addTags(functionHere):
def wrapper(*args):
return "NumberTag" + functionHere(*args) + "NumberTagOver"
return wrapper
The *args positional argument acts as a catch-all, all positional arguments to the function are now passed on to the wrapped function.
There is an equivalent for keyword arguments too:
def addTags(functionHere):
def wrapper(*args, **kw):
return "NumberTag" + functionHere(*args, **kw) + "NumberTagOver"
return wrapper