I'm not sure if this is possible in Python and I was wondering, is there any method for checking at runtime whether a parameter is passed to a Python function without doing some sort of check on the parameter value?
def my_func(req, opt=0):
return was_opt_passed() # Returns bool
print(my_func(0)) # Prints False
print(my_func(0, 0)) # Prints True
This would be superior in some circumstances, if possible, because it relieves the need to remember and check for sentinel values. Is it possible?
the standard method of detecting if an argument has been passed is sentinel values; however, if you're willing to lose your function signature, you can use **kwargs:
def my_func(**kwargs):
return 'opt' in kwargs
print(my_func()) #=> False
print(my_func(opt=0)) $=> True
As Mark has already stated in his comment the typical convention is to use the default value None. Then you can check if it’s still None upon calling.
def my_func(req, opt=None):
if opt is None:
#opt wasn’t passed.
return False
#opt was passed
return True
Although if you’d like to do more research on other options (most unconventional for most cases) feel free to check out these answers
One solution is to use decorators/wrappers. They allow you to interface with what's being passed to your function at runtime and then handle said things as you see fit Consider this code:
def check_keywords(func):
def wrapper(*args, **kwargs):
if kwargs:
print('Keyword was passed!')
return func(*args, **kwargs)
return wrapper
#check_keywords
def my_func(req, opt=0):
print(req)
check_keywords captures the function and if it detects keywords being passed into my_func, it then prints something. This print statement can be converted to any arbitrary code you want.
e.g.:
my_func(1)
>>> 1
my_func(1, opt = 1)
>>> Keyword was passed!
>>> 1
A great way would be to use *args or **kwargs.
def was_opt_passed(*args, **kwargs):
return len(args) > 0 or 'opt' in kwargs
def my_func(req, *args, **kwargs):
return was_opt_passed(*args, **kwargs) # Returns bool
print(my_func(0)) # Prints False
print(my_func(0, 0)) # Prints True
print(my_func(0, opt=0)) # Prints True
print(my_func(0, not_opt=0)) # Prints False
*args collects any positional arguments passed to your function that are not already enumerated, while **kwargs collects any keyword arguments passed to your function that are not already enumerated. If args contains a positional argument, we assume it was opt, and it must have been passed. Otherwise, if it is in kwargs, it was passed, and then if we didn't find it in either place, it must not have been passed.
See also https://docs.python.org/3/tutorial/controlflow.html#keyword-arguments
Using decorator
def check_opt_passed(methd):
def m(req, *args, **kwarg):
# this will check even if opt is passed as positional argument
# and check if opt is passed not any other keyword
if args or (kwarg and 'opt' in kwarg):
print('opt is passed')
else:
print('opt is not passed')
return methd(req, *args, **kwarg)
return m
#check_opt_passed
def my_func(req, opt=0):
# dummy expression for testing
return req * opt
print(my_func(1)) # opt is not passed
print(my_func(1, 0)) # opt is passed
print(my_func(1, opt=0)) # opt is passed
Related
In Python 3.4+, functools.wraps preserves the signature of the function it wraps. Unfortunately, if you create decorators that are meant to be stacked on top of each other, the second (or later) decorator in the sequence will be seeing the generic *args and **kwargs signature of the wrapper and not preserving the signature of the original function all the way at the bottom of the sequence of decorators. Here's an example.
from functools import wraps
def validate_x(func):
#wraps(func)
def wrapper(*args, **kwargs):
assert kwargs['x'] <= 2
return func(*args, **kwargs)
return wrapper
def validate_y(func):
#wraps(func)
def wrapper(*args, **kwargs):
assert kwargs['y'] >= 2
return func(*args, **kwargs)
return wrapper
#validate_x
#validate_y
def foo(x=1, y=3):
print(x + y)
# call the double wrapped function.
foo()
This gives
-------------------------------------------------------------------------
KeyError Traceback (most recent call last)
<ipython-input-5-69c17467332d> in <module>
22
23
---> 24 foo()
<ipython-input-5-69c17467332d> in wrapper(*args, **kwargs)
4 #wraps(func)
5 def wrapper(*args, **kwargs):
----> 6 assert kwargs['x'] <= 2
7 return func(*args, **kwargs)
8 return wrapper
KeyError: 'x'
and if you switch the order of the decorators, you get the same key error for 'y'.
I tried replacing wraps(func) with wraps(func.__wrapped__) in the second decorator, but this still doesn't work (not to mention it requires the programmer to explicitly know where in the stack of decorators they are working for given wrapper functionality).
I also took a look at inspect.signature(foo) and this seems to give the right thing, but I found that this is because inspect.signature has a follow_wrapped parameter that defaults to True so it somehow knows to follow the sequence of wrapped functions, but apparently the regular method call framework for invoking foo() will not follow this same protocol for resolve args and kwargs of the outer decorated wrapper.
How can I just have wraps faithfully passthrough the signature so that wraps(wraps(wraps(wraps(f)))) (so to speak) always faithfully replicated the signature of f?
You are not actually passing any arguments to you function foo so *args and **kwargs are empty for both decorators. If you pass arguments the decorators will work just fine
foo(x=2, y = 3) # prints 5
You can try to get default function arguments using inspect
You can't really get the default values without using inspect and you also need to account for positional args (*args) vs keyword args (**kwargs). So normalize the data if it's there if it's missing then inspect the function
import inspect
from functools import wraps
def get_default_args(func):
signature = inspect.signature(func)
return {
k: v.default
for k, v in signature.parameters.items()
if v.default is not inspect.Parameter.empty
}
def validate_x(func):
#wraps(func)
def wrapper(*args, **kwargs):
if args and not kwargs and len(args) == 2:
kwargs['x'] = args[0]
kwargs['y'] = args[1]
args = []
if not args and not kwargs:
kwargs = get_default_args(func)
assert kwargs['x'] <= 2
return func(*args, **kwargs)
return wrapper
def validate_y(func):
#wraps(func)
def wrapper(*args, **kwargs):
if args and not kwargs and len(args) == 2:
kwargs['x'] = args[0]
kwargs['y'] = args[1]
args = []
if not args and not kwargs:
kwargs = get_default_args(func)
assert kwargs['y'] >= 2
return func(*args, **kwargs)
return wrapper
#validate_x
#validate_y
def foo(x=1, y=3):
print(x + y)
# call the double wrapped function.
foo()
# call with positional args
foo(1, 4)
# call with keyword args
foo(x=2, y=10)
This prints
4
5
12
Your diagnosis is incorrect; in fact, functools.wraps preserves the signature of the double-decorated function:
>>> import inspect
>>> inspect.signature(foo)
<Signature (x=1, y=3)>
We can also observe that it is not a problem with calling the function with the wrong signature, since that would raise a TypeError, not a KeyError.
You seem to be under the impression that when just one decorator is used, kwargs will be populated with the argument default values. This doesn't happen at all:
def test_decorator(func):
#wraps(func)
def wrapper(*args, **kwargs):
print('args:', args)
print('kwargs:', kwargs)
return func(*args, **kwargs)
return wrapper
#test_decorator
def foo(x=1):
print('x:', x)
The output is:
>>> foo()
args: ()
kwargs: {}
x: 1
So as you can see, neither args nor kwargs receives the argument's default value, even when just one decorator is used. They are both empty, because foo() calls the wrapper function with no positional arguments and no keyword arguments.
The actual problem is that your code has a logical error. The decorators validate_x and validate_y expect the arguments to be passed as keyword arguments, but in fact they might be passed as positional arguments or not at all (so the default values would apply), in which case 'x' and/or 'y' won't be present in kwargs.
There is no easy way to make your decorators work with an argument which could be passed as either keyword or positional; if you make the arguments keyword-only, then you can test whether 'x' or 'y' are in kwargs before validating them.
def validate_x(func):
#wraps(func)
def wrapper(*args, **kwargs):
if 'x' in kwargs and kwargs['x'] > 2:
raise ValueError('Invalid x, should be <= 2, was ' + str(x))
return func(*args, **kwargs)
return wrapper
#validate_x
def bar(*, x=1): # keyword-only arg, prevent passing as positional arg
...
It's usually better to explicitly raise an error, instead of using assert, because your program can be run with assert disabled.
Beware also that it's possible to declare a function like #validate_x def baz(*, x=5): ... where the default x is invalid. This won't raise any error because the default argument value isn't checked by the decorator.
I'm writing several functions which accept an argument called policy, which is allowed only to have certain values (namely, 'allow' or 'deny'). If it doesn't, I would like a ValueError to be raised.
For brevity, I would like to define a decorator for this. So far, I have come up with the following:
def validate_policy(function):
'''Wrapper which ensures that if the function accepts a 'policy' argument, that argument is either 'allow' or 'deny'.'''
def wrapped_function(policy, *args, **kwargs):
if policy not in ['allow', 'deny']:
raise ValueError("The policy must be either 'allow' or 'deny'.")
return function(policy, *args, **kwargs)
return wrapped_function
The problem is that this only works if policy is the first positional argument of the function. However, I would like to allow for policy to appear at any position.
To be specific, here are some (dummy) functions called make_decision and make_informed_decision which accept an argument policy at different positions, and some test cases to go with them:
import pytest
#validate_policy
def make_decision(policy): # The 'policy' might be the first positional argument
if policy == 'allow':
print "Allowed."
elif policy == 'deny':
print "Denied."
#validate_policy
def make_informed_decision(data, policy): # It also might be the second one
if policy == 'allow':
print "Based on the data {data} it is allowed.".format(data=data)
elif policy == 'deny':
print "Based on the data {data} it is denied.".format(data=data)
'''Tests'''
def test_make_decision_with_invalid_policy_as_positional_argument():
with pytest.raises(ValueError):
make_decision('foobar')
def test_make_decision_with_invalid_policy_as_keyword_argument():
with pytest.raises(ValueError):
make_decision(policy='foobar')
def test_make_informed_decision_with_invalid_policy_as_positional_argument():
with pytest.raises(ValueError):
make_informed_decision("allow", "foobar")
def test_make_informed_decision_with_invalid_policy_as_keyword_argument():
with pytest.raises(ValueError):
make_informed_decision(data="allow", policy="foobar")
if __name__ == "__main__":
pytest.main([__file__])
Currently all the tests pass except the third one, because the first positional argument 'allow' is interpreted as the policy rather than as the data as it should be.
How can I adapt the validate_policy decorator such that all the tests pass?
You can use the inspect module's Signature.bind function:
import inspect
def validate_policy(function):
'''Wrapper which ensures that if the function accepts a 'policy' argument, that argument is either 'allow' or 'deny'.'''
signature= inspect.signature(function)
def wrapped_function(*args, **kwargs):
bound_args= signature.bind(*args, **kwargs)
bound_args.apply_defaults()
if bound_args.arguments.get('policy') not in ['allow', 'deny']:
raise ValueError("The policy must be either 'allow' or 'deny'.")
return function(*args, **kwargs)
return wrapped_function
Here is another solution using inspect.getcallargs:
def validate_policy(function):
'''Wrapper which ensures that if the function accepts a 'policy' argument, that argument is either 'allow' or 'deny'.'''
def wrapped_function(*args, **kwargs):
call_args = inspect.getcallargs(function, *args, **kwargs)
if 'policy' in call_args:
if call_args['policy'] not in ['allow', 'deny']:
raise ValueError("The policy must be either 'allow' or 'deny'.")
return function(*args, **kwargs)
return wrapped_function
It makes all the tests pass.
I wanted an easy-to-use decorator class that can be used to force certain parameters to be pre-defined types lest an error be raised. The issue is that I have to specify each variable twice, one for each args and one for kwargs. This is true even with default parameters, provided there is no single splat parameter in the original function.
Simplified:
class ParamConstraint:
def __init__(self, *args, error=TypeError, **kwargs):
self.args = args
self.kwargs = kwargs
self.error = error
def __call__(self, function):
def wrap(*args, **kwargs):
for (arg, constraint) in zip(args, self.args):
if not isinstance(arg, constraint):
raise self.error(
"Value '{}' is not of type '{}'.".format(arg, constraint.__name__)
)
for kwarg in self.kwargs:
if kwarg in kwargs:
assert isinstance(kwargs[kwarg], self.kwargs[kwarg])
return function(*args, **kwargs)
return __import__("functools").wraps(function)(wrap)
This is easy enough to use. Say I wanted a function that would multiply a number by 2, but I only want integers, not floats. I can use:
#ParamConstraint(int)
def foo(x):
return x*2
print(foo(5)) # => prints 10
print(foo(x=5.0)) # => prints 10.0
print(foo(5.0)) # => raises TypeError
Now, I can prevent this by passing both positional and non-positional arguments into a function, but it gets very tedious after a while. For example, I'd have to write out 6 constraints for 3 parameters:
#ParamConstraint((int, float), int, FunctionType, value=(int, float), count=int, func=FunctionType):
def foo(value, count, func):
for _ in range(count):
value = func(value)
return value
Ideally, I should be able to just pass it by KWarg and not provide any positional arguments. However, this does not work because I have no way of finding out what the positional arguments of the original function are. I have used the inspect module minorly to no avail.
How can I remediate this?
I am new to python and decorators and am stumped in writing a decorator which reports not only passed args and kwargs but ALSO the unchanged default kwargs.
This is what I have so far.
def document_call(fn):
def wrapper(*args, **kwargs):
print 'function %s called with positional args %s and keyword args %s' % (fn.__name__, args, kwargs)
return fn(*args, **kwargs)
return wrapper
#document_call
def square(n, trial=True, output=False):
# kwargs are a bit of nonsense to test function
if not output:
print 'no output'
if trial:
print n*n
square(6) # with this call syntax, the default kwargs are not reported
# function square called with positional args (6,) and keyword args {}
# no output
36
square(7,output=True) # only if a kwarg is changed from default is it reported
# function square called with positional args (7,) and keyword args {'output': True}
49
The 'problem' is that this decorator reports the args that are passed in the call to square but does not report the default kwargs defined in the square definition. The only way kwargs are reported is if they're changed from their default i.e. passed to the square call.
Any recommendations for how I get the kwargs in the square definition reported too?
Edit after following up on the inspect suggestions, which helped me to the solution below. I changed the output of positional params to include their names because I thought it made the output easier to understand.
import inspect
def document_call(fn):
def wrapper(*args, **kwargs):
argspec = inspect.getargspec(fn)
n_postnl_args = len(argspec.args) - len(argspec.defaults)
# get kwargs passed positionally
passed = {k:v for k,v in zip(argspec.args[n_postnl_args:], args[n_postnl_args:])}
# update with kwargs
passed.update({k:v for k,v in kwargs.iteritems()})
print 'function %s called with \n positional args %s\n passed kwargs %s\n default kwargs %s' % (
fn.__name__, {k:v for k,v in zip(argspec.args, args[:n_postnl_args])},
passed,
{k:v for k,v in zip(argspec.args[n_postnl_args:], argspec.defaults) if k not in passed})
return fn(*args, **kwargs)
return wrapper
That was a good learning experience. It's neat to see three different solutions to the same problem. Thanks to the Answerers!
You'll have to introspect the function that you wrapped, to read the defaults. You can do this with the inspect.getargspec() function.
The function returns a tuple with, among others, a sequence of all argument names, and a sequence of default values. The last of the argument names pair up with the defaults to form name-default pairs; you can use this to create a dictionary and extract unused defaults from there:
import inspect
argspec = inspect.getargspec(fn)
positional_count = len(argspec.args) - len(argspec.defaults)
defaults = dict(zip(argspec.args[positional_count:], argspec.defaults))
You'll need to take into account that positional arguments can specify default arguments too, so the dance to figure out keyword arguments is a little more involved but looks like this:
def document_call(fn):
argspec = inspect.getargspec(fn)
positional_count = len(argspec.args) - len(argspec.defaults)
defaults = dict(zip(argspec.args[positional_count:], argspec.defaults))
def wrapper(*args, **kwargs):
used_kwargs = kwargs.copy()
used_kwargs.update(zip(argspec.args[positional_count:], args[positional_count:]))
print 'function %s called with positional args %s and keyword args %s' % (
fn.__name__, args[:positional_count],
{k: used_kwargs.get(k, d) for k, d in defaults.items()})
return fn(*args, **kwargs)
return wrapper
This determines what keyword paramaters were actually used from both the positional arguments passed in, and the keyword arguments, then pulls out default values for those not used.
Demo:
>>> square(39)
function square called with positional args (39,) and keyword args {'trial': True, 'output': False}
no output
1521
>>> square(39, False)
function square called with positional args (39,) and keyword args {'trial': False, 'output': False}
no output
>>> square(39, False, True)
function square called with positional args (39,) and keyword args {'trial': False, 'output': True}
>>> square(39, False, output=True)
function square called with positional args (39,) and keyword args {'trial': False, 'output': True}
Starting with Python 3.5 you can use BoundArguments.apply_defaults to fill in missing arguments with their default value:
import inspect
def document_call(fn):
def wrapper(*args, **kwargs):
bound = inspect.signature(fn).bind(*args, **kwargs)
bound.apply_defaults()
print(f'{fn.__name__} called with {bound}')
return fn(*args, **kwargs)
return wrapper
Since the decorator function wrapper takes any argument and just passes everything on, of course it does not know anything about the parameters of the wrapped function and its default values.
So without actually looking at the decorated function, you will not get this information. Fortunately, you can use the inspect module to figure out the default arguments of the wrapped function.
You can use the inspect.getargspec function to get the information about the default argument values in the function signature. You just need to match them up properly with the parameter names:
def document_call(fn):
argspec = inspect.getargspec(fn)
defaultArguments = list(reversed(zip(reversed(argspec.args), reversed(argspec.defaults))))
def wrapper(*args, **kwargs):
all_kwargs = kwargs.copy()
for arg, value in defaultArguments:
if arg not in kwargs:
all_kwargs[arg] = value
print 'function %s called with positional args %s and keyword args %s' % (fn.__name__, args, all_kwargs)
# still make the call using kwargs, to let the function handle its default values
return fn(*args, **kwargs)
return wrapper
Note that you could still improve this as right now you are handling positional and named arguments separately. For example, in your square function, you could also set trial by passing it as a positional argument after n. This will make it not appear in the kwargs. So you’d have to match the positional arguments with your kwargs to get the full information. You can get all the information about the positions from the argspec.
In python 3.6 I did it using inspect.getfullargspec:
def document_call(func):
#wraps(func)
def decorator(*args, **kwargs):
fullargspec = getfullargspec(func)
default_kwargs = fullargspec.kwonlydefaults
print('Default kwargs', default_kwargs)
print('Passed kwargs', kwargs)
return func(*args, **kwargs)
return decorator
Be aware about using the * separator when defining the decorated function for this to work
#document_call
def square(n, *, trial=True, output=False):
# kwargs are a bit of nonsense to test function
if not output:
print 'no output'
if trial:
print n*n
Here is the code modified to work with python3
import inspect
import decorator
#decorator.decorator
def log_call(fn,*args, **kwargs):
sign = inspect.signature(fn)
arg_names = list(sign.parameters.keys())
passed = {k:v for k,v in zip(arg_names[:len(args)], args)}
passed.update({k:v for k,v in kwargs.items()})
params_str = ", ".join([f"{k}={passed.get(k, '??')}" for k in arg_names])
print (f"{fn.__name__}({params_str})")
return fn(*args, **kwargs)
Note I'm using additional library "decorator" as it preserves the function signature.
I have a decorator that validates some parameters and passes an validated key to various functions:
from functools import wraps
ref validate(f):
#wraps(f) # This is to ensure docstrings are passed through the decorated function
def redirect_if_invalid(request, *args, **kwargs):
if request.valid == False:
return HttpResponseRedirect('/login')
else:
newkwargs = { 'key': request.key }
return f(request, *args, **newkwargs)
return redirect_if_invalid
This is used by some other functions:
#validate
def genericHandler(request, key)
pass
I'd call the function like this:
genericHandler(request)
And the decorator generates the 'key' kwarg. However, I'd like to optionally pass in the key at some other point, ie call:
genericHandler(request, 'keydata')
Currently this gives me an error:
TypeError: genericHandler() got multiple values for keyword argument 'key'
How can I get around this? To reiterate, the main goal is to be able to call genericHandler() with or without the optional parameter, and have the decorator generate the parameter only if it's missing.
So far inside the decorator, I can't figure out how to determine whether the 'key' parameter was passed in or not because functools.wraps() seems to hide it.
There's not any reasonable way to do this if you want your wrapper's signature to still be (request, *args, **kwargs). On the other hand, it looks like your decorator already assumes that the wrapped function takes a key parameter, so why not rewrite the wrapper to take one as well? In that case it becomes trivial to check if it's been passed or not.
def validate(f):
#wraps(f)
def redirect_if_invalid(request, key=None):
# do the validation
if key is None:
key = request.key
return f(request, key)
return redirect_if_invalid
You can add the *args and **kwargs parameters back if you like, of course.
So the best way for me to do this was to explicitly pass kwargs as kwargs. So decorated functions should actually be:
#validate
def genericHandler(request, **kwargs)
key = kwargs.get('key')
pass
This way, I can call the function either with or without args:
genericHandler(request)
or
genericHandler(request, **{ 'key' : key })
And the actual decorated looks like:
def validate(f):
#wraps(f) # This is to ensure docstrings are passed through the decorated function
def redirect_if_invalid(request, *args, **kwargs):
key = kwargs.get('key')
if not key:
kwargs.set('key', request.key)
return f(request, *args, **kwargs)
return redirect_if_invalid