If I have a function
def foo(x, y):
pass
how can I tell, from inside the function, whether y was passed positionally or with its keyword?
I'd like to have something like
def foo(x, y):
if passed_positionally(y):
print('y was passed positionally!')
else:
print('y was passed with its keyword')
so that I get
>>> foo(3, 4)
y was passed positionally
>>> foo(3, y=4)
y was passed with its keyword
I realise I didn't originally specify this, but is it possible to do this whilst preserving type annotations? The top answer so far suggests using a decorator - however, that does not preserve the return type
You can create a decorator, like this:
def checkargs(func):
def inner(*args, **kwargs):
if 'y' in kwargs:
print('y passed with its keyword!')
else:
print('y passed positionally.')
result = func(*args, **kwargs)
return result
return inner
>>> #checkargs
...: def foo(x, y):
...: return x + y
>>> foo(2, 3)
y passed positionally.
5
>>> foo(2, y=3)
y passed with its keyword!
5
Of course you can improve this by allowing the decorator to accept arguments. Thus you can pass the parameter you want to check for. Which would be something like this:
def checkargs(param_to_check):
def inner(func):
def wrapper(*args, **kwargs):
if param_to_check in kwargs:
print('y passed with its keyword!')
else:
print('y passed positionally.')
result = func(*args, **kwargs)
return result
return wrapper
return inner
>>> #checkargs(param_to_check='y')
...: def foo(x, y):
...: return x + y
>>> foo(2, y=3)
y passed with its keyword!
5
I think adding functools.wraps would preserve the annotations, following version also allows to perform the check over all arguments (using inspect):
from functools import wraps
from inspect import signature
def checkargs(func):
#wraps(func)
def inner(*args, **kwargs):
for param in signature(func).parameters:
if param in kwargs:
print(param, 'passed with its keyword!')
else:
print(param, 'passed positionally.')
result = func(*args, **kwargs)
return result
return inner
>>> #checkargs
...: def foo(x, y, z) -> int:
...: return x + y
>>> foo(2, 3, z=4)
x passed positionally.
y passed positionally.
z passed with its keyword!
9
>>> inspect.getfullargspec(foo)
FullArgSpec(args=[], varargs='args', varkw='kwargs', defaults=None,
kwonlyargs=[], kwonlydefaults=None, annotations={'return': <class 'int'>})
_____________HERE____________
Update Python 3.10
In Python 3.10+ new ParamSpec type annotation was introduced (PEP 612), for better specifying parameter types in higher-order functions. As of now, the correct way to annotate this decorator would be like this:
from functools import wraps
from inspect import signature
from typing import Callable, ParamSpec, TypeVar, TYPE_CHECKING
T = TypeVar("T")
P = ParamSpec("P")
def check_args(func: Callable[P, T]) -> Callable[P, T]:
"""
Decorator to monitor whether an argument is passed
positionally or with its keyword, during function call.
"""
#wraps(func)
def inner(*args: P.args, **kwargs: P.kwargs) -> T:
for param in signature(func).parameters:
if param in kwargs:
print(param, 'passed with its keyword!')
else:
print(param, 'passed positionally.')
return func(*args, **kwargs)
return inner
Which correctly preserves type annotation:
if TYPE_CHECKING:
reveal_type(foo(2, 3))
# ─❯ mypy check_kwd.py
# check_kwd.py:34: note: Revealed type is "builtins.int"
# Success: no issues found in 1 source file
At the end, if you are going to do something like this:
def foo(x, y):
if passed_positionally(y):
raise Exception("You need to pass 'y' as a keyword argument")
else:
process(x, y)
You can do this:
def foo(x, *, y):
pass
>>> foo(1, 2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: foo() takes 1 positional argument but 2 were given
>>> foo(1, y=2) # works
Or only allow them to be passed positionally:
def foo(x, y, /):
pass
>>> foo(x=1, y=2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: foo() got some positional-only arguments passed as keyword arguments: 'x, y'
>>> foo(1, 2) # works
See PEP 570 and PEP 3102 for more.
Adapted from #Cyttorak 's answer, here's a way to do it which maintains the types:
from typing import TypeVar, Callable, Any, TYPE_CHECKING
T = TypeVar("T", bound=Callable[..., Any])
from functools import wraps
import inspect
def checkargs() -> Callable[[T], T]:
def decorate(func):
#wraps(func)
def inner(*args, **kwargs):
for param in inspect.signature(func).parameters:
if param in kwargs:
print(param, 'passed with its keyword!')
else:
print(param, 'passed positionally.')
result = func(*args, **kwargs)
return result
return inner
return decorate
#checkargs()
def foo(x, y) -> int:
return x+y
if TYPE_CHECKING:
reveal_type(foo(2, 3))
foo(2, 3)
foo(2, y=3)
Output is:
$ mypy t.py
t.py:27: note: Revealed type is 'builtins.int'
$ python t.py
x passed positionally.
y passed positionally.
x passed positionally.
y passed with its keyword!
It is not ordinarily possible. In a sense: the language is not designed to allow you to distinguish both ways.
You can design your function to take different parameters - positional, and named, and check which one was passed, in a thing like:
def foo(x, y=None, /, **kwargs):
if y is None:
y = kwargs.pop(y)
received_as_positional = False
else:
received_as_positional = True
The problem is that, although by using positional only parameters as abov, you could get y both ways,
it would be shown not once for a user (or IDE) inspecting the
function signature.
I hav a feeling you just want to know this for the sake of knowing - if
you really intend this for design of an API, I'd suggest you'd rethink
your API - there should be no difference in the behavior, unless both
are un-ambiguously different parameters from the user point of view.
That said, the way to go would be to inspect the caller frame, and check
the bytecode around the place the function is called:
In [24]: import sys, dis
In [25]: def foo(x, y=None):
...: f = sys._getframe().f_back
...: print(dis.dis(f.f_code))
...:
In [26]: foo(1, 2)
1 0 LOAD_NAME 0 (foo)
2 LOAD_CONST 0 (1)
4 LOAD_CONST 1 (2)
6 CALL_FUNCTION 2
8 PRINT_EXPR
10 LOAD_CONST 2 (None)
12 RETURN_VALUE
None
In [27]: foo(1, y=2)
1 0 LOAD_NAME 0 (foo)
2 LOAD_CONST 0 (1)
4 LOAD_CONST 1 (2)
6 LOAD_CONST 2 (('y',))
8 CALL_FUNCTION_KW 2
10 PRINT_EXPR
12 LOAD_CONST 3 (None)
14 RETURN_VALUE
So, as you can see, when y is called as named parameter, the opcode for the call is CALL_FUNCTION_KW , and the name of the parameter is loaded into the stack imediately before it.
You can trick the user and add another argument to the function like this:
def foo(x,y1=None,y=None):
if y1 is not None:
print('y was passed positionally!')
else:
print('y was passed with its keyword')
I don't recommend doing it but it does work
In foo, you can pass the call stack from traceback to positionally, which will then parse the lines, find the line where foo itself is called, and then parse the line with ast to locate positional parameter specifications (if any):
import traceback, ast, re
def get_fun(name, ast_obj):
if isinstance(ast_obj, ast.Call) and ast_obj.func.id == name:
yield from [i.arg for i in getattr(ast_obj, 'keywords', [])]
for a, b in getattr(ast_obj, '__dict__', {}).items():
yield from (get_fun(name, b) if not isinstance(b, list) else \
[i for k in b for i in get_fun(name, k)])
def passed_positionally(stack):
*_, [_, co], [trace, _] = [re.split('\n\s+', i.strip()) for i in stack]
f_name = re.findall('(?:line \d+, in )(\w+)', trace)[0]
return list(get_fun(f_name, ast.parse(co)))
def foo(x, y):
if 'y' in passed_positionally(traceback.format_stack()):
print('y was passed with its keyword')
else:
print('y was passed positionally')
foo(1, y=2)
Output:
y was passed with its keyword
Notes:
This solution does not require any wrapping of foo. Only the traceback needs to be captured.
To get the full foo call as a string in the traceback, this solution must be run in a file, not the shell.
When using functools.wraps, the signature claims one set of defaults, but the (wrapped) function uses the original (wrappee) defaults.
Why is that a good thing?
from functools import wraps
from inspect import signature
def g(a: float, b=10):
return a * b
def f(a: int, b=1):
return a * b
assert f(3) == 3
f = wraps(g)(f)
assert str(signature(f)) == '(a: float, b=10)' # signature says that b defaults to 10
assert f.__defaults__ == (1,) # ... but it's a lie!
assert f(3) == 3 != g(3) == 30 # ... and one that can lead to problems!
Why is this so? Because functools.wraps updates the __signature__ (including annotations and defaults), __annotations__, but not __defaults__, which python apparently looks to in order to assign defaults.
One solution is to politely ask wraps to include these defaults.
from functools import wraps, WRAPPER_ASSIGNMENTS, partial
my_wraps = partial(wraps, assigned=(list(WRAPPER_ASSIGNMENTS) + ['__defaults__', '__kwdefaults__']))
def g(a: float, b=10):
return a * b
def f(a: int, b=1):
return a * b
assert f(3) == 3
f = my_wraps(g)(f)
assert f(3) == 30 == g(3)
assert f.__defaults__ == (10,) # ... because now got g defaults!
But why oh why don't I get that for free? When would I ever want the defaults that are actually used be different than those mentioned in the signature?!?
I tried to use function decorators, but in this example it dooesn't work for me, can you give me the solution ?
def multiply_by_three(f):
def decorator():
return f() * 3
return decorator
#multiply_by_three
def add(a, b):
return a + b
print(add(1,2)) # returns (1 + 2) * 3 = 9
Interpreter prints error: "TypeError: decorator() takes 0 positional arguments but 2 were given"
When you use a decorator, the function you return from the decorator replaces the old function. In other words, the decorator function in multiply_by_three replaces the add function.
This means that each functions signature's should match, including their arguments. However, in your code add takes two arguments while decorator takes none. You need to let decorator receive two arguments as well. You can do this easily by using *args and **kwargs:
def multiply_by_three(f):
def decorator(*args, **kwargs):
return f(*args, **kwargs) * 3
return decorator
If you now decorate your function and run it, you can see it works:
#multiply_by_three
def add(a, b):
return a + b
print(add(1,2)) # 9
How can I check if a function can always be called with the same arguments as another function? For example, b can be called with all arguments provided to a.
def a(a, b, c=None):
pass
def b(a, *args, d=4,**kwargs):
pass
The reason I want this is that I have a base function:
def f(a, b):
print('f', a, b)
and a list of callbacks:
def g(b, a):
print('g', a, b)
def h(*args, **kwargs):
print('h', args, kwargs)
funcs = [g, h]
and a wrapper function that accepts anything:
def call(*args, **kwargs):
f(*args, **kwargs)
for func in funcs:
func(*args, **kwargs)
Now I want to check if all callbacks will accept the arguments provided to call(), assuming they're valid for f().
For performance reasons, I don't want to check the arguments every time call() is called, but rather check each callback before adding it to the list of callbacks.
For example, those calls are ok:
call(1, 2)
call(a=1, b=3)
But this one should fail because g has arguments in wrong order:
call(1, b=3)
This took a bit of fun research, but i think i've covered the corner cases. A number of them arise to keep things compatible with python 2 while new syntax being added.
Most problematic part is the fact that some named (keyword) parameters can be passed in as positional argument or be required based on order passed in.
For more see comments.
Below code will ensure that function b can be called using any possible combination of valid arguments to function a. (does not imply inverse).
Uncomment/add try except block to get true/valse result and not an AssertionError.
import inspect
def check_arg_spec(a,b):
"""
attrs of FullArgSpec object:
sp.args = pos or legacy keyword arguments, w/ keyword at back
sp.varargs = *args
sp.varkw = **kwargs
sp.defaults = default values for legacy keyword arguments #
sp.args
sp.kwdonly = keyword arguments follow *args or *, must be passed in by name
sp.kwdonlydefaults = {name: default_val, .... }
sp.annotatons -> currently not in use, except as standard flag for outside applications
Consume order:
(1) Positional arguments
(2) legacy keyword argument = default (can be filled by both keyword or positional parameter)
[
(3) *args
[
(4) keyword only arguments [=default]
]
]
(5) **kwds
"""
a_sp = inspect.getfullargspec(a)
b_sp = inspect.getfullargspec(b)
kwdfb = b_sp.kwonlydefaults or {}
kwdfa = a_sp.kwonlydefaults or {}
kwddefb = b_sp.defaults or []
kwddefa = a_sp.defaults or []
# try:
akwd = a_sp.kwonlyargs
if len(kwddefa):
akwd += a_sp.args[-len(kwddefa):]
bkwd = b_sp.kwonlyargs
if len(kwddefb):
bkwd += b_sp.args[-len(kwddefb):]
# all required arguments in b must have name match in a spec.
for bkey in (set(b_sp.args) ^ set(bkwd)) & set(b_sp.args) :
assert bkey in a_sp.args
# all required positional in b can be met by a
assert (len(a_sp.args)-len(kwddefb)) >= (len(b_sp.args)-len(kwddefb))
# if a has *args spec, so must b
assert not ( a_sp.varargs and b_sp.varargs is None )
# if a does not take *args, max number of pos args passed to a is len(a_sp.args). b must accept at least this many positional args unless it can consume *args
if b_sp.varargs is None:
# if neither a nor b accepts *args, check that total number of pos plus py2 style keyword arguments for sg of b is more than a can send its way.
assert len(a_sp.args) <= len(b_sp.args)
# Keyword only arguments of b -> they are required, must be present in a.
akws = set(a_sp.kwonlyargs) | set(a_sp.args[-len(kwddefa):])
for nmreq in (set(b_sp.kwonlyargs)^set(kwdfb)) & set(b_sp.kwonlyargs):
assert nmreq in akws
# if a and b both accept an arbitrary number of positional arguments or if b can but a cannot, no more checks neccessary here
# if a accepts optional arbitrary, **kwds, then so must b
assert not (a_sp.varkw and b_sp.varkw is None)
if b_sp.varkw is None:
# neither a nor b can consume arbitrary keyword arguments
# then b must be able to consume all keywords that a can be called w/th.
for akw in akwd:
assert akw in bkwd
# if b accepts **kwds, but not a, then there is no need to check further
# if both accept **kwds, then also no need to check further
# return True
#
# except AssertionError:
#
# return False
Not sure what you are really looking for and I'm pretty sure your issue could be solved in a better way, but anyway:
from inspect import getargspec
def foo(a, b, c=None):
pass
def bar(a, d=4, *args, **kwargs):
pass
def same_args(func1, func2):
return list(set(getargspec(func1)[0]).intersection(set(getargspec(func2)[0])))
print same_args(foo, bar)
# => ['a']
same_args just check arguments from func1 and func2 and returns a new list with only same arguments in both func1 and func2.
For a decorator I am writing I would like to manipulate a specific named parameter of a function. Consider the following decorator:
def square_param(param):
def func_decorator(func):
def func_caller(*args,**kwargs):
kwargs[param] = kwargs[param] * kwargs[param]
return func(*args,**kwargs)
return func_caller
return func_decorator
Applied on the next function:
#square_param('dividend')
def quotient(divisor=1,dividend=0):
return dividend/divisor
This will work if dividend is called as a keyword argument e.g.:
>>> quotient(dividend=2)
4
However, when given as a positional argument this will fail.
>>> quotient(3,4)
TypeError: quotient() got multiple values for keyword argument 'dividend'
With Python 3 I could solve this by forcing the parameter to be always given as a keyword:
#square_param('dividend')
def quotient(divisor=1,*,dividend=0):
return dividend/divisor
but I would like to support Python 2 and also I would like to put as little restrictions on the function.
Is there a way that I can fix this behaviour in my decorator?
Firstly, your square_param decorator doesn't work because it doesn't return the functions. It needs to be:
def square_param(param):
def func_decorator(func):
def func_caller(*args,**kwargs):
kwargs[param] = kwargs[param] * kwargs[param]
return func(*args,**kwargs)
return func_caller
return func_decorator
Now I took #Dirk's advice and looked into the inspect module. You can do it by checking first if the parameter is one of the function's positional arguments, and second if that positional argument has been specified, and then modifying that argument position. You also need to make sure you only modify kwargs if the parameter was supplied as a keyword argument.
import inspect
def square_param(param):
def func_decorator(func):
def func_caller(*args,**kwargs):
funparams = inspect.getargspec(func).args
if param in funparams:
i = funparams.index(param)
if len(args) > i:
args = list(args) # Make mutable
args[i] = args[i] * args[i]
if param in kwargs:
kwargs[param] = kwargs[param] * kwargs[param]
return func(*args,**kwargs)
return func_caller
return func_decorator
even without using Inspect we can get function params
>>> func = lambda x, y, args: (x, y, {})
>>> func.func_code.co_argcount
3
>>> func.func_code.co_varnames
('x', 'y', 'args')
This may only be tangentially related, but I found it useful to solve a similar problem. I wanted to meld *args and **kwargs into a single dictionary so that my following code could process without regard to how the args came in, and I didn't want to mutate the existing kwargs variable, otherwise I just would have use kwargs.update().
all_args = {**kwargs, **{k: v for k, v in zip(list(inspect.signature(func).parameters), args)}}
# optionally delete `self`
del (all_args['self'])
Update: While this works, this answer has a better technique. In part:
bound_args = inspect.signature(f).bind(*args, **kwargs)
bound_args.apply_defaults()