Related
If I have a function
def foo(x, y):
pass
how can I tell, from inside the function, whether y was passed positionally or with its keyword?
I'd like to have something like
def foo(x, y):
if passed_positionally(y):
print('y was passed positionally!')
else:
print('y was passed with its keyword')
so that I get
>>> foo(3, 4)
y was passed positionally
>>> foo(3, y=4)
y was passed with its keyword
I realise I didn't originally specify this, but is it possible to do this whilst preserving type annotations? The top answer so far suggests using a decorator - however, that does not preserve the return type
You can create a decorator, like this:
def checkargs(func):
def inner(*args, **kwargs):
if 'y' in kwargs:
print('y passed with its keyword!')
else:
print('y passed positionally.')
result = func(*args, **kwargs)
return result
return inner
>>> #checkargs
...: def foo(x, y):
...: return x + y
>>> foo(2, 3)
y passed positionally.
5
>>> foo(2, y=3)
y passed with its keyword!
5
Of course you can improve this by allowing the decorator to accept arguments. Thus you can pass the parameter you want to check for. Which would be something like this:
def checkargs(param_to_check):
def inner(func):
def wrapper(*args, **kwargs):
if param_to_check in kwargs:
print('y passed with its keyword!')
else:
print('y passed positionally.')
result = func(*args, **kwargs)
return result
return wrapper
return inner
>>> #checkargs(param_to_check='y')
...: def foo(x, y):
...: return x + y
>>> foo(2, y=3)
y passed with its keyword!
5
I think adding functools.wraps would preserve the annotations, following version also allows to perform the check over all arguments (using inspect):
from functools import wraps
from inspect import signature
def checkargs(func):
#wraps(func)
def inner(*args, **kwargs):
for param in signature(func).parameters:
if param in kwargs:
print(param, 'passed with its keyword!')
else:
print(param, 'passed positionally.')
result = func(*args, **kwargs)
return result
return inner
>>> #checkargs
...: def foo(x, y, z) -> int:
...: return x + y
>>> foo(2, 3, z=4)
x passed positionally.
y passed positionally.
z passed with its keyword!
9
>>> inspect.getfullargspec(foo)
FullArgSpec(args=[], varargs='args', varkw='kwargs', defaults=None,
kwonlyargs=[], kwonlydefaults=None, annotations={'return': <class 'int'>})
_____________HERE____________
Update Python 3.10
In Python 3.10+ new ParamSpec type annotation was introduced (PEP 612), for better specifying parameter types in higher-order functions. As of now, the correct way to annotate this decorator would be like this:
from functools import wraps
from inspect import signature
from typing import Callable, ParamSpec, TypeVar, TYPE_CHECKING
T = TypeVar("T")
P = ParamSpec("P")
def check_args(func: Callable[P, T]) -> Callable[P, T]:
"""
Decorator to monitor whether an argument is passed
positionally or with its keyword, during function call.
"""
#wraps(func)
def inner(*args: P.args, **kwargs: P.kwargs) -> T:
for param in signature(func).parameters:
if param in kwargs:
print(param, 'passed with its keyword!')
else:
print(param, 'passed positionally.')
return func(*args, **kwargs)
return inner
Which correctly preserves type annotation:
if TYPE_CHECKING:
reveal_type(foo(2, 3))
# ─❯ mypy check_kwd.py
# check_kwd.py:34: note: Revealed type is "builtins.int"
# Success: no issues found in 1 source file
At the end, if you are going to do something like this:
def foo(x, y):
if passed_positionally(y):
raise Exception("You need to pass 'y' as a keyword argument")
else:
process(x, y)
You can do this:
def foo(x, *, y):
pass
>>> foo(1, 2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: foo() takes 1 positional argument but 2 were given
>>> foo(1, y=2) # works
Or only allow them to be passed positionally:
def foo(x, y, /):
pass
>>> foo(x=1, y=2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: foo() got some positional-only arguments passed as keyword arguments: 'x, y'
>>> foo(1, 2) # works
See PEP 570 and PEP 3102 for more.
Adapted from #Cyttorak 's answer, here's a way to do it which maintains the types:
from typing import TypeVar, Callable, Any, TYPE_CHECKING
T = TypeVar("T", bound=Callable[..., Any])
from functools import wraps
import inspect
def checkargs() -> Callable[[T], T]:
def decorate(func):
#wraps(func)
def inner(*args, **kwargs):
for param in inspect.signature(func).parameters:
if param in kwargs:
print(param, 'passed with its keyword!')
else:
print(param, 'passed positionally.')
result = func(*args, **kwargs)
return result
return inner
return decorate
#checkargs()
def foo(x, y) -> int:
return x+y
if TYPE_CHECKING:
reveal_type(foo(2, 3))
foo(2, 3)
foo(2, y=3)
Output is:
$ mypy t.py
t.py:27: note: Revealed type is 'builtins.int'
$ python t.py
x passed positionally.
y passed positionally.
x passed positionally.
y passed with its keyword!
It is not ordinarily possible. In a sense: the language is not designed to allow you to distinguish both ways.
You can design your function to take different parameters - positional, and named, and check which one was passed, in a thing like:
def foo(x, y=None, /, **kwargs):
if y is None:
y = kwargs.pop(y)
received_as_positional = False
else:
received_as_positional = True
The problem is that, although by using positional only parameters as abov, you could get y both ways,
it would be shown not once for a user (or IDE) inspecting the
function signature.
I hav a feeling you just want to know this for the sake of knowing - if
you really intend this for design of an API, I'd suggest you'd rethink
your API - there should be no difference in the behavior, unless both
are un-ambiguously different parameters from the user point of view.
That said, the way to go would be to inspect the caller frame, and check
the bytecode around the place the function is called:
In [24]: import sys, dis
In [25]: def foo(x, y=None):
...: f = sys._getframe().f_back
...: print(dis.dis(f.f_code))
...:
In [26]: foo(1, 2)
1 0 LOAD_NAME 0 (foo)
2 LOAD_CONST 0 (1)
4 LOAD_CONST 1 (2)
6 CALL_FUNCTION 2
8 PRINT_EXPR
10 LOAD_CONST 2 (None)
12 RETURN_VALUE
None
In [27]: foo(1, y=2)
1 0 LOAD_NAME 0 (foo)
2 LOAD_CONST 0 (1)
4 LOAD_CONST 1 (2)
6 LOAD_CONST 2 (('y',))
8 CALL_FUNCTION_KW 2
10 PRINT_EXPR
12 LOAD_CONST 3 (None)
14 RETURN_VALUE
So, as you can see, when y is called as named parameter, the opcode for the call is CALL_FUNCTION_KW , and the name of the parameter is loaded into the stack imediately before it.
You can trick the user and add another argument to the function like this:
def foo(x,y1=None,y=None):
if y1 is not None:
print('y was passed positionally!')
else:
print('y was passed with its keyword')
I don't recommend doing it but it does work
In foo, you can pass the call stack from traceback to positionally, which will then parse the lines, find the line where foo itself is called, and then parse the line with ast to locate positional parameter specifications (if any):
import traceback, ast, re
def get_fun(name, ast_obj):
if isinstance(ast_obj, ast.Call) and ast_obj.func.id == name:
yield from [i.arg for i in getattr(ast_obj, 'keywords', [])]
for a, b in getattr(ast_obj, '__dict__', {}).items():
yield from (get_fun(name, b) if not isinstance(b, list) else \
[i for k in b for i in get_fun(name, k)])
def passed_positionally(stack):
*_, [_, co], [trace, _] = [re.split('\n\s+', i.strip()) for i in stack]
f_name = re.findall('(?:line \d+, in )(\w+)', trace)[0]
return list(get_fun(f_name, ast.parse(co)))
def foo(x, y):
if 'y' in passed_positionally(traceback.format_stack()):
print('y was passed with its keyword')
else:
print('y was passed positionally')
foo(1, y=2)
Output:
y was passed with its keyword
Notes:
This solution does not require any wrapping of foo. Only the traceback needs to be captured.
To get the full foo call as a string in the traceback, this solution must be run in a file, not the shell.
It seems like functools.wraps breaks the wrapped function when keyword only kinds are involved. Did I find a stlib bug, or is there something I'm missing
Consider this code:
from functools import wraps, WRAPPER_ASSIGNMENTS, partial
# First, I need to add `__defaults__` and `__kwdefaults__` to wraps, because they don't come for free...
my_wraps = partial(wraps, assigned=(list(WRAPPER_ASSIGNMENTS) + ['__defaults__', '__kwdefaults__']))
def g(a: float, b=10):
return a * b
def f(a: int, *, b=1):
return a * b
# all is well (for now)...
assert f(3) == 3
assert g(3) == 30
Whether I do this:
g = my_wraps(f)(g)
g(3) # raises TypeError (missing required positional argument 'b'), expected
or this:
f = my_wraps(g)(f)
f(3) # raises TypeError (missing required positional argument 'b'), expected
my wrapped function breaks.
def f(x, y):
return x & 1 == 0 and y > 0
g = lambda x, y: x & 1 == 0 and y > 0
Now the same thing in Haskell:
import Data.Bits
f :: Int -> Int -> Bool
f x y = (.&.) x 1 == 0 && y > 0
That works, however this doesn't:
g = \x y -> (.&.) x 1 == 0 && y > 0
Here's the error this gives:
someFunc :: IO ()
someFunc = putStrLn $ "f 5 7: " ++ ( show $ f 5 7 ) ++ "\tg 5 7: " ++ ( show $ g 5 7 )
• Ambiguous type variable ‘a0’ arising from the literal ‘1’
prevents the constraint ‘(Num a0)’ from being solved.
Relevant bindings include
x :: a0 (bound at src/Lib.hs:13:6)
g :: a0 -> Integer -> Bool (bound at src/Lib.hs:13:1)
Probable fix: use a type annotation to specify what ‘a0’ should be.
These potential instances exist:
instance Num Integer -- Defined in ‘GHC.Num’
instance Num Double -- Defined in ‘GHC.Float’
instance Num Float -- Defined in ‘GHC.Float’
...plus two others
...plus one instance involving out-of-scope types
(use -fprint-potential-instances to see them all)
• In the second argument of ‘(.&.)’, namely ‘1’
In the first argument of ‘(==)’, namely ‘(.&.) x 1’
In the first argument of ‘(&&)’, namely ‘(.&.) x 1 == 0’
|
13 | g = \x y -> (.&.) x 1 == 0 && y > 0
| ^
How do I get the same error in Python? - How do I get errors when the input doesn't match expectations?
To be specific, how do I say that a function/lambda MUST have:
arity of 5
each argument must be numerical
each argument must implement __matmul__ (#)
return bool
I know that I can roughly do this with: (docstrings and/or PEP484) with abc; for classes. But what can I do for 'loose' functions in a module?
Generally you have three possible approaches:
Let the runtime produce errors for incompatible operations (e.g. 1 + 'foo' is a TypeError).
Do explicit runtime checks for certain attributes; though this is usually discouraged since Python uses duck typing a lot.
Use type annotations and a static type checker.
Your specific points:
arity of 5
Define five parameters:
def f(a, b, c, d, e): ...
each argument must be numerical
Either static type annotations:
def f(a: int, b: int, c: int, d: int, e: int): ...
And/or runtime checks:
def f(a, b, c, d, e):
assert all(isinstance(i, int) for i in (a, b, c, d, e))
def f(a, b, c, d, e):
if not all(isinstance(i, int) for i in (a, b, c, d, e)):
raise TypeError
asserts are for debugging purposes and can be disabled, an explicit if..raise cannot. Given the verboseness of this and the duck typing philosophy, these approaches are not very pythonic.
each argument must implement __matmul__ (#)
The most practical way is probably to let the runtime raise an error intrinsically if the passed values do not support the operation, i.e. just do:
def f(a, b):
return a # b # TypeError: unsupported operand type(s) for #: ... and ...
If you want static type checking for this, you can use a typing.Protocol:
from typing import Protocol
class MatMullable(Protocol):
def __matmul__(self, other) -> int:
pass
def f(a: MatMullable, ...): ...
In practice you probably want to combine this with your previous "each argument must be numerical" and type hint for a type that fulfils both these requirements.
return bool
def f(...) -> bool: ...
Especially given that the # operator is mostly used by 3rd party packages like numpy, in practice the most pythonic implementation of such a function is probably something along these lines:
import numpy as np
from numpy import ndarray
def f(a: ndarray, b: ndarray, c: ndarray, d: ndarray, e: ndarray) -> bool:
return np.linalg.det(a # b # c # d # e) > 0 # or whatever
You can't directly translate the same typing expectations from Haskell to Python. Haskell is an insanely strongly typed language, while Python is almost the complete opposite.
To type hint a higher order function that accepts such a function as argument, use typing.Callable:
from typing import Callable
def hof(f: Callable[[ndarray, ndarray, ndarray, ndarray, ndarray], bool]): ...
I have a function with two optional parameters:
def func(a=0, b=10):
return a+b
Somewhere else in my code I am doing some conditional argument passing like:
if a and b:
return func(a, b)
elif a:
return func(a)
elif b:
return func(b=b)
else:
return func()
Is there anyway to simplify code in this pattern?
EDIT:
Let's assume that I'm not allowed to implement default parameter logic inside func.
I may have several functions like func: func1, func2 and func3 would all contain the
a = a or 0
b = b or 10
statements.
But I'm invoking these series of functions to eliminate duplication. (using a decorator)
If you don't want to change anything in func then the sensible option would be passing a dict of arguments to the function:
>>> def func(a=0,b=10):
... return a+b
...
>>> args = {'a':15,'b':15}
>>> func(**args)
30
>>> args={'a':15}
>>> func(**args)
25
>>> args={'b':6}
>>> func(**args)
6
>>> args = {}
>>> func(**args)
10
or just:
>>>func(**{'a':7})
17
You can add a decorator that would eliminate None arguments:
def skip_nones(fun):
def _(*args, **kwargs):
for a, v in zip(fun.__code__.co_varnames, args):
if v is not None:
kwargs[a] = v
return fun(**kwargs)
return _
#skip_nones
def func(a=10, b=20):
print a, b
func(None, None) # 10 20
func(11, None) # 11 20
func(None, 33) # 10 33
Going by the now-deleted comments to the question that the check is meant to be for the variables being None rather than being falsey, change func so that it handles the arguments being None:
def func(a=None, b=None):
if a is None:
a = 0
if b is None:
b = 10
And then just call func(a, b) every time.
to solve your specific question I would do:
args = {'a' : a, 'b' : b}
for varName, varVal in args.items():
if not varVal:
del args[varName]
f(**args)
But the most pythonic way would be to use None as the default value in your function:
f(a=None, b=None):
a = 10 if a is None else a
...
and just call f(a, b)
By default, all methods in Python take variable arguments.
When you define an argument in the signature of the method, you explicity make it required. In your snippet, what you are doing is giving it a default value - not making them optional.
Consider the following:
>>> def func(a,b):
... a = a if a else 0
... b = b if b else 10
... return a+b
...
>>> a = b = None
>>> func(a,b)
10
>>> a = 5
>>> b = 2
>>> func(a,b)
7
>>> func()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: func() takes exactly 2 arguments (0 given)
In this snippet, both a and b are required, since I didn't define any default values. They are not optional.
However in your snippet, because you have given them defaults in the method signature they look like they are optional, but in fact they just have defaults assigned to them.
>>> def func(a=0,b=10):
... return a+b
...
>>> func()
10
You can confirm this by checking the argument list inside the body of the method:
>>> def func(a=b,b=10):
... print locals().keys()
...
>>> func()
['a', 'b']
One way to have your function accept any number of arguments (in other words, making them all optional):
>>> def func(*args,**kwargs):
... print len(args),args
... print len(kwargs),kwargs
...
>>> func("hello","world",a=5,b=10)
2 ('hello', 'world')
2 {'a': 5, 'b': 10}
>>> func()
0 ()
0 {}
>>> func(1,2)
2 (1, 2)
0 {}
>>> func(a)
1 (5,)
0 {}
>>> func(foo="hello")
0 ()
1 {'foo': 'hello'}
Why not pass that logic to the function?
def func(a, b):
a = a or 0
b = b or 10
return a + b
To answer your literal question:
func(a or 0, b or 10)
Edit:
See comment why this doesn't answer the question.
You can use the ternary if-then operator to pass conditional arguements into functions
https://www.geeksforgeeks.org/ternary-operator-in-python/
For example, if you want to check if a key actually exists before passing it you can do something like:
def func(arg):
print(arg)
kwargs = {'name':'Adam')
func( kwargs['name'] if 'name' in kwargs.keys() else '' ) #Expected Output: 'Adam'
func( kwargs['notakey'] if 'notakey' in kwargs.keys() else '' ) #Expected Output: ''
This might work:
def f(**kwargs):
a = get(kwargs, 0)
b = get(kwargs, 10)
return a + b
One of the changes between python 2 and 3 is that the latter delegates to x.__round__([n]) the operation round(x, n). In python 2, for my classes implementing __round__ and __float__, when I call round(x), x.__float__ is called.
How can I know that round(x) (and not float(x)) was called to reroute the call appropriate in python 2 and obtain a python 3 like behaviour.
thanks
Update: I came up with an ugly hack. I am sure that:
it can be improved.
it will not always work.
The ndigits parameter is not handled in python 2.
it should not be used in production.
but it was interesting to build it anyway. Thanks for all the clarifications.
import dis
import sys
import inspect
import functools
#'CALL_FUNCTION', 'CALL_FUNCTION_VAR', 'CALL_FUNCTION_KW', 'CALL_FUNCTION_VAR_KW'
HUNGRY = (131, 140, 141, 142)
if sys.version < '3':
def is_round(frame):
"""Disassemble a code object."""
co = frame.f_code
lasti = frame.f_lasti
code = co.co_code
i, n = 0, len(code)
extended_arg = 0
free = None
codes = list()
while i < n:
c = code[i]
op = ord(c)
tmp = [op, ]
i += 1
if op >= dis.HAVE_ARGUMENT:
oparg = ord(code[i]) + ord(code[i + 1]) * 256 + extended_arg
extended_arg = 0
i += 2
if op == dis.EXTENDED_ARG:
extended_arg = oparg * long(65536)
tmp.append(oparg)
if op in dis.hasconst:
tmp.append(repr(co.co_consts[oparg]))
elif op in dis.hasname:
tmp.append(co.co_names[oparg])
elif op in dis.hasjrel:
tmp.append(repr(i + oparg)),
elif op in dis.haslocal:
tmp.append(co.co_varnames[oparg])
elif op in dis.hascompare:
tmp.append(dis.cmp_op[oparg])
elif op in dis.hasfree:
if free is None:
free = co.co_cellvars + co.co_freevars
tmp.append(free[oparg])
else:
tmp.append(None)
else:
tmp.append(None)
tmp.append(None)
codes.append(tmp)
if i > lasti:
break
pending = 1
for (opcode, arguments, param) in reversed(codes):
pending -= 1
if opcode in HUNGRY:
pending += arguments + 1
if not pending:
seen = dict(frame.f_builtins)
seen.update(frame.f_globals)
seen.update(frame.f_locals)
while param in seen:
param = seen[param]
return param == round
def round_check(func):
#functools.wraps(func)
def wrapped(self):
if is_round(inspect.currentframe().f_back):
return self.__round__()
return func(self)
return wrapped
else:
def round_check(func):
return func
class X():
#round_check
def __float__(self):
return 1.0
def __round__(self, ndigits=0):
return 2.0
x = X()
r = round
f = float
assert round(x) == 2.0
assert float(x) == 1.0
assert r(x) == 2.0
assert f(x) == 1.0
assert round(float(x)) == 1.0
assert float(round(x)) == 2.0
You could always redefine round to try __round__ first. Unfortunately this isn't a __future__ import, so I don't think there's much else you can do.
>>> class X(object):
... def __round__(self, n=0): return 1.
... def __float__(self): return 2.
...
>>> x = X()
>>> round(x)
2.0
>>> float(x)
2.0
>>> old_round = round
>>> def round(x, n=0):
... try:
... return x.__round__(n)
... except AttributeError:
... return old_round(x)
...
>>>
>>> round(x)
1.0
>>> float(x)
2.0
>>>
Note that this is at least a documented change:
The round() function rounding strategy and return type have changed.
Exact halfway cases are now rounded to the nearest even result instead
of away from zero. (For example, round(2.5) now returns 2 rather than
3.) round(x[, n])() now delegates to x.__round__([n]) instead of always returning a float. It generally returns an integer when called
with a single argument and a value of the same type as x when called
with two arguments.
In Python 2, you can not override what round() does. It does not delegate to __float__; it first calls float() (which in turn delegates to __float__), then does the rounding. There is therefore no point in knowing if __float__ is called from round() or not, as it will do the rounding for you. You can't delegate it.
If you want to implement your own custom rounding in Python 2, you should implement a custom_round() method that does the custom rounding, and use that instead of round().
How can I know that round(x) (and not float(x)) was called to reroute the call appropriate in python 2 and obtain a python 3 like behaviour.
You don't need to. if round(x) calls your __float__ method, it will round the returned floats using the normal logic for floats. You do not need to consider it in the __float__ implementation; you should return the same thing regardless of the calling context. Everything else is the calling context's responsibility.
>>> class hax(object):
... def __float__(self): return 2.6
...
>>> round(hax())
3.0