When forgetting to pass certain arguments to a function, Python gives the only-somewhat-helpful message "myfunction() takes X arguments (Y given)". Is there a way to figure out the names of the missing arguments, and tell the user? Something like:
try:
#begin blackbox
def f(x,y):
return x*y
f(x=1)
#end blackbox
except Exception as e:
#figure out the missing keyword argument is called "y" and tell the user so
Assuming that the code between begin blackbox and end blackbox is unknown to the exception handler.
Edit: As its been pointed out to me below, Python 3 already has this functionality built in. Let me extend the question then, is there a (probably ugly and hacky) way to do this in Python 2.x?
A much cleaner way to do this would be to wrap the function in another function, pass through the *args, **kwargs, and then use those values when you need them, instead of trying to reconstruct them after the fact. But if you don't want to do that…
In Python 3.x (except very early versions), this is easy, as poke's answer explains. Even easier with 3.3+, with inspect.signature, inspect.getargvalues, and inspect.Signature.bind_partial and friends.
In Python 2.x, there is no way to do this. The exception only has the string 'f() takes exactly 2 arguments (1 given)' in its args.
Except… in CPython 2.x specifically, it's possible with enough ugly and brittle hackery.
You've got a traceback, so you've got its tb_frame and tb_lineno… which is everything you need. So as long as the source is available, the inspect module makes it easy to get the actual function call expression. Then you just need to parse it (via ast) to get the arguments passed, and compare to the function's signature (which, unfortunately, isn't nearly as easy to get in 2.x as in 3.3+, but between f.func_defaults, f.func_code.co_argcount, etc., you can reconstruct it).
But what if the source isn't available? Well, between tb_frame.f_code and tb_lasti, you can find out where the function call was in the bytecode. And the dis module makes that relatively easy to parse. In particular, right before the call, the positional arguments and the name-value pairs for keyword arguments were all pushed on the stack, so you can easily see which names got pushed, and how many positional values, and reconstruct the function call that way. Which you compare to the signature in the same way.
Of course that relies on the some assumptions about how CPython's compiler builds bytecode. It would be perfectly legal to do things in all kinds of different orders as long as the stack ended up with the right values. So, it's pretty brittle. But I think there are already better reasons not to do it.
I would argue that doing this doesn’t really make that much sense. Such an exception is thrown because the programmer missed specifying the argument. So if you knowingly catch the exception, then you could just as well just fix it in the first place.
That being said, in current Python 3 versions, the TypeError that is being thrown does mention which arguments are missing from the call:
"f() missing 1 required positional argument: 'y'"
Unfortunately, the argument name is not mentioned separately, so you would have to extract it from the string:
try:
f(x=1)
except TypeError as e:
if 'required positional argument' in e.args[0]:
argumentNames = e.args[0].split("'")[1::2]
print('Missing arguments are ' + argumentNames)
else:
raise # Re-raise other TypeErrors
As Joran Beasley pointed out in the comments, Python 2 does not tell you which arguments are missing but just how many are missing. So there is no way to tell from the exception which arguments were missing in the call.
except TypeError as e:
import inspect
got_args = int(re.search("\d+.*(\d+)",str(e)).groups()[0])
print "missing args:",inspect.getargspec(f).args[got_args:]
a better method would be a decorator
def arg_decorator(fn):
def func(*args,**kwargs):
try:
return fn(*args,**kwargs)
except TypeError:
arg_spec = inspect.getargspec(fn)
missing_named = [a for a in arg_spec.args if a not in kwargs]
if arg_spec.defaults:
missing_args = missing_named[len(args): -len(arg_spec.defaults) ]
else:
missing_args = missing_named[len(args):]
print "Missing:",missing_args
return func
#arg_decorator
def fn1(x,y,z):
pass
def fn2(x,y):
pass
arged_fn2 = arg_decorator(fn2)
fn1(5,y=2)
arged_fn2(x=1)
With purely the exception to deal with it is not possible to do what you want and handle keyword arguments. This is of course wrt Python 2.7.
The code that generates this message in Python is:
PyErr_Format(PyExc_TypeError,
"%.200s() takes %s %d "
"argument%s (%d given)",
PyString_AsString(co->co_name),
defcount ? "at most" : "exactly",
co->co_argcount,
co->co_argcount == 1 ? "" : "s",
argcount + kwcount);
Taken from lines 3056-3063 from http://hg.python.org/cpython/file/0e5df5b62488/Python/ceval.c
As you can see, there is just not enough information given to the exception as to what arguments are missing. co in this context is the PyCodeObject being called. The only thing given is a string (which you could parse if you like) with the function name, whether or not there is a vararg, how many arguments are expected, and how many arguments were given. As has been pointed out, this does not give you sufficient information as to what argument(s) were not given (in the case of keyword arguments).
Something like inspect or the other debugging modules might be able to give you enough information as to what function was called and how it was called, from which you could figure out what arguments were not given.
I should also mention however that almost certainly, whatever solution you come up with will not be able to handle at least some extension module methods (those written in C) because they don't provide argument information as part of their object.
Related
Background
I am new to python and I am writing a simple function but I am also interested in learning to do things the correct / pythonic way as I progress in my journey.
Lets consider the function below
def test_func(nested_lists,val):
return
I am expecting two arguments. One argument would be a list containing more lists. Something like this [[1,2,3,],[4,5,6,]...]. The second argument could be a value like 1.
If someone say for instance passes in a single value as the first argument and an array as the second argument. My code as it is currently returning the correct output which is 0 , However is there another way that i should be handle this?
For example should I be doing something like this
if(type(value) == list):
return 0
Or do i not need to do anything because my function is returning 0 anyway.
I know this maybe a very basic question so please forgive me but coming from a java background I am new to python so i am not sure how to handle such scenarios in python.
The other answer illustrates the proper way to check in advance for problems you can foresee. I'll provide a different approach.
The idiomatic solution in python is to "ask forgiveness, not permission". There are a lot of things that can go wrong, and where other languages might ask you to foresee all those problems and address them manually, python encourages just handling them as they happen. I would recommend doing:
def test_func(nested_lists, val):
try:
...
except TypeError:
# do whatever error-handling behavior you need to
# either throw a custom exception or return a specific value or whatever
return 0
and then designing your code in such a way that, if nested_lists and values are not compatible types, then they throw a TypeError (e.g. trying to iterate through nested_lists should fail if nested_lists is not a list. You can experiment with this behavior in a python console, but in general trying to do something to a variable that doesn't work because it's not the right type will produce a TypeError).
If your current code is working correctly, there is no pressing need to change anything. However, there are some reasons you might want to code more defensively;
If the code will seem to work correctly when you pass in bogus values, it would be better if it raised an exception instead of return a bogus value. The responsibility to call it correctly lies squarely with the caller, but enforcing it can help make sure the code is correct.
if not isinstance(nested_lists,list):
raise ValueError('Need a list, got {0!r}'.format(nested_lists))
This has the drawback that it hardcodes list as the type for the first argument; properly reusable code should work with any type, as long as it has the required methods and behaviors to remain compatible with your implementation. Perhaps instead check for a behavior:
try:
something involving nested_lists[0][0]
except (IndexError, AttributeError):
raise ValueError('Expected nested list but got {0!r}'.format(nested_lists))
(The try is not strictly necessary here; but see below.)
If you get a traceback when you call the code incorrectly, but it is opaque or misleading, it is more helpful to catch and explicitly point out the error earlier. #or example, the snippet above (without the try wrapper) would produce
Traceback (most recent call last):
module __main__ line 141
traceback.print_exc()
module <module> line 1
test_func(1,1)
module <module> line 2
AttributeError: 'int' object has no attribute '__getitem__'
which is somewhat unobvious to debug.
If the code will be used by third parties, both of the above considerations will be more important from a support point of view, too.
Notice how the code raises an exception when called incorrectly. This is generally better than silently returning some garbage value, and the caller can similarly trap the error with a try/except if this is well-defined (i.e. documented!) behavior.
Finally, since Python 3.5, you have the option to use type annotations:
def test_func(nested_lists: list, val: int) -> int:
...
As noted in the documentation, the core language does not (yet?) enforce these type checks, but they can help static code analysis tools point out possible errors.
I recently bumped into a "feature" of Python which I found not really well thought-out: Python raises TypeError if a function is called with the wrong number of arguments. I would like to understand what is the reason behind this choice.
Further details:
I find myself setting-up a callback system and, among other things, I check if a (user-defined) callback can accept or not an argument. I have to check this up-front when I am given the callback, because if I simply do something like:
try:
call_the_callback(one_argument)
except TypeError:
call_the_callback()
except Exception:
do_something_else()
I find myself in the uncomfortable situation of catching also a TypeError thrown by the inside-workings of the callback (maybe it expected a string instead of an integer, or something), which I definitely need to discriminate.
In the end, I bypassed this with a check as follows (I guess this is only for Python 3):
callsign = signature(action)
nargs = len(callsign.parameters)
and then using the appropriate call. However, apart from this specific problem, I simply cannot understand why putting under the same umbrella a call with the wrong number of arguments and, say, 1 / "three".
NOTE: I understand why an error is raised for a call with the wrong number of arguments, I definitely want that! My problem is with understanding why a TypeError is raised and not something more specific. I want to have a deeper understanding of this to better design strategies to tackle this kind of situation.
References:
Too generic, only saying that a call with the wrong number of arguments is "some kind of TypeError": Why TypeError when too many / too few args in Python function call
Discusses the issue but does not answer the question: https://bytes.com/topic/python/answers/45841-incorrect-number-arguments
I think almost all programming languages have mandatory parameters for a function and if those parameters are not received an exception is raised. This is good for enforcing a proper usage for that function. If function parameters are not mandatory you need more handling logic inside the function in order to cover all cases.
But you can use optional parameters or generic parameters along with or without mandatory params.
For example bellow is a function which allow you to have any number of parameters (simple or keyword parameters) and no param is mandatory:
def f1(*args, **kwargs):
print args
print kwargs
f1(1,'param', a=4)
For more details see:
generic function in python - calling a method with unknown number of arguments
I'm currently building up a library of personal debugging functions, and wanted to make a debugging print function that would provide more quality-of-life functionality than adding print(thing) lines everywhere. My current goal is to have this pseudocode function translated to Python:
def p(*args, **kwargs)
for arg in args:
print(uneval(arg), ": ", arg)
return args
The types of **kwargs and how they'd effect the output isn't as important to this question, hence why I don't use them here.
My main problem comes from trying to have some method of uneval-ing the arguments, such that I can get the string of code that actually produced them. As a use case:
>>> p(3 + 5) + 1
3 + 5: 8 # printed by p
9 # Return value of the entire expression
There's a couple of ways that I see that I can do this, and not only am I not sure if any of them work, I'm also concerned about how Pythonic any solution that actually implements these could possibly be.
Pass the argument as a string, eval it using the previous context's local identifier dictionary (how could I get that without passing it as another argument, which I definitely don't want to do?)
Figure out which line it's being run from in the .py file to extract the uneval'd strings that way (is that even possible?)
Find some metadata magic that has the information I need already in it (if it exists, which is unlikely at best).
I'm getting stuck with this
I have a python file which is imported from elsewhere as a module, in order to use some functions provided by it. I'm trying a way to call it form CLI, giving it 0 or 5 arguments.
def simulate(index, sourcefile, temperature_file, save=0, outfile='fig.png'):
(...)
# do calculations and spit a nice graph file.
if __name__ == '__main__':
if (len(sys.argv) == 6 ):
# ugly code alert
simulate(sys.argv[1], sys.argv[2], sys.argv[3], sys.argv[4], sys.argv[5])
else:
(...)
#do some other things and don't bother me
I was wondering if there's a clean way to pass all but first argument to a function.
I tried simulate(sys.argv[1:]) but it throws a single object (list), and since simulate function expects 4 arguments, it doesn't work: TypeError: 'simulate() takes at least 3 arguments (1 given)'
Tried also with simulate(itertools.chain(sys.argv[1:])) with same result.
Since this file is imported elsewhere as a module and this function is being called many times, it seems a bad idea to change the function's signature to recieve a single argument
simulate(*sys.argv[1:])
See "Unpacking Argument Lists" in the tutorial
What you want to use is called "Packing/Unpacking" in Python:
foo(*sys.argv)
See: http://en.wikibooks.org/wiki/Python_Programming/Tuples#Packing_and_Unpacking
If you want "all but first argument":
foo(*sys.argv[1:])
This is called "slicing". See: http://docs.python.org/2.3/whatsnew/section-slices.html
Suppose in python you have a routine that accepts three named parameters (as **kwargs), but any two out of these three must be filled in. If only one is filled in, it's an error. If all three are, it's an error. What kind of error would you raise? RuntimeError, a specifically created exception, or other?
Remember that you can subclass Python's built-in exception classes (and TypeError would surely be the right built-in exception class to raise here -- that's what Python raises if the number of arguments does not match the signature, in normal cases without *a or **k forms in the signature). I like having every package define its own class Error(Exception), and then specific exceptions as needed can multiply inherit as appropriate, e.g.:
class WrongNumberOfArguments(thispackage.Error, TypeError):
Then, I'd raise WrongNumberOfArguments when I detect such a problem situation.
This way, any caller who's aware of this package can catch thispackage.Error, if they need to deal with any error specific to the package, while other callers (presumably up higher in the call chain) call still catch the more generic TypeError to deal with any errors such as "wrong number of arguments used in a function call".
Why not just do what python does?
>>> abs(1, 2, 3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: abs() takes exactly one argument (3 given)
If (as you say in one of the comments) that this is a programmer error, then you can raise AssertionError:
def two(**kwargs):
assert len(kwargs) == 2, "Please only provide two args"
BTW, if you only have three named arguments, **kwargs seems like an odd way to do it. More natural might be:
def two(a=None, b=None, c=None):
pass
I would make a specific one. You can catch it and deal with that specific exception since it is a special circumstance that you created :)
I would use a ValueError, or a subclass thereof: "Raised when a built-in operation or function receives an argument that has the right type but an inappropriate value, and the situation is not described by a more precise exception such as IndexError."
Passing 3 or 1 values when exactly 2 are required would technically be an inappropriate value if you consider all of the arguments a single tuple... At least in my opinion! :)
I recommend a custom exception. Like so:
class NeedExactlyTwo(ValueError):
pass
Then you can raise NeedExactlyTwo in your code.
Be sure to document this in the docstring for your function.