In Python, 'Inspecting' callable objects that aren't strictly [functions?] - python

I've found a number of questions and answers dealing with determining the parameters/arguments needed for functions* in python. Python has a number of callable objects (using callable) that will return a value for which inspect.getfullargspec returns an error (inspect.Signature seems less reliable by my informal testing?).
For example:
inspect.getfullargspec(max)
Returns an Error:
TypeError: unsupported callable
Is there an alternative to inspect that will work for otherwise "unsupported callable" [functions/objects?]
Looking at a number of builtin callable objects it seems like many are in the "unsupported" category, i can only guess at objects in other modules/libraries.
*I may be mincing my terms here, apologies - any way to determine specifically which is which?

In CPython, builtin functions are implemented in C, and thus have many quirks. This is one of them. Taking some examples from PEP 570:
In current versions of Python, many CPython "builtin" and standard
library functions only accept positional-only parameters. The
resulting semantics can be easily observed by calling one of these
functions using keyword arguments:
>>> help(pow)
...
pow(x, y, z=None, /)
...
>>> pow(x=5, y=3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: pow() takes no keyword arguments
pow() expresses that its parameters are positional-only via the / marker.
However, this is only a documentation convention; Python developers
cannot use this syntax in code.
There are functions with other interesting semantics:
range(), an overloaded function, accepts an optional parameter to the
left of its required parameter.
dict(), whose mapping/iterator
parameter is optional and semantically must be positional-only. Any
externally visible name for this parameter would occlude that name
going into the **kwarg keyword variadic parameter dict.
Currently (while the PEP is not implemented) Python has no official way to express these unusual semantics such as positional only arguments.

Related

Code style in using argument names

What's the recommended style for argument names?
open(file='myfile.txt')
or
open('myfile.txt')
PEP8 Style Guide For Python doesn't mention anything about this. When do I use one or the other?
The main benefit I see in using explicit names is that the modification of a function's signature will have less impact on code relying on it.
For instance, say you're using a module's function, defined as:
their_function(a, b, c=1)
You're calling it with the c keyword argument by:
their_function(myA, myB, myC)
But now, the module's developers find it useful to have another keyword argument, and in their mind, it makes more sense that it comes before c.
So now, the function is:
their_function(a, b, d=2, c=1)
Everywhere you call their_function(myA, myB, myC), now myC is passed to their_function as d, and everything is messed up.
On the other hand, had you called it by their_function(myA, myB, c=myC), the mapping of the keyword arguments would have been so that myC would have still been passed to their_function as c.
Of course, this is probably overkill for obvious functions, such as print or open whose, positional argument is natural.
But I find it really reassuring to call open(path, 'r', encoding='utf8'), rather than open(path, 'r', 'utf8'), because even if I got the order wrong, the behaviour would still be as expected.
As for me, except in a few cases where it would be counter-intuitive, I tend to force the use of the names for the keyword arguments.
Python 3, from some version on, allows you to do the following:
def my_function(a, b, *, c=1):
pass
Here, the use of the splat operator * alone tells Python that no positional argument can be found after the third one.
This will result in a TypeError when passing a fourth argument as a positional one, ie without naming it.
>>> my_function(1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: my_function() missing 1 required positional argument: 'b'
>>> my_function(1, 2)
# OK
>>> my_function(1, 2, 3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: my_function() takes 2 positional arguments but 3 were given
>>> my_function(1, 2, c=3)
# OK
This helps you make your code a little more fool-proof, especially when defining functions with many positional or keyword arguments.
You'll get different opinions. Personally I would argue that using keyword arguments whenever possible is strictly better because it attaches human readable semantics to a function call. The reader has a decent shot at guessing what the arguments are supposed to be without inspecting the code/docs any further.
However, I do sometimes omit the keyword(s) when using well-known functions.

Why can't I use named arguments in Python 2 for the functions from the standard library [duplicate]

I'm new at Python, and I'm trying to basically make a hash table that checks if a key points to a value in the table, and if not, initializes it to an empty array. The offending part of my code is the line:
converted_comments[submission.id] = converted_comments.get(submission.id, default=0)
I get the error:
TypeError: get() takes no keyword arguments
But in the documentation (and various pieces of example code), I can see that it does take a default argument:
https://docs.python.org/2/library/stdtypes.html#dict.get
http://www.tutorialspoint.com/python/dictionary_get.htm
Following is the syntax for get() method:
dict.get(key, default=None)
There's nothing about this on The Stack, so I assume it's a beginner mistake?
Due to the way the Python C-level APIs developed, a lot of built-in functions and methods don't actually have names for their arguments. Even if the documentation calls the argument default, the function doesn't recognize the name default as referring to the optional second argument. You have to provide the argument positionally:
>>> d = {1: 2}
>>> d.get(0, default=0)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: get() takes no keyword arguments
>>> d.get(0, 0)
0
The error message says that get takes no keyword arguments but you are providing one with default=0
converted_comments[submission.id] = converted_comments.get(submission.id, 0)
Many docs and tutorials, for instance https://www.tutorialspoint.com/python/dictionary_get.htm, erroneously specify the syntax as
dict.get(key, default = None)
instead of
dict.get(key, default)

Is there a complete list of built-in functions that cannot be called with keyword argument?

People mentioned in answers a1, a2 that
Due to the way the Python C-level APIs developed, a lot of built-in
functions and methods don't actually have names for their arguments.
I found it really annoying cause I'm not be able to know it by looking at the doc. For instance, eval
eval(expression, globals=None, locals=None)
Then I wrote this line of code
print(eval('a+b', globals={'a':1, 'b':2}))
and got TypeError: eval() takes no keyword arguments. So is there a complete list of functions of this kind? How do I know if a function is allowed to have keyword arguments?
In Python 3.5 you can inspect the __text_signature__ of the built-in function:
>>> eval.__text_signature__
'($module, source, globals=None, locals=None, /)'
or
>>> abs.__text_signature__
'($module, x, /)'
>>> abs(x=5)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: abs() takes no keyword arguments
(x cannot be used as a keyword argument)
The / means that the arguments following that can be used as keyword arguments. C.f.
>>> compile.__text_signature__
'($module, /, source, filename, mode, flags=0,\n dont_inherit=False, optimize=-1)'
>>> compile(source='foo', filename='bar', mode='exec')
<code object <module> at 0x7f41c58f0030, file "bar", line 1>
Of course there are bugs even in 3.5:
>>> sorted.__text_signature__
'($module, iterable, key=None, reverse=False)'
though according to issue 26729 in the Python bug tracker, there ought to be / after the iterable as the iterable cannot be used as a keyword argument.
Unfortunately this information is not yet available in the Python documentation itself.

Explanation of Python doc argument syntax

Is anyone able to help me to understand the syntax of arguments being passed to some methods in the Python doc?
Examples of the type of things that are confusing me would be from the iter() function
iter(o[, sentinel])
From my understanding this is equivalent to
iter(o, sentinel)
but as to why I really don't understand.
function(mandatory_argument[, optional_argument] represents an optional argument that will alter the function if provided. In the iter() documentation:
The first argument is interpreted very differently depending on the presence of the second argument.
In what way optional arguments alters the function should be described in the documentation to it.
Optional arguments can be nested, so you may see something like (source):
bytearray([source[, encoding[, errors]]])
That means that each of the arguments are optional, but builtin upon the previous ones. So the following are all valid calls:
bytearray(source)
bytearray(source, encoding)
bytearray(source, encoding, errors)
But this is not:
bytearray(source, errors=errors)
There is a second way to indicate that arguments are optional:
__import__(name, globals=None, locals=None, fromlist=(), level=0)
This tells us that all these arguments (but name) are optional and also tells us the default for when we do not provide arguments for them.
On the code site in pure python you can get optional arguments by:
def iter(o, sentinel=None):
[do something]
But this would not be documented the way above, as we can see in the example of __import__:
__import__(name, globals=None, locals=None, fromlist=(), level=0)
To see why iter is different read the section at the end of my post.
Also note that in the example of the iter() builtin you can not provide sentinel as keyword argument and trying will raise a TypeError:
>>> iter([], sentinel=None)
Traceback (most recent call last):
File '<stdin>', line1, in <module>
TypeError: iter() takes no keyword arguments
in other cases though it may be possible:
>>> bytearray('', encoding='UTF-8')
bytearray(b'')
It is still true that providing a later argument without the previous ones will raise an error.
>>> bytearray('', errors='')
Traceback (most recent call last):
File '<stdin>', line 1, in <module>
TypeError: string argument without an encoding
The "keyword like syntax" is the "normal" way to document optional arguments in python. Why is iter different? iter is a builtin and not implemented in python but in C. If we look at the source code of it we see that it treats the arguments as a tuple that may have one or two arguments.
builtin_iter(PyObject *self, PyObject *args)
{
PyObject *v, *w = NULL;
if (!PyArg_UnpackTuple(args, "iter", 1, 2, &v, &w))
return NULL;
if (w == NULL)
return PyObject_GetIter(v);
if (!PyCallable_Check(v)) {
PyErr_SetString(PyExc_TypeError,
"iter(v, w): v must be callable");
return NULL;
}
return PyCallIter_New(v, w);
}
This might explain the "list-like syntax". It seems that the [optional_argument] notation is only used in modules that are programmed in C. For the normal user it makes no difference if there is
function([optional_argument])
or
function(optional_argument=True)
Brackets means that the argument in question is optional.
iter(o[, sentinel])
I.e. the above example means that iter is a function which takes one mandatory argument o and one optional argument sentinel.
This means that you can call this function like this:
iter(o) # Method 1
or like this:
iter(o, sentinel) # Method 2
The behavior of the function depending on if you use method 1 or 2 is described by the text in the docs:
Return an iterator object. The first argument is interpreted very
differently depending on the presence of the second argument. Without
a second argument, o must be a collection object which supports the
iteration protocol (the iter() method), or it must support the
sequence protocol (the getitem() method with integer arguments
starting at 0). If it does not support either of those protocols,
TypeError is raised. If the second argument, sentinel, is given, then
o must be a callable object. The iterator created in this case will
call o with no arguments for each call to its next() method; if the
value returned is equal to sentinel, StopIteration will be raised,
otherwise the value will be returned.

Python method overload based on argument count?

If I call QApplication's init without arguments i get
TypeError: arguments did not match any overloaded call:
QApplication(list-of-str): not enough arguments
QApplication(list-of-str, bool): not enough arguments
QApplication(list-of-str, QApplication.Type): not enough arguments
QApplication(Display, int visual=0, int colormap=0): not enough arguments
QApplication(Display, list-of-str, int visual=0, int cmap=0): not enough arguments
very interesting! How can I write a class like that?? I mean, every trick for this kind of function overloading I saw did not involve explicit signatures.
TypeError is just another Exception. You can take *args **kwargs, check those, and raise a TypeError yourself, specify the text displayed - e.g. listing the expected call.
That being said, PyQt is a bunch of .pyd == native python extension, written in C or C++ (using Boost::Python). At least the latter supports "real" overloads afaik.
Either way, you shouldn't do this unless you have a really good reason. Python is duck-typed, embrace it.
It's quite possible that its init is simply using __init__(self, *args, **kwargs) and then doing its own signature testing against the args list and kwargs dict.

Categories