We Keep Coding

Python dataclass evaluating type-hinted Dict fields [duplicate]

I'm trying to generate some JavaScript based on the type annotations I have provided in on some Python functions by using the signature() function in the inspect module.
This part works as I expect when the type is a simple builtin class:
import inspect
def my_function() -> dict:
pass
signature = inspect.signature(my_function)
signature.return_annotation is dict # True
Though I'm not sure how to unwrap and inspect more complex annotations e.g:
from typing import List
import inspect
def my_function() -> List[int]:
pass
signature = inspect.signature(my_function)
signature.return_annotation is List[int] # False
Again similar problem with forward referencing a custom class:
def my_function() -> List['User']:
pass
...
signature.return_annotation # typing.List[_ForwardRef('User')]
What I'm looking to get out is something like this - so I can branch appropriately while generating the JavaScript:
type = signature.return_annotation... # list
member_type = signature.return_annotation... # int / 'User'
Thanks.

Python 3.8 provides typing.get_origin() and typing.get_args() for this!
assert get_origin(Dict[str, int]) is dict
assert get_args(Dict[int, str]) == (int, str)
assert get_origin(Union[int, str]) is Union
assert get_args(Union[int, str]) == (int, str)
See https://docs.python.org/3/library/typing.html#typing.get_origin

List is not a map of types to GenericMeta, despite the syntax. Each access to it generates a new instance:
>>> [ id(List[str]) for i in range(3) ]
[33105112, 33106872, 33046936]
This means that even List[int] is not List[int]. To compare two instances, you have multiple options:
Use ==, i.e., signature.return_annotation == List[int].
Store an instance of your type in a global variable and check against that, i.e.,
a = List[int]
def foo() -> a:
pass
inspect.signature(foo).return_annotation is a
Use issubclass. The typing module defines that. Note that this might do more than you'd like, make sure to read the _TypeAlias documentation if you use this.
Check against List only and read the contents yourself. Though the property is internal, it is unlikely that the implementation will change soon: List[int].__args__[0] contains the type argument starting from Python 3.5.2, and in earlier versions, its List[int].__parameters__[0].
If you'd like to write generic code for your exporter, then the last option is probably best. If you only need to cover a specific use case, I'd personally go with using ==.

Take note, this applies to Python 3.5.1
For Python 3.5.2 take a look at phillip's answer.
You shouldn't be checking with the identity operator as Phillip stated, use equality to get this right.
To check if a hint is a subclass of a list you could use issubclass checks (even though you should take note that this can be quirky in certain cases and is currently worked on):
issubclass(List[int], list) # True
To get the members of a type hint you generally have two watch out for the cases involved.
If it has a simple type, as in List[int] the value of the argument is located in the __parameters__ value:
signature.return_annotation.__parameters__[0] # int
Now, in more complex scenarios i.e a class supplied as an argument with List[User] you must again extract the __parameter__[0] and then get the __forward_arg__. This is because Python wraps the argument in a special ForwardRef class:
d = signature.return_annotation.__parameter__[0]
d.__forward_arg__ # 'User'
Take note, you don't need to actually use inspect here, typing has a helper function named get_type_hints that returns the type hints as a dictionary (it uses the function objects __annotations__ attribute).

Can a decorator access variable type declaration in python? [duplicate]

does python 3.5 provide functions that allow to test whether a given
argument would fit the type hints given in the function declaration?
if i have e.g. this function:
def f(name: List[str]):
pass
is there a python method that can check whether
name = ['a', 'b']
name = [0, 1]
name = []
name = None
...
fit the type hints?
i know that 'no type checking happens at runtime' but can i still check the
validity of these arguments by hand in python?
or if python does not provide that functionality itself: what is the tool i'd
need to use?

Python itself doesn't provide such functions, you can read more about it here:
I wrote a decorator for that. This is the code of my decorator:
from typing import get_type_hints
def strict_types(function):
def type_checker(*args, **kwargs):
hints = get_type_hints(function)
all_args = kwargs.copy()
all_args.update(dict(zip(function.__code__.co_varnames, args)))
for argument, argument_type in ((i, type(j)) for i, j in all_args.items()):
if argument in hints:
if not issubclass(argument_type, hints[argument]):
raise TypeError('Type of {} is {} and not {}'.format(argument, argument_type, hints[argument]))
result = function(*args, **kwargs)
if 'return' in hints:
if type(result) != hints['return']:
raise TypeError('Type of result is {} and not {}'.format(type(result), hints['return']))
return result
return type_checker
You can use it like that:
#strict_types
def repeat_str(mystr: str, times: int):
return mystr * times
Though it's not very pythonic to restrict your function to accept only one type. Though you can use abc (abstract base classes) like number (or custom abc) as type-hints and restrict your functions to accept not only one type, but whatever combination of types you want.
Added a github repo for it, if anybody wants to use it.

This is an old question, but there is a tool I've written for doing run time type checking based on type hints: https://pypi.org/project/typeguard/

Python: check a function's default arguments

Given a function (e.g. create_instances) which takes many keyword arguments, how do I know which arguments have default values, and what exactly are the default values?
To give full context, I'm using the boto3 AWS Python API:
import boto3
ec2 = boto3.resource('ec2')
ec2.create_instances( ... )
For example, the create_instances function takes an argument MaxCount, but I want to know whether it has a default value. I looked around in the inspect module but wasn't did not find anything useful.
The documentation of create_instances is at https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2.html#EC2.ServiceResource.create_instances

inspect.getfullargspec(ec2.create_instance) should normally give you everything you need. Align args and defaults on the right side. For example:
def foo(a, b=3, *c, d=5):
m = a + b
return m
argspec = inspect.getfullargspec(ec2.create_instance)
{**dict(zip(argspec.args[-len(argspec.defaults):], argspec.defaults)),
**argspec.kwonlydefaults}
# => {'b': 3, 'd': 5}
As #9769953 says, if a parameter is bundled in **kwargs, it is processed by the function's code, and is thus not found in the function signature.

What are __signature__ and __text_signature__ used for in Python 3.4

If one does dir() on some builtin callables (class constructors, methods, etc) on CPython 3.4, one finds out that many of them often have a special attribute called __text_signature__, for example:
>>> object.__text_signature__
'()'
>>> int.__text_signature__
>>> # was None
However the documentation for this is nonexistent. Furthermore, googling for the attribute name suggests that there is also another possible special attribute __signature__, though I did not find any built-in functions that would have it.
I do know they are related to the function argument signature, but nothing beyond that, what do their values signify and what is the use of them?

These attributes are there to enable introspection for Python objects defined in C code. The C-API Argument Clinic provides the data, to assist the inspect module when building Signature objects. Introspection of C-API functions was not supported before.
See the internal inspect._signature_fromstr() function on how the __text_signature__ value is used.
Currently, the __text_signature__ attribute is filled from the internal docstring set on objects in the C-API; a simple text search is done for objectname(...)\n--\n\n, where the \n--\n\n is typical of Attribute Clinic-generated documentation strings. Take a look at the type object slots if you wanted to find some examples. Or you could look at the audioop module source to see how the Argument Clinic is being used to define signatures; the Argument Clinic script is run on those when building to generate the docstrings (in the accompanying audioop.c.h file).
The __signature__ attribute, if present, would be a inspect.Signature() object; instead of providing a text version a C-API can provide a fully parsed Signature instance instead.

Quick Summary
Those two attributes are used by the inspect.signature() function to retrieve metadata about a function or method's call signature.
Real-world Application
One use case for manually specifying one of these attributes is to provide useful tooltips for a function that uses *args.
In this example, the randrange() method uses *args to accept a variable number of inputs. However, we want the signature provided to help() and tooltips to show the meaning of each argument so that it matches the corresponding range() function.
import random
class Random(random.Random):
def randrange(self, /, *args):
'Choose a random value from range(start[, stop[, step]]).'
return self.choice(range(*args))
randrange.__text_signature__ = '($self, start, stop=None, step=1, /)'
The __text_signature__ attribute informs the creation of the Signature object:
>>> inspect.signature(Random.randrange)
<Signature (self, start, stop=None, step=1, /)>
This makes the help() output more useful:
>>> help(Random.randrange)
Help on function randrange in module __main__:
randrange(self, start, stop=None, step=1, /)
Choose a random value from range(start[, stop[, step]]).

How can I find the number of arguments of a Python function?

How can I find the number of arguments of a Python function? I need to know how many normal arguments it has and how many named arguments.
Example:
def someMethod(self, arg1, kwarg1=None):
pass
This method has 2 arguments and 1 named argument.

The previously accepted answer has been deprecated as of Python 3.0. Instead of using inspect.getargspec you should now opt for the Signature class which superseded it.
Creating a Signature for the function is easy via the signature function:
from inspect import signature
def someMethod(self, arg1, kwarg1=None):
pass
sig = signature(someMethod)
Now, you can either view its parameters quickly by string it:
str(sig) # returns: '(self, arg1, kwarg1=None)'
or you can also get a mapping of attribute names to parameter objects via sig.parameters.
params = sig.parameters
print(params['kwarg1']) # prints: kwarg1=20
Additionally, you can call len on sig.parameters to also see the number of arguments this function requires:
print(len(params)) # 3
Each entry in the params mapping is actually a Parameter object that has further attributes making your life easier. For example, grabbing a parameter and viewing its default value is now easily performed with:
kwarg1 = params['kwarg1']
kwarg1.default # returns: None
similarly for the rest of the objects contained in parameters.
As for Python 2.x users, while inspect.getargspec isn't deprecated, the language will soon be :-). The Signature class isn't available in the 2.x series and won't be. So you still need to work with inspect.getargspec.
As for transitioning between Python 2 and 3, if you have code that relies on the interface of getargspec in Python 2 and switching to signature in 3 is too difficult, you do have the valuable option of using inspect.getfullargspec. It offers a similar interface to getargspec (a single callable argument) in order to grab the arguments of a function while also handling some additional cases that getargspec doesn't:
from inspect import getfullargspec
def someMethod(self, arg1, kwarg1=None):
pass
args = getfullargspec(someMethod)
As with getargspec, getfullargspec returns a NamedTuple which contains the arguments.
print(args)
FullArgSpec(args=['self', 'arg1', 'kwarg1'], varargs=None, varkw=None, defaults=(None,), kwonlyargs=[], kwonlydefaults=None, annotations={})

import inspect
inspect.getargspec(someMethod)
see the inspect module

func.__code__.co_argcount gives you the number of any arguments BEFORE *args
func.__kwdefaults__ gives you a dict of the keyword arguments AFTER *args
func.__code__.co_kwonlyargcount is equal to len(func.__kwdefaults__)
func.__defaults__ gives you the values of optional arguments that appears before *args
Here is a simple illustration:
>>> def a(b, c, d, e, f=1, g=3, h=None, *i, j=2, k=3, **L):
pass
>>> a.__code__.co_argcount
7
>>> a.__defaults__
(1, 3, None)
>>> len(a.__defaults__)
3
>>>
>>>
>>> a.__kwdefaults__
{'j': 2, 'k': 3}
>>> len(a.__kwdefaults__)
2
>>> a.__code__.co_kwonlyargcount
2

someMethod.func_code.co_argcount
or, if the current function name is undetermined:
import sys
sys._getframe().func_code.co_argcount

inspect.getargspec()
Get the names and default values of a function’s arguments. A tuple of four things is returned: (args, varargs, varkw, defaults). args is a list of the argument names (it may contain nested lists). varargs and varkw are the names of the * and ** arguments or None. defaults is a tuple of default argument values or None if there are no default arguments; if this tuple has n elements, they correspond to the last n elements listed in args.
Changed in version 2.6: Returns a named tuple ArgSpec(args, varargs, keywords, defaults).
See can-you-list-the-keyword-arguments-a-python-function-receives.

Adding to the above, I've also seen that the most of the times help() function really helps
For eg, it gives all the details about the arguments it takes.
help(<method>)
gives the below
method(self, **kwargs) method of apiclient.discovery.Resource instance
Retrieves a report which is a collection of properties / statistics for a specific customer.
Args:
date: string, Represents the date in yyyy-mm-dd format for which the data is to be fetched. (required)
pageToken: string, Token to specify next page.
parameters: string, Represents the application name, parameter name pairs to fetch in csv as app_name1:param_name1, app_name2:param_name2.
Returns:
An object of the form:
{ # JSON template for a collection of usage reports.
"nextPageToken": "A String", # Token for retrieving the next page
"kind": "admin#reports#usageReports", # Th

Good news for folks who want to do this in a portable way between Python 2 and Python 3.6+: use inspect.getfullargspec() method. It works in both Python 2.x and 3.6+
As Jim Fasarakis Hilliard and others have pointed out, it used to be like this:
1. In Python 2.x: use inspect.getargspec()
2. In Python 3.x: use signature, as getargspec() and getfullargspec() were deprecated.
However, starting Python 3.6 (by popular demand?), things have changed towards better:
From the Python 3 documentation page:
inspect.getfullargspec(func)
Changed in version 3.6: This method was previously documented as deprecated in favour of signature() in Python 3.5, but that decision has been reversed in order to restore a clearly supported standard interface for single-source Python 2/3 code migrating away from the legacy getargspec() API.

You get the number of arguments by (replace "function" by the name of your function):
function.__code__.co_argcount ## 2
And the names for the arguments by:
function.__code__.co_varnames ## ('a', 'b')

As other answers suggest, getargspec works well as long as the thing being queried is actually a function. It does not work for built-in functions such as open, len, etc, and will throw an exception in such cases:
TypeError: <built-in function open> is not a Python function
The below function (inspired by this answer) demonstrates a workaround. It returns the number of args expected by f:
from inspect import isfunction, getargspec
def num_args(f):
if isfunction(f):
return len(getargspec(f).args)
else:
spec = f.__doc__.split('\n')[0]
args = spec[spec.find('(')+1:spec.find(')')]
return args.count(',')+1 if args else 0
The idea is to parse the function spec out of the __doc__ string. Obviously this relies on the format of said string so is hardly robust!

inspect.getargspec() to meet your needs
from inspect import getargspec
def func(a, b):
pass
print len(getargspec(func).args)

The accepted answer by Dimitris Fasarakis Hilliard suggests getting parameters in the string format but I think one can make a mistake when parsing this string and thus I created rather a list of the parameters directly using the inspect module
import inspect
def my_function(a,b,c):
#some code
pass
result=list(inspect.signature(my_function).parameters.keys())
print(result)
['a','b','c']

Assuming you may be dealing with class based methods or simply functions, you could do something like the following.
This will automatically subtract one input if the input is a class method (and therefore includes self).
import types
def get_arg_count(fn):
extra_method_input_count=1 if isinstance(fn, types.MethodType) else 0
return fn.__code__.co_argcount-extra_method_input_count
Then you can apply as you need to functions or methods:
def fn1(a, b, c):
return None
class cl1:
def fn2(self, a, b, c):
return None
print(get_arg_count(fn1)) #=> 3
print(get_arg_count(cl1().fn2)) #=> 3

In:
import inspect
class X:
def xyz(self, a, b, c):
return
print(len(inspect.getfullargspec(X.xyz).args))
Out:
4
Note: If xyz wasn't inside class X and had no "self" and just "a, b, c", then it would have printed 3.
For python below 3.5, you may want to replace inspect.getfullargspec by inspect.getargspec in the code above.

This is a solution to getting the number of mandatory arguments of a function (*)
Many of the solutions proposed here do not work for this purpose if some more uncommon parameter specifications are used (positional-only parameters with defaults, keyword-only parameters without defaults, etc.)
from typing import Callable, Any
import inspect
def get_mandatory_argcount(f: Callable[..., Any]) -> int:
"""Get the number of mandatory arguments of a function."""
sig = inspect.signature(f)
def parameter_is_mandatory(p: inspect.Parameter) -> bool:
return p.default is inspect.Parameter.empty and p.kind not in (
inspect.Parameter.VAR_POSITIONAL,
inspect.Parameter.VAR_KEYWORD,
)
return sum(parameter_is_mandatory(p) for p in sig.parameters.values())
# mandatory keyword-only
def f1(b=2, *args, c, d=1, **kwds): pass
print(get_mandatory_argcount(f1))
# positional only with default
def f2(a=1, /, b=3, *args, **kwargs): pass
print(get_mandatory_argcount(f2))
(*) I would have liked to put this as an answer to Programmatically determining amount of parameters a function requires - Python instead, but for some reason this question is marked as duplicate to this one despite it asking specifically about the number of required arguments whereas this question only asks about the general number of arguments.