In Python's typing module, it seems that anything that is not a generic type does not support subscripting:
>>> import typing
>>> from yarl import URL
>>> StrOrURL = typing.Union[URL, str]
>>> typing.List[StrOrURL] # OK; List is generic
typing.List[typing.Union[yarl.URL, str]]
>>> SetOrDict = typing.Union[set, dict]
>>> SetOrDict[StrOrURL] # Raises
TypeError: typing.Union[set, dict] is not a generic class
This will be the case whether SetOrDict is defined with typing.Union or typing.TypeVar. (In Python 3.7, at least.)
It looks like there is a way to subclass Generic, presumably like:
class SetOrDict(typing.Generic[StrOrURL]):
# ...
But this will immediately raise TypeError: Parameters to Generic[...] must all be type variables.
Is there a way to accommodate the above? I.e., what is the recommended practice for subscripting user-defined (non-generic) typedefs?
You can't use a Union[...], alias or otherwise, as a generic type, no. Your union contents are also not generic, you can't state what types a dict or set object can contain by using dict or set directly, you'd use Set or Dict.
You have a new union:
URLorStrSetOrDict = typing.Union[Set[StrOrUrl], Dict[StrOrUrl, Any]]
Note that a dictionary has keys and values, you need to provide information on both. If this used as the input for an API, then consider using the more generic and immutable AbstractSet and Mapping types; this makes it clear that your API only needs to read information.
Personally, I'd look hard at refactoring a codebase that is this muddy about what kinds of objects it can accept. Requiring that an API accepts only sets and existing code has to pass in set(dictionary) instead of dictionary is not an arduous requirement, or perhaps the API really would work with any iterable of StrOrUrl.
Related
I'm trying to generate some JavaScript based on the type annotations I have provided in on some Python functions by using the signature() function in the inspect module.
This part works as I expect when the type is a simple builtin class:
import inspect
def my_function() -> dict:
pass
signature = inspect.signature(my_function)
signature.return_annotation is dict # True
Though I'm not sure how to unwrap and inspect more complex annotations e.g:
from typing import List
import inspect
def my_function() -> List[int]:
pass
signature = inspect.signature(my_function)
signature.return_annotation is List[int] # False
Again similar problem with forward referencing a custom class:
def my_function() -> List['User']:
pass
...
signature.return_annotation # typing.List[_ForwardRef('User')]
What I'm looking to get out is something like this - so I can branch appropriately while generating the JavaScript:
type = signature.return_annotation... # list
member_type = signature.return_annotation... # int / 'User'
Thanks.
Python 3.8 provides typing.get_origin() and typing.get_args() for this!
assert get_origin(Dict[str, int]) is dict
assert get_args(Dict[int, str]) == (int, str)
assert get_origin(Union[int, str]) is Union
assert get_args(Union[int, str]) == (int, str)
See https://docs.python.org/3/library/typing.html#typing.get_origin
List is not a map of types to GenericMeta, despite the syntax. Each access to it generates a new instance:
>>> [ id(List[str]) for i in range(3) ]
[33105112, 33106872, 33046936]
This means that even List[int] is not List[int]. To compare two instances, you have multiple options:
Use ==, i.e., signature.return_annotation == List[int].
Store an instance of your type in a global variable and check against that, i.e.,
a = List[int]
def foo() -> a:
pass
inspect.signature(foo).return_annotation is a
Use issubclass. The typing module defines that. Note that this might do more than you'd like, make sure to read the _TypeAlias documentation if you use this.
Check against List only and read the contents yourself. Though the property is internal, it is unlikely that the implementation will change soon: List[int].__args__[0] contains the type argument starting from Python 3.5.2, and in earlier versions, its List[int].__parameters__[0].
If you'd like to write generic code for your exporter, then the last option is probably best. If you only need to cover a specific use case, I'd personally go with using ==.
Take note, this applies to Python 3.5.1
For Python 3.5.2 take a look at phillip's answer.
You shouldn't be checking with the identity operator as Phillip stated, use equality to get this right.
To check if a hint is a subclass of a list you could use issubclass checks (even though you should take note that this can be quirky in certain cases and is currently worked on):
issubclass(List[int], list) # True
To get the members of a type hint you generally have two watch out for the cases involved.
If it has a simple type, as in List[int] the value of the argument is located in the __parameters__ value:
signature.return_annotation.__parameters__[0] # int
Now, in more complex scenarios i.e a class supplied as an argument with List[User] you must again extract the __parameter__[0] and then get the __forward_arg__. This is because Python wraps the argument in a special ForwardRef class:
d = signature.return_annotation.__parameter__[0]
d.__forward_arg__ # 'User'
Take note, you don't need to actually use inspect here, typing has a helper function named get_type_hints that returns the type hints as a dictionary (it uses the function objects __annotations__ attribute).
I've been trying to tighten up a few classes of code with type hinting. I'm struggling with a nesting problem. The parameter of a function of interest is an iterable container of objects with known types. I'm having trouble getting my IDE to recognize the inner types (which is a nice convenience.)
I'm using python 3.9 on PyCharm, but I'm getting similar auto-complete results in Ipython.
When using the Iterable class out of typing module, the IDE cannot "see through" to the inner types. Similarly for Collection. But it can when using either list or tuple on the outer container.
Is this an IDE issue or is there another way to package this? It would be nice to be able to send this function any type of iterable, rather than hard code it...
from datetime import datetime
from typing import Iterable
Data_collection = Iterable[tuple[datetime, str]]
Data_list = list[tuple[datetime, str]]
def foo(bar: Data_collection):
bar[1][0]. # no type-hint on inner obj
def foo2(bar: Data_list):
bar[1][0]. # good type-hint on inner obj
Gets me (PyCharm) results like this:
You are using the wrong abstract base class. Iterable only promises/requires __iter__ to be implemented, not __getitem__. Statically speaking, bar[1] isn't guaranteed to be defined.
To specify any type that supports indexing, use Sequence instead.
from typing import Sequence
Data_collection = Sequence[tuple[datetime, str]]
Now, regardless of the runtime type of bar, your IDE can assume that bar.__getitem__ is defined and returns a tuple. (Whether your specific IDE does make that assumption depends on the IDE, not your code.)
I'm trying to use generic type hints in python, however I'm not sure if a behaviour I want is possible or not. I've done a lot of reading on the type hints documentation, but cannot find what I'm looking for.
I want to be able to specify class type arguments with default types if the type isn't defined. For instance, I would imagine the code would look something like this.
T = TypeVar('T')
S = TypeVar('S')
class Attribute(Generic[T, S=int]):
...
class FooBar:
a: Attribute[str, str]
b: Attribute[str]
So in this case type S would default to int if it is not specified in a type hint. I am also happy with a solution which uses metaprogramming to make this possible.
I think you're gonna have to wait till Python 3.12 if I'm understanding this correctly.
PEP 696 – Type defaults for TypeVarLikes | peps.python.org
The whole idea behind generic types stems from statically-typed languages (eg. Scala or Java) and it is to allow flexibility for types when defining code, and prevents unnecessary excessive overloading for functions (which is not needed in Python being a dynamically-typed language). Hence defining default or fallback types kind of deviates from the concepts of generic altogether. With that being said, I see that the Union[] is a better fit to answer your question.
If you would like to combine that with generic types (which does not make much sense given the logic behind generics), you can do something like:
from typing import Union, TypeVar
T = TypeVar('T')
def foo(var: Union[T, str]):
if var:
return "boo"
res = foo("bar")
print(res)
However, I suggest that when defining your code, try to be specific and strict when possible regarding types, and to define better generics. That can be accomplished by defining more strict generic/abstract types:
T = TypeVar('T') # Can be anything
A = TypeVar('A', str, bytes) # Must be str or bytes
The latter will allow you to remain generic,with slightly more control over your types.
I have a function that accepts a class that derives from NamedTuple and converts it into a schema. However when I run MyPy on the following code it fails with Argument 1 to "to_schema" has incompatible type "Type[Foo]"; expected "Type[NamedTuple]"
from typing import NamedTuple, Type
def to_schema(named_tuple: Type[NamedTuple]):
pass
class Foo(NamedTuple):
pass
to_schema(Foo)
Is there a way to properly type the code so that it typechecks with MyPy?
Edit:
Python documentation states that Type[Foo] accepts any subclasses of Foo (https://docs.python.org/3/library/typing.html#typing.Type). I have multiple subclasses of NamedTuple, for entities in our data model, so I'm looking for a way to annotate the function in a way that would typecheck.
The root issue with your code is that NamedTuple is not an actual type -- it's actually just a special sort of "type constructor" that synthesizes an entirely new class and type. E.g. if you try printing out the value of Foo.__mro__, you'll see (<class '__main__.Foo'>, <class 'tuple'>, <class 'object'>) -- NamedTuple is not present there at all.
That means that NamedTuple isn't actually a valid type to use at all -- in that regard, it's actually a little surprising to me that mypy just silently lets you construct Type[NamedTuple] to begin with.
To work around this, you have several potential approaches:
Rather then using Type[NamedTuple], use either Type[tuple] or Type[Tuple[Any]].
Your Foo genuinely is a subtype of a tuple, after all.
If you need methods or fields that are specifically present only in namedtuples, use a custom protocol. For example, if you particularly need the _asdict method in namedtuples, you could do:
from typing_extensions import Protocol
class NamedTupleProto(Protocol):
def _asdict(self) -> Dict[str, Any]: ...
def to_schema(x: Type[NamedTupleProto]) -> None: pass
class Foo(NamedTuple):
pass
to_schema(Foo)
Note that you will need to install the typing_extensions third party library to use this, though there are plans to formalize Protocols and add it to Python at some point. (I forget if the plan was Python 3.7 or 3.8).
Add a type ignore or a cast on the call to to_schema to silence mypy. This isn't the greatest solution, but is also the quickest.
For related discussion, see this issue. Basically, there's consensus on the mypy team that somebody ought to do something about this NamedTuple thing, whether it's by adding an error message or by adding an officially sanctioned protocol, but I think people are too busy with other tasks/bugs to push this forward. (So if you're bored and looking for something to do...)
I'm trying to generate some JavaScript based on the type annotations I have provided in on some Python functions by using the signature() function in the inspect module.
This part works as I expect when the type is a simple builtin class:
import inspect
def my_function() -> dict:
pass
signature = inspect.signature(my_function)
signature.return_annotation is dict # True
Though I'm not sure how to unwrap and inspect more complex annotations e.g:
from typing import List
import inspect
def my_function() -> List[int]:
pass
signature = inspect.signature(my_function)
signature.return_annotation is List[int] # False
Again similar problem with forward referencing a custom class:
def my_function() -> List['User']:
pass
...
signature.return_annotation # typing.List[_ForwardRef('User')]
What I'm looking to get out is something like this - so I can branch appropriately while generating the JavaScript:
type = signature.return_annotation... # list
member_type = signature.return_annotation... # int / 'User'
Thanks.
Python 3.8 provides typing.get_origin() and typing.get_args() for this!
assert get_origin(Dict[str, int]) is dict
assert get_args(Dict[int, str]) == (int, str)
assert get_origin(Union[int, str]) is Union
assert get_args(Union[int, str]) == (int, str)
See https://docs.python.org/3/library/typing.html#typing.get_origin
List is not a map of types to GenericMeta, despite the syntax. Each access to it generates a new instance:
>>> [ id(List[str]) for i in range(3) ]
[33105112, 33106872, 33046936]
This means that even List[int] is not List[int]. To compare two instances, you have multiple options:
Use ==, i.e., signature.return_annotation == List[int].
Store an instance of your type in a global variable and check against that, i.e.,
a = List[int]
def foo() -> a:
pass
inspect.signature(foo).return_annotation is a
Use issubclass. The typing module defines that. Note that this might do more than you'd like, make sure to read the _TypeAlias documentation if you use this.
Check against List only and read the contents yourself. Though the property is internal, it is unlikely that the implementation will change soon: List[int].__args__[0] contains the type argument starting from Python 3.5.2, and in earlier versions, its List[int].__parameters__[0].
If you'd like to write generic code for your exporter, then the last option is probably best. If you only need to cover a specific use case, I'd personally go with using ==.
Take note, this applies to Python 3.5.1
For Python 3.5.2 take a look at phillip's answer.
You shouldn't be checking with the identity operator as Phillip stated, use equality to get this right.
To check if a hint is a subclass of a list you could use issubclass checks (even though you should take note that this can be quirky in certain cases and is currently worked on):
issubclass(List[int], list) # True
To get the members of a type hint you generally have two watch out for the cases involved.
If it has a simple type, as in List[int] the value of the argument is located in the __parameters__ value:
signature.return_annotation.__parameters__[0] # int
Now, in more complex scenarios i.e a class supplied as an argument with List[User] you must again extract the __parameter__[0] and then get the __forward_arg__. This is because Python wraps the argument in a special ForwardRef class:
d = signature.return_annotation.__parameter__[0]
d.__forward_arg__ # 'User'
Take note, you don't need to actually use inspect here, typing has a helper function named get_type_hints that returns the type hints as a dictionary (it uses the function objects __annotations__ attribute).