We Keep Coding

Python type hints - better syntax for cast()?

I recently started using type hints in my code, and have so far found them to be (mostly) very helpful.
However, one thing that I really do not like is the syntax to force the type checker to assume that a variable is of a certain type. Given this example:
import itertools
from typing import Iterable, Tuple
x: Iterable[Tuple[str, str]] = itertools.combinations('abc', 2)
# error: Incompatible types in assignment (expression has type "Iterable[Tuple[str, ...]]", variable has type "List[Tuple[str, str]]")
As far as I can tell, the recommended way to work around this is to explicitly cast the object to force the type checker to use the specified type, e.g.:
import itertools
from typing import Iterable, Tuple, cast
x = cast(Iterable[Tuple[str, str]], itertools.combinations('abc', 2))
I personally find this solution to be a bit gross. My primary concern is that, to the inexperienced reader, it is not clear that the cast is purely there to help the static analyzer. (If I didn't already know, I would assume based on the name and context that it is converting and doing a copy into an object of the specified type, when really there is no runtime cost.)
cast looks like any old function call. When I see that a function is being called on a value, I expect the value to be mutated and/or some other side-effects to occur, but in this case the only side effect is that mypy stops complaining. Type hints themselves have a distinct syntax, but I feel that this blurs the lines with a mixture of the new typing syntax and traditional python syntax. (It's already a bit blurry since you have to import the types and can compose them, but that's another discussion.)
Is there an alternative syntax for cast-like behavior? I haven't found anything, but I was hoping for something like:
x1 = itertools.combinations('abc', 2)) # cast: Iterable[Tuple[str, str]]
x2: Iterable[Tuple[str, str]] = itertools.combinations('abc', 2)) # type: cast
x3: Cast[Iterable[Tuple[str, str]]] = itertools.combinations('abc', 2))

Actually the latest version of Mypy does return the correct type Iterator[Tuple[str, str]].
This change was introduced to Typeshed in PR https://github.com/python/typeshed/pull/4309.
If you cannot update mypy to the latest version you can checkout the latest version from typeshed and use the config option custom_typeshed_dir.
See https://mypy.readthedocs.io/en/stable/config_file.html#confval-custom_typeshed_dir for more details.

How to specify method return type list of (what) in Python?

Let's say I have a method like the following
def validate(self, item:dict, attrs:dict)-> list:
If I want to be more specific and tell that my return type is a list of ValidationMessages?
How should I / Can I achieve that?
(I would check off the mark as a duplicate, since this is not about extending a list or flattening) I'm asking about how to be more specific on identifying the return type of a method...

With Python 3.6, the built-in typing package will do the job.
from typing import List
def validate(self, item:dict, attrs:dict)-> List[str]:
...
The notation is a bit weird, since it uses brackets but works out pretty well.
Edit: With the new 3.9 version of Python, you can annotate types without importing from the typing module. The only difference is that you use real type names instead of defined types in the typing module.
def validate(self, item:dict, attrs:dict)-> list[str]:
...
NOTE: Type hints are just hints that help IDE. Those types are not enforced. You can add a hint for a variable as str and set an int to it like this:
a:str = 'variable hinted as str'
a = 5 # See, we can set an int
Your IDE will warn you but you will still be able to run the code. Because those are just hints. Python is not a type strict language. Instead, it employs dynamic typing.

Problems with “typing.ClassVar” when using a factory function to generate annotation types in Python 3.7

I am trying to use a factory function to generate some type annotations – specifically for tuple types. I have one version of the factory that works fine (as in, it compiles, runs, and checks out satisfactorily in MyPy):
import typing as tx
HomogenousTypeVar = tx.TypeVar('HomogenousTypeVar')
TupleTypeReturnType = tx.Type[tx.Tuple[HomogenousTypeVar, ...]]
def TupleType(length: int,
tuptyp: tx.Type[HomogenousTypeVar] = str) -> TupleTypeReturnType:
""" Create a type annotation for a tuple of a given type and length """
assert length > 0
return tx.Tuple[tuple(tuptyp for idx in range(length))]
… for which the usage is like e.g.:
class Thing(object):
__slots__: TupleType(2) = ('yo', 'dogg')
other_fields: TupleType(4) = ('i', 'heard',
'you', 'like')
# etc, or what have you
… however, I was unsuccessful when I tried to add support for the typing.ClassVar annotation, which looked like this:
import typing as tx
HomogenousTypeVar = tx.TypeVar('HomogenousTypeVar')
TupleTypeReturnType = tx.Union[tx.Type[tx.Tuple[HomogenousTypeVar, ...]],
tx.Type[tx.ClassVar[tx.Tuple[HomogenousTypeVar, ...]]]]
def TupleType(length: int,
tuptyp: tx.Type[HomogenousTypeVar] = str,
clsvar: bool = False) -> TupleTypeReturnType:
""" Create a type annotation for a tuple of a given type and length,
specifying additionally whether or not it is a ClassVar """
assert length > 0
out = tx.Tuple[tuple(tuptyp for idx in range(length))]
return clsvar and tx.ClassVar[out] or out
… after this change, the code won’t even initially compile – it fails to do so with a TypeError from deep within the typing module:
TypeError: typing.ClassVar[typing.Tuple[~HomogenousTypeVar, ...]] is
not valid as type argument
… which, as errors go, strikes me as a little phoned-in; I mean, is not everything in typing supposed to be a valid type argument in some fashion, give-or-take?
In the typing source code related to ClassVar, there are a handful of restrictions to its use mentioned in the docstring – but this is not one of them. Is there something obvious I am missing? Is my attempt to use this annotation in this fashion a quixotic one? What else might I try?

Are you sure your original code snippet actually type-checks with mypy? When I try running it using either Mypy 0.620 or the latest version from github, I get the following errors:
test.py:13: error: invalid type comment or annotation
test.py:13: note: Suggestion: use TupleType[...] instead of TupleType(...)
test.py:14: error: invalid type comment or annotation
test.py:14: note: Suggestion: use TupleType[...] instead of TupleType(...)
I'm also not able to reproduce the error you get with your ClassVar code -- when I try running it, I get the following errors:
test.py:4: error: Invalid type: ClassVar nested inside other type
test.py:6: error: Incompatible default for argument "tuptyp" (default has type "Type[str]", argument has type "Type[HomogenousTypeVar]")
test.py:12: error: Invalid type alias
test.py:13: warning: Returning Any from function declared to return "Union[Type[Tuple[HomogenousTypeVar?, ...]], Type[Tuple[HomogenousTypeVar?, ...]]]"
test.py:15: error: Name 'Thing' is not defined
test.py:16: error: Revealed type is 'Any'
Are you sure you're actually running mypy, as opposed to just running the code? E.g. if you only run python3 test.py, you're basically skipping all type checks (apart from some bare minimum sanity checks built into the typing module).
If you want to type-check your code, you need to pip-install mypy and run python3 -m mypy test.py.
In any case, all of these error messages are expected behavior -- mypy (and any other PEP 484 compliant type checker) can only analyze your code statically and will not attempt to run or analyze any factory functions/any type-hint generation functions you may try and write.
So, this means that unfortunately you won't be able to use your generated type hint with ClassVars if you want PEP 484 compliant tools to be able to analyze your code -- they can't understand/interpret your original set of type hints, and adding ClassVars certainly won't help.
If you want to generate type hints, the only real option I can think of is to invent some sort of mini-language or macro system on top of Python that when run, will generate Python code. You would then run and typecheck that generated code instead of your macrofied Python language.
But I really don't recommend doing this -- it's a very fragile hack.
More broadly, whenever you start running into these sorts of type-related limitations, I think it's a sign that your code is too complicated. I would either look into simplifying your code or (if that's not possible) switch to a language like Haskell or Idris which would let you use a more expressive (albeit more complex) type system.
For example, in this case, you're trying to generalize the Tuple type -- that leads me to infer that your codebase contains many different kinds of tuples of different arities and types.
That strikes me as being a bit suspicious -- I would instead look into converting some of those tuples into either regular classes or (if you still need tuple-like functionality) a namedtuple. Dataclasses (which are new as of Python 3.7) could also be convenient here.
Those solutions would also help make your code a little more readable -- you can now give concrete names and meanings to each distinct kind of tuple.
Alternatively, if you have only a few distinct types of tuples but use those tuples all over the place, you could try using type aliases so you don't have to repeatedly re-type the same (long) type over and over. E.g. instead of doing:
def foo(x: Tuple[int, int, int, int]) -> None: ...
...you could do:
IpAddress = Tuple[int, int, int, int]
def foo(x: IpAddress) -> None: ...

Unpacking Python's Type Annotations

I'm trying to generate some JavaScript based on the type annotations I have provided in on some Python functions by using the signature() function in the inspect module.
This part works as I expect when the type is a simple builtin class:
import inspect
def my_function() -> dict:
pass
signature = inspect.signature(my_function)
signature.return_annotation is dict # True
Though I'm not sure how to unwrap and inspect more complex annotations e.g:
from typing import List
import inspect
def my_function() -> List[int]:
pass
signature = inspect.signature(my_function)
signature.return_annotation is List[int] # False
Again similar problem with forward referencing a custom class:
def my_function() -> List['User']:
pass
...
signature.return_annotation # typing.List[_ForwardRef('User')]
What I'm looking to get out is something like this - so I can branch appropriately while generating the JavaScript:
type = signature.return_annotation... # list
member_type = signature.return_annotation... # int / 'User'
Thanks.

Python 3.8 provides typing.get_origin() and typing.get_args() for this!
assert get_origin(Dict[str, int]) is dict
assert get_args(Dict[int, str]) == (int, str)
assert get_origin(Union[int, str]) is Union
assert get_args(Union[int, str]) == (int, str)
See https://docs.python.org/3/library/typing.html#typing.get_origin

List is not a map of types to GenericMeta, despite the syntax. Each access to it generates a new instance:
>>> [ id(List[str]) for i in range(3) ]
[33105112, 33106872, 33046936]
This means that even List[int] is not List[int]. To compare two instances, you have multiple options:
Use ==, i.e., signature.return_annotation == List[int].
Store an instance of your type in a global variable and check against that, i.e.,
a = List[int]
def foo() -> a:
pass
inspect.signature(foo).return_annotation is a
Use issubclass. The typing module defines that. Note that this might do more than you'd like, make sure to read the _TypeAlias documentation if you use this.
Check against List only and read the contents yourself. Though the property is internal, it is unlikely that the implementation will change soon: List[int].__args__[0] contains the type argument starting from Python 3.5.2, and in earlier versions, its List[int].__parameters__[0].
If you'd like to write generic code for your exporter, then the last option is probably best. If you only need to cover a specific use case, I'd personally go with using ==.

Take note, this applies to Python 3.5.1
For Python 3.5.2 take a look at phillip's answer.
You shouldn't be checking with the identity operator as Phillip stated, use equality to get this right.
To check if a hint is a subclass of a list you could use issubclass checks (even though you should take note that this can be quirky in certain cases and is currently worked on):
issubclass(List[int], list) # True
To get the members of a type hint you generally have two watch out for the cases involved.
If it has a simple type, as in List[int] the value of the argument is located in the __parameters__ value:
signature.return_annotation.__parameters__[0] # int
Now, in more complex scenarios i.e a class supplied as an argument with List[User] you must again extract the __parameter__[0] and then get the __forward_arg__. This is because Python wraps the argument in a special ForwardRef class:
d = signature.return_annotation.__parameter__[0]
d.__forward_arg__ # 'User'
Take note, you don't need to actually use inspect here, typing has a helper function named get_type_hints that returns the type hints as a dictionary (it uses the function objects __annotations__ attribute).

How to properly function annotate / type hint a list of strings

I am trying to figure out how to properly function annotate or type hint a list of strings. For example, if I had a function like this:
def send_email(self, from_address: str, to_addresses: list[str]):
pass
to_addresses should be a list of strings. But when I try to use that annotation I get the following error in my Python 3.4.3 interpreter:
TypeError: 'type' object is not subscriptable
I am positive the list[str] is causing the issue, because if I change it to str the error goes away, but that doesn't properly reflect my intentions for the parameter.

Python 3.4 doesn't specify a format for its function annotations, it merely provides a mechanism that allows you to use any expression as the annotation. How the annotations are interpreted is up to you and the libraries you use.
Python 3.5 standardizes the way function annotations are used for type hinting, as documented in PEP 484. To annotate a list of strings, you use List[str], where List is imported from the typing module. You can also use Sequence[str] if your function accepts any list-like sequence, or Iterable[str] for any iterable.
Starting with Python 3.9, you can use list[str] as a type annotation, which doesn't require importing anything.

In Python 3.9+, list (with a lowercase l) can be used in type annotations and your code should work as is. On older versions of Python you need to import typing.List and use it instead
from typing import List
to_addresses: List[str]
Note the capital L.
You might want to consider something more specific, e.g.
import typing
Address = typing.NewType("Address")
See NewType docs
The static type checker will treat the new type as if it were a subclass of the original type

This syntax is now valid in Python 3.9+:
In type annotations you can now use built-in collection types such as list and dict as generic types instead of importing the corresponding capitalized types (e.g. List or Dict) from typing.
Prior to 3.9 though, you need to use the imported List or in Python 3.7+ you can add
from __future__ import annotations
at the top of your file, which allows using list[int] (for example). Note though, this import only affects annotations:
from __future__ import annotations only affects annotations -- just the things after the colon. It makes it so that annotations are never evaluated
It still doesn't allow list[int] in arbitrary contexts.