Both typing and collections.abc includes similar type such as Mapping, Sequence, etc.
Based on the python documentation, it seems that collections.abc is preferred for type checking:
This module provides abstract base classes that can be used to test whether a class provides a particular interface; for example, whether it is hashable or whether it is a mapping.
https://docs.python.org/3/library/collections.abc.html
but using typing also works and I'd rather not import Mapping from both typing and collections.abc. So is there any catch in using typing with isinstance()?
As of python 3.9, the entire suite of collections.abc aliases in the typing module is deprecated, meaning that your preferred option for python >=3.9 should be to use the versions in the collections.abc module. The classes will not be removed from the typing module for a few years to come, however.
Many of the typing generic classes are just aliases to the abc ones. Just as an example from the docs, Hashable:
class typing.Hashable
An alias to collections.abc.Hashable
Also,
isinstance(abc.Hashable, typing.Hashable)
isinstance(typing.Hashable, abc.Hashable)
are both True, making it clear they are equivalent in terms of the class hierarchy. In Python you can check an alias's origin using the origin field:
>>> typing.Hashable.__origin__
<class 'collections.abc.Hashable'>
No reason I can see to import abc if you do not need it. The two packages provide close but different uses, so I would import only what you need. This is more of an opinion, but if your use-case is only checking against isinstance (i.e. no interesting argument annotation) both seem equivalent to me, though abc may be the more usual tool for this.
There is no catch. The typing library uses stubs from the collections library. You are totally allowed to use isinstance with the typing library, certainly for the types you mentioned (Mapping and Sequence).
Related
In python, if I am writing a function, is this the best way to type hint a list of strings:
def sample_def(var:list[str]):
I would use the typing module
from typing import List
def foo(bar: List[str]):
pass
The reason is typing contains so many type hints and the ability to create your own, specify callables, etc. Definitely check it out.
Edit:
I guess as of Python 3.9 typing is deprecated (RIP). Instead it looks like you can use collections.abc.*. So you can do this if you're on Python 3.9+:
from collections.abc import Iterable
def foo(bar: Iterable[str]):
pass
You can take a look at https://docs.python.org/3/library/collections.abc.html for a list of ABCs that might fit your need. For instance, it might make more sense to specify Sequence[str] depending on your needs of that function.
The latest typing docs has a lot of deprecation notices like the following:
class typing.Deque(deque, MutableSequence[T])
A generic version of collections.deque.
New in version 3.5.4.
New in version 3.6.1.
Deprecated since version 3.9: collections.deque now supports []. See PEP 585 and Generic Alias Type.
What does this means? Should we not use the generic type Deque (and several others) anymore? I've looked at the references and didn't connect the dots (could be because I'm an intermediate Python user).
It means that you should be transitioning to using built-in types / types from the standard library instead of the ones provided by typing. So for example collections.deque[int] instead of typing.Deque[int]. The same for list, tuple, etc. So tuple[int, str] is the preferred way.
See PEP585 and Generic Alias Type.
Changes in Python 3.9 remove the necessity for a parallel type hierarchy in the typing module, so that you can just use collections.deque directly when annotating a deque-like type.
For example, it means that annotating a deque of ints like
def foo(d: typing.Deque[int]):
...
Should be changed to:
def foo(d: collections.deque[int]):
...
Let's say we have a function definition like this:
def f(*, model: Optional[Type[pydantic.BaseModel]] = None)
So the function doesn't require pydantic to be installed until you pass something as a model. Now let's say we want to pack the function into pypi package. And my question is if there's a way to avoid bringing pydantic into the package dependencies only the sake of type checking?
I tried to follow dspenser's advice, but I found mypy still giving me Name 'pydantic' is not defined error. Then I found this chapter in the docs and it seems to be working in my case too:
from typing import TYPE_CHECKING
if TYPE_CHECKING:
import pydantic
You can use normal clases (instead of string literals) with __future__.annotations (python 3.8.1):
from __future__ import annotations
from typing import TYPE_CHECKING, Optional, Type
if TYPE_CHECKING:
import pydantic
def f(*, model: Optional[Type[pydantic.BaseModel]] = None):
pass
If for some reason you can't use __future__.annotations, e.g. you're on python < 3.7, use typing with string literals from dspenser's solution.
Python's typing module can use string type names, as well as the type itself, as you have in your example.
If you don't want the type to be evaluated when the code is run, and an exception thrown if the module has not been imported, then you may prefer to use the string name. Your code snippet would then be:
def f(*, model: Optional[Type["pydantic.BaseModel"]] = None)
Your static type checking using mypy should continue to work, but your package will no longer always require the dependency on pydantic.
As you discovered in the docs, python's typing module includes a way to specify that some imports are only required for (string literal) type annotations:
from typing import TYPE_CHECKING
if TYPE_CHECKING:
import pydantic
This will execute the import pydantic statement only when TYPE_CHECKING is True, primarily when using mypy to analyze the types in the code. When running the program normally, the import statement is not executed. For this reason, the option is only to be used with string literal type hints; otherwise, the missing import will cause the program's normal execution to fail.
As lazy evaluation of type hints is to be implemented in Python 4.0,
from __future__ import annotations
can be used to enable this behaviour in Python 3. With this import statement, string literal types are not required.
I would do something like
try:
from pydantic import BaseModel as PydanticBaseModel
except ImportError:
PydanticBaseModel = None
I don't think it's better than what dspencer proposed, but sometimes some weird styleguide may forbid you from using string type names, so I just wanted to point another solution out.
I would like to add the (Python3) type hint for a module (class 'module'). The typing package doesn't provide one, and types.ModuleType() is a constructor that returns a module object for a specific name.
Example:
import types
def foo(module: types.ModuleType):
pass
at least in PyCharm results in
"Cannot find reference ModuleType in types.pyi".
Note that Python typing for module type doesn't answer my question, as it does not explain that ModuleType is both a constructor as well as a type, as answered below.
and types.ModuleType() is a constructor.
That doesn't matter. types.ModuleType is still a reference to a type, just like str and int are. There is no need for a generic Module[typehint] annotation, so types.ModuleType is exacly what you need to use here.
For example, the official Python typeshed project provides a type hint annotation for sys.modules as:
from types import FrameType, ModuleType, TracebackType
# ...
modules: Dict[str, ModuleType]
Don't be confused by the name here; types.ModuleType is a reference to the module type. It is not a separate factory function or something. The CamelCase name follows the convention of that module, and you use that reference because the type object is not otherwise available as a built-in. The types module assigns the value of type(sys) to the name.
If PyCharm is having issues with finding the types.ModuleType stubs, then that's either a problem with PyCharm itself (a bug), or the stubs currently bundled are outdated, or you used an incomplete typeshed set of stubs. See the PyCharm documentation on how to use custom stubs to provide a fresh set.
If that doesn't work, it may be a bug in PyCharm dealing with the concept of exporting type hints. Typeshed currently defines the ModuleType type hints in a separate module, which are then imported into the types.pyi stubfile using the from module import name as name syntax. PEP 484 states that imported type hints are not part of the stub unless you use the as syntax:
Modules and variables imported into the stub are not considered exported from the stub unless the import uses the import ... as ... form or the equivalent from ... import ... as ... form.
It may be that PyCharm doesn't yet correctly handle such cases.
I have a function that accepts a class that derives from NamedTuple and converts it into a schema. However when I run MyPy on the following code it fails with Argument 1 to "to_schema" has incompatible type "Type[Foo]"; expected "Type[NamedTuple]"
from typing import NamedTuple, Type
def to_schema(named_tuple: Type[NamedTuple]):
pass
class Foo(NamedTuple):
pass
to_schema(Foo)
Is there a way to properly type the code so that it typechecks with MyPy?
Edit:
Python documentation states that Type[Foo] accepts any subclasses of Foo (https://docs.python.org/3/library/typing.html#typing.Type). I have multiple subclasses of NamedTuple, for entities in our data model, so I'm looking for a way to annotate the function in a way that would typecheck.
The root issue with your code is that NamedTuple is not an actual type -- it's actually just a special sort of "type constructor" that synthesizes an entirely new class and type. E.g. if you try printing out the value of Foo.__mro__, you'll see (<class '__main__.Foo'>, <class 'tuple'>, <class 'object'>) -- NamedTuple is not present there at all.
That means that NamedTuple isn't actually a valid type to use at all -- in that regard, it's actually a little surprising to me that mypy just silently lets you construct Type[NamedTuple] to begin with.
To work around this, you have several potential approaches:
Rather then using Type[NamedTuple], use either Type[tuple] or Type[Tuple[Any]].
Your Foo genuinely is a subtype of a tuple, after all.
If you need methods or fields that are specifically present only in namedtuples, use a custom protocol. For example, if you particularly need the _asdict method in namedtuples, you could do:
from typing_extensions import Protocol
class NamedTupleProto(Protocol):
def _asdict(self) -> Dict[str, Any]: ...
def to_schema(x: Type[NamedTupleProto]) -> None: pass
class Foo(NamedTuple):
pass
to_schema(Foo)
Note that you will need to install the typing_extensions third party library to use this, though there are plans to formalize Protocols and add it to Python at some point. (I forget if the plan was Python 3.7 or 3.8).
Add a type ignore or a cast on the call to to_schema to silence mypy. This isn't the greatest solution, but is also the quickest.
For related discussion, see this issue. Basically, there's consensus on the mypy team that somebody ought to do something about this NamedTuple thing, whether it's by adding an error message or by adding an officially sanctioned protocol, but I think people are too busy with other tasks/bugs to push this forward. (So if you're bored and looking for something to do...)