I've been trying to tighten up a few classes of code with type hinting. I'm struggling with a nesting problem. The parameter of a function of interest is an iterable container of objects with known types. I'm having trouble getting my IDE to recognize the inner types (which is a nice convenience.)
I'm using python 3.9 on PyCharm, but I'm getting similar auto-complete results in Ipython.
When using the Iterable class out of typing module, the IDE cannot "see through" to the inner types. Similarly for Collection. But it can when using either list or tuple on the outer container.
Is this an IDE issue or is there another way to package this? It would be nice to be able to send this function any type of iterable, rather than hard code it...
from datetime import datetime
from typing import Iterable
Data_collection = Iterable[tuple[datetime, str]]
Data_list = list[tuple[datetime, str]]
def foo(bar: Data_collection):
bar[1][0]. # no type-hint on inner obj
def foo2(bar: Data_list):
bar[1][0]. # good type-hint on inner obj
Gets me (PyCharm) results like this:
You are using the wrong abstract base class. Iterable only promises/requires __iter__ to be implemented, not __getitem__. Statically speaking, bar[1] isn't guaranteed to be defined.
To specify any type that supports indexing, use Sequence instead.
from typing import Sequence
Data_collection = Sequence[tuple[datetime, str]]
Now, regardless of the runtime type of bar, your IDE can assume that bar.__getitem__ is defined and returns a tuple. (Whether your specific IDE does make that assumption depends on the IDE, not your code.)
Related
In python, if I am writing a function, is this the best way to type hint a list of strings:
def sample_def(var:list[str]):
I would use the typing module
from typing import List
def foo(bar: List[str]):
pass
The reason is typing contains so many type hints and the ability to create your own, specify callables, etc. Definitely check it out.
Edit:
I guess as of Python 3.9 typing is deprecated (RIP). Instead it looks like you can use collections.abc.*. So you can do this if you're on Python 3.9+:
from collections.abc import Iterable
def foo(bar: Iterable[str]):
pass
You can take a look at https://docs.python.org/3/library/collections.abc.html for a list of ABCs that might fit your need. For instance, it might make more sense to specify Sequence[str] depending on your needs of that function.
I'm new to Python development, and trying to get a fix on things. I am using Pycharm for development. I am currently trying to annotate the types of variables for easier access with autocompletion and suggestions. I've tried iterations of the code with mixed results.
This is the code that has a problem:
path = os.path.dirname(os.path.realpath(__file__)) # type: str
components = path.split(os.sep) # type: list[str]
The first problem displayed is at the opening brace of the type annotation of the second line. It says:
Class 'type' does not define '__getitem__', so the '[]' operator cannot be used on its instances.
I've googled around, and although the problem seems clear, opening the code for the list class clearly shows a method __getitem__:
class list(object):
"""
list() -> new empty list
list(iterable) -> new list initialized from iterable's items
"""
....
def __getitem__(self, y): # real signature unknown; restored from __doc__
""" x.__getitem__(y) <==> x[y] """
pass
Okay, maybe that's not trivial to understand and there's some other loading mechanism at work. And besides, the "problem" seems to be that I used list[str] and not List[str]. So I altered the code:
path = os.path.dirname(os.path.realpath(__file__)) # type: str
components = path.split(os.sep) # type: List[str]
Now everything breaks: The second line now complains about this:
Expected type 'List[str]', got 'List[str]' instead`
The previous problem regarding __getitem__ yet persists.
Is there a way to annotate these variables without causing problems for the checker? I am not very happy with the Python documentation in that regard, not explicitly stating the return types of their built-in methods. I have to rely on the information Pycharm provides in the documentation pop-ups (Ctrl+q).
It was a right solution to use List[str] instead of list[str] since builtin types could not be used in type hints, the corresponding PEP has not been accepted yet.
From what module do you import List? I'm unable to reproduce an issue in 2019.3.
In Python's typing module, it seems that anything that is not a generic type does not support subscripting:
>>> import typing
>>> from yarl import URL
>>> StrOrURL = typing.Union[URL, str]
>>> typing.List[StrOrURL] # OK; List is generic
typing.List[typing.Union[yarl.URL, str]]
>>> SetOrDict = typing.Union[set, dict]
>>> SetOrDict[StrOrURL] # Raises
TypeError: typing.Union[set, dict] is not a generic class
This will be the case whether SetOrDict is defined with typing.Union or typing.TypeVar. (In Python 3.7, at least.)
It looks like there is a way to subclass Generic, presumably like:
class SetOrDict(typing.Generic[StrOrURL]):
# ...
But this will immediately raise TypeError: Parameters to Generic[...] must all be type variables.
Is there a way to accommodate the above? I.e., what is the recommended practice for subscripting user-defined (non-generic) typedefs?
You can't use a Union[...], alias or otherwise, as a generic type, no. Your union contents are also not generic, you can't state what types a dict or set object can contain by using dict or set directly, you'd use Set or Dict.
You have a new union:
URLorStrSetOrDict = typing.Union[Set[StrOrUrl], Dict[StrOrUrl, Any]]
Note that a dictionary has keys and values, you need to provide information on both. If this used as the input for an API, then consider using the more generic and immutable AbstractSet and Mapping types; this makes it clear that your API only needs to read information.
Personally, I'd look hard at refactoring a codebase that is this muddy about what kinds of objects it can accept. Requiring that an API accepts only sets and existing code has to pass in set(dictionary) instead of dictionary is not an arduous requirement, or perhaps the API really would work with any iterable of StrOrUrl.
I'm wanting to go through a list of objects so that my PyCharm IDE knows what type each list item is:
For example, say I know that each item in a list is an class instance of type 'myClass' - how do I use this to cast my objects so that my ide can help with code completion?
for i in range(len(myList)):
myClass(myList[i]).myClassProperty .....
I know how to do it in Delphi (something like the above) but not in python.
Thanks
In PyCharm, you can use Type Hinting:
class Bar:
def __init__(self,bar):
self.bar = bar
def do_bar(self):
return self.bar
def foo(x):
for el in x: # type: Bar
el.do_bar()
bars = [Bar('hello'), Bar('World')]
foo(bars)
You can't get code completion similar to Java or C++ in dynamically typed, interpreted language.There is no casting, because you don't need it in python. A function works for a given object if it has needed methods implemented, type is irrelevant to the language at this point. It is good practice though to leave some runtime checks using isinstance, if you expect your argument to be e.g. a dict. Otherwise you will end up with many difficult bugs.
As for code completion in python there are two solutions I find useful. The best IDEs around here are probably PyCharm https://www.jetbrains.com/pycharm/ and PyDev the Eclipse Plugin http://www.pydev.org/manual_101_install.html. They provide some code completion.
The other is interactive console Jupyter http://jupyter.org/. As you write your code, you could execute it in chunks (cells) and easily see object methods or fields, using not the type information, but the object itself existing in memory. This is very good for data analysis or playing with framework you don't know well.
I am trying to figure out how to properly function annotate or type hint a list of strings. For example, if I had a function like this:
def send_email(self, from_address: str, to_addresses: list[str]):
pass
to_addresses should be a list of strings. But when I try to use that annotation I get the following error in my Python 3.4.3 interpreter:
TypeError: 'type' object is not subscriptable
I am positive the list[str] is causing the issue, because if I change it to str the error goes away, but that doesn't properly reflect my intentions for the parameter.
Python 3.4 doesn't specify a format for its function annotations, it merely provides a mechanism that allows you to use any expression as the annotation. How the annotations are interpreted is up to you and the libraries you use.
Python 3.5 standardizes the way function annotations are used for type hinting, as documented in PEP 484. To annotate a list of strings, you use List[str], where List is imported from the typing module. You can also use Sequence[str] if your function accepts any list-like sequence, or Iterable[str] for any iterable.
Starting with Python 3.9, you can use list[str] as a type annotation, which doesn't require importing anything.
In Python 3.9+, list (with a lowercase l) can be used in type annotations and your code should work as is. On older versions of Python you need to import typing.List and use it instead
from typing import List
to_addresses: List[str]
Note the capital L.
You might want to consider something more specific, e.g.
import typing
Address = typing.NewType("Address")
See NewType docs
The static type checker will treat the new type as if it were a subclass of the original type
This syntax is now valid in Python 3.9+:
In type annotations you can now use built-in collection types such as list and dict as generic types instead of importing the corresponding capitalized types (e.g. List or Dict) from typing.
Prior to 3.9 though, you need to use the imported List or in Python 3.7+ you can add
from __future__ import annotations
at the top of your file, which allows using list[int] (for example). Note though, this import only affects annotations:
from __future__ import annotations only affects annotations -- just the things after the colon. It makes it so that annotations are never evaluated
It still doesn't allow list[int] in arbitrary contexts.