What causes this mysterious pylint E1101 error? - python

I have following code:
#!/usr/bin/env python
"""pylint behavior test"""
def autodetect_method(method, data):
"""autodetect method"""
if not method:
method = 'POST' if data else 'GET'
else:
method = method.upper()
return method
pylint produces the following error:
tt.py:10:17: E1101: Class 'method' has no 'upper' member (no-member)
Error is not reported if I rename method variable to something else, f.e. to method_name!
So, I know several ways to get rid of this error message. But I am very curious what is so special with variable name method and why the error is generated?
Just in case this problem is version-specific, my versions are:
$ pylint --version
pylint 2.4.4
astroid 2.3.3
Python 3.8.5 (default, Jan 27 2021, 15:41:15)
[GCC 9.3.0]

There is the way you could satisfy your curiosity - debugging.
You could run pylint from the same file and trace its behavior.
if __name__ == "__main__":
import pylint
import sys
sys.argv.append(__file__)
pylint.run_pylint()
Actually it's easier to say than to do, I've tried, but was unable to understand what exactly happens in details. In high-level terms, pylint tries to infer the type for method in the expression method.upper(), unable to do it itself, falling into astroid library, and there the method type inferred as ClassDef.method. Obviously the word method means something special for astroid, as some other words. For example, the same E1101 error would detected if you use name function instead of method.
Looks like it's behavior somehow relate to builtin's names, but function and method are not builtins. I guess astroid treats them as some kind if "aliases" for ast classes, but I'm not sure.

Related

declare variable type inside function

I am defining a function that gets pdf in bytes, so I wrote:
def documents_extractos(pdf_bytes: bytes):
pass
When I call the function and unfortunately pass a wrong type, instead of bytes let's say an int, why I don't get an error? I have read the documentation regarding typing but I don't get it. Why is the purpose of telling the function that the variable shoudl be bytes but when you pass and int there is no error? This could be handle by a isinstance(var, <class_type>) right? I don't understand it =(
Type hints are ignored at runtime.
At the top of the page, the documentation that you've linked contains a note that states (emphasis mine):
The Python runtime does not enforce function and variable type annotations. They can be used by third party tools such as type checkers, IDEs, linters, etc.
The purpose of type hints is for static typechecking tools (e.g. mypy), which use static analysis to verify that your code respects the written type hints. These tools must be run as a separate process. Their primary use is to ensure that new changes in large codebases do not introduce potential typing issues (which can eventually become latent bugs that are difficult to resolve).
If you want explicit runtime type checks (e.g. to raise an Exception if a value of a wrong type is passed into a function), use isinstance().
By default python ignores type hints at runtime, however python preserves the type information when the code is executed. Thanks to this library authors can implement runtime type checking packages such as typeguard, pydantic or beartype.
If you don't want to use isinstance checks yourself, you can use one of those libraries.
Typeguard example:
main.py:
from typeguard import importhook
importhook.install_import_hook('mypack')
import mypack
mypack.documents_extractos("test")
mypack.py
def documents_extractos(pdf_bytes: bytes):
pass
When you run python3 main.py you will get error TypeError: type of argument "pdf_bytes" must be bytes-like; got str instead

Typing NamedTuple as a class to include argument hints in PyCharm

EDIT: Type[X] does type hint to the class itself. The repl now has an example of that, too. PyCharm just doesn't use the __init__ information within the class to type hint an initialization elsewhere. If anyone knows why that is, I'll accept that answer. The below question is now just for context:
Let's say I have the following code:
from typing import NamedTuple, Type
class SubTuple(NamedTuple):
a: str
b: int
def get_tuple_type() -> Type[SubTuple]:
return SubTuple
test_tuple = get_tuple_type()
test_tuple_instance = test_tuple('test', 1)
Is there a way to type hint so that PyCharm will give me argument hints? Right now it gives me this:
which is rather useless. Hilariously, it does know the arguments, because when I type:
test_tuple.
it recommends all of the fields in the SubTuple. It just won't recommend them to me when I go to create an instance of the thing.
Also, https://repl.it/repls/BurdensomeInnocentKnowledge just so that you can see that the above code actually does work. The repl IDE also has no idea what to do with this thing, and is recommending that I call .a and .b on the class itself, which leads me to believe this might just be a general issue with what I'm trying to do in Python?
EDIT: I'm using Python 3.7; updated code to include what I've tried. It has the same result. Will update the repl, too.
This is a bug in PyCharm's static type checker, there are a few open bugs about NamedTuple on JetBrains bugtracker. The most up-to-date thread on the NamedTuple datatype is PY-18246. Fixes have been rolled out during 2020 but this specific case of correctly inferring the NamedTuple members after typing a class object has neither been reported nor solved.
The warning has also changed, if you try the same code with PyCharm 2020.3.3 Pro you'll get a different hint.
I tried checking the code with Mypy and it works correctly without any warnings. If you try causing an error by changing the types:
variable = test_tuple(1, 'test')
Mypy warns and correctly hints you about it:
error: Argument 1 to "SubTuple" has incompatible type "int"; expected "str"
error: Argument 2 to "SubTuple" has incompatible type "str"; expected "int"
Found 2 errors in 1 file (checked 1 source file)
Is there a way to type hint so that PyCharm will give me argument hints?
You are already type hinting correctly as PEP 484 recommends. If you use the Mypy static type checker it works, as for PyCharm it's a matter of waiting for a bug fix. There's nothing wrong with your type hinting and it shouldn't be changed in this case.

Is there a python linter that checks types according to type hints?

I'm looking for a Python linter, that can check types usage according to the type hints in the code.
The purpose is to run a single check that verifies style, logic, and type errors.
I need to run this on CI server, and as a file watcher during development.
For example, I need this code to output an error for passing the wrong type argument -
def double(x: int):
return x * 2
result = double('hello')
I've checked the documentation of PyLint and flake8, and couldn't find any support for type checking.
With PyLint I also verified there are no errors when checking the above code.
Yes, there is, it's called mypy

Tell pylint what the return type of a method is

To give details: this is a constrained environment with Python 2.7.14, Pylint 1.8.1, and Astroid 1.6.0 and various other modules, but I can't easily install new modules such as mypy (not even with virtualenv) or make major changes to the codebase.
Due to https://github.com/PyCQA/pylint/issues/400 I'm getting errors from pylint on some of my code. I've worked around this issue in a hacky way (by removing the integer argument to the wait() method) and now I don't get errors, but no checking is done (pylint can't determine what the type of the variable is at all).
What I'd really like to do is tell pylint what the return type is. I've tried reading the Astroid docs, perusing the various astroid/brains files, and looking at other SO questions and answers such as Set multiple inferred types based on arguments for pylint plugin and pylint, coroutines, decorators and type inferencing but I guess I'm just not getting it. I have added my own load-plugins and I know it's loaded during pylint (because it had syntax errors at first :)). But I'm just not sure what it needs to do.
Suppose I had this code:
class Peer(object):
def get_domain(self):
return "foo"
class TestListener(object):
def __init__(self):
self._latest = None
def get(self, timeout):
# tricks here
return self._latest
def peer_joined(self, peer):
self._latest = peer
peer = TestListener().get(3)
print(peer.get_domain())
print(peer.get_foo())
During # tricks here we are really waiting on a threading.Event() during which another thread will invoke peer_joined() and pass an object of type Peer(), but pylint doesn't grok this.
What I'd like to do is annotate the TestListener.get() method to tell pylint that it will return a Peer object, so that pylint will catch the error in the second print call.
I've tried this but clearly I'm missing something fundamental, since it appears my transform method is never even invoked (if I put a bogus method call there no error is printed):
from astroid import MANAGER, register_module_extender
from astroid.builder import AstroidBuilder
def TestListener_transform():
return AstroidBuilder(MANAGER).string_build('''
class TestListener(object):
def get(self, timeout):
return Peer()
''')
register_module_extender(MANAGER, 'TestListener', TestListener_transform)
# for pylint load-plugins
def register(linter):
pass
def get(self, timeout):
"""
:param timeout: How many seconds to wait
:type timeout: int
:rtype: threading.Event
"""
# tricks here
return self._latest
This is a reStructuredText format for writing a python docstring (a multiline string that appear right after function definition.
You can document there all parameter types, and what the function return along with their datatypes.
pylint can parse that docstring to figure out the datatype of that function's return value.
This stackoverflow answer has more comprehensive explanation
What is the standard Python docstring format?
One thing to mention is that pylint currently has limited capabilities for type-checking, as those seen in mypy. It relies more on inferring values and types without type hints, although the intent is to have support for PEP-484 typing at some point.
Now regarding the question, I see that you are using a module extender, while in fact you should use a class transform, as in:
astroid.MANAGER.register_transform(astroid.ClassDef, _class_transform,
optional_transform_predicate)

Python - curly braces in type hints

What do these mean?
def f(a: {int, float}):
pass
I've seen this syntax used in some standard Python modules when fetching documentation via PyCharm, and I have no idea what it means. What's the hinted type for a in my example? What types can I pass to this function?
The particular example where I've seen this is in tkinter's Frame __init__ method, where the master parameter is of type {tk, _w}.
It's a hint telling you it wants an object with the named attributes 'int' and 'float' -- or more specifically for tkinter 'tk' and '_w'
I coded up a minimal example in pycharm:
Inpecting the python library sources -- You can see that there are attempted accesses to master.tk and master._w. That's all that pycharm was able to infer about the type of the parameter master so it floated it up to the IDE in this manner.

Categories