pylint complain if we use pandas class instances - python

pylint complain about pandas class instances::
I have db instance that has data (a panda Dataframe) as instance.
If I call e.g. iloc or shape on it::
cols = db.data.shape
xxx = db.data.iloc[1:4, 0:9]
pylint complain about::
E: 36,18: Instance of 'TextFileReader' has no 'iloc' member (no-member)
E: 92,30: Instance of 'TextFileReader' has no 'shape' member (no-member)
E: 92,30: Instance of 'tuple' has no 'shape' member (no-member)
I've try How do I get PyLint to recognize numpy members? and Disabling Pylint no member- E1101 error for specific libraries with no success.

This seems to be an open issue in pylint that has been marked high priority as of March 14th 2022.
The issue stems from the underlying AST engine that pylint uses that has a limit to the number of inferences it can make at runtime as a bid to improve performance.
For posteriority in case the link above ever dies, the workaround is as follows:
pylint some_file_to_lint.py --init-hook "import astroid; astroid.context.InferenceContext.max_inferred = 500"
Or in .pylintrc:
[MASTER]
init-hook = "import astroid; astroid.context.InferenceContext.max_inferred = 500"

Related

What causes this mysterious pylint E1101 error?

I have following code:
#!/usr/bin/env python
"""pylint behavior test"""
def autodetect_method(method, data):
"""autodetect method"""
if not method:
method = 'POST' if data else 'GET'
else:
method = method.upper()
return method
pylint produces the following error:
tt.py:10:17: E1101: Class 'method' has no 'upper' member (no-member)
Error is not reported if I rename method variable to something else, f.e. to method_name!
So, I know several ways to get rid of this error message. But I am very curious what is so special with variable name method and why the error is generated?
Just in case this problem is version-specific, my versions are:
$ pylint --version
pylint 2.4.4
astroid 2.3.3
Python 3.8.5 (default, Jan 27 2021, 15:41:15)
[GCC 9.3.0]
There is the way you could satisfy your curiosity - debugging.
You could run pylint from the same file and trace its behavior.
if __name__ == "__main__":
import pylint
import sys
sys.argv.append(__file__)
pylint.run_pylint()
Actually it's easier to say than to do, I've tried, but was unable to understand what exactly happens in details. In high-level terms, pylint tries to infer the type for method in the expression method.upper(), unable to do it itself, falling into astroid library, and there the method type inferred as ClassDef.method. Obviously the word method means something special for astroid, as some other words. For example, the same E1101 error would detected if you use name function instead of method.
Looks like it's behavior somehow relate to builtin's names, but function and method are not builtins. I guess astroid treats them as some kind if "aliases" for ast classes, but I'm not sure.

Unresolved attribute reference 'cfg' for class 'Runattr'

I have declared a class as follows:
class Runattr:
pass
run = Runattr()
run.cfg = 'some str'
When I am trying to access run.cfg further in my code, PyCharm gives me the following warning and I am not able to autocomplete run.cfg:
Unresolved attribute reference 'cfg' for class 'Runattr' Inspection info: This inspection detects names that should resolve but don't. Due to dynamic dispatch and duck typing, this is possible in a limited but useful number of cases. Top-level and class-level items are supported better than instance items.
I am new to using classes. Can someone tell me why I am seeing this warning and can this be modified to get rid of the warning?
It's just a PyCharm warning (not an error!), since it can't figure out whether that class really should have a cfg attribute.
Assuming you're using a modern Python version, add an annotation for such a field:
class Runattr:
cfg: str # denotes there would/could be a string `cfg`
run = Runattr()
run.cfg = 'some str'
Classes and instances can nevertheless have any attributes (and heck, you can even do setattr(run, "oh this is fun", True) to have an attribute that's not a valid Python identifier), it's just that IDEs can't be infinitely smart about the dynamic nature here.
you have to declare on a field named cfg
class Runattr:
cfg = ''
run = Runattr()
run.cfg = 'some str'

Python how to iterate over StrEnum without linter (PyCharm inspections) unresolved references

I have implemented a StrEnum (aka enforcing Enum members to be str) per the instructions within the Python 3.6 enum docs, section 8.13.13.4 Others.
from enum import Enum
class StrEnum(str, Enum):
"""Enum where members are strings."""
I have a subclass of StrEnum where I use a for loop to iterate through it.
class TestStrEnum(StrEnum):
"""Test string enum."""
A = "Apple"
B = "Bananas"
for enum_ in TestStrEnum:
print(f"name = {enum_.name}, value = {enum_.value}.")
This works fine at runtime. However, my linter complains about an unresolved reference when referring to the name and value attributes:
How can I get my linter to not complain?
The problem seems to be that the linter is not properly inspecting within the StrEnum._member_map_. I think the solution may be to override a dunder method within StrEnum, but am not sure which one to use.
Workaround Solution
Here is a workaround I figured out by using the __members__ attribute to access the members in an explicit manner.
for enum_member_name, enum_member in TestStrEnum.__members__.items():
print(f"name = {enum_member.name}, value = {enum_member.value}.")
My linter is PyCharm 2019.2.6 CE's code inspections.
**EDIT**
Per #user2235698's answer, it seems this is a known issue with PyCharm: PY-36205, raised in May 2019. Since this issue only currently has 6 upvotes (as of April 11th, 2020), I am not sure it will be fixed anytime soon.
If anyone has a good solution that can be used until the issue is resolved, I am still interested.
It is a known issue, please vote for https://youtrack.jetbrains.com/issue/PY-36205 (thumbs up near the issue title)

Suppress warnings from Python module's destructor order

I am getting a series of warnings that occurs during interpreter shutdown of a Python 2.7 script when using libvirt. This is reproducible on a variety of distros including python2-libvirt 3.7.0-1 on Fedora 27 to libvirt-python 3.2.0-1 on Centos 7.4.1708. The warnings I get when the script is exitings is:
Exception AttributeError: "'NoneType' object has no attribute 'virDomainFree'" in <bound method virDomain.__del__ of <libvirt.virDomain object at 0x7f34a194ee10>> ignored
Exception AttributeError: "'NoneType' object has no attribute 'virDomainFree'" in <bound method virDomain.__del__ of <libvirt.virDomain object at 0x7f34a194ed90>> ignored
Exception AttributeError: "'NoneType' object has no attribute 'virConnectClose'" in <bound method virConnect.__del__ of <libvirt.virConnect object at 0x7f34a176a590>> ignored
Drilling down into the library, it seems to be an issue with assumptions in destructor order as in this code from libvirt.py:
class virDomain(object):
def __del__(self):
if self._o is not None:
libvirtmod.virDomainFree(self._o)
self._o = None
libvirtmod is a global created by an import at the top of the libvirt.py module. When the __del__() destructor for virDomain is finally run, libvirtmod has been replaced by the value None causing the code above to fail with a warning. We have been using this Python module for some time now, but only recently have these warning started showing up after we refactored the code quite heavily. What can I do to suppress these warnings from standard error or avoid the situation from occurring? Is there a way to ensure objects from libvirt.py are cleaned up before libvirtmod.so goes away?
We test on a variety of distros and would like to stick with using the stock (but updated) packages that come with the distro.
The standard trick is to equip __del__ with default arguments that hold whatever global values it needs (here, libvirtmod itself would suffice). There is a variation using weakref, which can also benefit from default arguments.

How to specify the object type so it is recognized prior to running the code in Python?

Suppose I have the following code excerpt:
import pickle
with open('my_object.pkl','r') as f:
object = pickle.load(f)
My question is:
Suppose object is from a class I defined previously, how can I specify this in the code such that my interpreter knows prior to running the code what the object class is ? My goal here is to have the auto-completion of my IDE (I use VSCode) recognize the object so I can auto-complete and easily search the methods and attributes of that object.
It depends on the version of Python and IDE, but in general looks like an additional statement with assertion instance type is the only way so far. This will trigger VS autocomplete settings
import pickle
with open('my_object.pkl','r') as f:
object = pickle.load(f)
assert isinstance(object, YourType)
# and now you can use autocompletion with the object
The following issue is tracking that feature: #82.

Categories