We Keep Coding

How pylint-pytest throws F6401 Can-enumerate-pytest-fixtures

Can you explain the prompt 'F6401' when I run pylint pylint-pytest plugin cannot enumerate and collect pytest fixtures. Please run `pytest --fixtures --collect-only path/to/current/module.py` and resolve any potential syntax error or package dependency issues (Can-enumerate-pytest-fixtures) is the reason?
I would like to know how it works, or why it appears, and sometimes has different outputs. The same code, sometimes two, sometimes more. I was depressed.
I did run pytest --fixtures --collect-only without any unusual hints and my tests were normal.
Description:
After I fine-tune my existing code, including running pylint, pytest, and isort, everything works. I added a new package executor with three modules, one is the abstract module of base.py, two are corresponding to different implementation modules(local.py, docker.py).
Then I run isort, and pylint works fine
Then I import the base class and two implementation classes in the module's __init__.py file, and add a factory method.
When I run pylint again, the input tells me that some of the test modules have problems with F6401.
Again, I want to emphasize that everything was fine until I added this module. But now I just added the source code of this module, this exception will appear.
What makes it even more confusing to me is that the module I'm prompted doesn't include any fixtures. I ran pylint again and found that F6401 has more test modules (several times more than last time).
I've been using PyLint for a new project to check for a mode-by-module migration, and when I migrate to this module, I can't continue.
OS env
python 3.7
os: Deepin(base Debian)
IDE: Pycharm
Package versions
pylint 3.0.0a3
pylint-pytest 1.1.2
pyparsing 2.4.7
pytest 6.2.3
pytest-asyncio 0.14.0
pytest-cov 2.11.1
pytest-mock 3.5.1
ISSUE about this question.

After debugging the source code, I found out that the cause of my problems was an error in pylint-pytest when running pytest to collect fixtures from source code, and then pylint-pytest passed the error to PyLint.
My source code had a type annotation error that caused pytest to look for a fixture from that module that was wrong, and the error was passed to pylint. But why there is a different output is not clear to me.
From debugging the source code, we know that pylint-pytest registers itself with pylint, and when pylint checks all files, it passes the files to pylint-pytest's FixtureChecker.
https://github.com/reverbc/pylint-pytest/blob/62676386f80989cc0373d77bc5dc74acc635fd7a/pylint_pytest/checkers/fixture.py#L92-L142
The visit_module method in the FixtureChecker passes the file to pytest, running pytest <module_file> --fixtures --collect-only, At the same time load the FixtureCollector plug-in into pytest.
https://github.com/reverbc/pylint-pytest/blob/62676386f80989cc0373d77bc5dc74acc635fd7a/pylint_pytest/checkers/fixture.py#L125-L131
In pytest_collectreport , if an error is reported by pytest, it is logged and the error information is passed to pytest.
https://github.com/reverbc/pylint-pytest/blob/62676386f80989cc0373d77bc5dc74acc635fd7a/pylint_pytest/checkers/fixture.py#L24-L34
I don't think this logic makes sense. Pytest should only collect fixtures from the test modules, and instead of collecting fixtures from all modules, Pylint-Pytest should filter out the source code when PyLint checks.
At this point, my doubts have disappeared. Thanks.

Python linter in vscode with incomplete type checking (Expected... got...)

I use vscode and I found the Python linter(s) do not detect when a python function is invoked with a parameter of different type than it's signature defines.
In this case, the function expects a typing.Type[Exception] instead of Exception:
Pycharm detects this situation:
I think mypy does not detect this one because it uses the typing module (mypy detects type mismatch in builtin types).
Is there a way to configure mypy to detect cases like this?
Which is the linter pycharm uses? Does it exist in vscode?
Is there another linter that could do the job?
Note: I followed the flag suggestion from this SO post (--check-untyped-defs) but it didn't work.

After following the suggestions in the comment section, I was able to do this in vscode by:
Installing the Pylance plugin and
Setting the following params on the User Settings:
"python.languageServer": "Pylance",
"python.analysis.typeCheckingMode": "strict",

VSCode/pylint reports module win32event as unresolved import when module is installed

It's a small annoyance, but, pywin32 modules get reported as unresolved imports by the python linter in VSCode.
I've got pywin32 installed, and I'm able to import the module(s) in a terminal session, and when the script runs.
My python.pythonPath is set correctly in my settings.json.
The linter just can't seem to find the modules.
But if I use the following syntax the linter works, but the import obviously fails at run time.
import win32.lib.win32event
Any ideas on how I can "make" VSCode or the linter match the correct import?

Pylint has a configuration file, loaded by the command-line switch --rcfile. One of the entries in the configuration file is ignored-modules=. A comment line describes this option as a "list of module names for which member attributes should not be checked". There are several other options that are similar.
These options are needed because pylint is a static type checker, meaning that it doesn't load the module but merely inspects its source code. In the case of a module like the win32 collection, it uses .dll files that pylint cannot inspect. Therefore there is no way that pylint can figure out what names are exposed by win32. So the best you can do is tell pylint to suppress error messages.

pylint 1.4 reports E1101(no-member) on all C extensions

We've been long-time fans of pylint. Its static analysis has become a critical part of all our python projects and has saved tons of time chasing obscure bugs. But after upgrading from 1.3 -> 1.4, almost all compiled c extensions result in E1101(no-member) errors.
Projects that previously run perfectly clean through pylint 1.3 now complain about almost every C extension member with E1101. We've been forced to disable E1101 errors, but this materially detracts from the usefulness of pylint.
For example, this perfectly valid use of the lxml package
r"""valid.py: demonstrate pylint 1.4 error"""
from lxml import etree
print etree.Element('mydoc')
Run this through pylint, and it reports:
$ pylint -rn valid.py
No config file found, using default configuration
************* Module valid
E: 3, 6: Module 'lxml.etree' has no 'Element' member (no-member)
But it is perfectly valid:
$ python valid.py
<Element mydoc at 7fddf67b1ba8>
Here's where it gets really weird. A very small handful of C extensions seem to work just fine through pylint, e.g.:
r"""valid2.py: this one works fine"""
import sqlite3
print sqlite3.version
$ pylint -rn valid2.py
No config file found, using default configuration
My question is, has anyone else witnessed this? And if so, would you be willing to share your workaround/solution?
We've experimented with trying to create plugins to suppress these warnings
(http://docs.pylint.org/plugins.html#enter-plugin), but we're having difficulty making heads or tails of the docs -- and the astroid base class is uber-complex and has defied our attempts to grok it.
For real bonus points (and our eternal gratitude) we'd love to understand what changed in pylint. We'd be happy to fix the code (or at least publish a best practice document for C extension authors) that would satisfy pylint.
Platform details
$ pylint --version
No config file found, using default configuration
pylint 1.4.0,
astroid 1.3.2, common 0.63.2
Python 2.7.5 (default, Jul 1 2013, 18:09:11)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-3)]

Shortly after posting my question, I found the answer. The change was in fact done on purpose as a security measure. Pylint imports modules to effectively identify valid methods and attributes. It was decided that importing c extensions that are not part of the python stdlib is a security risk and could introduce malicious code.
This was done in the release of Astroid 1.3.1 https://mail.python.org/pipermail/code-quality/2014-November/000394.html
Only C extensions from trusted sources (the standard library) are
loaded into the examining Python process to build an AST from the live
module.
There are four solutions if you want to use pylint on projects that import non-stdlib c extensions.
1) Disable safety using the --unsafe-load-any-extension=y command line option. This feature is undocumented and classified as a hidden option (https://mail.python.org/pipermail/code-quality/2014-November/000439.html).
2) Disable safety using the pylint.rc setting unsafe-load-any-extensions=yes. This is recommended over option 1 and includes full documentation in the default pylint.rc file (created with --generate-rcfile).
3) Specifically list packages or modules names that you trust to be loaded by pylint in the pylint.rc file using the extension-pkg-whitelist= option.
4) Create a plugin to manipulate the AST (I have no idea how to effect this -- but it's regularly discussed on on the pylint mailing list).
We opted for Option 3. We added the following line to our project pylint.rc file:
extension-pkg-whitelist=lxml

#user590028, thanks a lot for your answer! I just ran into this same problem with the libraries win32api, win32evtlog, win32file, win32gui, and win32process, and your solution worked.
I used another method I think is worth posting here, which is to call pylint and pass the whitelisted packages as a parameter:
pylint --extension-pkg-whitelist=win32api,win32evtlog,win32file,win32gui,win32process myfile.py

For those of you using VS Code, it's a bit tricky to find where to put the command as I couldn't find my executable.
In VS Code;
click on File > Preferences > Settings.
Scroll down to "Python Configurations" in the left window
scroll down to "Python Linting: Mypy Args" in the right window
click on "Edit in settings.json" link
edit the json to include:
"--extension-pkg-whitelist="
I had to do all this because PyLint isn't executable from my Windows command line...

If you're using VS Code for Mac, this is what you need to do in order to edit the settings.json file:
Click on Code (i.e. the Visual Studio Code tab which is on the left of the 'File' tab) -> Preferences - > Settings
Scroll down to Extensions and click on Python in the list.
Click on any of the Edit in settings.json links. This opens up settings.json for editing.
Add the line "python.linting.pylintArgs": ["----extension-pkg-whitelist=1xml"].

PyLint 1.0.0 with PyDev + Eclipse: "include-ids" option no longer allowed, breaks Eclipse integration

As noted in this question: How do I get Pylint message IDs to show up after pylint-1.0.0?
pylint 1.0.0 no longer accepts "include-ids" option. (It returns "lint.py: error: no such option: --include-ids"). Unfortunately, in the integation with PyDev/Eclipse, there is this little nugget:
"The --include-ids=y is always included...".
How to disable that argument so that Pylint will work with Eclipse?
[I know, other alternatives include installing an older version of Pylint or running pylint from command line without that option (which does work), but I'd like to have the integration with Eclipse.]

This should be already fixed in the latest nightly build. Please grab it there.
See: http://pydev.org/download.html for details on how to get it.

To make ID's appear in the Problem View on Eclispe using PyDev, make sure you use
msg-template={msg_id}:{line:3d},{column}: {obj}: {msg}
in pylintrc or on the command line options. Eclipse needs this to parse the PyLint output.

The following option works with pylint v1.4.3 under pydev v3.9.2:
--msg-template="{msg_id}:{line:3d},{column:2d}:{msg}"
Note: Don't put space(s) after the semicolons otherwise it does not work.