I am trying to measure code coverage by my pytest tests. I tried following the quick start guide of coverage (https://coverage.readthedocs.io/en/6.4.1/)
When I run my test with the following command, everything seems fine
coverage run -m pytest tests/
===================================== test session starts ======================================
platform linux -- Python 3.10.4, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/arnaud/Documents/Github/gotcha
collected 4 items
tests/preprocessing/test_preprocessing.py .... [100%]
====================================== 4 passed in 0.30s =======================================
However, when I try to access the report with either of those commands,
coverage report
coverage html
I get the following message:
No source for code: '<project_directory>/config-3.py'.
I did not find an appropriate solution to this problem so far
It is possible to ignore errors using the command
coverage html -i
which solved my issue
This issue is usually caused by older coverage result files, so you can either:
remove the old coverage results files or...
run coverage command with -i flag in order to ignore the errors - you can read more about that in coverage official docs: https://coverage.readthedocs.io/en/6.4.1/cmd.html#reporting
Another possible solution is to specify the source attribute. In my case, rather than the whole project (source = .), I specified the actual source folder (e.g. src). This can either be done on the commandline:
coverage run --source=src
or include it in your .coveragerc file:
[run]
source = src
...
I was getting this same issue because of a specific library I was importing*, but I never figured out why that library affected coverage, and others didn't.
Though this might just be a workaround, it makes sense to just check your source folder, and ignoring all errors (with -i) isn't much better.
* The library uses opencv-python-headless, which I think has the root cause of this issue.
Related
Background
I am wrangling some legacy code into shape.
I use PDM to manage dependencies, which places all dependent packages in a __pypackages__ folder directly under the repo root level. PDM also uses the relatively new pyproject.toml package config file.
I am trying to adopt pre-commit Git hooks so that I can have automated checks for formatting and style before trying to commit, merge, and/or create PRs.
I am asking pre-commit to use only a few Python tools for now: pylint and black.
Issue
Most of that toolset works great together. However, pylint cannot find any of the modules that are stored in the __pypackages__ folder. Most of what I have read suggests that I alter my $PYTHONPATH to find the modules.
This solution seems very outdated. But also, I am not sure how I can do this in a robust way across the team. I can alter the Git hooks, but the $PYTHONPATH may be different for each engineer, so this will only work for my machine.
I would like to be able to add something in the pyproject.toml file to have pylint find it. I am not sure what to write, though, so that it generically works across the whole team. Something like
[tools.pylint]
pypackages = "./__pypackages__"
Any ideas how I can do this?
Details
I am not sure more details are needed, but here it is:
My actions:
> pre-commit run --all-files # The --all-files flag is just to allow me to test without a commit
Trim Trailing Whitespace.................................................Passed
Fix End of Files.........................................................Passed
Check Yaml...........................................(no files to check)Skipped
Check for added large files..............................................Passed
black....................................................................Passed
pylint...................................................................Failed
- hook id: pylint
- exit code: 30
************* Module testfile
testfile.py:18:0: E0401: Unable to import 'boto3' (import-error)
boto3 is in the __pypackages__ mentioned above. None of the modules can be imported, but I limited the output for clarity.
I can pdm run ... everything correctly and VS Code sees the modules fine. But pylint is not finding it because it cannot find this __pypackages__ folder.
You can get around this by updating the PYTHONPATH environment variable used by the extension, by creating a file named .env in your workspace (project folder) and adding the following entry:
PYTHONPATH=D:/commonScripts
Note: Relative paths are also supported.
Further info on .env files can be found here https://code.visualstudio.com/docs/python/environments#_environment-variable-definitions-file
Can you explain the prompt 'F6401' when I run pylint pylint-pytest plugin cannot enumerate and collect pytest fixtures. Please run `pytest --fixtures --collect-only path/to/current/module.py` and resolve any potential syntax error or package dependency issues (Can-enumerate-pytest-fixtures) is the reason?
I would like to know how it works, or why it appears, and sometimes has different outputs. The same code, sometimes two, sometimes more. I was depressed.
I did run pytest --fixtures --collect-only without any unusual hints and my tests were normal.
Description:
After I fine-tune my existing code, including running pylint, pytest, and isort, everything works. I added a new package executor with three modules, one is the abstract module of base.py, two are corresponding to different implementation modules(local.py, docker.py).
Then I run isort, and pylint works fine
Then I import the base class and two implementation classes in the module's __init__.py file, and add a factory method.
When I run pylint again, the input tells me that some of the test modules have problems with F6401.
Again, I want to emphasize that everything was fine until I added this module. But now I just added the source code of this module, this exception will appear.
What makes it even more confusing to me is that the module I'm prompted doesn't include any fixtures. I ran pylint again and found that F6401 has more test modules (several times more than last time).
I've been using PyLint for a new project to check for a mode-by-module migration, and when I migrate to this module, I can't continue.
OS env
python 3.7
os: Deepin(base Debian)
IDE: Pycharm
Package versions
pylint 3.0.0a3
pylint-pytest 1.1.2
pyparsing 2.4.7
pytest 6.2.3
pytest-asyncio 0.14.0
pytest-cov 2.11.1
pytest-mock 3.5.1
ISSUE about this question.
After debugging the source code, I found out that the cause of my problems was an error in pylint-pytest when running pytest to collect fixtures from source code, and then pylint-pytest passed the error to PyLint.
My source code had a type annotation error that caused pytest to look for a fixture from that module that was wrong, and the error was passed to pylint. But why there is a different output is not clear to me.
From debugging the source code, we know that pylint-pytest registers itself with pylint, and when pylint checks all files, it passes the files to pylint-pytest's FixtureChecker.
https://github.com/reverbc/pylint-pytest/blob/62676386f80989cc0373d77bc5dc74acc635fd7a/pylint_pytest/checkers/fixture.py#L92-L142
The visit_module method in the FixtureChecker passes the file to pytest, running pytest <module_file> --fixtures --collect-only, At the same time load the FixtureCollector plug-in into pytest.
https://github.com/reverbc/pylint-pytest/blob/62676386f80989cc0373d77bc5dc74acc635fd7a/pylint_pytest/checkers/fixture.py#L125-L131
In pytest_collectreport , if an error is reported by pytest, it is logged and the error information is passed to pytest.
https://github.com/reverbc/pylint-pytest/blob/62676386f80989cc0373d77bc5dc74acc635fd7a/pylint_pytest/checkers/fixture.py#L24-L34
I don't think this logic makes sense. Pytest should only collect fixtures from the test modules, and instead of collecting fixtures from all modules, Pylint-Pytest should filter out the source code when PyLint checks.
At this point, my doubts have disappeared. Thanks.
I am trying to use the following pre-commit hook to check my code quality.
https://github.com/sebdah/git-pylint-commit-hook
I followed the instruction and installed it but it's displaying me a 0 score for all file. ex.
Running pylint on make_postreq.py (file 5/15).. 0/10.00 FAILED ************* Module make_postreq
Whereas if I run pylint from the console, I get a decent score.
Global evaluation
Your code has been rated at 8.75/10 (previous run: 8.75/10, +0.00)
I feel it might be a configuration issue, but can't seem to make it work.
Or is there some other way we can check our Python code quality before committing in GitHub?
This issue may be caused by setting the files-output parameter in your pylintrc to yes. I just ran into this where I transferred a configuration file over. The result processing in git-pylint-commit-hook requires that pylint print the final status to stdout, which is suppressed with files-output.
Try to
set 'reports=yes' in [REPORTS] section
don't disable 'RP0004'
I used to be able to get code coverage for unit testing a Google App Engine test via a commandline like:
coverage run --omit=/Applications --source=../mycode --branch /usr/local/bin/dev_appserver.py ...
[This uses Ned Batchelder's coverage.py] But, after recently updating to the latest SDK (after a long spell of not working on the code), I find that this does not work any more. The server process must run the application code in a subprocess or somesuch.
I tried following this http://nedbatchelder.com/code/coverage/subprocess.html#subprocess
I see another semi-recent question about this with a comment that suggests that coverage.py just won't work. Getting coverage with dev_appserver.py excludes my project files
I've spent a few frustrating hours googling around and trying some things with no luck. So...is this still impossible? Has anyone gotten code coverage to work in any manner? Is there some other tool that can figure out code coverage?
A short term fix might be to run the old dev_appserver.py
https://developers.google.com/appengine/docs/python/tools/old_devserver#Running_the_Old_Development_Web_Server
dragonx's suggestion to use old_dev_appserver.py worked well for me. More specifically, here's what I did using App Engine 1.9.6, coverage 3.7.1, and Python 2.7 on MacOS X 10.9.3:
MyAppDir is the directory containing app.yaml.
--omit is optional. You may well not need it. I had already moved my test code out of MyAppDir because I did not want appcfg.py to upload it.
--branch is optional but useful.
old_dev_appserver.py ships (for now) with App Engine. There is no need to download or install a copy.
# One time:
sudo pip install coverage
# Start the server:
APP=MyAppDir
coverage run \
--source=$APP \
--omit='$APP/exclude/*' \
--branch \
/usr/local/bin/old_dev_appserver.py \
$APP
# Run your tests in a separate tab. In my case I use this command:
webdriver/system_tests.py
# Kill the server with Control-C once the tests are finished.
# Display a quick text summary:
coverage report -m
# Generate and open an HTML report linking to line by line coverage:
coverage html
open htmlcov/index.html
My relatively straightforward app (email, full text search, ndb, urlfetch, webapp2) did not need any changes to work with old_dev_appserver. I did remove the flags I passed to dev_appserver. I was able to live without them. --port is supported if you need it, as are a few others.
If you'd like to see code coverage support in future versions of dev_appserver.py please vote up Add support for code coverage tests and some documentation, formerly https://code.google.com/p/googleappengine/issues/detail?id=4936.
I use coverage.py to check the test coverage of my django application. However since I use South for my database migrations, all those files show up with 0% and mess up the overall percentage.
I already tried using --omit=*migrations* in both run and report (and both) but that didn't work.
I tried versions 3.4 and latest revision from Bitbucket as of Dec 20th 2010 with the same result.
Any ideas how I can get coverage.py to actually ignore the migrations folders?
The solution was:
[run]
omit = ../*migrations*
You should be able to match against the migrations directory to omit those files. Have you tried quoting the argument? Depending on your OS and shell, it may be expanding those asterisks prematurely. Try it like this:
--omit='*migrations*'
Alternately, you could put the switch into a .coveragerc file:
[run]
omit = *migrations*
Latest version of django-jenkins has new option COVERAGE_WITH_MIGRATIONS that would exclude migrations. It's not in PyPI yet so you need to install it with pip/easy_install specyfing url git url as source.
Have you tried django_coverage. I think it handles this kind of problem.
This worked for me:
coverage run --source='.' --omit='*/migrations/*.py' manage.py test
try:
coverage run --source=. manage.py test app_name
this ignores third party code and fixes your % problem