https://coverage.readthedocs.io/en/coverage-4.5.1a/source.html#source
My coverage is also including “venv” folder and I would like to exclude it
no matter what I do even with --include or omit nothing works
coverage run --omit /venv/* tests.py
This runs the test but still adds "venv" folder and dependencies and their % coverage
When I do
coverage run --include tests.py
To run only tests - it says
Nothing to do.
It is pretty annoying... can someone please help?
The help text for the --omit option says (documentation)
--omit=PAT1,PAT2,... Omit files whose paths match one of these patterns.
Accepts shell-style wildcards, which must be quoted.
It will not work without quoting the wildcard, as bash will expand the wildcards before handing the argument list to the coverage binary. Use single-quotes to avoid bash wildcard expansion.
To run my tests without getting coverage from any files within venv/*:
$ coverage run --omit 'venv/*' -m unittest tests/*.py && coverage report -m
........
----------------------------------------------------------------------
Ran 8 tests in 0.023s
OK
Name Stmts Miss Cover Missing
-------------------------------------------------------
ruterstop.py 84 8 90% 177, 188, 191-197, 207
tests/test_ruterstop.py 108 0 100%
-------------------------------------------------------
TOTAL 192 8 96%
If you usually use plain python -m unittest to run your tests you can of course omit the test target argument as well.
$ coverage run --omit 'venv/*' -m unittest
$ coverage report -m
For those who don't want to pass --omit each time when executing coverage you can define following in .coveragerc or in pyproject.toml. Example for .coveragerc:
# .coveragerc file content
[run]
omit = [
.venv/*
tests/*
]
Example for pyproject.toml:
# pyproject.toml file content
[tool.coverage.run]
omit = [
"tests/*",
".venv/*",
]
The command:
coverage run --omit /venv/* tests.py
omits coverage from /venv (ie: venv is off of root).
You should instead try a relative directory like:
coverage run --omit venv tests.py
Use the * for /venv/ and it will eliminate all files within your virtual environment.
coverage run tests.py && coverage report --omit=*/venv/*
Related
Using the following setup, the calculated coverage is less than if I use a single thread without parallelization. Coverage creates only 1 coverage file in the project root directory, which I expect is where the problem lies.
I cannot identify what I am doing wrong, the reported coverage is less than if I simply run coverage -m pytest (on a single thread). The tests themselves run in parallel just fine.
Can anyone identify my mistake? I wonder if an environment variable is missing. I run the command from the project root, which contains .coveragerc and sitecustomize.py.
coverage erase && COVERAGE_PROCESS_START=./.coveragerc coverage run --concurrency=multiprocessing --parallel-mode -m pytest -n 8 && coverage combine && coverage report
sitecustomize.py
import coverage
coverage.process_startup()
.coveragerc
[run]
include =
lettergun/*
omit =
*migrations*
*tests*
*.html
plugins = django_coverage_plugin
parallel = True
concurrency = multiprocessing
branch = True
pytest.ini
[pytest]
addopts = --ds=config.settings.test --reuse-db -n 8
python_files = test_*.py
norecursedirs = node_modules
DJANGO_SETTINGS_MODULE = config.settings.test
https://github.com/nedbat/coveragepy/issues/1341 provides some context (and possible solution to this), specially this comment: https://github.com/nedbat/coveragepy/issues/1341#issuecomment-1302863172
I tried to add the coverage-enable-subprocess package but didn't get it to work.
I switched to use pytest-cov and got coverage report when using pytest-xdist)
When I run the coverage.py commands, the only results in the report are from the test directory. Obviously, the value of the report is from code coverage in the source directory. How do I view that result?
I'm using:
coverage run -m unittest test.tests1.sometestfile1
and all the imports/functions from the source directory execute and pass, but the report looks like this:
$ coverage report
Name Stmts Miss Cover
-----------------------------------------------
test/__init__.py 1 0 100%
test/../sometestfile1.py 116 69 41%
test/../sometestfile2.py 116 69 41%
test/../sometestfile3.py 116 69 41%
...
-----------------------------------------------
TOTAL 373 137 63%
I've experimented with adding the source dir to coverage's --source and --include options, but it didn't resolve the issue.
How can I view the coverage from the actual source files?
The solution is to use the --source= option for pytest.
you have:
coverage run -m unittest test.tests1.sometestfile1
which is showing you only the test files coverage.
You need to ensure that the files from your test/Lib/site-packages are getting coverage. (I'm assuming you're making a package for a user to import and maybe for PyPi to hold).
These are being created by tox in the .tox/test subdirectory.
This is why coverage run -m --source=. unittest test.tests1.sometestfile1 does not work.
You need to use the lib name.
coverage run -m --source=<module> unittest test.tests1.sometestfile1
E.g. for my project pyweaving which uses this directory structure:
pyweaving/
tests/
docs/
src/
pyweaving/
__init__.py
foo.py
data/
generators/
__init__.py
bar.py
setup.py (just the stub code)
setup.cfg
tox.ini
my coverage command line looks like this:
coverage run -m --source=pyweaving pytest --basetemp="{envtmpdir}" {posargs}
I have my tests at src/com/xyz/tests/bla_tests/file1.py. I also have file2.py at that path.
I have the following code in file1.py
class HelloWorld():
#pytest.mark.regressiontest
def test_001_bla?():
# Some code here.
From command line I invoke the test by :
pytest -v -m regressiontest but no items are collected. It says : collected 0 items.
My aim is to mark multiple tests across various files as regressiontest and be able to run them from the Command line. (On Mac).
The following command is able to collect the tests though
pytest -q file1.py -v -m regressiontest
This says: collected 20 items / 11 deselected / 9 selected
Can someone help me out?
Got the resolution here:
pytest by default doesn't look for tests in files not named like
test_*.py
also using --basetemp=. actually does a rm -rf . so that's def not a
good idea, we should warn about that
In my case, my file name was not starting from test_*
When you invoke the pytest -v -m regressiontest , are you on your current folder tests ?
I believe you should do pytest src/com/xyz/tests/bla_tests/ -v -m regressiontest. Hope this helps.
Pytest + coverage are showing very strange coverage statistics.
They are counting only those modules where tests were added, but other Python modules are not calculated for some reason.
I have a simple Python Microservice with a structure similar to:
README.rst
Dockerfile
manage.py
api_service/
setup.py
requirements.txt
tests/
Where api_service contains all the logic, and tests contains unit tests.
API is written in Python 3.X
Unit tests - Pytest 3.10.0
I'm running these commands to get a code coverage statistics:
python coverage run pytest -v --junit-xml=junit-report.xml tests/
python coverage xml --fail-under 80
python coverage report
It shows really strange and unexpected results for me.
e.g. there are empty init.py modules in the final report (with 100% coverage) and they affects the final coverage percentage.
Also, it adds a lot of modules with just abstract classes, etc.
But what is really not expected at all - it's not counting Python modules without tests. It's awful!
Are there any commands, flags etc. to handle this situation is a proper way?
I've tried also to run something like:
python coverage run --source=service_api -v --junit-xml=junit-report.xml tests/
But it also returns not expected results.
CD into prj directory and run:
pytest --cov=. tests/ --cov-report xml
in order to get the code coverage for your source files in xml format.
prereq:
pip install pytest pytest-cov
I'm have the following script section in my .travis.yml file:
script:
# run all tests in mymodule/tests and check coverage of the mymodule dir
# don't report on coverage of files in the mymodule/tests dir itself
- coverage run -m --source mymodule --omit mymodule/tests/* py.test mymodule/tests -v
This works fine on my own (Windows) machine, but throws an error on both Linux and OSX on the Travis build. The error is:
Import by filename is not supported.
With the flags in a different order I see a different error (only on the Linux build - the OSX tests pass with this order of the flags):
-coverage run --source eppy --omit eppy/tests/* -m py.test eppy/tests -v
Can't find '__main__' module in 'mymodule/tests/geometry_tests'
What am I doing wrong here?
Solved by changing from using coverage directly to using pytest-cov.
script:
# run all tests in mymodule/tests and check coverage of the mymodule dir
- py.test -v --cov-config .coveragerc --cov=mymodule mymodule/tests
And the .coveragerc file:
# .coveragerc to control coverage.py
[run]
# don't report on coverage of files in the tests dir itself
omit =
mymodule/tests/*
I don't know why this works where the previous approach didn't, but this at least solves the problem.