I have a Python Django project on GitHub and I'm using CodeCov with this project.
I have two apps in this Django project, a general app, and a general_api app. For some reason, all of the changes made in the general_api app files are being ignored.
I had YAML settings like this to ignore my test cases:
codecov:
require_ci_to_pass: false
ignore:
- (?s:test_[^\/]+\.py.*)\Z
- (?s:tests_[^\/]+\.py.*)\Z
- ^test.py.*
- ^tests.py.*
However, I've removed them with the same issue.
Is there some other way to ignore, or set the ignore parameters in CodeCov other than the YAML settings?
The root of my problems was actually in how I was using coverage to generate my coverage report.
I was previously using the command line: coverage run -m pytest
This only creates a coverage report on files that have test cases against them or files that are interacted with while testing. Files that have no interaction or test cases would be completely omitted.
I found through the coverage documentation, that I needed to add --source=. to my command line if I wanted the coverage report to include untested files. This is now showing all files in my root source.
The final command was: coverage run --source=. -m pytest
I'm trying to use a fixture from a subdirectory that is defined in the parent directory's conftest.py however, I'm just constantly getting a "fixture [name] not found" error when doing so. Example of the directory structure:
Tests/
conftest.py
test_1.py
test_subdirectory_1/
test_subdirectory_2/
conftest.py
test_2.py
I'd like to use a fixture that has a session scope in the original conftest.py (in the Tests/ directory) in the test_2.py file. However whenever I run test_2.py from the test_subdirectory_2 folder, it gives me a fixture not found error.
I'm doing this from the command line:
root#[ip]:/Test/test_subdirectory_1/test_subdirectory/2# pytest test_2.py
and it spits out:
E Fixture not found.
however, if I move over to the root directory and I do:
root#[ip]:/Test# pytest -k "test_name_from_test_2"
it works perfectly fine. I need to be able to run it from its own directory by just specifying the file name.
Pytest documentation says this type of file structure should be acceptable as tests are supposed to be able to look upward in parent directories for additional fixtures. Can anyone tell me why this is happening?
Here's the platform/version im using: platform linux -- Python 3.8.5, pytest-6.1.0, py-1.9.0, pluggy-0.13.1
I want to perform testing on a script that interacts with my Django application, namely the database. Normally, when we want to test something in Django we simply fire up the built in test suite. With this test suite we even get nice command line switches such as overriding the built in settings.py with a different settings file:
python manage.py test myApp --settings='settings_test'
Here is the problem:
1) I want to test said script which is not part of an app, so I know of no way to invoke the test suite using manage.py. Is this possible? I.e.:
python manage.py test /path/myScript.py --settings='settings_test'
I would suggest to use a different test runner.
You can do pip install django-nose and then set the following setting in your test_settings.py
TEST_RUNNER = `django_nose.NoseTestSuiteRunner`
Now you can run the tests with
./manage.py test --settings=yourproject.test_settings.py
and the Nose testrunner will search all subfolders for folders called tests and in those folders it will search for files that end with _tests.py (and in those files it will search for classes that derive from TestCase, as usual).
So your project structure should look something like this:
- Project-Root/
- Your-Non-App-Code/
- __init__.py
- non_app_code.py
- tests/
- __init__.py
- non_app_code_tests.py
For more info on how to install django-nose, check their Github repo: https://github.com/django-nose/django-nose
I can run tests in workflow folder with nosetests:
workflow maks$ nosetests
..........
----------------------------------------------------------------------
Ran 10 tests in 0.093s
OK
my tests live in test folder:
workflow maks$ ls
__pycache__ iterations.py test
data iterationsClass.py testData
env iterationsClass.pyc
But when I move to parent dir:
(py3env)Makss-Mac:workflow maks$ cd ..
It cannot find tests.
(py3env)Makss-Mac:Packages maks$ nosetests
----------------------------------------------------------------------
Ran 0 tests in 0.005s
OK
So how to make nosetest search tests in all subdirectories?
If you make your workflow folder a module by placing __init__.py in it, nose should be able to find your tests.
If you don't want to make your folder a module, put a setup.cfg in the directory you want to run nosetests which automatically passes the tests option to nosetests to specify relative location of tests. Like so:
#setup.cfg contents:
[nosetests]
exe = True
tests = workflow/
You can also pass multiple folders/tests using this same mechanism:
#setup.cfg contents:
[nosetests]
exe = True
tests = workflow/, other/dir/full/of/tests/, direct/file/my_test.py
I'm kind of a rookie with python unit testing, and particularly coverage.py. Is it desirable to have coverage reports include the coverage of your actual test files?
Here's a screenshot of my HTML report as an example.
You can see that the report includes tests/test_credit_card. At first I was trying to omit the tests/ directory from the reports, like so:
coverage html --omit=tests/ -d tests/coverage
I tried several variations of that command but I could not for the life of me get the tests/ excluded. After accepting defeat, I began to wonder if maybe the test files are supposed to be included in the report.
Can anyone shed some light on this?
coverage html --omit="*/test*" -d tests/coverage
Create .coveragerc file in your project root folder, and include the following:
[run]
omit = *tests*
Leaving this here in case if any Django developer needs a .coveragerc for their project.
[run]
source = .
omit = ./venv/*,*tests*,*apps.py,*manage.py,*__init__.py,*migrations*,*asgi*,*wsgi*,*admin.py,*urls.py
[report]
omit = ./venv/*,*tests*,*apps.py,*manage.py,*__init__.py,*migrations*,*asgi*,*wsgi*,*admin.py,*urls.py
Create a file named .coveragerc on your projects root directory, paste the above code and then just run the command:
coverage run manage.py test
In addition, if you want the tests to execute faster run this command instead.
coverage run manage.py test --keepdb --parallel
This will preserve the test DB and will run the tests in parallel.
You can specify the directories you want to exclude by creating a .coveragerc in the project root.
It supports wildcards (which you can use to exclude virtual environment) and comments (very useful for effective tracking).
The below code block shows how omit can be used (taken from the latest documentation) with multiple files and directories.
[run]
omit =
# omit anything in a .local directory anywhere
*/.local/*
# omit everything in /usr
/usr/*
# omit this single file
utils/tirefire.py
In your case, you could have the following in your .coveragerc:
[run]
omit =
# ignore all test cases in tests/
tests/*
For your question on coverage reports, you can think about testing and coverage in the following manner:
When you run pytest or unittest, all the test cases for your source code are executed
When you run coverage, it shows the part of the source code that isn't being used.
When you run pytest with coverage (something like pytest -v --cov), it runs all test cases and shows the part of the source code that isn't being used.
Extra:
You can also specify the location of your HTML report in the configuration file like:
[html]
directory = tests/coverage/html_report/
This is going to create html, js, css, etc. inside tests/coverage/html_report/ everytime you run coverage or pytest -v --cov
You can also explicitly specify which directory has the code you want coverage on instead of saying which things to omit. In a .coveragerc file, if the directory of interest is called demo, this looks like
[run]
source = demo