Pytest Not Deselecting Certain Tests (but should be in pytest.ini) - python

I've got a test suite set up and have been using pytest and pytest-django.
To give some background: I am trying to do some integration testing with a headless browser and have pytest ignore certain tests that are used where I am using a standard (non-headless) browser; I'd like to deselect those tests using a real, visual browser so that they don't trigger in my CI/CD pipeline.
Given the example pytest.ini below, if I run pytest launcher it shows as I would expect that there is 1 test being deselected (the StandardBrowserTestCases class only has 1 test in it).
However, if I run pytest other_app (which also has a StandardBrowserTestCases class) it does not show anything being deselected and StandardBrowserTestCases is ran with the other tests and not deselected.
[pytest]
addopts =
--nomigrations
--cov-config=.coveragerc
--cov=my_project
--cov=other_app
--cov=launcher
--cov=people
; # Ignore the StandardBrowserTestCases - these are only used for local
; # development / visual debug and contain no real valuable tests
--deselect=other_app/tests/test_integrations.py::StandardBrowserTestCases
--deselect=launcher/tests/test_integrations.py::StandardBrowserTestCases
--deselect=people/tests/test_integrations.py::StandardBrowserTestCases
--junitxml=./test-results/junit.xml
--cov-report html:./test-results/htmlcov
--html=./test-results/test_results.html
--self-contained-html
DJANGO_SETTINGS_MODULE = my_project.unit-test-settings
python_files = tests.py test_*.py *_tests.py
Am I using --deselect right? How can I figure out (troubleshoot) to see why it works on one app (launcher) but not the other (other_app)? How can I get pytest to deselect those tests from each app instead of just one?

Related

Python add pytest --black to test suite

I use pytest for testing my Python project. What I want to do is to add to my test suite a function to check whether my code is formatted in "Black" or not. When I press the command "pytest --black" my whole project is tested as I want to. How can I add this function in my tests suite and check the same thing when I run the "python setup.py pytest" command?
pytest.ini has an addopts key which does literally what it says on the tin: it adds options to the pytest command line you used.
So at the root of your program, you should be able to just create a pytest.ini file containing:
[pytest]
addopts = --black
and it should automatically enable black-as-test on your pytest runs.

How to disable Django's Test Discovery?

This is an unusual situation - most Django users want Django's test runner to find all of their tests.
I'm working on a python library with several test suites that are run with different configurations, so I don't want the discovery to find and run tests from the wrong configuration. How do I disable discovery entirely and rely on the pre-1.6 behavior of only running the tests for apps explicitly declared in INSTALLED_APPS?
My library structure:
library/ # django app used by others
tests/ # custom test suites here
core/ # tests of core functionality
custom/ # tests of a custom feature requiring separate config
contrib/ # tests for assorted contrib features, also requiring separate config
manage_core.py # separate manage.py files for each "project"
manage_custom.py # these specify settings file to use.
manage_contrib.py
settings.py # base settings for all tests
settings_core.py # settings for 'core' tests including unique INSTALLED_APPS
settings_custom.py # settings for 'custom' tests; different INSTALLED_APPS
settings_contrib.py # settings for 'contrib' tests; different INSTALLED_APPS
The problem is that this command, which should only run tests for the 'contrib' test suite, is also finding and running tests for 'core':
./manage_contrib.py test contrib.tests
It's missing from the Django docs, but the command-line has an option, found via ./manage.py help test:
-t TOP_LEVEL, --top-level-directory TOP_LEVEL
Top level of project for unittest discovery.
Confusingly, specifying the module to test doesn't appear to prevent test discovery, but specifying a sub-directory does, like this:
./manage_contrib.py test contrib.tests -t ./contrib/
That appears to prevent the discovery of tests located outside of contrib.
Hmm I'm unfortunately not aware of a settings parameter that might let you tell unittest to only run from individual apps (a-la "settings.TEST_DIRECTORIES=settings.INSTALLED_APPS") but if you're able to give your tests a unique naming convention, you could use the --pattern= option when running the test suite.
For example, if you have
/myapp/tests/test_a_models.py
/myapp/tests/test_b_models.py
You could only run a with ./manage.py test --pattern='*_a_*' and then run b with ./manage.py test --pattern='*_b_*'
Definitely not ideal, but might get the job done depending on how much flexibility you have with the test naming conventions in your own app.

running unittests \ integration tests in python

I have a Django project with multiple apps. Each app has a set of unittests. I'm using pytest as my test runner. We have gotten to a point that we want to start writing integration tests. I was wondering if there is any way to keep the naming convention and thus the auto discovery of pytest but still be able (via flag maybe?) to run the different test types. The most intuitive solution that comes to mind is some sort of decorator on test methods or even TestCase classes (something like Category in JUnit).
something like:
#testtype('unittest')
def test_my_test(self):
# do some testing
#testtype('integration')
def test_my_integration_test(self):
# do some integration testing
and then i could run the test like:
py.test --type=integration
py.test --type=unittest
Is there such a thing?
If not, the only other solution i can think about is to add a django command and "manually" build a testsuite and run it with pytest... I would prefer not to use this option. Is there any other solution that can help me?
Thanks
You can mark test functions.
import pytest
#pytest.mark.unittest
def test_my_test(self):
# do some testing
#pytest.mark.integration
def test_my_integration_test(self):
# do some integration testing
These custom markers must be registered in your pytest.ini file.
Then use the -m flag to run the marked tests
py.test -v -m unittest
Another option would be to split your tests into unittest and integration directories. Then you can run tests in a specific directory with:
py.test -v unittest
or
py.test -v integration
Another way to do this (without any config or code)
pytest -o "python_functions=*_integration_test"
You can also do this in module/class level, e.g.,
python_files = integration_test_*.py
python_classes = IntegrationTest
Ref:
https://docs.pytest.org/en/latest/example/pythoncollection.html#changing-naming-conventions

Django testing external script

I want to perform testing on a script that interacts with my Django application, namely the database. Normally, when we want to test something in Django we simply fire up the built in test suite. With this test suite we even get nice command line switches such as overriding the built in settings.py with a different settings file:
python manage.py test myApp --settings='settings_test'
Here is the problem:
1) I want to test said script which is not part of an app, so I know of no way to invoke the test suite using manage.py. Is this possible? I.e.:
python manage.py test /path/myScript.py --settings='settings_test'
I would suggest to use a different test runner.
You can do pip install django-nose and then set the following setting in your test_settings.py
TEST_RUNNER = `django_nose.NoseTestSuiteRunner`
Now you can run the tests with
./manage.py test --settings=yourproject.test_settings.py
and the Nose testrunner will search all subfolders for folders called tests and in those folders it will search for files that end with _tests.py (and in those files it will search for classes that derive from TestCase, as usual).
So your project structure should look something like this:
- Project-Root/
- Your-Non-App-Code/
- __init__.py
- non_app_code.py
- tests/
- __init__.py
- non_app_code_tests.py
For more info on how to install django-nose, check their Github repo: https://github.com/django-nose/django-nose

Teamcity pytest plugin & unit test reporting

We have recently moved away from our custom test runner / discovery tool in favor of py.test. For proper unit test reporting when running under teamcity there exists a pytest plugin: https://github.com/JetBrains/teamcity-python
When installed with:
python setup.py install
The plugin is discovered correctly by pytest. However, we don't want to install pytest & this plugin on our build machines. Instead we would rather have them packaged as part of our projects "tools" directory.
How do install / configure py.test to "discover" this plugin. We have tried adding pytest_plugins = "teamcity" with several variations to the setup.cfg file of pytest with no success.
There is no "pytest_plugins" configuration variable (see output at the end of "py.test -h). However, the env var "PYTEST_PLUGINS" contains comma-separated python import paths. So if you plugin is in "teamcity/util/myplugin.py" you would set it like this:
export PYTEST_PLUGINS=teamcity.util.myplugin
You can also use a command line option "-p teamcity.util.myplugin" to achieve a similar effect. And then add it to your setup.cfg's section:
[pytest]
addopts = -p teamcity.util.myplugin

Categories