I use pytest for testing my Python project. What I want to do is to add to my test suite a function to check whether my code is formatted in "Black" or not. When I press the command "pytest --black" my whole project is tested as I want to. How can I add this function in my tests suite and check the same thing when I run the "python setup.py pytest" command?
pytest.ini has an addopts key which does literally what it says on the tin: it adds options to the pytest command line you used.
So at the root of your program, you should be able to just create a pytest.ini file containing:
[pytest]
addopts = --black
and it should automatically enable black-as-test on your pytest runs.
Related
pytest is issuing a warning due to an unknown custom mark when running my test suite despite (hopefully) registering it correctly as per the pytest documentation (see https://docs.pytest.org/en/stable/mark.html#registering-marks).
My Python project's structure is (simplified for the sake of this query):
my_project/
src/
tests/
integration/
unit/
conftest.py
pytest.ini
My pytest.ini is (again, simplified):
# pytest.ini
[pytest]
markers =
incremental: marks related sequential tests to xfail after an earlier failure
because my conftest.py contains the #pytest.mark.incremental recipe outlined in the pytest documentation (see https://docs.pytest.org/en/latest/example/simple.html#incremental-testing-test-steps).
When I run pytest from the command line from within the root directory of my project (i.e. /my_project/ $ pytest), pytest issues the following warning:
PytestUnknownMarkWarning: Unknown pytest.mark.incremental - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/mark.html
#pytest.mark.incremental
-- Docs: https://docs.pytest.org/en/stable/warnings.html
However, if I tell pytest to run the tests directory (i.e. /my_project/ $ pytest tests) I get no such warning. Similarly if I move my pytest.ini file from within the tests directory back to the project root directory then again I don't get the error.
Is there a way to configure pytest so that I can keep my structure as tests/pytest.ini to avoid over-cluttering the project root directory, while still ensuring that my pytest.ini is read correctly by pytest when I just run /my_project/ $ pytest from the command line?
Extra info incase useful:
(my-project-env) /Users/me/Documents/my_project/ $ pytest
============================================= test session starts =============================================
platform darwin -- Python 3.8.5, pytest-6.0.2, py-1.9.0, pluggy-0.13.1
rootdir: /Users/me/Documents/my_project
I know this is an old question, but may I suggest that you run pytest from inside the folder where you put your pytest.ini file?
cd tests
pytest
I think that may solve your problem.
I've got a test suite set up and have been using pytest and pytest-django.
To give some background: I am trying to do some integration testing with a headless browser and have pytest ignore certain tests that are used where I am using a standard (non-headless) browser; I'd like to deselect those tests using a real, visual browser so that they don't trigger in my CI/CD pipeline.
Given the example pytest.ini below, if I run pytest launcher it shows as I would expect that there is 1 test being deselected (the StandardBrowserTestCases class only has 1 test in it).
However, if I run pytest other_app (which also has a StandardBrowserTestCases class) it does not show anything being deselected and StandardBrowserTestCases is ran with the other tests and not deselected.
[pytest]
addopts =
--nomigrations
--cov-config=.coveragerc
--cov=my_project
--cov=other_app
--cov=launcher
--cov=people
; # Ignore the StandardBrowserTestCases - these are only used for local
; # development / visual debug and contain no real valuable tests
--deselect=other_app/tests/test_integrations.py::StandardBrowserTestCases
--deselect=launcher/tests/test_integrations.py::StandardBrowserTestCases
--deselect=people/tests/test_integrations.py::StandardBrowserTestCases
--junitxml=./test-results/junit.xml
--cov-report html:./test-results/htmlcov
--html=./test-results/test_results.html
--self-contained-html
DJANGO_SETTINGS_MODULE = my_project.unit-test-settings
python_files = tests.py test_*.py *_tests.py
Am I using --deselect right? How can I figure out (troubleshoot) to see why it works on one app (launcher) but not the other (other_app)? How can I get pytest to deselect those tests from each app instead of just one?
I'm using the pytest-flake8 plugin to lint my Python code.
Everytime I run the linting like this:
pytest --flake8
In addition to the linting, all tests are run.
But I would like to run the linter checks only.
How can I configure pytest so it only lints the code but skips all my tests, preferrably via commandline (or conftest.py) - without having to add skip markers to my tests?
flake8 tests are marked with the flake8 marker, so you can select only those by running:
pytest --flake8 -m flake8
Pytests --ignore <<path>> option also works well here if all of your tests are in one directory.
I usually hide this behind a make command. In this case my Makefile and tests directory are both at the root of the repository.
.PHONY: lint
lint:
pytest --flake8 --ignore tests
I had the same problem and after some digging, I realized that I just want to run flake8:
flake8 <path to folder>
That's it. No need to run anything else as your flake8 configuration is independent of PyTest.
You can alter the test run logic yourself, for example by ignoring collected tests when --flake8 arg passed:
# conftest.py
def pytest_collection_modifyitems(session, config, items):
if config.getoption('--flake8'):
items[:] = [item for item in items if item.get_closest_marker('flake8')]
Now only flake8 tests will be executed, the rest will be just ignored.
After some more thoughts, this is the solution I came up with - and it works with pytest 5.3.5 (get_marker from https://stackoverflow.com/a/52891274/319905 does not exist anymore).
It allows me to run specific linting checks via commandline.
Since I still like to keep the option of running both linting checks and tests, I added a flag telling pytest if it should only do linting.
Usage:
# Run only flake8 and mypy, no tests
pytest --lint-only --flake8 --mypy
# Run tests and flake8
pytest --flake8
Code:
# conftest.py
def pytest_addoption(parser):
parser.addoption(
"--lint-only",
action="store_true",
default=False,
help="Only run linting checks",
)
def pytest_collection_modifyitems(session, config, items):
if config.getoption("--lint-only"):
lint_items = []
for linter in ["flake8", "black", "mypy"]:
if config.getoption(f"--{linter}"):
lint_items.extend(
[item for item in items if item.get_closest_marker(linter)]
)
items[:] = lint_items
I have a directory with a unittest-module and a logs-directory where debug-informations are written.
since I use the -f / --looponfail option, I need changes in the logs-directory to be ignored.
tests$ ls
logs __pycache__ pytest.ini test_basic.py
tests$ cat pytest.ini
[pytest]
norecursedirs = logs
python_files = test_*.py
tests$ py.test -h
(...)
[pytest] ini-options in the next pytest.ini|tox.ini|setup.cfg file:
markers (linelist) markers for test functions
norecursedirs (args) directory patterns to avoid for recursion
usefixtures (args) list of default fixtures to be used with this project
python_files (args) glob-style file patterns for Python test module discovery
python_classes (args) prefixes for Python test class discovery
python_functions (args) prefixes for Python test function and method discovery
addopts (args) extra command line options
minversion (string) minimally required pytest version
rsyncdirs (pathlist) list of (relative) paths to be rsynced for remote distributed testing.
rsyncignore (pathlist) list of (relative) glob-style paths to be ignored for rsyncing.
looponfailroots (pathlist) directories to check for changes
Eventually the failing test is invoked in a loop, because changes in logs/test_basic.log are monitored and I wonder why this is so. The documentation seems clear to me.
looponfailroots now solves this problem. Put this in your setup.cfg:
[tool:pytest]
looponfailroots=mylib
pytest-xdist has no option that influences what files are ignored on --looponfail. The norecursedirs only influences what files are collected as tests.
As #hpk42 points out the norecursedirs only affects test-collection, not filesystem-monitoring.
So, a possibility to deal with it is to move the logs out of the test-directory.
We have recently moved away from our custom test runner / discovery tool in favor of py.test. For proper unit test reporting when running under teamcity there exists a pytest plugin: https://github.com/JetBrains/teamcity-python
When installed with:
python setup.py install
The plugin is discovered correctly by pytest. However, we don't want to install pytest & this plugin on our build machines. Instead we would rather have them packaged as part of our projects "tools" directory.
How do install / configure py.test to "discover" this plugin. We have tried adding pytest_plugins = "teamcity" with several variations to the setup.cfg file of pytest with no success.
There is no "pytest_plugins" configuration variable (see output at the end of "py.test -h). However, the env var "PYTEST_PLUGINS" contains comma-separated python import paths. So if you plugin is in "teamcity/util/myplugin.py" you would set it like this:
export PYTEST_PLUGINS=teamcity.util.myplugin
You can also use a command line option "-p teamcity.util.myplugin" to achieve a similar effect. And then add it to your setup.cfg's section:
[pytest]
addopts = -p teamcity.util.myplugin