I have my tests at src/com/xyz/tests/bla_tests/file1.py. I also have file2.py at that path.
I have the following code in file1.py
class HelloWorld():
#pytest.mark.regressiontest
def test_001_bla?():
# Some code here.
From command line I invoke the test by :
pytest -v -m regressiontest but no items are collected. It says : collected 0 items.
My aim is to mark multiple tests across various files as regressiontest and be able to run them from the Command line. (On Mac).
The following command is able to collect the tests though
pytest -q file1.py -v -m regressiontest
This says: collected 20 items / 11 deselected / 9 selected
Can someone help me out?
Got the resolution here:
pytest by default doesn't look for tests in files not named like
test_*.py
also using --basetemp=. actually does a rm -rf . so that's def not a
good idea, we should warn about that
In my case, my file name was not starting from test_*
When you invoke the pytest -v -m regressiontest , are you on your current folder tests ?
I believe you should do pytest src/com/xyz/tests/bla_tests/ -v -m regressiontest. Hope this helps.
Related
https://coverage.readthedocs.io/en/coverage-4.5.1a/source.html#source
My coverage is also including “venv” folder and I would like to exclude it
no matter what I do even with --include or omit nothing works
coverage run --omit /venv/* tests.py
This runs the test but still adds "venv" folder and dependencies and their % coverage
When I do
coverage run --include tests.py
To run only tests - it says
Nothing to do.
It is pretty annoying... can someone please help?
The help text for the --omit option says (documentation)
--omit=PAT1,PAT2,... Omit files whose paths match one of these patterns.
Accepts shell-style wildcards, which must be quoted.
It will not work without quoting the wildcard, as bash will expand the wildcards before handing the argument list to the coverage binary. Use single-quotes to avoid bash wildcard expansion.
To run my tests without getting coverage from any files within venv/*:
$ coverage run --omit 'venv/*' -m unittest tests/*.py && coverage report -m
........
----------------------------------------------------------------------
Ran 8 tests in 0.023s
OK
Name Stmts Miss Cover Missing
-------------------------------------------------------
ruterstop.py 84 8 90% 177, 188, 191-197, 207
tests/test_ruterstop.py 108 0 100%
-------------------------------------------------------
TOTAL 192 8 96%
If you usually use plain python -m unittest to run your tests you can of course omit the test target argument as well.
$ coverage run --omit 'venv/*' -m unittest
$ coverage report -m
For those who don't want to pass --omit each time when executing coverage you can define following in .coveragerc or in pyproject.toml. Example for .coveragerc:
# .coveragerc file content
[run]
omit = [
.venv/*
tests/*
]
Example for pyproject.toml:
# pyproject.toml file content
[tool.coverage.run]
omit = [
"tests/*",
".venv/*",
]
The command:
coverage run --omit /venv/* tests.py
omits coverage from /venv (ie: venv is off of root).
You should instead try a relative directory like:
coverage run --omit venv tests.py
Use the * for /venv/ and it will eliminate all files within your virtual environment.
coverage run tests.py && coverage report --omit=*/venv/*
I made a small project called demo, with a single test in it
import unittest
class Test(unittest.TestCase):
def testName1(self):
self.assertEqual(5+9, 14)
if __name__ == "__main__":
#import sys;sys.argv = ['', 'Test.testName']
unittest.main()
However, from command line
ThinkPad-T520:~/workspacep/demo$ python -m unittest
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
Why doesn't this work? In general, how can I run all unit tests from command line with a single line?
The structure of the directory is
demo
tests
demo_test1.py __init__.py
There are three gotcha's that I know of:
Your tests in your TestCases need to be named test_*
Your test files need to be named: test*.py (by default, you can change it with the -p flag when running the tests). e.g. test_demo1.py
Your tests folder needs to have an __init__.py file in it, or else it won't be considered a valid location to import from.
So, for #1, you need to rename the test to test_name_1. And for #2, there's two options:
A - Restructure your files like this:
demo
tests
__init__.py
test_demo1.py
Then run python -m unittest and it should find the test cases.
B - Just run it like: python -m unittest discover -p *test.py
I fought with the same exact problem a while ago and I solved it by using test discovery command.
python -m unittest discover -s .
You can pass in your test file pattern as well and a whole other options https://docs.python.org/2/library/unittest.html#test-discovery
You need to pass in a list of modules.
For example, if your test file is foo.py, then you can run python -m unittest foo.
For me (running tests from IntelliJ IDEA) I had to remove the class' 'Run configuration'. Earlier on I had wrongly imported the unittest as _pytest.unittest and ran the test class. Of course that didn't work.
I corrected the module import, but the 'run configuration' was still there, causing it to run as a Python script and not as 'Python tests'.
I had a similar issue and figured out that I had to run python3 -m unittest instead as I had forgotten python defaults to Python 2 on my system
I'm looking for a way of running python -m unittest discover, which will discover tests in, say, directories A, B and C. However, directories A, B and C have directories named dependencies inside each of them, in which there are also some tests which, however, I don't want to run.
Is there a way to run my tests satisfying these constraints without having to create a script for this?
I ran into the same problem and was eventually able to find these handy arguments to pass to unittest discover that resolved my issue.
It is documented here: https://docs.python.org/2/library/unittest.html#test-discovery
-s, --start-directory directory
Directory to start discovery (. default)
-p, --pattern pattern
Pattern to match test files (test*.py default)
So I modified my command to be:
python -m unittest discover -s test
since all of the tests I actually want to run are in the one module, test. You could also use the -p to in theory match regex that only hits your tests, ignoring all of the rest it may find.
I've managed to do it this way (In *NIX):
find `pwd` -name '*_test.py' -not -path '*unwanted_path*' \
| xargs python3 -m unittest -v
That is, the tests are discovered by find, which allows for options like path pattern exclusions, then they're passed to the unittest command as argument list.
Note that I had to switch to find pwd, where usually I can write find ., since relative paths in the form ./xxx aren't accepted by unittest (module not found).
It would seem that python -m unittest descends into module directories but not in other directories.
It quickly tried the following structure
temp
+ a
- test_1.py
+ dependencies
- test_a.py
With the result
>python -m unittest discover -s temp\a
test_1
.
----------------------------------------------------------------------
Ran 1 test in 0.002s
OK
However, if the directory is a module directory (containing a file __init__.py) the situation is different.
temp
+ a
- __init__.py
- test_1.py
+ dependencies
- __init__.py
- test_a.py
Here the result was
>python -m unittest discover -s temp\a
test_a
.test_1
.
----------------------------------------------------------------------
Ran 2 tests in 0.009s
OK
The usefulness of this answer for you now depends on whether it is acceptable for your folder dependencies not to be a module directory.
EDIT: After seeing your comment
Would using pytest be an option? This test runner has many command arguments, one specifically to exclude tests.
See Changing standard (Python) test discovery
From their site
Ignore paths during test collection
You can easily ignore certain test directories and modules during collection by passing the --ignore=path option on the cli. pytest allows multiple --ignore options
I am porting a series of tests from nosetests + python unittest to py.test. I was pleasantly surprised to learn that py.test supports python unittests and running existing tests with py.test is just as easy as calling py.test instead of nosetests on the command line. However I am having problems with specifying the working directory for the tests. They are not in the root project dir but in a subdir. Currently the tests are run like this:
$ nosetests -w test-dir/ tests.py
which changes the current working directory to test-dir and run all the tests in tests.py. However when I use py.test
$ py.test test-dir/tests.py
all the tests in tests.py are run but the current working directory is not changed to test-dir. Most tests assume that the working directory is test-dir and try to open and read files from it which obviously fails.
So my question is how to change the current working directory for all tests when using py.test.
The are a lot of tests and I don't want to invest the time to fix them all and make them work regardless of the cwd.
Yes, I can simply do cd test-dir; py.test tests.py but I am used to working from the project root directory and don't want to cd every time I want to run a test.
Here is some code that may give you better idea what I am trying to achieve:
content of tests.py:
import unittest
class MyProjectTestCase(unittest.TestCase):
def test_something(self):
with open('testing-info.txt', 'r') as f:
test something with f
directory layout:
my-project/
test-dir/
tests.py
testing-info.txt
And then when I try to run the tests:
$ pwd
my-project
$ nosetests -w test-dir tests.py
# all is fine
$ py.test ttest-dir/tests.py
# tests fail because they cannot open testing-info.txt
So this is the best I could come up with:
# content of conftest.py
import pytest
import os
def pytest_addoption(parser):
parser.addoption("-W", action="store", default=".",
help="Change current working dir before running the collected tests.")
def pytest_sessionstart(session):
os.chdir(session.config.getoption('W'))
And then when running the tests
$ py.test -W test-dir test-dir/tests.py
It's not clean but it will do the trick until I fix all the tests.
I'm a newbie when it comes to python unit testing, but I'm eager to learn!
I just read python setup.py test can run all suites derived from unittest classes. I wonder if I also can use setup.py to run a single suite and/or a single test case, maybe adding some modifier to the previous command like python setup.py tests suitename. If so, can you please point me to any docs/examples?
You guys are all wrong, setup.py test can be used with the -s option the same way python -m unittest does:
cd root_of_your_package
python setup.py test -s tests.TestClass.test_method
The setup.py test runner is rather limited; it only supports letting you specify a specific module. The documentation for the command-line switches is given when you use the --help switch:
python setup.py test --help
Common commands: (see '--help-commands' for more)
[ ... cut ... ]
Options for 'test' command:
--test-module (-m) Run 'test_suite' in specified module
--test-suite (-s) Test suite to run (e.g. 'some_module.test_suite')
[ ... more cut ... ]
so python setup.py test -m your.package.tests.test_module would limit running the tests from the test_module.py file only.
All the test command does, really, is make sure your egg has been built already, extract the test_suite value from setup() metadata, configure a test loader that understands about zipped eggs, then run the unittest.main() function.
If you need to run a single test only, have already built your egg, are not running this with a zipped egg, then you can also just use the unittest command line interface, which does pretty much everything else:
python -m unittest yourpackage.tests.TestClass.test_method
would instruct unittest to only run a very specific test method.
setup.py test
setup.py test is not that flexible, but here's an alternative:
The unittest module can run specific test methods
From the Documentation on unittest
The unittest module can be used from the command line to run tests from modules, classes or even individual test methods:
python -m unittest test_module1 test_module2
python -m unittest test_module.TestClass
python -m unittest test_module.TestClass.test_method
You can pass in a list with any combination of module names, and fully qualified class or method names.
You can run tests with more detail (higher verbosity) by passing in the -v flag:
python -m unittest -v test_module
For a list of all the command-line options:
python -m unittest -h
in case of pytest run specific test:
python setup.py test --addopts tests/jobs/test_data_monitoring.py::TestDataMonitoring::test_dataframe__bulk_update
In case you use pytest for unit testing you can run a file directly by avoiding the setup.py file.
Running all tests for me is:
python setup.py test
Running a specific test:
pytest --pyargs path/to/test.py
Hope it helps.
(I know this post is already a bit old, but I came here looking for running a way to run pytest tests alone, figured it might be of use to someone)