We Keep Coding

Running unittest discover ignoring specific directory

I'm looking for a way of running python -m unittest discover, which will discover tests in, say, directories A, B and C. However, directories A, B and C have directories named dependencies inside each of them, in which there are also some tests which, however, I don't want to run.
Is there a way to run my tests satisfying these constraints without having to create a script for this?

I ran into the same problem and was eventually able to find these handy arguments to pass to unittest discover that resolved my issue.
It is documented here: https://docs.python.org/2/library/unittest.html#test-discovery
-s, --start-directory directory
Directory to start discovery (. default)
-p, --pattern pattern
Pattern to match test files (test*.py default)
So I modified my command to be:
python -m unittest discover -s test
since all of the tests I actually want to run are in the one module, test. You could also use the -p to in theory match regex that only hits your tests, ignoring all of the rest it may find.

I've managed to do it this way (In *NIX):
find `pwd` -name '*_test.py' -not -path '*unwanted_path*' \
| xargs python3 -m unittest -v
That is, the tests are discovered by find, which allows for options like path pattern exclusions, then they're passed to the unittest command as argument list.
Note that I had to switch to find pwd, where usually I can write find ., since relative paths in the form ./xxx aren't accepted by unittest (module not found).

It would seem that python -m unittest descends into module directories but not in other directories.
It quickly tried the following structure
temp
+ a
- test_1.py
+ dependencies
- test_a.py
With the result
>python -m unittest discover -s temp\a
test_1
.
----------------------------------------------------------------------
Ran 1 test in 0.002s
OK
However, if the directory is a module directory (containing a file __init__.py) the situation is different.
temp
+ a
- __init__.py
- test_1.py
+ dependencies
- __init__.py
- test_a.py
Here the result was
>python -m unittest discover -s temp\a
test_a
.test_1
.
----------------------------------------------------------------------
Ran 2 tests in 0.009s
OK
The usefulness of this answer for you now depends on whether it is acceptable for your folder dependencies not to be a module directory.
EDIT: After seeing your comment
Would using pytest be an option? This test runner has many command arguments, one specifically to exclude tests.
See Changing standard (Python) test discovery
From their site
Ignore paths during test collection
You can easily ignore certain test directories and modules during collection by passing the --ignore=path option on the cli. pytest allows multiple --ignore options

running unittests \ integration tests in python

I have a Django project with multiple apps. Each app has a set of unittests. I'm using pytest as my test runner. We have gotten to a point that we want to start writing integration tests. I was wondering if there is any way to keep the naming convention and thus the auto discovery of pytest but still be able (via flag maybe?) to run the different test types. The most intuitive solution that comes to mind is some sort of decorator on test methods or even TestCase classes (something like Category in JUnit).
something like:
#testtype('unittest')
def test_my_test(self):
# do some testing
#testtype('integration')
def test_my_integration_test(self):
# do some integration testing
and then i could run the test like:
py.test --type=integration
py.test --type=unittest
Is there such a thing?
If not, the only other solution i can think about is to add a django command and "manually" build a testsuite and run it with pytest... I would prefer not to use this option. Is there any other solution that can help me?
Thanks

You can mark test functions.
import pytest
#pytest.mark.unittest
def test_my_test(self):
# do some testing
#pytest.mark.integration
def test_my_integration_test(self):
# do some integration testing
These custom markers must be registered in your pytest.ini file.
Then use the -m flag to run the marked tests
py.test -v -m unittest
Another option would be to split your tests into unittest and integration directories. Then you can run tests in a specific directory with:
py.test -v unittest
or
py.test -v integration

Another way to do this (without any config or code)
pytest -o "python_functions=*_integration_test"
You can also do this in module/class level, e.g.,
python_files = integration_test_*.py
python_classes = IntegrationTest
Ref:
https://docs.pytest.org/en/latest/example/pythoncollection.html#changing-naming-conventions

Is there a way to specify which pytest tests to run from a file?

Is there a way to select pytest tests to run from a file?
For example, a file foo.txt containing a list of tests to be executed:
tests_directory/foo.py::test_001
tests_directory/bar.py::test_some_other_test
Or, is there a way to select multiple tests, having no common pattern in test name, from different directories with pytest?
pytest -k <pattern> allows a single pattern.
One option is to have a pytest.mark against each test, but my requirement is to run different combination of tests from different files.
Is there a way to specify multiple patterns and a test file name for each pattern?
Or
Is there a way to specify the exact test paths in a file and feed that file as an input to pytest?
Or
Is there a hook function that can be utilized for this purpose?

You can use -k option to run test cases with different patterns:
py.test tests_directory/foo.py tests_directory/bar.py -k 'test_001 or test_some_other_test'
This will run test cases with name test_001 and test_some_other_test deselecting the rest of the test cases.
Note: This will select any test case starting with test_001 or test_some_other_test. For example, if you have test case test_0012 it will also be selected.

Specifying tests / selecting tests
Pytest supports several ways to run and select tests from the command-line.
Run tests in a module
pytest test_mod.py
Run tests in a directory
pytest testing/
Run tests by keyword expressions
pytest -k "MyClass and not method"
This will run tests which contain names that match the given string expression, which can include Python operators that use filenames, class names and function names as variables. The example above will run TestMyClass.test_something but not TestMyClass.test_method_simple.
Run tests by node ids
Each collected test is assigned a unique nodeid which consist of the module filename followed by specifiers like class names, function names and parameters from parametrization, separated by :: characters.
To run a specific test within a module:
pytest test_mod.py::test_func
Another example specifying a test method in the command line:
pytest test_mod.py::TestClass::test_method
Run tests by marker expressions
pytest -m slow
Will run all tests which are decorated with the #pytest.mark.slow decorator.
For more information see marks.
Run tests from packages
pytest --pyargs pkg.testing
This will import pkg.testing and use its filesystem location to find and run tests from.
Source: https://docs.pytest.org/en/latest/usage.html#specifying-tests-selecting-tests

My answer provides a ways to run a subset of test in different scenarios.
Run all tests in a project
pytest
Run tests in a Single Directory
To run all the tests from one directory, use the directory as a parameter to
pytest:
pytest tests/my-directory
Run tests in a Single Test File/Module
To run a file full of tests, list the file with the relative path as a parameter to pytest:
pytest tests/my-directory/test_demo.py
Run a Single Test Function
To run a single test function, add :: and the test function name:
pytest -v tests/my-directory/test_demo.py::test_specific_function
-v is used so you can see which function was run.
Run a Single Test Class
To run just a class, do like we did with functions and add ::, then the class name to the file parameter:
pytest -v tests/my-directory/test_demo.py::TestClassName
Run a Single Test Method of a Test Class
If you don't want to run all of a test class, just one method, just add
another :: and the method name:
pytest -v tests/my-directory/test_demo.py::TestClassName::test_specific_method
Run a Set of Tests Based on Test Name
The -k option enables you to pass in an expression to run tests that have
certain names specified by the expression as a substring of the test name.
It is possible to use and, or, and not to create complex expressions.
For example, to run all of the functions that have _raises in their name:
pytest -v -k _raises

Method 1: Randomly selected tests. Long and ugly.
python -m pytest test/stress/test_performance.py::TestPerformance::test_continuous_trigger test/integration/test_config.py::TestConfig::test_valid_config
Method 2: Use Keyword Expressions.
Note: I am searching by testcase names. Same is applicable to TestClass names.
Case 1: The below will run whichever is found. Since we have used 'OR' .
python -m pytest -k 'test_password_valid or test_no_configuration'
Lets say the two above are actually correct, 2 tests will be run.
Case 2: Now an incorrect name and another correct name.
python -m pytest -k 'test_password_validzzzzzz or test_no_configuration'
Only one is found and run.
Case 3: If you want to run either all tests mentioned, or None, then use AND
python -m pytest -k 'test_password_valid and test_no_configuration'
Both will be run if correct or none.
Case 4: Run test only in one folder:
python -m pytest test/project1/integration -k 'test_password_valid or test_no_configuration'
Case 5: Run test from only one file.
python -m pytest test/integration/test_authentication.py -k 'test_password_expiry or test_incorrect_password'
Case 6: Run all tests except the match.
python -m pytest test/integration/test_authentication.py -k 'not test_incorrect_password'

If you have the same method name in two different classes and you just want to run one of them, this works:
pytest tests.py -k "TestClassName and test_method_name"

According to the doc about Run tests by node ids
since you have all node ids in foo.txt, just run
pytest $(tr '\n' ' ' <foo.txt)
this is same with below command (with file content in the question)
pytest tests_directory/foo.py::test_001 tests_directory/bar.py::test_some_other_test

Maybe using pytest_collect_file() hook you can parse the content of a .txt o .yaml file where the tests are specify as you want, and return them to the pytest core.
A nice example is shown in the pytest documentation. I think what you are looking for.

Try:
pytest path/file_test.py::test_name
Using -k as others have suggested will match any test with that substring, not just a specific test.

Say for example you have a file called test.py. In this file you have a class called TestSomeService. This class has 2 functions namely test_service1 and test_service2.
Now to run specific function test you can this
pytest test.py::TestSomeService::test_service1
pytest test.py::TestSomeService::test_service2
To run specific class
pytest test.py::TestSomeService

Here's a possible partial answer, because it only allows selecting the test scripts, not individual tests within those scripts.
And it also limited by my using legacy compatibility mode vs unittest scripts, so not guaranteeing it would work with native pytest.
Here goes:
create a new dictory, say subset_tests_directory.
ln -s tests_directory/foo.py
ln -s tests_directory/bar.py
be careful about imports which implicitly assume files are in test_directory. I had to fix several of those by running python foo.py, from within subset_tests_directory and correcting as needed.
Once the test scripts execute correctly, just cd subset_tests_directory and pytest there. Pytest will only pick up the scripts it sees.
Another possibility is symlinking within your current test directory, say as ln -s foo.py subset_foo.py then pytest subset*.py. That would avoid needing to adjust your imports, but it would clutter things up until you removed the symlinks. Worked for me as well.

Assuming the test names are unique, you have to remove the test file's name:
cut -d : -f 3 foo.txt > FAILED_TESTS.txt
As others pointed out use -k, but you have to pass the file's content (i.e., list of test names) as a single string:
pytest -k "$(awk '$1=$1' RS= OFS=' or ' FAILED_TESTS.txt)"
awk will replace the new lines with a delimiter or so that the test names are joined in a format that pytest expects.

Django unit tests: Can I specify default label/location?

I know that when I run
$ ./manage.py test
then it will discover tests in any file named test*.py under the current working directory.
I also know that I can specify a test label like this:
$ ./manage.py test animals.tests
Then all the tests in the animals.tests module will be run.
Another option is to specify a directory like this:
$ ./manage.py test animals/
And then all the tests in the animals/ directory will be discovered and run.
(The examples above are from the docs.)
I'd like to know if there is a way to specify a default test label to be tested or a directory to be used for tests discovery.
Let's say I'd like to run the tests in animals/ directory by default (i.e. if not specified otherwise). Then the following two commands would be equal:
$ ./manage.py test
$ ./manage.py test animals/
Although I would still like to be able to run another tests like this:
$ ./manage.py test people/
Thank you.
P.S. I know that I could redefine the test management command but I'd rather avoid doing this. I'm looking for a cleaner solution - such as a configuration option etc.

Does unittest allow single case/suite testing through "setup.py test"?

I'm a newbie when it comes to python unit testing, but I'm eager to learn!
I just read python setup.py test can run all suites derived from unittest classes. I wonder if I also can use setup.py to run a single suite and/or a single test case, maybe adding some modifier to the previous command like python setup.py tests suitename. If so, can you please point me to any docs/examples?

You guys are all wrong, setup.py test can be used with the -s option the same way python -m unittest does:
cd root_of_your_package
python setup.py test -s tests.TestClass.test_method

The setup.py test runner is rather limited; it only supports letting you specify a specific module. The documentation for the command-line switches is given when you use the --help switch:
python setup.py test --help
Common commands: (see '--help-commands' for more)
[ ... cut ... ]
Options for 'test' command:
--test-module (-m) Run 'test_suite' in specified module
--test-suite (-s) Test suite to run (e.g. 'some_module.test_suite')
[ ... more cut ... ]
so python setup.py test -m your.package.tests.test_module would limit running the tests from the test_module.py file only.
All the test command does, really, is make sure your egg has been built already, extract the test_suite value from setup() metadata, configure a test loader that understands about zipped eggs, then run the unittest.main() function.
If you need to run a single test only, have already built your egg, are not running this with a zipped egg, then you can also just use the unittest command line interface, which does pretty much everything else:
python -m unittest yourpackage.tests.TestClass.test_method
would instruct unittest to only run a very specific test method.

setup.py test
setup.py test is not that flexible, but here's an alternative:
The unittest module can run specific test methods
From the Documentation on unittest
The unittest module can be used from the command line to run tests from modules, classes or even individual test methods:
python -m unittest test_module1 test_module2
python -m unittest test_module.TestClass
python -m unittest test_module.TestClass.test_method
You can pass in a list with any combination of module names, and fully qualified class or method names.
You can run tests with more detail (higher verbosity) by passing in the -v flag:
python -m unittest -v test_module
For a list of all the command-line options:
python -m unittest -h

in case of pytest run specific test:
python setup.py test --addopts tests/jobs/test_data_monitoring.py::TestDataMonitoring::test_dataframe__bulk_update

In case you use pytest for unit testing you can run a file directly by avoiding the setup.py file.
Running all tests for me is:
python setup.py test
Running a specific test:
pytest --pyargs path/to/test.py
Hope it helps.
(I know this post is already a bit old, but I came here looking for running a way to run pytest tests alone, figured it might be of use to someone)