We Keep Coding

Running unittest discover ignoring specific directory

I'm looking for a way of running python -m unittest discover, which will discover tests in, say, directories A, B and C. However, directories A, B and C have directories named dependencies inside each of them, in which there are also some tests which, however, I don't want to run.
Is there a way to run my tests satisfying these constraints without having to create a script for this?

I ran into the same problem and was eventually able to find these handy arguments to pass to unittest discover that resolved my issue.
It is documented here: https://docs.python.org/2/library/unittest.html#test-discovery
-s, --start-directory directory
Directory to start discovery (. default)
-p, --pattern pattern
Pattern to match test files (test*.py default)
So I modified my command to be:
python -m unittest discover -s test
since all of the tests I actually want to run are in the one module, test. You could also use the -p to in theory match regex that only hits your tests, ignoring all of the rest it may find.

I've managed to do it this way (In *NIX):
find `pwd` -name '*_test.py' -not -path '*unwanted_path*' \
| xargs python3 -m unittest -v
That is, the tests are discovered by find, which allows for options like path pattern exclusions, then they're passed to the unittest command as argument list.
Note that I had to switch to find pwd, where usually I can write find ., since relative paths in the form ./xxx aren't accepted by unittest (module not found).

It would seem that python -m unittest descends into module directories but not in other directories.
It quickly tried the following structure
temp
+ a
- test_1.py
+ dependencies
- test_a.py
With the result
>python -m unittest discover -s temp\a
test_1
.
----------------------------------------------------------------------
Ran 1 test in 0.002s
OK
However, if the directory is a module directory (containing a file __init__.py) the situation is different.
temp
+ a
- __init__.py
- test_1.py
+ dependencies
- __init__.py
- test_a.py
Here the result was
>python -m unittest discover -s temp\a
test_a
.test_1
.
----------------------------------------------------------------------
Ran 2 tests in 0.009s
OK
The usefulness of this answer for you now depends on whether it is acceptable for your folder dependencies not to be a module directory.
EDIT: After seeing your comment
Would using pytest be an option? This test runner has many command arguments, one specifically to exclude tests.
See Changing standard (Python) test discovery
From their site
Ignore paths during test collection
You can easily ignore certain test directories and modules during collection by passing the --ignore=path option on the cli. pytest allows multiple --ignore options

Is there a way to specify which pytest tests to run from a file?

Is there a way to select pytest tests to run from a file?
For example, a file foo.txt containing a list of tests to be executed:
tests_directory/foo.py::test_001
tests_directory/bar.py::test_some_other_test
Or, is there a way to select multiple tests, having no common pattern in test name, from different directories with pytest?
pytest -k <pattern> allows a single pattern.
One option is to have a pytest.mark against each test, but my requirement is to run different combination of tests from different files.
Is there a way to specify multiple patterns and a test file name for each pattern?
Or
Is there a way to specify the exact test paths in a file and feed that file as an input to pytest?
Or
Is there a hook function that can be utilized for this purpose?

You can use -k option to run test cases with different patterns:
py.test tests_directory/foo.py tests_directory/bar.py -k 'test_001 or test_some_other_test'
This will run test cases with name test_001 and test_some_other_test deselecting the rest of the test cases.
Note: This will select any test case starting with test_001 or test_some_other_test. For example, if you have test case test_0012 it will also be selected.

Specifying tests / selecting tests
Pytest supports several ways to run and select tests from the command-line.
Run tests in a module
pytest test_mod.py
Run tests in a directory
pytest testing/
Run tests by keyword expressions
pytest -k "MyClass and not method"
This will run tests which contain names that match the given string expression, which can include Python operators that use filenames, class names and function names as variables. The example above will run TestMyClass.test_something but not TestMyClass.test_method_simple.
Run tests by node ids
Each collected test is assigned a unique nodeid which consist of the module filename followed by specifiers like class names, function names and parameters from parametrization, separated by :: characters.
To run a specific test within a module:
pytest test_mod.py::test_func
Another example specifying a test method in the command line:
pytest test_mod.py::TestClass::test_method
Run tests by marker expressions
pytest -m slow
Will run all tests which are decorated with the #pytest.mark.slow decorator.
For more information see marks.
Run tests from packages
pytest --pyargs pkg.testing
This will import pkg.testing and use its filesystem location to find and run tests from.
Source: https://docs.pytest.org/en/latest/usage.html#specifying-tests-selecting-tests

My answer provides a ways to run a subset of test in different scenarios.
Run all tests in a project
pytest
Run tests in a Single Directory
To run all the tests from one directory, use the directory as a parameter to
pytest:
pytest tests/my-directory
Run tests in a Single Test File/Module
To run a file full of tests, list the file with the relative path as a parameter to pytest:
pytest tests/my-directory/test_demo.py
Run a Single Test Function
To run a single test function, add :: and the test function name:
pytest -v tests/my-directory/test_demo.py::test_specific_function
-v is used so you can see which function was run.
Run a Single Test Class
To run just a class, do like we did with functions and add ::, then the class name to the file parameter:
pytest -v tests/my-directory/test_demo.py::TestClassName
Run a Single Test Method of a Test Class
If you don't want to run all of a test class, just one method, just add
another :: and the method name:
pytest -v tests/my-directory/test_demo.py::TestClassName::test_specific_method
Run a Set of Tests Based on Test Name
The -k option enables you to pass in an expression to run tests that have
certain names specified by the expression as a substring of the test name.
It is possible to use and, or, and not to create complex expressions.
For example, to run all of the functions that have _raises in their name:
pytest -v -k _raises

Method 1: Randomly selected tests. Long and ugly.
python -m pytest test/stress/test_performance.py::TestPerformance::test_continuous_trigger test/integration/test_config.py::TestConfig::test_valid_config
Method 2: Use Keyword Expressions.
Note: I am searching by testcase names. Same is applicable to TestClass names.
Case 1: The below will run whichever is found. Since we have used 'OR' .
python -m pytest -k 'test_password_valid or test_no_configuration'
Lets say the two above are actually correct, 2 tests will be run.
Case 2: Now an incorrect name and another correct name.
python -m pytest -k 'test_password_validzzzzzz or test_no_configuration'
Only one is found and run.
Case 3: If you want to run either all tests mentioned, or None, then use AND
python -m pytest -k 'test_password_valid and test_no_configuration'
Both will be run if correct or none.
Case 4: Run test only in one folder:
python -m pytest test/project1/integration -k 'test_password_valid or test_no_configuration'
Case 5: Run test from only one file.
python -m pytest test/integration/test_authentication.py -k 'test_password_expiry or test_incorrect_password'
Case 6: Run all tests except the match.
python -m pytest test/integration/test_authentication.py -k 'not test_incorrect_password'

If you have the same method name in two different classes and you just want to run one of them, this works:
pytest tests.py -k "TestClassName and test_method_name"

According to the doc about Run tests by node ids
since you have all node ids in foo.txt, just run
pytest $(tr '\n' ' ' <foo.txt)
this is same with below command (with file content in the question)
pytest tests_directory/foo.py::test_001 tests_directory/bar.py::test_some_other_test

Maybe using pytest_collect_file() hook you can parse the content of a .txt o .yaml file where the tests are specify as you want, and return them to the pytest core.
A nice example is shown in the pytest documentation. I think what you are looking for.

Try:
pytest path/file_test.py::test_name
Using -k as others have suggested will match any test with that substring, not just a specific test.

Say for example you have a file called test.py. In this file you have a class called TestSomeService. This class has 2 functions namely test_service1 and test_service2.
Now to run specific function test you can this
pytest test.py::TestSomeService::test_service1
pytest test.py::TestSomeService::test_service2
To run specific class
pytest test.py::TestSomeService

Here's a possible partial answer, because it only allows selecting the test scripts, not individual tests within those scripts.
And it also limited by my using legacy compatibility mode vs unittest scripts, so not guaranteeing it would work with native pytest.
Here goes:
create a new dictory, say subset_tests_directory.
ln -s tests_directory/foo.py
ln -s tests_directory/bar.py
be careful about imports which implicitly assume files are in test_directory. I had to fix several of those by running python foo.py, from within subset_tests_directory and correcting as needed.
Once the test scripts execute correctly, just cd subset_tests_directory and pytest there. Pytest will only pick up the scripts it sees.
Another possibility is symlinking within your current test directory, say as ln -s foo.py subset_foo.py then pytest subset*.py. That would avoid needing to adjust your imports, but it would clutter things up until you removed the symlinks. Worked for me as well.

Assuming the test names are unique, you have to remove the test file's name:
cut -d : -f 3 foo.txt > FAILED_TESTS.txt
As others pointed out use -k, but you have to pass the file's content (i.e., list of test names) as a single string:
pytest -k "$(awk '$1=$1' RS= OFS=' or ' FAILED_TESTS.txt)"
awk will replace the new lines with a delimiter or so that the test names are joined in a format that pytest expects.

Why doesn't nosetests find anything?

I am switching from python's unittest framework to nosetests, trying to reuse my unittest.TestCases
After cding into my tests package I started nosetests as described on their homepage:
./test/$ nosetests
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
Why do I need to specify each module to have nose discover its tests like in the following example?
./test/$ nosetests test_all.py
.......
----------------------------------------------------------------------
Ran 7 tests in 0.002s
OK
Also running the tests one folder above doesn't change anything.
./tests/$ cd ..
./$ nosetests
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK

I can see in your repo that at least some of the files are executable, so that is at least part of the problem. By default, nose won't collect those: it's trying to avoid running scripts that might do something destructive on import. Try the --exe flag, or removing the executable bit from the test files.

You need to be in the directory above that if you want nose to run all the tests in that package.

In my case I had following line at the end of the test files:
unittest.main()
Removing this from all my tests solved my issue.

Anyone know how nosetest's -m, -i and -e work?

I am trying to get nosetests to identify my tests but it is not running any of my tests properly.
I have the following file structure
Project
+----Foo/
+----__init__.py
+----bar.py
+----test/
+----__init__.py
+----unit/
+----__init__.py
+----bar_test.py
+----functional/
+----__init__.py
+----foo_test.py
Within bar_test.py
class BarTest(unittest.TestCase):
def bar_1_test():
...
Within foo_test.py
class FooFTest.py
def foo_1_test():
...
Using -m, -i, -e options of nosetests
What is the regex I need to run only unit tests (under unit/, tests in class BarTest)
What is the regex I need run only functional tests (under functional/, tests in class FooFTest)
I've tried various combinations and can't seem to get nosetests to do what I want consistently

The easiest way to run only the tests under Project/test/unit is to use --where. For example:
nosetests --where=Project/test/unit
Then use --match (-m) or --exclude (-e) to refine the list, if needed.
If you still want to use the regex selectors, you can probably do it like this (not tested):
nosetests --match='^Foo[\b_\./-])[Tt]est'
Executing this script from the Project directory would run all tests that start with "Foo" and end in "[Tt]est".
As a general rule, you probably want to use either --match or --exclude, but not both. These parameters both specify the pattern of the function names to match. You can refine either one by using --ignore-files which, naturally, allows you to ignore whole files.

Given your directory structure, you can easily run segments of the tests using the --exclude option.
Run all tests:
nosetests Project
Run unit tests:
nosetests Project -e functional
Run functional tests:
nosetests Project -e unit
If you have more complex test execution needs, I'd recommend marking the tests and using the attrib package.

Nose: test all modules in a given package

I have a package with several sub-packages, one of them for tests (named tests). Since the sub-package name makes it clear that contained modules are test modules, I prefer not to pollute module names with the test-name pattern nose expects to include them for testing. That's the setup I'm thinking of:
- foo
- __init__.py
...
- bar
- __init__.py
...
- baz
- __init__.py
...
- tests
- __init__.py
- something.py
Now, by default nose does not run tests found in foo.tests.something. I know nose accepts the -i option to define regular expressions for additional stuff to search tests in. So nose -i something does the job here. However, I have a bunch of modules in the tests package and do not want to name them explicitely. nose -i tests\..* does not work, it looks like nose only matches against module base-names. As a last resort I could run nose --all-modules, but this also inspects foo.bar and foo.baz -- I'd like to avoid this.
So, how could I instruct nose to look for tests in all modules within a given package (tests in my case)? I could write a nose plugin for this task, but I'm looking for a standard solution.

If you name the files in tests with a test_ prefix (i.e. rename something.py to test_something.py, running nose should pick them up by default.
You say "I prefer not to pollute module names with the test-name pattern nose expects to include them for testing", but "something" isn't descriptive of the file, because the file tests that something. What's the problem with using the non-confusing, standard way of naming your tests?

You should be able to import the files into your __init__.py and have them picked up. I had this setup:
- app:
- __init__.py
- models.py
- views.py
- tests:
- __init__.py
- models.py # contain tests
- views.py # contain tests
In the tests/__init__.py file I had the following:
from app.tests.models import *
from app.tests.views import *
Which defeats one of the benefits of using a regex to find tests (a goal of nose) but worked.
You can also use the decorator #istest to flag an individual def as a test if you want to avoid the naming of methods to match the regex. I've not tried to do this for modules (py files) that don't also match the regex but I doubt it would work without the above import.
Note, I've moved away from the import in __init__.py and just prefix my test methods and filenames with test_ and postfix my classes with Test. I find this makes the code more self-describing as even in a Test class there may be setup methods and helper methods (e.g. generators).

with Nose:
nosetests --all-modules
With py.test it can be done by put this in setup.cfg (configure file):
[pytest]
python_files=*.py
check this doc: pytest docs.