We Keep Coding

Running unittest discover ignoring specific directory

I'm looking for a way of running python -m unittest discover, which will discover tests in, say, directories A, B and C. However, directories A, B and C have directories named dependencies inside each of them, in which there are also some tests which, however, I don't want to run.
Is there a way to run my tests satisfying these constraints without having to create a script for this?

I ran into the same problem and was eventually able to find these handy arguments to pass to unittest discover that resolved my issue.
It is documented here: https://docs.python.org/2/library/unittest.html#test-discovery
-s, --start-directory directory
Directory to start discovery (. default)
-p, --pattern pattern
Pattern to match test files (test*.py default)
So I modified my command to be:
python -m unittest discover -s test
since all of the tests I actually want to run are in the one module, test. You could also use the -p to in theory match regex that only hits your tests, ignoring all of the rest it may find.

I've managed to do it this way (In *NIX):
find `pwd` -name '*_test.py' -not -path '*unwanted_path*' \
| xargs python3 -m unittest -v
That is, the tests are discovered by find, which allows for options like path pattern exclusions, then they're passed to the unittest command as argument list.
Note that I had to switch to find pwd, where usually I can write find ., since relative paths in the form ./xxx aren't accepted by unittest (module not found).

It would seem that python -m unittest descends into module directories but not in other directories.
It quickly tried the following structure
temp
+ a
- test_1.py
+ dependencies
- test_a.py
With the result
>python -m unittest discover -s temp\a
test_1
.
----------------------------------------------------------------------
Ran 1 test in 0.002s
OK
However, if the directory is a module directory (containing a file __init__.py) the situation is different.
temp
+ a
- __init__.py
- test_1.py
+ dependencies
- __init__.py
- test_a.py
Here the result was
>python -m unittest discover -s temp\a
test_a
.test_1
.
----------------------------------------------------------------------
Ran 2 tests in 0.009s
OK
The usefulness of this answer for you now depends on whether it is acceptable for your folder dependencies not to be a module directory.
EDIT: After seeing your comment
Would using pytest be an option? This test runner has many command arguments, one specifically to exclude tests.
See Changing standard (Python) test discovery
From their site
Ignore paths during test collection
You can easily ignore certain test directories and modules during collection by passing the --ignore=path option on the cli. pytest allows multiple --ignore options

Is there a way to specify which pytest tests to run from a file?

Is there a way to select pytest tests to run from a file?
For example, a file foo.txt containing a list of tests to be executed:
tests_directory/foo.py::test_001
tests_directory/bar.py::test_some_other_test
Or, is there a way to select multiple tests, having no common pattern in test name, from different directories with pytest?
pytest -k <pattern> allows a single pattern.
One option is to have a pytest.mark against each test, but my requirement is to run different combination of tests from different files.
Is there a way to specify multiple patterns and a test file name for each pattern?
Or
Is there a way to specify the exact test paths in a file and feed that file as an input to pytest?
Or
Is there a hook function that can be utilized for this purpose?

You can use -k option to run test cases with different patterns:
py.test tests_directory/foo.py tests_directory/bar.py -k 'test_001 or test_some_other_test'
This will run test cases with name test_001 and test_some_other_test deselecting the rest of the test cases.
Note: This will select any test case starting with test_001 or test_some_other_test. For example, if you have test case test_0012 it will also be selected.

Specifying tests / selecting tests
Pytest supports several ways to run and select tests from the command-line.
Run tests in a module
pytest test_mod.py
Run tests in a directory
pytest testing/
Run tests by keyword expressions
pytest -k "MyClass and not method"
This will run tests which contain names that match the given string expression, which can include Python operators that use filenames, class names and function names as variables. The example above will run TestMyClass.test_something but not TestMyClass.test_method_simple.
Run tests by node ids
Each collected test is assigned a unique nodeid which consist of the module filename followed by specifiers like class names, function names and parameters from parametrization, separated by :: characters.
To run a specific test within a module:
pytest test_mod.py::test_func
Another example specifying a test method in the command line:
pytest test_mod.py::TestClass::test_method
Run tests by marker expressions
pytest -m slow
Will run all tests which are decorated with the #pytest.mark.slow decorator.
For more information see marks.
Run tests from packages
pytest --pyargs pkg.testing
This will import pkg.testing and use its filesystem location to find and run tests from.
Source: https://docs.pytest.org/en/latest/usage.html#specifying-tests-selecting-tests

My answer provides a ways to run a subset of test in different scenarios.
Run all tests in a project
pytest
Run tests in a Single Directory
To run all the tests from one directory, use the directory as a parameter to
pytest:
pytest tests/my-directory
Run tests in a Single Test File/Module
To run a file full of tests, list the file with the relative path as a parameter to pytest:
pytest tests/my-directory/test_demo.py
Run a Single Test Function
To run a single test function, add :: and the test function name:
pytest -v tests/my-directory/test_demo.py::test_specific_function
-v is used so you can see which function was run.
Run a Single Test Class
To run just a class, do like we did with functions and add ::, then the class name to the file parameter:
pytest -v tests/my-directory/test_demo.py::TestClassName
Run a Single Test Method of a Test Class
If you don't want to run all of a test class, just one method, just add
another :: and the method name:
pytest -v tests/my-directory/test_demo.py::TestClassName::test_specific_method
Run a Set of Tests Based on Test Name
The -k option enables you to pass in an expression to run tests that have
certain names specified by the expression as a substring of the test name.
It is possible to use and, or, and not to create complex expressions.
For example, to run all of the functions that have _raises in their name:
pytest -v -k _raises

Method 1: Randomly selected tests. Long and ugly.
python -m pytest test/stress/test_performance.py::TestPerformance::test_continuous_trigger test/integration/test_config.py::TestConfig::test_valid_config
Method 2: Use Keyword Expressions.
Note: I am searching by testcase names. Same is applicable to TestClass names.
Case 1: The below will run whichever is found. Since we have used 'OR' .
python -m pytest -k 'test_password_valid or test_no_configuration'
Lets say the two above are actually correct, 2 tests will be run.
Case 2: Now an incorrect name and another correct name.
python -m pytest -k 'test_password_validzzzzzz or test_no_configuration'
Only one is found and run.
Case 3: If you want to run either all tests mentioned, or None, then use AND
python -m pytest -k 'test_password_valid and test_no_configuration'
Both will be run if correct or none.
Case 4: Run test only in one folder:
python -m pytest test/project1/integration -k 'test_password_valid or test_no_configuration'
Case 5: Run test from only one file.
python -m pytest test/integration/test_authentication.py -k 'test_password_expiry or test_incorrect_password'
Case 6: Run all tests except the match.
python -m pytest test/integration/test_authentication.py -k 'not test_incorrect_password'

If you have the same method name in two different classes and you just want to run one of them, this works:
pytest tests.py -k "TestClassName and test_method_name"

According to the doc about Run tests by node ids
since you have all node ids in foo.txt, just run
pytest $(tr '\n' ' ' <foo.txt)
this is same with below command (with file content in the question)
pytest tests_directory/foo.py::test_001 tests_directory/bar.py::test_some_other_test

Maybe using pytest_collect_file() hook you can parse the content of a .txt o .yaml file where the tests are specify as you want, and return them to the pytest core.
A nice example is shown in the pytest documentation. I think what you are looking for.

Try:
pytest path/file_test.py::test_name
Using -k as others have suggested will match any test with that substring, not just a specific test.

Say for example you have a file called test.py. In this file you have a class called TestSomeService. This class has 2 functions namely test_service1 and test_service2.
Now to run specific function test you can this
pytest test.py::TestSomeService::test_service1
pytest test.py::TestSomeService::test_service2
To run specific class
pytest test.py::TestSomeService

Here's a possible partial answer, because it only allows selecting the test scripts, not individual tests within those scripts.
And it also limited by my using legacy compatibility mode vs unittest scripts, so not guaranteeing it would work with native pytest.
Here goes:
create a new dictory, say subset_tests_directory.
ln -s tests_directory/foo.py
ln -s tests_directory/bar.py
be careful about imports which implicitly assume files are in test_directory. I had to fix several of those by running python foo.py, from within subset_tests_directory and correcting as needed.
Once the test scripts execute correctly, just cd subset_tests_directory and pytest there. Pytest will only pick up the scripts it sees.
Another possibility is symlinking within your current test directory, say as ln -s foo.py subset_foo.py then pytest subset*.py. That would avoid needing to adjust your imports, but it would clutter things up until you removed the symlinks. Worked for me as well.

Assuming the test names are unique, you have to remove the test file's name:
cut -d : -f 3 foo.txt > FAILED_TESTS.txt
As others pointed out use -k, but you have to pass the file's content (i.e., list of test names) as a single string:
pytest -k "$(awk '$1=$1' RS= OFS=' or ' FAILED_TESTS.txt)"
awk will replace the new lines with a delimiter or so that the test names are joined in a format that pytest expects.

Emulating "nosetests -w working-dir" behavior using py.test

I am porting a series of tests from nosetests + python unittest to py.test. I was pleasantly surprised to learn that py.test supports python unittests and running existing tests with py.test is just as easy as calling py.test instead of nosetests on the command line. However I am having problems with specifying the working directory for the tests. They are not in the root project dir but in a subdir. Currently the tests are run like this:
$ nosetests -w test-dir/ tests.py
which changes the current working directory to test-dir and run all the tests in tests.py. However when I use py.test
$ py.test test-dir/tests.py
all the tests in tests.py are run but the current working directory is not changed to test-dir. Most tests assume that the working directory is test-dir and try to open and read files from it which obviously fails.
So my question is how to change the current working directory for all tests when using py.test.
The are a lot of tests and I don't want to invest the time to fix them all and make them work regardless of the cwd.
Yes, I can simply do cd test-dir; py.test tests.py but I am used to working from the project root directory and don't want to cd every time I want to run a test.
Here is some code that may give you better idea what I am trying to achieve:
content of tests.py:
import unittest
class MyProjectTestCase(unittest.TestCase):
def test_something(self):
with open('testing-info.txt', 'r') as f:
test something with f
directory layout:
my-project/
test-dir/
tests.py
testing-info.txt
And then when I try to run the tests:
$ pwd
my-project
$ nosetests -w test-dir tests.py
# all is fine
$ py.test ttest-dir/tests.py
# tests fail because they cannot open testing-info.txt

So this is the best I could come up with:
# content of conftest.py
import pytest
import os
def pytest_addoption(parser):
parser.addoption("-W", action="store", default=".",
help="Change current working dir before running the collected tests.")
def pytest_sessionstart(session):
os.chdir(session.config.getoption('W'))
And then when running the tests
$ py.test -W test-dir test-dir/tests.py
It's not clean but it will do the trick until I fix all the tests.

Django testing external script

I want to perform testing on a script that interacts with my Django application, namely the database. Normally, when we want to test something in Django we simply fire up the built in test suite. With this test suite we even get nice command line switches such as overriding the built in settings.py with a different settings file:
python manage.py test myApp --settings='settings_test'
Here is the problem:
1) I want to test said script which is not part of an app, so I know of no way to invoke the test suite using manage.py. Is this possible? I.e.:
python manage.py test /path/myScript.py --settings='settings_test'

I would suggest to use a different test runner.
You can do pip install django-nose and then set the following setting in your test_settings.py
TEST_RUNNER = `django_nose.NoseTestSuiteRunner`
Now you can run the tests with
./manage.py test --settings=yourproject.test_settings.py
and the Nose testrunner will search all subfolders for folders called tests and in those folders it will search for files that end with _tests.py (and in those files it will search for classes that derive from TestCase, as usual).
So your project structure should look something like this:
- Project-Root/
- Your-Non-App-Code/
- __init__.py
- non_app_code.py
- tests/
- __init__.py
- non_app_code_tests.py
For more info on how to install django-nose, check their Github repo: https://github.com/django-nose/django-nose

Anyone know how nosetest's -m, -i and -e work?

I am trying to get nosetests to identify my tests but it is not running any of my tests properly.
I have the following file structure
Project
+----Foo/
+----__init__.py
+----bar.py
+----test/
+----__init__.py
+----unit/
+----__init__.py
+----bar_test.py
+----functional/
+----__init__.py
+----foo_test.py
Within bar_test.py
class BarTest(unittest.TestCase):
def bar_1_test():
...
Within foo_test.py
class FooFTest.py
def foo_1_test():
...
Using -m, -i, -e options of nosetests
What is the regex I need to run only unit tests (under unit/, tests in class BarTest)
What is the regex I need run only functional tests (under functional/, tests in class FooFTest)
I've tried various combinations and can't seem to get nosetests to do what I want consistently

The easiest way to run only the tests under Project/test/unit is to use --where. For example:
nosetests --where=Project/test/unit
Then use --match (-m) or --exclude (-e) to refine the list, if needed.
If you still want to use the regex selectors, you can probably do it like this (not tested):
nosetests --match='^Foo[\b_\./-])[Tt]est'
Executing this script from the Project directory would run all tests that start with "Foo" and end in "[Tt]est".
As a general rule, you probably want to use either --match or --exclude, but not both. These parameters both specify the pattern of the function names to match. You can refine either one by using --ignore-files which, naturally, allows you to ignore whole files.

Given your directory structure, you can easily run segments of the tests using the --exclude option.
Run all tests:
nosetests Project
Run unit tests:
nosetests Project -e functional
Run functional tests:
nosetests Project -e unit
If you have more complex test execution needs, I'd recommend marking the tests and using the attrib package.