We Keep Coding

In "unittest discover", what does the "-t" option do?

The Python documentation states that the -t option controls the:
Top level directory of project (defaults to start directory)
Usually people use the -s option (python -m unittest discover tests/ is equivalent to python -m unittest discover -s tests), and I have never seen anyone use -t before. The brief description in the documentation is not enlightening to me.
What does the "top level directory" mean in this particular context? What exactly does the -t option do?

My understanding is that although the top-level directory defaults to whatever the starting directory is, the starting directory must be contained by the test directory.
From the first paragraph on test discovery:
Unittest supports simple test discovery. In order to be compatible with test discovery, all of the test files must be modules or packages (including namespace packages) importable from the top-level directory of the project (this means that their filenames must be valid identifiers).
Suppose you have a directory structure like
./
tests1/
tests2/
If ./ is the top-level directory and tests1 is the starting directory, no tests will discovered under tests2, even though tests2 is importable from the top-level directory.
The purpose of -s would be to discover only a subset of tests for a particular project. The purpose of -t might be to choose a particular "subproject" to run tests for.

Python unittest does not run tests

I made a small project called demo, with a single test in it
import unittest
class Test(unittest.TestCase):
def testName1(self):
self.assertEqual(5+9, 14)
if __name__ == "__main__":
#import sys;sys.argv = ['', 'Test.testName']
unittest.main()
However, from command line
ThinkPad-T520:~/workspacep/demo$ python -m unittest
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
Why doesn't this work? In general, how can I run all unit tests from command line with a single line?
The structure of the directory is
demo
tests
demo_test1.py __init__.py

There are three gotcha's that I know of:
Your tests in your TestCases need to be named test_*
Your test files need to be named: test*.py (by default, you can change it with the -p flag when running the tests). e.g. test_demo1.py
Your tests folder needs to have an __init__.py file in it, or else it won't be considered a valid location to import from.
So, for #1, you need to rename the test to test_name_1. And for #2, there's two options:
A - Restructure your files like this:
demo
tests
__init__.py
test_demo1.py
Then run python -m unittest and it should find the test cases.
B - Just run it like: python -m unittest discover -p *test.py

I fought with the same exact problem a while ago and I solved it by using test discovery command.
python -m unittest discover -s .
You can pass in your test file pattern as well and a whole other options https://docs.python.org/2/library/unittest.html#test-discovery

You need to pass in a list of modules.
For example, if your test file is foo.py, then you can run python -m unittest foo.

For me (running tests from IntelliJ IDEA) I had to remove the class' 'Run configuration'. Earlier on I had wrongly imported the unittest as _pytest.unittest and ran the test class. Of course that didn't work.
I corrected the module import, but the 'run configuration' was still there, causing it to run as a Python script and not as 'Python tests'.

I had a similar issue and figured out that I had to run python3 -m unittest instead as I had forgotten python defaults to Python 2 on my system

Is there a way to specify which pytest tests to run from a file?

Is there a way to select pytest tests to run from a file?
For example, a file foo.txt containing a list of tests to be executed:
tests_directory/foo.py::test_001
tests_directory/bar.py::test_some_other_test
Or, is there a way to select multiple tests, having no common pattern in test name, from different directories with pytest?
pytest -k <pattern> allows a single pattern.
One option is to have a pytest.mark against each test, but my requirement is to run different combination of tests from different files.
Is there a way to specify multiple patterns and a test file name for each pattern?
Or
Is there a way to specify the exact test paths in a file and feed that file as an input to pytest?
Or
Is there a hook function that can be utilized for this purpose?

You can use -k option to run test cases with different patterns:
py.test tests_directory/foo.py tests_directory/bar.py -k 'test_001 or test_some_other_test'
This will run test cases with name test_001 and test_some_other_test deselecting the rest of the test cases.
Note: This will select any test case starting with test_001 or test_some_other_test. For example, if you have test case test_0012 it will also be selected.

Specifying tests / selecting tests
Pytest supports several ways to run and select tests from the command-line.
Run tests in a module
pytest test_mod.py
Run tests in a directory
pytest testing/
Run tests by keyword expressions
pytest -k "MyClass and not method"
This will run tests which contain names that match the given string expression, which can include Python operators that use filenames, class names and function names as variables. The example above will run TestMyClass.test_something but not TestMyClass.test_method_simple.
Run tests by node ids
Each collected test is assigned a unique nodeid which consist of the module filename followed by specifiers like class names, function names and parameters from parametrization, separated by :: characters.
To run a specific test within a module:
pytest test_mod.py::test_func
Another example specifying a test method in the command line:
pytest test_mod.py::TestClass::test_method
Run tests by marker expressions
pytest -m slow
Will run all tests which are decorated with the #pytest.mark.slow decorator.
For more information see marks.
Run tests from packages
pytest --pyargs pkg.testing
This will import pkg.testing and use its filesystem location to find and run tests from.
Source: https://docs.pytest.org/en/latest/usage.html#specifying-tests-selecting-tests

My answer provides a ways to run a subset of test in different scenarios.
Run all tests in a project
pytest
Run tests in a Single Directory
To run all the tests from one directory, use the directory as a parameter to
pytest:
pytest tests/my-directory
Run tests in a Single Test File/Module
To run a file full of tests, list the file with the relative path as a parameter to pytest:
pytest tests/my-directory/test_demo.py
Run a Single Test Function
To run a single test function, add :: and the test function name:
pytest -v tests/my-directory/test_demo.py::test_specific_function
-v is used so you can see which function was run.
Run a Single Test Class
To run just a class, do like we did with functions and add ::, then the class name to the file parameter:
pytest -v tests/my-directory/test_demo.py::TestClassName
Run a Single Test Method of a Test Class
If you don't want to run all of a test class, just one method, just add
another :: and the method name:
pytest -v tests/my-directory/test_demo.py::TestClassName::test_specific_method
Run a Set of Tests Based on Test Name
The -k option enables you to pass in an expression to run tests that have
certain names specified by the expression as a substring of the test name.
It is possible to use and, or, and not to create complex expressions.
For example, to run all of the functions that have _raises in their name:
pytest -v -k _raises

Method 1: Randomly selected tests. Long and ugly.
python -m pytest test/stress/test_performance.py::TestPerformance::test_continuous_trigger test/integration/test_config.py::TestConfig::test_valid_config
Method 2: Use Keyword Expressions.
Note: I am searching by testcase names. Same is applicable to TestClass names.
Case 1: The below will run whichever is found. Since we have used 'OR' .
python -m pytest -k 'test_password_valid or test_no_configuration'
Lets say the two above are actually correct, 2 tests will be run.
Case 2: Now an incorrect name and another correct name.
python -m pytest -k 'test_password_validzzzzzz or test_no_configuration'
Only one is found and run.
Case 3: If you want to run either all tests mentioned, or None, then use AND
python -m pytest -k 'test_password_valid and test_no_configuration'
Both will be run if correct or none.
Case 4: Run test only in one folder:
python -m pytest test/project1/integration -k 'test_password_valid or test_no_configuration'
Case 5: Run test from only one file.
python -m pytest test/integration/test_authentication.py -k 'test_password_expiry or test_incorrect_password'
Case 6: Run all tests except the match.
python -m pytest test/integration/test_authentication.py -k 'not test_incorrect_password'

If you have the same method name in two different classes and you just want to run one of them, this works:
pytest tests.py -k "TestClassName and test_method_name"

According to the doc about Run tests by node ids
since you have all node ids in foo.txt, just run
pytest $(tr '\n' ' ' <foo.txt)
this is same with below command (with file content in the question)
pytest tests_directory/foo.py::test_001 tests_directory/bar.py::test_some_other_test

Maybe using pytest_collect_file() hook you can parse the content of a .txt o .yaml file where the tests are specify as you want, and return them to the pytest core.
A nice example is shown in the pytest documentation. I think what you are looking for.

Try:
pytest path/file_test.py::test_name
Using -k as others have suggested will match any test with that substring, not just a specific test.

Say for example you have a file called test.py. In this file you have a class called TestSomeService. This class has 2 functions namely test_service1 and test_service2.
Now to run specific function test you can this
pytest test.py::TestSomeService::test_service1
pytest test.py::TestSomeService::test_service2
To run specific class
pytest test.py::TestSomeService

Here's a possible partial answer, because it only allows selecting the test scripts, not individual tests within those scripts.
And it also limited by my using legacy compatibility mode vs unittest scripts, so not guaranteeing it would work with native pytest.
Here goes:
create a new dictory, say subset_tests_directory.
ln -s tests_directory/foo.py
ln -s tests_directory/bar.py
be careful about imports which implicitly assume files are in test_directory. I had to fix several of those by running python foo.py, from within subset_tests_directory and correcting as needed.
Once the test scripts execute correctly, just cd subset_tests_directory and pytest there. Pytest will only pick up the scripts it sees.
Another possibility is symlinking within your current test directory, say as ln -s foo.py subset_foo.py then pytest subset*.py. That would avoid needing to adjust your imports, but it would clutter things up until you removed the symlinks. Worked for me as well.

Assuming the test names are unique, you have to remove the test file's name:
cut -d : -f 3 foo.txt > FAILED_TESTS.txt
As others pointed out use -k, but you have to pass the file's content (i.e., list of test names) as a single string:
pytest -k "$(awk '$1=$1' RS= OFS=' or ' FAILED_TESTS.txt)"
awk will replace the new lines with a delimiter or so that the test names are joined in a format that pytest expects.

Anyone know how nosetest's -m, -i and -e work?

I am trying to get nosetests to identify my tests but it is not running any of my tests properly.
I have the following file structure
Project
+----Foo/
+----__init__.py
+----bar.py
+----test/
+----__init__.py
+----unit/
+----__init__.py
+----bar_test.py
+----functional/
+----__init__.py
+----foo_test.py
Within bar_test.py
class BarTest(unittest.TestCase):
def bar_1_test():
...
Within foo_test.py
class FooFTest.py
def foo_1_test():
...
Using -m, -i, -e options of nosetests
What is the regex I need to run only unit tests (under unit/, tests in class BarTest)
What is the regex I need run only functional tests (under functional/, tests in class FooFTest)
I've tried various combinations and can't seem to get nosetests to do what I want consistently

The easiest way to run only the tests under Project/test/unit is to use --where. For example:
nosetests --where=Project/test/unit
Then use --match (-m) or --exclude (-e) to refine the list, if needed.
If you still want to use the regex selectors, you can probably do it like this (not tested):
nosetests --match='^Foo[\b_\./-])[Tt]est'
Executing this script from the Project directory would run all tests that start with "Foo" and end in "[Tt]est".
As a general rule, you probably want to use either --match or --exclude, but not both. These parameters both specify the pattern of the function names to match. You can refine either one by using --ignore-files which, naturally, allows you to ignore whole files.

Given your directory structure, you can easily run segments of the tests using the --exclude option.
Run all tests:
nosetests Project
Run unit tests:
nosetests Project -e functional
Run functional tests:
nosetests Project -e unit
If you have more complex test execution needs, I'd recommend marking the tests and using the attrib package.

`python -m unittest discover` does not discover tests

Python's unittest discover does not find my tests!
I have been using nose to discover my unit tests and it is working fine. From the top level of my project, if I run nosetests I get:
Ran 31 tests in 0.390s
Now that Python 2.7 unittest has discovery, I have tried using
python -m unittest discover
but I get
Ran 0 tests in 0.000s
My directory structure is:
myproj/
reporter/
__init__.py
report.py
[other app modules]
tests/
__init__.py
test-report.py
[other test modules]
Do you have any ideas why unittest's discovery algorithm can't find the tests?
I'm using Python 2.7.1 and nose 1.0.0 on Windows 7.

The behaviour is intentional, but the documentation could make this clearer. If you look at the first paragraph in the test discovery section, it says:
For a project’s tests to be compatible with test discovery they must all be importable from the top level directory of the project (in other words, they must all be in Python packages).
A corollary to that is that the file names must also be valid Python module names. test-report.py fails that test, since test-report is not a legal Python identifier.
A docs bug suggesting that this be mentioned explicitly in the documentation for the -p pattern option would probably be a good way forward.

I had this problem because some directories in a project were missing __init__.py. I thought I don't need them in Python 3.7.
Just add __init__.py to every directory and python3 -m unittest will find tests automatically.

As someone relatively new to Python, the naming convention in the docs implied the opposite. Ben's comment was very helpful: the default discovery pattern looks for test-modules prefixed with the string "test"
I thought the introspection would just look for class names and not require a specific file naming convention.
Here is what the docs say:
https://docs.python.org/3/library/unittest.html
python -m unittest discover -s project_directory -p "_test.py"
I couldn't get this to work, but by changing my file names to be "test_.py" - success!