I'm a newbie when it comes to python unit testing, but I'm eager to learn!
I just read python setup.py test can run all suites derived from unittest classes. I wonder if I also can use setup.py to run a single suite and/or a single test case, maybe adding some modifier to the previous command like python setup.py tests suitename. If so, can you please point me to any docs/examples?
You guys are all wrong, setup.py test can be used with the -s option the same way python -m unittest does:
cd root_of_your_package
python setup.py test -s tests.TestClass.test_method
The setup.py test runner is rather limited; it only supports letting you specify a specific module. The documentation for the command-line switches is given when you use the --help switch:
python setup.py test --help
Common commands: (see '--help-commands' for more)
[ ... cut ... ]
Options for 'test' command:
--test-module (-m) Run 'test_suite' in specified module
--test-suite (-s) Test suite to run (e.g. 'some_module.test_suite')
[ ... more cut ... ]
so python setup.py test -m your.package.tests.test_module would limit running the tests from the test_module.py file only.
All the test command does, really, is make sure your egg has been built already, extract the test_suite value from setup() metadata, configure a test loader that understands about zipped eggs, then run the unittest.main() function.
If you need to run a single test only, have already built your egg, are not running this with a zipped egg, then you can also just use the unittest command line interface, which does pretty much everything else:
python -m unittest yourpackage.tests.TestClass.test_method
would instruct unittest to only run a very specific test method.
setup.py test
setup.py test is not that flexible, but here's an alternative:
The unittest module can run specific test methods
From the Documentation on unittest
The unittest module can be used from the command line to run tests from modules, classes or even individual test methods:
python -m unittest test_module1 test_module2
python -m unittest test_module.TestClass
python -m unittest test_module.TestClass.test_method
You can pass in a list with any combination of module names, and fully qualified class or method names.
You can run tests with more detail (higher verbosity) by passing in the -v flag:
python -m unittest -v test_module
For a list of all the command-line options:
python -m unittest -h
in case of pytest run specific test:
python setup.py test --addopts tests/jobs/test_data_monitoring.py::TestDataMonitoring::test_dataframe__bulk_update
In case you use pytest for unit testing you can run a file directly by avoiding the setup.py file.
Running all tests for me is:
python setup.py test
Running a specific test:
pytest --pyargs path/to/test.py
Hope it helps.
(I know this post is already a bit old, but I came here looking for running a way to run pytest tests alone, figured it might be of use to someone)
Related
My very simple example project contains:
addtest/
setup.py
addtest/
__init__.py
__main__.py
app.py
My app.py is just:
def main():
raise SystemExit("Command line entry point called.")
My __main__.py is just:
from addtest.app import main
main()
My setup.py contains:
from setuptools import setup, find_packages
setup(
name='AddTest',
version='1.0',
packages=find_packages(),
entry_points={
'console_scripts': ['addtest = addtest.app:main']
},
)
I would expect that running python setup.py test would do nothing, since no unit tests are written. However, running it in a clean virtualenv (Python 3.6.6 on Ubuntu 18.04.1) gives me:
$ python setup.py test
running test
running egg_info
writing AddTest.egg-info/PKG-INFO
writing dependency_links to AddTest.egg-info/dependency_links.txt
writing entry points to AddTest.egg-info/entry_points.txt
writing top-level names to AddTest.egg-info/top_level.txt
reading manifest file 'AddTest.egg-info/SOURCES.txt'
writing manifest file 'AddTest.egg-info/SOURCES.txt'
running build_ext
Command line entry point called.
Note the Command line entry point called. which means it's invoking the console script it generates from my __main__.py (or maybe just calling python -m addtest).
Why is setup.py calling the console script when I want it to run tests? Further inspection of the script's execution shows that sys.argv is ['setup.py','test'] - why?
The test scanner in setuptools will look for tests in any *.py file found in your sub-directories, except for __init__.py. Yes, this includes __main__.py, it will call __import__() on it, causing its main suite to be executed.
If you want to be able to run python -m addtest and have that run your __main__.py code, you may want to add the standard 'run only if this is really main' protection:
if __name__ == "__main__":
...
(then, if you do python -m the code will run, but it won't run if the file is loaded by setup.py test)
The setuptools docs explains how setuptools scans for unit tests by default.
Since, as you say, no unit tests are written or specified, setuptools can be using default behavior.
A couple references from the setuptools docs that elaborates on this as well as specifies how to set up test suites are below:
test_loader If you would like to use a
different way of finding tests to run than what setuptools normally
uses, you can specify a module name and class name in this argument.
...
The module name and class name must be separated by a :. The default value of this argument is "setuptools.command.test:ScanningLoader". If you want to use the default unittest behavior, you can specify "unittest:TestLoader" as your test_loader argument instead. This will prevent automatic scanning of submodules and subpackages.
... test_suite A string naming a unittest.TestCase subclass (or a package
or module containing one or more of them, or a method of such a
subclass), or naming a function that can be called with no arguments
and returns a unittest.TestSuite. If the named suite is a module, and
the module has an additional_tests() function, it is called and the
results are added to the tests to be run. If the named suite is a
package, any submodules and subpackages are recursively added to the
overall test suite.
Specifying this argument enables use of the test command to run the
specified test suite, e.g. via setup.py test. See the section on the
test command below for more details.
...
setup(
# ...
test_suite="my_package.tests.test_all"
)
This, in conjunction with another answer on this thread leaves you with at least a couple of options to ensure that python setup.py test doesn't run the console script:
Configure how setuptools looks for test suites.
Add if __name__ == "__main__": around main()
Yet another option may be given by the unit testing library you're using. For example, Pytest has an integration guide for setuptools that replaces the test command with its own.
Regarding the sysargs
sys.argv is ['setup.py','test'] because you invoked python with the args setup.py test
I think you have to specify the entry point for test in your setup.py. Otherwise it passes it to the main.py.
I made a small project called demo, with a single test in it
import unittest
class Test(unittest.TestCase):
def testName1(self):
self.assertEqual(5+9, 14)
if __name__ == "__main__":
#import sys;sys.argv = ['', 'Test.testName']
unittest.main()
However, from command line
ThinkPad-T520:~/workspacep/demo$ python -m unittest
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
Why doesn't this work? In general, how can I run all unit tests from command line with a single line?
The structure of the directory is
demo
tests
demo_test1.py __init__.py
There are three gotcha's that I know of:
Your tests in your TestCases need to be named test_*
Your test files need to be named: test*.py (by default, you can change it with the -p flag when running the tests). e.g. test_demo1.py
Your tests folder needs to have an __init__.py file in it, or else it won't be considered a valid location to import from.
So, for #1, you need to rename the test to test_name_1. And for #2, there's two options:
A - Restructure your files like this:
demo
tests
__init__.py
test_demo1.py
Then run python -m unittest and it should find the test cases.
B - Just run it like: python -m unittest discover -p *test.py
I fought with the same exact problem a while ago and I solved it by using test discovery command.
python -m unittest discover -s .
You can pass in your test file pattern as well and a whole other options https://docs.python.org/2/library/unittest.html#test-discovery
You need to pass in a list of modules.
For example, if your test file is foo.py, then you can run python -m unittest foo.
For me (running tests from IntelliJ IDEA) I had to remove the class' 'Run configuration'. Earlier on I had wrongly imported the unittest as _pytest.unittest and ran the test class. Of course that didn't work.
I corrected the module import, but the 'run configuration' was still there, causing it to run as a Python script and not as 'Python tests'.
I had a similar issue and figured out that I had to run python3 -m unittest instead as I had forgotten python defaults to Python 2 on my system
I have a Django project with multiple apps. Each app has a set of unittests. I'm using pytest as my test runner. We have gotten to a point that we want to start writing integration tests. I was wondering if there is any way to keep the naming convention and thus the auto discovery of pytest but still be able (via flag maybe?) to run the different test types. The most intuitive solution that comes to mind is some sort of decorator on test methods or even TestCase classes (something like Category in JUnit).
something like:
#testtype('unittest')
def test_my_test(self):
# do some testing
#testtype('integration')
def test_my_integration_test(self):
# do some integration testing
and then i could run the test like:
py.test --type=integration
py.test --type=unittest
Is there such a thing?
If not, the only other solution i can think about is to add a django command and "manually" build a testsuite and run it with pytest... I would prefer not to use this option. Is there any other solution that can help me?
Thanks
You can mark test functions.
import pytest
#pytest.mark.unittest
def test_my_test(self):
# do some testing
#pytest.mark.integration
def test_my_integration_test(self):
# do some integration testing
These custom markers must be registered in your pytest.ini file.
Then use the -m flag to run the marked tests
py.test -v -m unittest
Another option would be to split your tests into unittest and integration directories. Then you can run tests in a specific directory with:
py.test -v unittest
or
py.test -v integration
Another way to do this (without any config or code)
pytest -o "python_functions=*_integration_test"
You can also do this in module/class level, e.g.,
python_files = integration_test_*.py
python_classes = IntegrationTest
Ref:
https://docs.pytest.org/en/latest/example/pythoncollection.html#changing-naming-conventions
Behold, my setup.py:
https://github.com/mongodb/motor/blob/master/setup.py
... and setup.cfg:
https://github.com/mongodb/motor/blob/master/setup.cfg
I'd like to be able to run one suite, like:
python setup.py nosetests test.test_motor_ssl
But I get "invalid command name 'test.test_motor_ssl'". With this, on the other hand:
python setup.py nosetests --tests test.test_motor_ssl
... nosetests runs every test in my project. How can I tell nosetests, when it is running in setup.py, how to run a subset of tests?
Apparently this is a known bug in nose 1.2.1 and they already have a fix in the master branch. You can either wait for the next version or use the nosetests command directly.
source: https://github.com/nose-devs/nose/issues/556
The only thing that works now is actually the directory approach. It is still not possible to specify the path...
python setup.py nosetests -w tests/test_folder
Using nose==1.3.1 I'm able to run a single test class/test case via:
python setup.py nosetests --tests tests/test_file.py:TestClass.test_case
Python's unittest discover does not find my tests!
I have been using nose to discover my unit tests and it is working fine. From the top level of my project, if I run nosetests I get:
Ran 31 tests in 0.390s
Now that Python 2.7 unittest has discovery, I have tried using
python -m unittest discover
but I get
Ran 0 tests in 0.000s
My directory structure is:
myproj/
reporter/
__init__.py
report.py
[other app modules]
tests/
__init__.py
test-report.py
[other test modules]
Do you have any ideas why unittest's discovery algorithm can't find the tests?
I'm using Python 2.7.1 and nose 1.0.0 on Windows 7.
The behaviour is intentional, but the documentation could make this clearer. If you look at the first paragraph in the test discovery section, it says:
For a project’s tests to be compatible with test discovery they must all be importable from the top level directory of the project (in other words, they must all be in Python packages).
A corollary to that is that the file names must also be valid Python module names. test-report.py fails that test, since test-report is not a legal Python identifier.
A docs bug suggesting that this be mentioned explicitly in the documentation for the -p pattern option would probably be a good way forward.
I had this problem because some directories in a project were missing __init__.py. I thought I don't need them in Python 3.7.
Just add __init__.py to every directory and python3 -m unittest will find tests automatically.
As someone relatively new to Python, the naming convention in the docs implied the opposite. Ben's comment was very helpful: the default discovery pattern looks for test-modules prefixed with the string "test"
I thought the introspection would just look for class names and not require a specific file naming convention.
Here is what the docs say:
https://docs.python.org/3/library/unittest.html
python -m unittest discover -s project_directory -p "_test.py"
I couldn't get this to work, but by changing my file names to be "test_.py" - success!