I typically write both unittests and doctests in my modules. I'd like to automatically run all of my doctests when running the test suite. I think this is possible, but I'm having a hard time with the syntax.
I have the test suite
import unittest
class ts(unittest.TestCase):
def test_null(self): self.assertTrue(True)
if __name__ == '__main__': unittest.main()
I'd like to add to this suite all of the doctests in module module1. How can I do this? I've read the python docs, but I'm not any closer to success, here. Adding the lines
import doctest
import module1
suite = doctest.DocTestSuite(module1)
doesn't work. unittest.main() searches through the current file scope and runs every test case it finds, right? But DocTestSuite produces a test suite. How do I get unittest.main() to run the additional cases in the suite? Or am I just confused and deluded??
Once again, I'd be grateful for any help anyone can offer.
An update to this old question: since Python version 2.7 there is the load_tests protocol and there is no longer a need to write custom code. It allows you to add a function load_tests(), which a test loader will execute to update its collection of unit tests for the current module.
Put a function like this in your code module to package the module's own doctests into a test suite for unittest:
def load_tests(loader, tests, ignore):
tests.addTests(doctest.DocTestSuite())
return tests
Or, put a function like this into your unit test module to add the doctests from another module (for example, package.code_module) into the tests suite which is already there:
def load_tests(loader, tests, ignore):
tests.addTests(doctest.DocTestSuite(package.code_module))
return tests
When unittest.TestLoader methods loadTestsFromModule(), loadTestsFromName() or discover() are used unittest uses a test suite including both unit tests and doctests.
In this code i combined unittests and doctests from imported module
import unittest
class ts(unittest.TestCase):
def test_null(self):
self.assertTrue(True)
class ts1(unittest.TestCase):
def test_null(self):
self.assertTrue(True)
testSuite = unittest.TestSuite()
testSuite.addTests(unittest.makeSuite(ts))
testSuite.addTest(unittest.makeSuite(ts1))
import doctest
import my_module_with_doctests
testSuite.addTest(doctest.DocTestSuite(my_module_with_doctests))
unittest.TextTestRunner(verbosity = 2).run(testSuite)
I would recommend to use pytest --doctest-modules without any load_test protocol. You can simply add both the files or directories with your normal pytests and your modules with doctests to that pytest call.
pytest --doctest-modules path/to/pytest/unittests path/to/modules
It discovers and runs all doctests as well.
See https://docs.pytest.org/en/latest/doctest.html
This code will automatically run the doctests for all the modules in a package without needing to manually add a test suite for each module. This can be used with Tox.
import doctest
import glob
import os
import sys
if sys.version_info < (2,7,):
import unittest2 as unittest
else:
import unittest
import mypackage as source_package
def load_module_by_path(path):
"""Load a python module from its path.
Parameters
----------
path : str
Path to the module source file.
Returns
-------
mod : module
Loaded module.
"""
import imp
module_file_basename = os.path.basename(path)
module_name, ext = os.path.splitext(module_file_basename)
mod = imp.load_source(module_name, path)
return mod
def file_contains_doctests(path):
"""Scan a python source file to determine if it contains any doctest examples.
Parameters
----------
path : str
Path to the module source file.
Returns
-------
flag : bool
True if the module source code contains doctest examples.
"""
with open(path) as f:
for line in f:
if ">>>" in line:
return True
return False
def load_tests(loader, tests, pattern):
"""Run doctests for all modules"""
source_dir = os.path.dirname(source_package.__path__[0])
python_source_glob = os.path.join(source_dir, source_package.__name__, "*.py")
python_source_files = glob.glob(python_source_glob)
for python_source_file in python_source_files:
if not file_contains_doctests(python_source_file):
continue
module = load_module_by_path(python_source_file)
tests.addTests(doctest.DocTestSuite(module))
return tests
First I tried accepted answer from Andrey, but at least when running in Python 3.10 and python -m unittest discover it has led to running the test from unittest twice. Then I tried to simplify it and use load_tests and to my surprise it worked very well:
So just write both load_tests and normal unittest tests in a single file and it works!
import doctest
import unittest
import my_module_with_doctests
class ts(unittest.TestCase):
def test_null(self):
self.assertTrue(False)
# No need in any other extra code here
# Load doctests as unittest, see https://docs.python.org/3/library/doctest.html#unittest-api
def load_tests(loader, tests, ignore):
tests.addTests(doctest.DocTestSuite(my_module_with_doctests))
return tests
The zope.testing module provide such a functionality.
See
http://www.veit-schiele.de/dienstleistungen/schulungen/testen/doctests
for examples.
Related
I am writing test cases for a module module.py that imports from another module legacy.py. The legacy.py reads os.environ["some_var"] at module level. When I am trying to run test cases for module.py, they are failing with KeyError for some_var in os.environ.
This is how code looks like:
module.py
from legacy import db
def fun():
pass
legacy.py
import os
env = os.environ["some_var"]
class db:
def __init__(self):
pass
When running test cases for module.py, I am getting KeyError: 'some_var'.
I tried patching os (at module level also putting it before importing from module.py in test file) but the import statement is run before it could be patched. I tried looking for similar question on StackOverflow but didn't find the exact problem I am facing here.
How can I mock it? Or any other suggestion to be able to run the test cases. Assume that I cannot modify the legacy.py file.
You could use the mock.patch.dict. There is an example in official doc.
Here is a fully functional example based in your question.
module.py
import os
def get_env_var_value():
env_var = os.environ['PATH']
return env_var
print(get_env_var_value())
test_module.py
from unittest.mock import patch
from unittest import main, TestCase
import module
class MyClassTestCase(TestCase):
#patch.dict('os.environ', {'PATH': '/usr/sbin'})
def test_get_env_var_value(self):
self.assertEqual(module.get_env_var_value(), '/usr/sbin')
if __name__ == '__main__':
main()
Doesn't matter if the environment var is loaded in outside or inside the class/method scope, because you will mock for all test scope.
I'm doing a simple script to run and test my code. How can i import dinamically and run my test classes?
This is the solution that I found to import and dynamically run my test classes.
import glob
import os
import imp
import unittest
def execute_all_tests(tests_folder):
test_file_strings = glob.glob(os.path.join(tests_folder, 'test_*.py'))
suites = []
for test in test_file_strings:
mod_name, file_ext = os.path.splitext(os.path.split(test)[-1])
py_mod = imp.load_source(mod_name, test)
suites.append(unittest.defaultTestLoader.loadTestsFromModule(py_mod))
text_runner = unittest.TextTestRunner().run(unittest.TestSuite(suites))
Install pytest, and run your tests with a command like:
py.test src
That's it. Py.test will load all test_*.py files, find all def test_* calls inside them, and run each one for you.
The board is having trouble answering your question because it's in "why is water wet?" territory; all test rigs come with runners that automatically do what your code snip does, so you only need read the tutorial for one to get started.
And major props for writing auto tests at all; they put you above 75% of all programmers.
This solution is too simple and perform what i want.
import unittest
def execute_all_tests(tests_folder):
suites = unittest.TestLoader().discover(tests_folder)
text_runner = unittest.TextTestRunner().run(suites)
I have written a package (http://github.com/anntzer/parsedcmd) that runs with both Python2 and Python3. However, I had to write separate (py.test) unit tests for Python2 and Python3 (mainly because I want to test extra features of Python3, in particular keyword-only arguments), so I have a test_py2.py and a test_py3.py in a test subpackage. Now, if I run, say py.test2 mypkg, test_py2 passes, but test_py3 fails with a SyntaxError. Likewise, for py.test3 mypkg, test_py3 passes but test_py2 fails (I could make this one work though, it's just an issue of StringIO having moved to io).
I can design the test subpackage so that import mypkg.test only imports the proper version of the tests, but apparently py.test doesn't care -- it just sees two files matching test_* and grabs all tests in both of them, ignoring what __init__.py tells him to import.
So right now I have to do both py.test2 mypkg/test/test_py2.py and py.test3 mypkg/test/test_py3.py. Is there a way to set up the whole thing so that py.test2 mypkg and py.test3 mypkg would "just work"?
Thanks.
If you can then making your modules importable on all interpreters and skipping tests as appropriate is a common solution. Otherwise you can put the following as "conftest.py" into the test directory:
import sys
py3 = sys.version_info[0] >= 3
class DummyCollector(pytest.collect.File):
def collect(self):
return []
def pytest_pycollect_makemodule(path, parent):
bn = path.basename
if "py3" in bn and not py3 or ("py2" in bn and py3):
return DummyCollector(path, parent=parent)
This gets picked up a project-specific plugin and will properly ignore a test module with a filename containing a "py2" or "py3" substring on the wrong interpreter version. Of course you can refine it to rather have an explicit list directly in the conftest.py file instead of checking the filename etc.pp.
HTH, holger
You can put your tests in different packages and run only the tests in the appropriate package. Or you can load the appropriate test module in a script:
import sys, unittest
cur_version = sys.version_info
if cur_version[0] < 3:
import myApp.test.test_py2
unittest.TestLoader().loadTestsFromModule(myApp.test.test_py2).run()
else:
import myApp.test.test_py3
unittest.TestLoader().loadTestsFromModule(myApp.test.test_py3).run()
Alternatively, use a setup.py file so you can run:
python setup.py test
and put the versioning logic in there:
versionedTestSuite = "parsedcmd.test.test_py2" # do something as above here
setup(name='parsedcmd',
...
test_suite=versionedTestSuite,
)
A problem I continue to have it "bootstrapping" my tests.
The problem that I have is exactly what this guy has.
The top solution talks about creating a "boostrap" script. I presume that I must then enumerate all of the tests to be run, or use test manifests in the __init__.py files using the __all__ keyword. However, I noticed that the most recent Python documentation on unittest does not talk about __all__ anymore.
In 2.7, we have the python command called "discovery"
python -m unittest discover
That works even nicer. Because:
1) There's no need for Nose
2) There's no need for test manifests
But it doesn't seem to have a way to "bootstrap"
Do I need to use another test runner? One that allows bootstrapping AND discovery?
Do I need py.test?
http://pytest.org/
The reason that I need bootstrapping, is the problem that this guy has. Basically, my import statements don't work right if I run the test directly. I want to execute my suite of tests from the top of my project, just like the app would when it runs normally.
After all, import statements are always relative to their physical location. (BTW, I think this is a hindrance in Python)
Definition: What is Bootstrapping?
Bootstrapping means that I want to do some setup before running any tests at all in the entire project. This is sort of like me asking for a "test setup" at the whole project level.
Update
Here is another posting about the same thing. Using this 2.7 command, we can avoid Nose. But how does one add bootstrapping?
I got it!
Using this one script that I wrote and called it "runtests.py" and placed in my project root, I was able to "bootstrap" that is to run some initialization code AND use discovery. Woot!
In my case, the "bootstrap" code is the two lines that say:
import sys
sys.path.insert(0, 'lib.zip')
Thanks!
#!/usr/bin/python
import unittest
import sys
sys.path.insert(0, 'lib.zip')
if __name__ == "__main__":
all_tests = unittest.TestLoader().discover('.')
unittest.TextTestRunner().run(all_tests)
Here's what I do, and I think it works quite well. For a file/directory structure similar to this:
main_code.py
run_tests.py
/Modules
__init__.py
some_module1.py
some_module2.py
/Tests
__init__.py
test_module1.py
test_module2.py
It's fairly easy to organize your run_tests.py file to bootstrap the tests. First every file with test (test_module1.py, etc.) should implement a function that generates a test suite. Something like:
def suite():
suite = unittest.TestSuite()
suite.addTest(unittest.makeSuite(Test_Length))
suite.addTest(unittest.makeSuite(Test_Sum))
return suite
at the end of your test code. Then, in the run_tests.py file, you aggregate these into an additional test_suite, and run that:
import unittest
import Tests.test_module1 as test_module1
import Tests.test_module2 as test_module2
module1_test_suite = test_module1.suite()
module2_test_suite = test_module2.suite()
aggregate_suite = unittest.TestSuite()
aggregate_suite.addTest(module1_test_suite)
aggregate_suite.addTest(module2_test_suite)
unittest.TextTestsRunner(verbosity = 2).run(aggregate_suite
Then to run all of these tests, from the command line, simply run
python run_tests.py
I am developing a Python module with several source files, each with its own test class derived from unittest right in the source. Consider the directory structure:
dirFoo\
test.py
dirBar\
__init__.py
Foo.py
Bar.py
To test either Foo.py or Bar.py, I would add this at the end of the Foo.py and Bar.py source files:
if __name__ == "__main__":
unittest.main()
And run Python on either source, i.e.
$ python Foo.py
...........
----------------------------------------------------------------------
Ran 11 tests in 2.314s
OK
Ideally, I would have "test.py" automagically search dirBar for any unittest derived classes and make one call to "unittest.main()". What's the best way to do this in practice?
I tried using Python to call execfile for every *.py file in dirBar, which runs once for the first .py file found & exits the calling test.py, plus then I have to duplicate my code by adding unittest.main() in every source file--which violates DRY principles.
As of Python 2.7, test discovery is automated in the unittest package. From the docs:
Unittest supports simple test discovery. In order to be compatible
with test discovery, all of the test files must be modules or packages
importable from the top-level directory of the project (this means
that their filenames must be valid identifiers).
Test discovery is implemented in TestLoader.discover(), but can also
be used from the command line. The basic command-line usage is:
cd project_directory
python -m unittest discover
By default it looks for packages named test*.py, but this can be changed so you might use something like
python -m unittest discover --pattern=*.py
In place of your test.py script.
Here is my test discovery code that seems to do the job. I wanted to make sure I can extend the tests easily without having to list them in any of the involved files, but also avoid writing all tests in one single Übertest file.
So the structure is
myTests.py
testDir\
__init__.py
testA.py
testB.py
myTest.py look like this:
import unittest
if __name__ == '__main__':
testsuite = unittest.TestLoader().discover('.')
unittest.TextTestRunner(verbosity=1).run(testsuite)
I believe this is the simplest solution for writing several test cases in one directory. The solution requires Python 2.7 or Python 3.
I knew there was an obvious solution:
dirFoo\
__init__.py
test.py
dirBar\
__init__.py
Foo.py
Bar.py
Contents of dirFoo/test.py
from dirBar import *
import unittest
if __name__ == "__main__":
unittest.main()
Run the tests:
$ python test.py
...........
----------------------------------------------------------------------
Ran 11 tests in 2.305s
OK
You should try nose. It's a library to help create tests and it integrates with unittest or doctest. All you need to do is run nosetests and it'll find all your unittests for you.
% nosetests # finds all tests in all subdirectories
% nosetests tests/ # find all tests in the tests directory
In case it happens to help anyone, here is the approach I arrived at for solving this problem. I had the use case where I have the following directory structure:
mypackage/
tests/
test_category_1/
tests_1a.py
tests_1b.py
...
test_category_2/
tests_2a.py
tests_2b.py
...
...
and I want all of the following to work in the obvious way and to be able to be supplied the same commandline arguments as are accepted by unittest:
python -m mypackage.tests
python -m mypackage.tests.test_category_1
python -m mypackage.tests.test_category_1.tests_1a
The solution was to set up mypackage/tests/__init__.py like this:
import unittest
def prepare_load_tests_function (the__path__):
test_suite = unittest.TestLoader().discover(the__path__[0])
def load_tests (_a, _b, _c):
return test_suite
return load_tests
and to set up mypackage/tests/__main__.py like this:
import unittest
from . import prepare_load_tests_function, __path__
load_tests = prepare_load_tests_function(__path__)
unittest.main()
and to copy and paste an empty __init__.py and the following __main__.py in each mypackage/tests/test_category_n/:
import unittest
from .. import prepare_load_tests_function
from . import __path__
load_tests = prepare_load_tests_function(__path__)
unittest.main()
and also to add the standard if __name__ == '__main__': unittest.main() in each actual tests file.
(Works for me on Python 3.3 on Windows, ymmv.)
I came up with a snippet that may do what you want. It walks a path that you provide looking for Python packages/modules and accumulates a set of test suites from those modules, which it then executes all at once.
The nice thing about this is that it will work on all packages nested under the directory you specify, and you won't have to manually change the imports as you add new components.
import logging
import os
import unittest
MODULE_EXTENSIONS = set('.py .pyc .pyo'.split())
def unit_test_extractor(tup, path, filenames):
"""Pull ``unittest.TestSuite``s from modules in path
if the path represents a valid Python package. Accumulate
results in `tup[1]`.
"""
package_path, suites = tup
logging.debug('Path: %s', path)
logging.debug('Filenames: %s', filenames)
relpath = os.path.relpath(path, package_path)
relpath_pieces = relpath.split(os.sep)
if relpath_pieces[0] == '.': # Base directory.
relpath_pieces.pop(0) # Otherwise, screws up module name.
elif not any(os.path.exists(os.path.join(path, '__init__' + ext))
for ext in MODULE_EXTENSIONS):
return # Not a package directory and not the base directory, reject.
logging.info('Base: %s', '.'.join(relpath_pieces))
for filename in filenames:
base, ext = os.path.splitext(filename)
if ext not in MODULE_EXTENSIONS: # Not a Python module.
continue
logging.info('Module: %s', base)
module_name = '.'.join(relpath_pieces + [base])
logging.info('Importing from %s', module_name)
module = __import__(module_name)
module_suites = unittest.defaultTestLoader.loadTestsFromModule(module)
logging.info('Got suites: %s', module_suites)
suites += module_suites
def get_test_suites(path):
""":return: Iterable of suites for the packages/modules
present under :param:`path`.
"""
logging.info('Base path: %s', package_path)
suites = []
os.path.walk(package_path, unit_test_extractor, (package_path, suites))
logging.info('Got suites: %s', suites)
return suites
if __name__ == '__main__':
logging.basicConfig(level=logging.WARN)
package_path = os.path.dirname(os.path.abspath(__file__))
suites = get_test_suites(package_path)
for suite in suites:
unittest.TextTestRunner(verbosity=2).run(suite)