A problem I continue to have it "bootstrapping" my tests.
The problem that I have is exactly what this guy has.
The top solution talks about creating a "boostrap" script. I presume that I must then enumerate all of the tests to be run, or use test manifests in the __init__.py files using the __all__ keyword. However, I noticed that the most recent Python documentation on unittest does not talk about __all__ anymore.
In 2.7, we have the python command called "discovery"
python -m unittest discover
That works even nicer. Because:
1) There's no need for Nose
2) There's no need for test manifests
But it doesn't seem to have a way to "bootstrap"
Do I need to use another test runner? One that allows bootstrapping AND discovery?
Do I need py.test?
http://pytest.org/
The reason that I need bootstrapping, is the problem that this guy has. Basically, my import statements don't work right if I run the test directly. I want to execute my suite of tests from the top of my project, just like the app would when it runs normally.
After all, import statements are always relative to their physical location. (BTW, I think this is a hindrance in Python)
Definition: What is Bootstrapping?
Bootstrapping means that I want to do some setup before running any tests at all in the entire project. This is sort of like me asking for a "test setup" at the whole project level.
Update
Here is another posting about the same thing. Using this 2.7 command, we can avoid Nose. But how does one add bootstrapping?
I got it!
Using this one script that I wrote and called it "runtests.py" and placed in my project root, I was able to "bootstrap" that is to run some initialization code AND use discovery. Woot!
In my case, the "bootstrap" code is the two lines that say:
import sys
sys.path.insert(0, 'lib.zip')
Thanks!
#!/usr/bin/python
import unittest
import sys
sys.path.insert(0, 'lib.zip')
if __name__ == "__main__":
all_tests = unittest.TestLoader().discover('.')
unittest.TextTestRunner().run(all_tests)
Here's what I do, and I think it works quite well. For a file/directory structure similar to this:
main_code.py
run_tests.py
/Modules
__init__.py
some_module1.py
some_module2.py
/Tests
__init__.py
test_module1.py
test_module2.py
It's fairly easy to organize your run_tests.py file to bootstrap the tests. First every file with test (test_module1.py, etc.) should implement a function that generates a test suite. Something like:
def suite():
suite = unittest.TestSuite()
suite.addTest(unittest.makeSuite(Test_Length))
suite.addTest(unittest.makeSuite(Test_Sum))
return suite
at the end of your test code. Then, in the run_tests.py file, you aggregate these into an additional test_suite, and run that:
import unittest
import Tests.test_module1 as test_module1
import Tests.test_module2 as test_module2
module1_test_suite = test_module1.suite()
module2_test_suite = test_module2.suite()
aggregate_suite = unittest.TestSuite()
aggregate_suite.addTest(module1_test_suite)
aggregate_suite.addTest(module2_test_suite)
unittest.TextTestsRunner(verbosity = 2).run(aggregate_suite
Then to run all of these tests, from the command line, simply run
python run_tests.py
Related
I have a python script with main, and a whole lot of helper methods. How do I write unit tests for these helper methods which all reside in the same file. A lot of the examples I see online involve creating libraries of helpers and then importing and testing that. In my case it is all one file.
Sample structure:
/user/
|-- pythonscript.py
and I want to write tests for pythonscript.py.
Sample pythonscript.py:
def getSum(i ,j):
return i+j
def main():
summer = getSum(1,1)
if __name__ == '__main__':
main()
For example, I want to write tests for methods like getSum in another file. (I recall there was a tester which would screen .py files and identify functions needing tests based on appending test_ or _test, but I cannot seem to find it anymore.)
It sounds like you're describing pytest, which can automatically discover tests that follow certain naming and path conventions. One of these is considering every function prefixed with test to be a test.
For your scenario, I'd recommend creating a file called test_pythonscript.py also at the root of your directory. Inside that file, you could import functions from pythonscript.py, then test them:
# test_pythonscript.py
from pythonscript import getSum
test_getSum():
assert getSum(1, 2) == 3
Once you've installed pytest as a dependency, you'd run pytest at the root of your project and it should be able to discover the above test.
I am using Python's unittest with simple code like so:
suite = unittest.TestSuite()
suite.addTest(unittest.defaultTestLoader.loadTestsFromTestCase(module1))
suite.addTest(unittest.defaultTestLoader.loadTestsFromTestCase(module2))
I want my test suite to automatically parse all the modules and searches for all the unit test cases files that we have written?
for e.g.
there are 5 files,
1). f1.py
2). f2.py
3). f3.py
4). f4.py
5). f5.py
we dont know which of this file is the unit test case file. I want a way through which each file will be parsed and only the name of the module that has unit test cases should be returned
NOTE:- I am using python 2.6.6 so could not really make use of unittest.TestLoaded.discover()
Consider using the nose tool, it completely changes your unit-testing life. You just run it in the source folder root like:
> nosetests
then it automatically finds all the test cases.
If you want also run all the doctests, use:
> nosetests --with-doctest
In case if you only want to find a list of modules programmatically, nose provides some API (unfortunately, not as convenient as TestLoader.discover()).
UPDATE: I've just discovered (pun intended) that there is a library called unittest2 that backports all the later unittest features to the earlier versions of Python. I'll keep the code below for the archaeologists, but I think, unittest2 is a better way to go.
import nose.loader
import nose.suite
import types
def _iter_modules(tests):
'''
Recursively find all the modules containing tests.
(Some may repeat)
'''
for item in tests:
if isinstance(item, nose.suite.ContextSuite):
for t in _iter_modules(item):
yield t
elif isinstance(item.context, types.ModuleType):
yield item.context.__name__
else:
yield item.context.__module__
def find_test_modules(basedir):
'''
Get a list of all the modules that contain tests.
'''
loader = nose.loader.TestLoader()
tests = loader.loadTestsFromDir(basedir)
modules = list(set(_iter_modules(tests))) # remove duplicates
return modules
Use the discovery feature of the unittest library:
$ python -m unittest discover --start-directory my_project
I have written a package (http://github.com/anntzer/parsedcmd) that runs with both Python2 and Python3. However, I had to write separate (py.test) unit tests for Python2 and Python3 (mainly because I want to test extra features of Python3, in particular keyword-only arguments), so I have a test_py2.py and a test_py3.py in a test subpackage. Now, if I run, say py.test2 mypkg, test_py2 passes, but test_py3 fails with a SyntaxError. Likewise, for py.test3 mypkg, test_py3 passes but test_py2 fails (I could make this one work though, it's just an issue of StringIO having moved to io).
I can design the test subpackage so that import mypkg.test only imports the proper version of the tests, but apparently py.test doesn't care -- it just sees two files matching test_* and grabs all tests in both of them, ignoring what __init__.py tells him to import.
So right now I have to do both py.test2 mypkg/test/test_py2.py and py.test3 mypkg/test/test_py3.py. Is there a way to set up the whole thing so that py.test2 mypkg and py.test3 mypkg would "just work"?
Thanks.
If you can then making your modules importable on all interpreters and skipping tests as appropriate is a common solution. Otherwise you can put the following as "conftest.py" into the test directory:
import sys
py3 = sys.version_info[0] >= 3
class DummyCollector(pytest.collect.File):
def collect(self):
return []
def pytest_pycollect_makemodule(path, parent):
bn = path.basename
if "py3" in bn and not py3 or ("py2" in bn and py3):
return DummyCollector(path, parent=parent)
This gets picked up a project-specific plugin and will properly ignore a test module with a filename containing a "py2" or "py3" substring on the wrong interpreter version. Of course you can refine it to rather have an explicit list directly in the conftest.py file instead of checking the filename etc.pp.
HTH, holger
You can put your tests in different packages and run only the tests in the appropriate package. Or you can load the appropriate test module in a script:
import sys, unittest
cur_version = sys.version_info
if cur_version[0] < 3:
import myApp.test.test_py2
unittest.TestLoader().loadTestsFromModule(myApp.test.test_py2).run()
else:
import myApp.test.test_py3
unittest.TestLoader().loadTestsFromModule(myApp.test.test_py3).run()
Alternatively, use a setup.py file so you can run:
python setup.py test
and put the versioning logic in there:
versionedTestSuite = "parsedcmd.test.test_py2" # do something as above here
setup(name='parsedcmd',
...
test_suite=versionedTestSuite,
)
What is the best way to create a command for setuptools which generates a code coverage report using coverage.py?
What you are looking for is Extending Distutils capabilities through extensions and it is covered in the docs that I have linked. The basic idea is to give the command and the entr5y point for execution and the entry point should follow some of setuptools Component based terminology. I think, you are luck here, because someone has already tried successfully ( Adding Test Code Coverage Analysis to a Python Project's setup Command ) integrating it for his project and you should be able to adopt it for your purposes.
Here is one simple solution which uses subprocess calls to the coverage executable. I assume you have a Python package called mycoolpackage which contains the code for which you want to measure test coverage, and you have a mytests package which exposes a suite function which returns the test suite.
First, create a run-tests.py file:
import os.path, sys
sys.path.insert(0, os.path.abspath(os.path.dirname(__file__)))
import unittest
from mytests import suite
unittest.main(defaultTest='suite')
then in setup.py create the following command and add it to the cmdclass argument of the setup() function.
class run_coverage(Command):
description = "Generate a test coverage report."
user_options = []
def initialize_options(self): pass
def finalize_options(self): pass
def run(self):
import subprocess
subprocess.call(['coverage', 'run', '--source=mycoolpackage', 'run-tests.py'])
subprocess.call(['coverage', 'html'])
Note that coverage.py does have an API which would allow you to accomplish this without using a subprocess call.
Here's what I want to do: I want to build a test suite that's organized into packages like tests.ui, tests.text, tests.fileio, etc. In each __init__.py in these packages, I want to make a test suite consisting of all the tests in all the modules in that package. Of course, getting all the tests can be done with unittest.TestLoader, but it seems that I have to add each module individually. So supposing that test.ui has editor_window_test.py and preview_window_test.py, I want the __init__.py to import these two files and get a list of the two module objects. The idea is that I want to automate making the test suites so that I can't forget to include something in the test suite.
What's the best way to do this? It seems like it would be an easy thing to do, but I'm not finding anything.
I'm using Python 2.5 btw.
Good answers here, but the best thing to do would be to use a 3rd party test discovery and runner like:
Nose (my favourite)
Trial (pretty nice, especially when testing async stuff)
py.test (less good, in my opinion)
They are all compatible with plain unittest.TestCase and you won't have to modify your tests in any way, neither would you have to use the advanced features in any of them. Just use as a suite discovery.
Is there a specific reason you want to reinvent the nasty stuff in these libs?
Solution to exactly this problem from our django project:
"""Test loader for all module tests
"""
import unittest
import re, os, imp, sys
def find_modules(package):
files = [re.sub('\.py$', '', f) for f in os.listdir(os.path.dirname(package.__file__))
if f.endswith(".py")]
return [imp.load_module(file, *imp.find_module(file, package.__path__)) for file in files]
def suite(package=None):
"""Assemble test suite for Django default test loader"""
if not package: package = myapp.tests # Default argument required for Django test runner
return unittest.TestSuite([unittest.TestLoader().loadTestsFromModule(m)
for m in find_modules(package)])
if __name__ == '__main__':
unittest.TextTestRunner().run(suite(myapp.tests))
EDIT: The benefit compared to bialix's solution is that you can place this loader anytwhere in the project tree, there's no need to modify init.py in every test directory.
You can use os.listdir to find all files in the test.* directory and then filter out .py files:
# Place this code to your __init__.py in test.* directory
import os
modules = []
for name in os.listdir(os.path.dirname(os.path.abspath(__file__))):
m, ext = os.path.splitext()
if ext == '.py':
modules.append(__import__(m))
__all__ = modules
The magic variable __file__ contains filepath of the current module. Try
print __file__
to check.