Here's what I want to do: I want to build a test suite that's organized into packages like tests.ui, tests.text, tests.fileio, etc. In each __init__.py in these packages, I want to make a test suite consisting of all the tests in all the modules in that package. Of course, getting all the tests can be done with unittest.TestLoader, but it seems that I have to add each module individually. So supposing that test.ui has editor_window_test.py and preview_window_test.py, I want the __init__.py to import these two files and get a list of the two module objects. The idea is that I want to automate making the test suites so that I can't forget to include something in the test suite.
What's the best way to do this? It seems like it would be an easy thing to do, but I'm not finding anything.
I'm using Python 2.5 btw.
Good answers here, but the best thing to do would be to use a 3rd party test discovery and runner like:
Nose (my favourite)
Trial (pretty nice, especially when testing async stuff)
py.test (less good, in my opinion)
They are all compatible with plain unittest.TestCase and you won't have to modify your tests in any way, neither would you have to use the advanced features in any of them. Just use as a suite discovery.
Is there a specific reason you want to reinvent the nasty stuff in these libs?
Solution to exactly this problem from our django project:
"""Test loader for all module tests
"""
import unittest
import re, os, imp, sys
def find_modules(package):
files = [re.sub('\.py$', '', f) for f in os.listdir(os.path.dirname(package.__file__))
if f.endswith(".py")]
return [imp.load_module(file, *imp.find_module(file, package.__path__)) for file in files]
def suite(package=None):
"""Assemble test suite for Django default test loader"""
if not package: package = myapp.tests # Default argument required for Django test runner
return unittest.TestSuite([unittest.TestLoader().loadTestsFromModule(m)
for m in find_modules(package)])
if __name__ == '__main__':
unittest.TextTestRunner().run(suite(myapp.tests))
EDIT: The benefit compared to bialix's solution is that you can place this loader anytwhere in the project tree, there's no need to modify init.py in every test directory.
You can use os.listdir to find all files in the test.* directory and then filter out .py files:
# Place this code to your __init__.py in test.* directory
import os
modules = []
for name in os.listdir(os.path.dirname(os.path.abspath(__file__))):
m, ext = os.path.splitext()
if ext == '.py':
modules.append(__import__(m))
__all__ = modules
The magic variable __file__ contains filepath of the current module. Try
print __file__
to check.
Related
I am writing a pytest plugin that should test software that's designed to work inside a set of specific environments.
The software I'm writing is run inside a bigger framework, which makes certain Python modules available only when running my Python software inside the framework.
In order to test my software, I'm required to "mock" or fake an entire module (actually, quite a few). I'll need to implement its functionality in some kind of similar-looking way, but my question is how should I make this fake Python module available to my software's code, using a py.test plugin?
For example, let's assume I have the following code in one of my source files:
import fwlib
def fw_sum(a, b):
return fwlib.sum(a, b)
However, the fwlib module is only made available by the framework I run my software from, and I cannot test inside it.
How would I make sure, from within a pytest plugin, that a module named fwlib is already defined in sys.modules? Granted, I'll need to implement fwlib.sum myself. I'm looking for recommendations on how to do just that.
pytest provides a fixture for this use-case: monkeypatch.syspath_prepend.
You may prepend a path to sys.path list of import locations. Write a fake fwlib.py and include it in your tests, appending the directory as necessary. Like the other test modules, it needn't be included with the distribution.
After playing with this myself, I couldn't actually figure out how to get the fixture to mock module level imports correctly from the library code. By the time the tests run, the library code was already imported and then it is too late to patch.
However, I can offer a different solution that works: you may inject the name from within conftest.py, which gets imported first. The subsequent import statement within the code under test will just re-use the object already present in sys.modules.
Package structure:
$ tree .
.
├── conftest.py
├── lib
│ └── my_lib.py
└── tests
└── test_my_lib.py
2 directories, 3 files
Contents of files:
# conftest.py
import sys
def fwlib_sum(a, b):
return a + b
module = type(sys)('fwlib')
module.sum = fwlib_sum
sys.modules['fwlib'] = module
library file:
# lib/my_lib.py
import fwlib
def fw_sum(a, b):
return fwlib.sum(a, b)
test file:
# lib/test_my_lib.py
import my_lib
def test_sum():
assert my_lib.fw_sum(1, 2) == 3
Just to provide a little more details to #wim's good answer, you can use it with submodules too, like so:
import sys
module = type(sys)("my_module_name")
module.submodule = type(sys)("my_submodule_name")
module.submodule.something = sommething
sys.modules["my_module_name"] = module
sys.modules["my_module_name.my_submodule_name"] = module.submodule
I am using Python's unittest with simple code like so:
suite = unittest.TestSuite()
suite.addTest(unittest.defaultTestLoader.loadTestsFromTestCase(module1))
suite.addTest(unittest.defaultTestLoader.loadTestsFromTestCase(module2))
I want my test suite to automatically parse all the modules and searches for all the unit test cases files that we have written?
for e.g.
there are 5 files,
1). f1.py
2). f2.py
3). f3.py
4). f4.py
5). f5.py
we dont know which of this file is the unit test case file. I want a way through which each file will be parsed and only the name of the module that has unit test cases should be returned
NOTE:- I am using python 2.6.6 so could not really make use of unittest.TestLoaded.discover()
Consider using the nose tool, it completely changes your unit-testing life. You just run it in the source folder root like:
> nosetests
then it automatically finds all the test cases.
If you want also run all the doctests, use:
> nosetests --with-doctest
In case if you only want to find a list of modules programmatically, nose provides some API (unfortunately, not as convenient as TestLoader.discover()).
UPDATE: I've just discovered (pun intended) that there is a library called unittest2 that backports all the later unittest features to the earlier versions of Python. I'll keep the code below for the archaeologists, but I think, unittest2 is a better way to go.
import nose.loader
import nose.suite
import types
def _iter_modules(tests):
'''
Recursively find all the modules containing tests.
(Some may repeat)
'''
for item in tests:
if isinstance(item, nose.suite.ContextSuite):
for t in _iter_modules(item):
yield t
elif isinstance(item.context, types.ModuleType):
yield item.context.__name__
else:
yield item.context.__module__
def find_test_modules(basedir):
'''
Get a list of all the modules that contain tests.
'''
loader = nose.loader.TestLoader()
tests = loader.loadTestsFromDir(basedir)
modules = list(set(_iter_modules(tests))) # remove duplicates
return modules
Use the discovery feature of the unittest library:
$ python -m unittest discover --start-directory my_project
I have written a package (http://github.com/anntzer/parsedcmd) that runs with both Python2 and Python3. However, I had to write separate (py.test) unit tests for Python2 and Python3 (mainly because I want to test extra features of Python3, in particular keyword-only arguments), so I have a test_py2.py and a test_py3.py in a test subpackage. Now, if I run, say py.test2 mypkg, test_py2 passes, but test_py3 fails with a SyntaxError. Likewise, for py.test3 mypkg, test_py3 passes but test_py2 fails (I could make this one work though, it's just an issue of StringIO having moved to io).
I can design the test subpackage so that import mypkg.test only imports the proper version of the tests, but apparently py.test doesn't care -- it just sees two files matching test_* and grabs all tests in both of them, ignoring what __init__.py tells him to import.
So right now I have to do both py.test2 mypkg/test/test_py2.py and py.test3 mypkg/test/test_py3.py. Is there a way to set up the whole thing so that py.test2 mypkg and py.test3 mypkg would "just work"?
Thanks.
If you can then making your modules importable on all interpreters and skipping tests as appropriate is a common solution. Otherwise you can put the following as "conftest.py" into the test directory:
import sys
py3 = sys.version_info[0] >= 3
class DummyCollector(pytest.collect.File):
def collect(self):
return []
def pytest_pycollect_makemodule(path, parent):
bn = path.basename
if "py3" in bn and not py3 or ("py2" in bn and py3):
return DummyCollector(path, parent=parent)
This gets picked up a project-specific plugin and will properly ignore a test module with a filename containing a "py2" or "py3" substring on the wrong interpreter version. Of course you can refine it to rather have an explicit list directly in the conftest.py file instead of checking the filename etc.pp.
HTH, holger
You can put your tests in different packages and run only the tests in the appropriate package. Or you can load the appropriate test module in a script:
import sys, unittest
cur_version = sys.version_info
if cur_version[0] < 3:
import myApp.test.test_py2
unittest.TestLoader().loadTestsFromModule(myApp.test.test_py2).run()
else:
import myApp.test.test_py3
unittest.TestLoader().loadTestsFromModule(myApp.test.test_py3).run()
Alternatively, use a setup.py file so you can run:
python setup.py test
and put the versioning logic in there:
versionedTestSuite = "parsedcmd.test.test_py2" # do something as above here
setup(name='parsedcmd',
...
test_suite=versionedTestSuite,
)
A problem I continue to have it "bootstrapping" my tests.
The problem that I have is exactly what this guy has.
The top solution talks about creating a "boostrap" script. I presume that I must then enumerate all of the tests to be run, or use test manifests in the __init__.py files using the __all__ keyword. However, I noticed that the most recent Python documentation on unittest does not talk about __all__ anymore.
In 2.7, we have the python command called "discovery"
python -m unittest discover
That works even nicer. Because:
1) There's no need for Nose
2) There's no need for test manifests
But it doesn't seem to have a way to "bootstrap"
Do I need to use another test runner? One that allows bootstrapping AND discovery?
Do I need py.test?
http://pytest.org/
The reason that I need bootstrapping, is the problem that this guy has. Basically, my import statements don't work right if I run the test directly. I want to execute my suite of tests from the top of my project, just like the app would when it runs normally.
After all, import statements are always relative to their physical location. (BTW, I think this is a hindrance in Python)
Definition: What is Bootstrapping?
Bootstrapping means that I want to do some setup before running any tests at all in the entire project. This is sort of like me asking for a "test setup" at the whole project level.
Update
Here is another posting about the same thing. Using this 2.7 command, we can avoid Nose. But how does one add bootstrapping?
I got it!
Using this one script that I wrote and called it "runtests.py" and placed in my project root, I was able to "bootstrap" that is to run some initialization code AND use discovery. Woot!
In my case, the "bootstrap" code is the two lines that say:
import sys
sys.path.insert(0, 'lib.zip')
Thanks!
#!/usr/bin/python
import unittest
import sys
sys.path.insert(0, 'lib.zip')
if __name__ == "__main__":
all_tests = unittest.TestLoader().discover('.')
unittest.TextTestRunner().run(all_tests)
Here's what I do, and I think it works quite well. For a file/directory structure similar to this:
main_code.py
run_tests.py
/Modules
__init__.py
some_module1.py
some_module2.py
/Tests
__init__.py
test_module1.py
test_module2.py
It's fairly easy to organize your run_tests.py file to bootstrap the tests. First every file with test (test_module1.py, etc.) should implement a function that generates a test suite. Something like:
def suite():
suite = unittest.TestSuite()
suite.addTest(unittest.makeSuite(Test_Length))
suite.addTest(unittest.makeSuite(Test_Sum))
return suite
at the end of your test code. Then, in the run_tests.py file, you aggregate these into an additional test_suite, and run that:
import unittest
import Tests.test_module1 as test_module1
import Tests.test_module2 as test_module2
module1_test_suite = test_module1.suite()
module2_test_suite = test_module2.suite()
aggregate_suite = unittest.TestSuite()
aggregate_suite.addTest(module1_test_suite)
aggregate_suite.addTest(module2_test_suite)
unittest.TextTestsRunner(verbosity = 2).run(aggregate_suite
Then to run all of these tests, from the command line, simply run
python run_tests.py
My program is broken down into two parts: the engine, which deals with user interface and other "main program" stuff, and a set of plugins, which provide methods to deal with specific input.
Each plugin is written in its own module, and provides a function that will allow me to send and retrieve data to and from the plugin.
The name of this function is the same across all plugins, so all I need is to determine which one to call and then the plugin will handle the rest.
I've placed all of the plugins in a sub-folder, wrote an __ init__.py that imports each plugin, and then I import the folder (I think it's called a package?)
Anyways currently I explicitly tell it what to import (which is basically "import this", "import that"). Is there a way for me to write it so that it will import everything in that folder that is a plug-in so that I can add additional plugins without having to edit the init file?
Here is the code I use to do this:
def _loadPackagePlugins(package):
"Load plugins from a specified package."
ppath = package.__path__
pname = package.__name__ + "."
for importer, modname, ispkg in pkgutil.iter_modules(ppath, pname):
module = __import__(modname, fromlist = "dummy")
The main difference from Jakob's answer is that it uses pkgutil.iter_modules instead of os.listdir. I used to use os.listdir and changed to doing it this way, but I don't remember why. It might have been that os.listdir failed when I packaged my app with py2exe and py2app.
You could always have a dict called plugins, use __import__ to import the modules and store them that way.
e.g.
plugins = {}
for plugin in os.listdir('plugins'):
plugin = plugin.split()[0]
plugins[plugin] = __import__(plugin)
This is assuming that every plugin is a single file. Personally I would go with something that looks in each folder for a __run__.py file, like a __init__.py in a package it would indicate a plugin, that code would look more like something like this
for root, dirs, files in os.walk('.'):
for dir in dirs:
if "__run__.py" in os.listdir(os.path.join(root, dir)):
plugins[dir] = __import__(dir)
Code written without testing. YMMV