I have written a package (http://github.com/anntzer/parsedcmd) that runs with both Python2 and Python3. However, I had to write separate (py.test) unit tests for Python2 and Python3 (mainly because I want to test extra features of Python3, in particular keyword-only arguments), so I have a test_py2.py and a test_py3.py in a test subpackage. Now, if I run, say py.test2 mypkg, test_py2 passes, but test_py3 fails with a SyntaxError. Likewise, for py.test3 mypkg, test_py3 passes but test_py2 fails (I could make this one work though, it's just an issue of StringIO having moved to io).
I can design the test subpackage so that import mypkg.test only imports the proper version of the tests, but apparently py.test doesn't care -- it just sees two files matching test_* and grabs all tests in both of them, ignoring what __init__.py tells him to import.
So right now I have to do both py.test2 mypkg/test/test_py2.py and py.test3 mypkg/test/test_py3.py. Is there a way to set up the whole thing so that py.test2 mypkg and py.test3 mypkg would "just work"?
Thanks.
If you can then making your modules importable on all interpreters and skipping tests as appropriate is a common solution. Otherwise you can put the following as "conftest.py" into the test directory:
import sys
py3 = sys.version_info[0] >= 3
class DummyCollector(pytest.collect.File):
def collect(self):
return []
def pytest_pycollect_makemodule(path, parent):
bn = path.basename
if "py3" in bn and not py3 or ("py2" in bn and py3):
return DummyCollector(path, parent=parent)
This gets picked up a project-specific plugin and will properly ignore a test module with a filename containing a "py2" or "py3" substring on the wrong interpreter version. Of course you can refine it to rather have an explicit list directly in the conftest.py file instead of checking the filename etc.pp.
HTH, holger
You can put your tests in different packages and run only the tests in the appropriate package. Or you can load the appropriate test module in a script:
import sys, unittest
cur_version = sys.version_info
if cur_version[0] < 3:
import myApp.test.test_py2
unittest.TestLoader().loadTestsFromModule(myApp.test.test_py2).run()
else:
import myApp.test.test_py3
unittest.TestLoader().loadTestsFromModule(myApp.test.test_py3).run()
Alternatively, use a setup.py file so you can run:
python setup.py test
and put the versioning logic in there:
versionedTestSuite = "parsedcmd.test.test_py2" # do something as above here
setup(name='parsedcmd',
...
test_suite=versionedTestSuite,
)
Related
I have a Python module that tries to import a module, and if it fails, adds sets up a lightweight class to replicate some of the functionality as a fallback.
How can I write a unit test that covers both cases - when the module is present, and when the module is not present?
try:
from foo import bar
except ImportError:
class bar(object):
def doAThing(self):
return "blah"
For example, if bar is present, the unit tests will report that only the 2nd line is covered, and the rest of the module isn't. Python's coverage will report missing tests on the line with "except" to the end.
Modules are loaded from the sys.path array. If you clear that array then any non-core import will fail. The following commands were run in ipython in a virtual env with access to django
import sys
sys.path = []
import django
... ImportError ...
import os
... No ImportError ...
Your other alternative is to use two virtual environments. One which has the module and one which does not. That would make it slightly more difficult to run the tests though.
If this doesn't work then you can add a path to the start of the sys.path that has a module that will produce an ImportError when loaded. This works because the first match found when searching the sys.path is loaded, so this can replace the real module. The following is sufficient:
test.py
import sys
sys.path.insert(0, '.')
import foo
foo/__init__.py
raise ImportError()
I am using Python's unittest with simple code like so:
suite = unittest.TestSuite()
suite.addTest(unittest.defaultTestLoader.loadTestsFromTestCase(module1))
suite.addTest(unittest.defaultTestLoader.loadTestsFromTestCase(module2))
I want my test suite to automatically parse all the modules and searches for all the unit test cases files that we have written?
for e.g.
there are 5 files,
1). f1.py
2). f2.py
3). f3.py
4). f4.py
5). f5.py
we dont know which of this file is the unit test case file. I want a way through which each file will be parsed and only the name of the module that has unit test cases should be returned
NOTE:- I am using python 2.6.6 so could not really make use of unittest.TestLoaded.discover()
Consider using the nose tool, it completely changes your unit-testing life. You just run it in the source folder root like:
> nosetests
then it automatically finds all the test cases.
If you want also run all the doctests, use:
> nosetests --with-doctest
In case if you only want to find a list of modules programmatically, nose provides some API (unfortunately, not as convenient as TestLoader.discover()).
UPDATE: I've just discovered (pun intended) that there is a library called unittest2 that backports all the later unittest features to the earlier versions of Python. I'll keep the code below for the archaeologists, but I think, unittest2 is a better way to go.
import nose.loader
import nose.suite
import types
def _iter_modules(tests):
'''
Recursively find all the modules containing tests.
(Some may repeat)
'''
for item in tests:
if isinstance(item, nose.suite.ContextSuite):
for t in _iter_modules(item):
yield t
elif isinstance(item.context, types.ModuleType):
yield item.context.__name__
else:
yield item.context.__module__
def find_test_modules(basedir):
'''
Get a list of all the modules that contain tests.
'''
loader = nose.loader.TestLoader()
tests = loader.loadTestsFromDir(basedir)
modules = list(set(_iter_modules(tests))) # remove duplicates
return modules
Use the discovery feature of the unittest library:
$ python -m unittest discover --start-directory my_project
A problem I continue to have it "bootstrapping" my tests.
The problem that I have is exactly what this guy has.
The top solution talks about creating a "boostrap" script. I presume that I must then enumerate all of the tests to be run, or use test manifests in the __init__.py files using the __all__ keyword. However, I noticed that the most recent Python documentation on unittest does not talk about __all__ anymore.
In 2.7, we have the python command called "discovery"
python -m unittest discover
That works even nicer. Because:
1) There's no need for Nose
2) There's no need for test manifests
But it doesn't seem to have a way to "bootstrap"
Do I need to use another test runner? One that allows bootstrapping AND discovery?
Do I need py.test?
http://pytest.org/
The reason that I need bootstrapping, is the problem that this guy has. Basically, my import statements don't work right if I run the test directly. I want to execute my suite of tests from the top of my project, just like the app would when it runs normally.
After all, import statements are always relative to their physical location. (BTW, I think this is a hindrance in Python)
Definition: What is Bootstrapping?
Bootstrapping means that I want to do some setup before running any tests at all in the entire project. This is sort of like me asking for a "test setup" at the whole project level.
Update
Here is another posting about the same thing. Using this 2.7 command, we can avoid Nose. But how does one add bootstrapping?
I got it!
Using this one script that I wrote and called it "runtests.py" and placed in my project root, I was able to "bootstrap" that is to run some initialization code AND use discovery. Woot!
In my case, the "bootstrap" code is the two lines that say:
import sys
sys.path.insert(0, 'lib.zip')
Thanks!
#!/usr/bin/python
import unittest
import sys
sys.path.insert(0, 'lib.zip')
if __name__ == "__main__":
all_tests = unittest.TestLoader().discover('.')
unittest.TextTestRunner().run(all_tests)
Here's what I do, and I think it works quite well. For a file/directory structure similar to this:
main_code.py
run_tests.py
/Modules
__init__.py
some_module1.py
some_module2.py
/Tests
__init__.py
test_module1.py
test_module2.py
It's fairly easy to organize your run_tests.py file to bootstrap the tests. First every file with test (test_module1.py, etc.) should implement a function that generates a test suite. Something like:
def suite():
suite = unittest.TestSuite()
suite.addTest(unittest.makeSuite(Test_Length))
suite.addTest(unittest.makeSuite(Test_Sum))
return suite
at the end of your test code. Then, in the run_tests.py file, you aggregate these into an additional test_suite, and run that:
import unittest
import Tests.test_module1 as test_module1
import Tests.test_module2 as test_module2
module1_test_suite = test_module1.suite()
module2_test_suite = test_module2.suite()
aggregate_suite = unittest.TestSuite()
aggregate_suite.addTest(module1_test_suite)
aggregate_suite.addTest(module2_test_suite)
unittest.TextTestsRunner(verbosity = 2).run(aggregate_suite
Then to run all of these tests, from the command line, simply run
python run_tests.py
Ok, I think whatever I'm doing wrong, it's probably blindingly obvious, but I can't figure it out. I've read and re-read the tutorial section on packages and the only thing I can figure is that this won't work because I'm executing it directly. Here's the directory setup:
eulerproject/
__init__.py
euler1.py
euler2.py
...
eulern.py
tests/
__init__.py
testeulern.py
Here are the contents of testeuler12.py (the first test module I've written):
import unittest
from .. import euler12
class Euler12UnitTests(unittest.TestCase):
def testtriangle(self):
"""
Ensure that the triangle number generator returns the first 10
triangle numbers.
"""
self.seq = [1,3,6,10,15,21,28,36,45,55]
self.generator = euler12.trianglegenerator()
self.results = []
while len(self.results) != 10:
self.results.append(self.generator.next())
self.assertEqual(self.seq, self.results)
def testdivisors(self):
"""
Ensure that the divisors function can properly factor the number 28.
"""
self.number = 28
self.answer = [1,2,4,7,14,28]
self.assertEqual(self.answer, euler12.divisors(self.number))
if __name__ == '__main__':
unittest.main()
Now, when I execute this from IDLE and from the command line while in the directory, I get the following error:
Traceback (most recent call last):
File "C:\Documents and Settings\jbennet\My Documents\Python\eulerproject\tests\testeuler12.py", line 2, in <module>
from .. import euler12
ValueError: Attempted relative import in non-package
I think the problem is that since I'm running it directly, I can't do relative imports (because __name__ changes, and my vague understanding of the packages description is that __name__ is part of how it tells what package it's in), but in that case what do you guys suggest for how to import the 'production' code stored 1 level up from the test code?
I had the same problem. I now use nose to run my tests, and relative imports are correctly handled.
Yeah, this whole relative import thing is confusing.
Generally you would have a directory, the name of which is your package name, somewhere on your PYTHONPATH. For example:
eulerproject/
euler/
__init__.py
euler1.py
...
tests/
...
setup.py
Then, you can either install this systemwide, or make sure to set PYTHONPATH=/path/to/eulerproject/:$PYTHONPATH when invoking your script.
An absolute import like this will then work:
from euler import euler1
Edit:
According to the Python docs, "modules intended for use as the main module of a Python application should always use absolute imports." (Cite)
So a test harness like nose, mentioned by the other answer, works because it imports packages rather than running them from the command line.
If you want to do things by hand, your runnable script needs to be outside the package hierarchy, like this:
eulerproject/
runtests.py
euler/
__init__.py
euler1.py
...
tests/
__init__.py
testeulern.py
Now, runtests.py can do from euler.tests.testeulern import TestCase and testeulern.py can do from .. import euler1
Here's what I want to do: I want to build a test suite that's organized into packages like tests.ui, tests.text, tests.fileio, etc. In each __init__.py in these packages, I want to make a test suite consisting of all the tests in all the modules in that package. Of course, getting all the tests can be done with unittest.TestLoader, but it seems that I have to add each module individually. So supposing that test.ui has editor_window_test.py and preview_window_test.py, I want the __init__.py to import these two files and get a list of the two module objects. The idea is that I want to automate making the test suites so that I can't forget to include something in the test suite.
What's the best way to do this? It seems like it would be an easy thing to do, but I'm not finding anything.
I'm using Python 2.5 btw.
Good answers here, but the best thing to do would be to use a 3rd party test discovery and runner like:
Nose (my favourite)
Trial (pretty nice, especially when testing async stuff)
py.test (less good, in my opinion)
They are all compatible with plain unittest.TestCase and you won't have to modify your tests in any way, neither would you have to use the advanced features in any of them. Just use as a suite discovery.
Is there a specific reason you want to reinvent the nasty stuff in these libs?
Solution to exactly this problem from our django project:
"""Test loader for all module tests
"""
import unittest
import re, os, imp, sys
def find_modules(package):
files = [re.sub('\.py$', '', f) for f in os.listdir(os.path.dirname(package.__file__))
if f.endswith(".py")]
return [imp.load_module(file, *imp.find_module(file, package.__path__)) for file in files]
def suite(package=None):
"""Assemble test suite for Django default test loader"""
if not package: package = myapp.tests # Default argument required for Django test runner
return unittest.TestSuite([unittest.TestLoader().loadTestsFromModule(m)
for m in find_modules(package)])
if __name__ == '__main__':
unittest.TextTestRunner().run(suite(myapp.tests))
EDIT: The benefit compared to bialix's solution is that you can place this loader anytwhere in the project tree, there's no need to modify init.py in every test directory.
You can use os.listdir to find all files in the test.* directory and then filter out .py files:
# Place this code to your __init__.py in test.* directory
import os
modules = []
for name in os.listdir(os.path.dirname(os.path.abspath(__file__))):
m, ext = os.path.splitext()
if ext == '.py':
modules.append(__import__(m))
__all__ = modules
The magic variable __file__ contains filepath of the current module. Try
print __file__
to check.