We Keep Coding

py.test skips test class if constructor is defined

I have following unittest code running via py.test.
Mere presence of the constructor make the entire class skip when running
py.test -v -s
collected 0 items / 1 skipped
Can anyone please explain to me this behaviour of py.test?
I am interested in understanding py.test behaviour, I know the constructor is not needed.
Thanks,
Zdenek
class TestClassName(object):
def __init__(self):
pass
def setup_method(self, method):
print "setup_method called"
def teardown_method(self, method):
print "teardown_method called"
def test_a(self):
print "test_a called"
assert 1 == 1
def test_b(self):
print "test_b called"
assert 1 == 1

The documentation for py.test says that py.test implements the following standard test discovery:
collection starts from the initial command line arguments which may be directories, filenames or test ids.
recurse into directories, unless they match norecursedirs
test_*.py or *_test.py files, imported by their package name.
Test prefixed test classes (without an __init__ method) [<-- notice this one here]
test_ prefixed test functions or methods are test items
So it's not that the constructor isn't needed, py.test just ignores classes that have a constructor. There is also a guide for changing the standard test discovery.

As already mentioned in the answer by Matti Lyra py.test purposely skips classes which have a constructor. The reason for this is that classes are only used for structural reasons in py.test and do not have any inherent behaviour, while when actually writing code it is the opposite and much rarer to not have an .__init__() method for a class. So in practice skipping a class with a constructor will likely be what was desired, usually it is just a class which happens to have a conflicting name.
Lastly py.test needs to instantiate the class in order to execute the tests. If the constructor takes any arguments it can't instantiate it, so again skipping is the right thing to do.

All the above answers clearly explain the underlying cause, I just thought to share my experience and workaround the warnings.
I got my test to work without the warnings by aliasing the imported Class
from app.core.utils import model_from_meta
from app.core.models import Panel, TestType as _TestType
from app.core.serializers import PanelSerializer, TestType as _TestTypeSerializer
def test_model_from_meta():
assert (Panel is model_from_meta(PanelSerializer))
assert (_TestType is model_from_meta(_TestTypeSerializer))
After importing the class using aliases the warnings no longer get printed
I hope this helps someone.

In my case, I just so happened to have a parameter's class names TestParams, which conflicts with pytest looking for classes beginning with the name test....
Solution: rename your own class
Source

Skip unittest test without decorator syntax

I have a suite of tests that I have loaded using TestLoader's (from the unittest module) loadTestsFromModule() method, i.e.,
suite = loader.loadTestsFromModule(module)
This gives me a perfectly ample list of tests that works fine. My problem is that the test harness I'm working with sometimes needs to skip certain tests based on various criteria. What I want to do is something like this:
for test in suite:
mark the test as 'to-skip' if it meets certain criteria
Note that I can't just remove the test from the list of tests because I want the unittest test runner to actually skip the tests, add them to the skipped count, and all of that jazz.
The unittest documentation suggests using decorators around the test methods or classes. Since I'm loading these tests from a module and determining to skip based on criteria not contained within the tests themselves, I can't really use decorators. Is there a way I can iterate over each individual test and some how mark it as a "to-skip" test without having to directly access the test class or methods within the class?

Using unittest.TestCase.skipTest:
import unittest
class TestFoo(unittest.TestCase):
def setUp(self): print('setup')
def tearDown(self): print('teardown')
def test_spam(self): pass
def test_egg(self): pass
def test_ham(self): pass
if __name__ == '__main__':
import sys
loader = unittest.loader.defaultTestLoader
runner = unittest.TextTestRunner(verbosity=2)
suite = loader.loadTestsFromModule(sys.modules['__main__'])
for ts in suite:
for t in ts:
if t.id().endswith('am'): # To skip `test_spam` and `test_ham`
setattr(t, 'setUp', lambda: t.skipTest('criteria'))
runner.run(suite)
prints
test_egg (__main__.TestFoo) ... setup
teardown
ok
test_ham (__main__.TestFoo) ... skipped 'criteria'
test_spam (__main__.TestFoo) ... skipped 'criteria'
----------------------------------------------------------------------
Ran 3 tests in 0.001s
OK (skipped=2)
----------------------------------------------------------------------
Ran 3 tests in 0.002s
OK (skipped=2)
UPDATE
Updated the code to patch setUp instead of test method. Otherwise, setUp/tearDown methods will be executed for test to be skipped.
NOTE
unittest.TestCase.skipTest (Test skipping) was introduced in Python 2.7, 3.1. So this method only work in Python 2.7+, 3.1+.

This is a bit of a hack, but because you only need to raise unittest.SkipTest you can walk through your suite and modify each test to raise it for you instead of running the actual test code:
import unittest
from unittest import SkipTest
class MyTestCase(unittest.TestCase):
def test_this_should_skip(self):
pass
def test_this_should_get_skipped_too(self):
pass
def _skip_test(reason):
raise SkipTest(reason)
if __name__ == '__main__':
suite = unittest.TestLoader().loadTestsFromTestCase(MyTestCase)
for test in suite:
skipped_test_method = lambda: _skip_test("reason")
setattr(test, test._testMethodName, skipped_test_method)
unittest.TextTestRunner(verbosity=2).run(suite)
When I run this, this is the output I get:
test_this_should_get_skipped_too (__main__.MyTestCase) ... skipped 'reason'
test_this_should_skip (__main__.MyTestCase) ... skipped 'reason'
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK (skipped=2)

Google brought me here.
I found the easiest way to do this is by raising a SkipTest exception when your skip criteria is met.
from unittest.case import SkipTest
def test_this_foo(self):
if <skip conditsion>:
raise SkipTest
And that test will be marked as skipped.

Some observations:
A test is a callable object with a __call__(result) method
TestCase provides a higher-level interface, allowing test methods to throw a SkipTest exception to skip themselves
The skip decorators do exactly this
Skipped tests are recorded calling the TestResult.addSkip(test, reason) method.
So you just need to replace the to-be-skipped tests with a custom test that calls addSkip:
class Skipper(object):
def __init__(self, test, reason):
self.test = test
self.reason = reason
def __call__(self, result):
result.addSkip(self.test, self.reason)

How do I write a nose2 plugin that separates different types of tests?

I'm writing a plugin that will separate treat my unit tests, functional tests & integration tests differently.
My tests folder will have the following structure exactly:
/tests
-- /unit
-- /functional
-- /integration
Each unit test will reside in the unit directory and each functional test will reside in the functional directory and so on.
I am familiar with the Layers plugin but I'd rather have my tests follow a convention.
Which hook exactly should I use to inject the appropriate Layer before tests are run?
Should it be the loadTestsFromModule hook? Can you show me an example?
I'd also like to separate the summary report for each type of test.
Which hook should I use?

I got this working with nose2 by using the nose2 attrib plugin for discovery and some code copied from the nose1 attrib plugin which allowed me to decorate my tests.
Using the nose2 attrib plugin
You will see the nose2 attrib plugin allows for custom attributes to be defined on test functions and classes.
For this to work, you have to specify the attributes of the tests after defining the test function.
class MyTestCase(unittest.TestCase):
def test_function(self):
self.assertEqual(1+1, 2)
test_function.custom_attr1 = True
test_function.custom_attr2 = ['foo', 'bar']
Then you can run a set of filtered tests by specifying -A or --attribute as a nose2 command-line argument to list the attribute(s) you to match against your test suite. You can even use the expression command-line argument of -E or --eval-attribute which allows more complex Python expressions for matching test attributes.
e.g. nose2 -v -A custom_attr1
will run all tests which have a custom_attr1 specified with a truthy value.
Using decorators to specify test attributes
This wasn't quite good enough for me though because I didn't like the idea of defining these attributes on tests after their definition. I wanted to use a decorator instead but nose2 didn't have a built-in decorator for doing this.
I went to the nose1 source code for its attrib plugin and copied the source for the attr function.
def attr(*args, **kwargs):
"""Decorator that adds attributes to classes or functions
for use with the Attribute (-a) plugin.
"""
def wrap_ob(ob):
for name in args:
setattr(ob, name, True)
for name, value in kwargs.iteritems():
setattr(ob, name, value)
return ob
return wrap_ob
I put this into a test/attrib_util.py file. Now I can specify attributes using the decorator instead. My original test class code from above can be converted to the (IMO) simpler:
from test.attrib_util import attr
class MyTestCase(unittest.TestCase):
#attr('custom_attr1', custom_attr2=['foo', 'bar'])
def test_function(self):
self.assertEqual(1+1, 2)
You will notice that the attributes can be specified as either args or kwargs; all args will get a default value of True.
You can also even use this attr decorator on a test class or base class and the attributes will be applied to all test functions defined within. This allows for very easy separation of unit and functional tests.
from test.attrib_util import attr
#attr('functional')
class FunctionalTestCase(unittest.TestCase):
pass
class MyFunctionalCase(FunctionalTestCase):
def test_function(self):
print 'this will be considered a "functional" test function'

You don't need to write a plug-in, the built-in attr module is designed for this purpose. It does not depend on your file hierarchy, however. Instead, you mark individual tests as unit, functional, or integration. This would look like:
from nose.plugins import attrib
#attrib.attr("functional")
class FunctionalTestCase(unittest.TestCase):
pass
To run only the functional tests, you would then do:
nosetests -a functional
If I were creating this test layout, I would probably have 3 unittest.TestCase subclasses, already marked with "unit", "functional", and "integration". New tests could easily inherit the proper test type.

If you already have the tests sorted into directories (as you mentioned), you could write a plugin that uses the wantDirectory method.
import os.path
from nose.plugins import Plugin
class TestCategory(Plugin):
"""
Run tests in a defined category (unittest, functional, integration. Always
runs uncategorized tests.
"""
def wantDirectory(self, dirname):
dirname = os.path.basename(dirname)
if (dirname in ('unit', 'functional', 'integration') and
dirname != self.category):
return False
return None
You will want to write options() and configure() methods for this plug-in to deal with enabling and disabling it and gleaning the user's choice of category. When running nosetests you would choose from the three categories:
nosetests --category functional
Since only one test category is run at a time, you would get a separate report for each test category. You could always, of course, run all tests by not enabling this plugin.
(adding as a different answer because it is a completely different approach).

Why is my setUpClass function not called when I load the test suite via defaultTestLoader.loadTestsFromModule?

Context: python 2.6.5 environment
I am using unittest.defaultTestLoader.loadTestsFromModule(module) to load tests.
However, when the following is loaded, the setUpClass method is not executed.
class MyTest(unittest.TestCase):
foo = None
def test_choice(self):
self.logger.info(' .. %s' % str(Full.foo))
self.assertTrue(1 == 1)
#classmethod
def setUpClass(cls):
logging.warn('setUpClass')
cls.foo = settings.INITIAL
The returned test suites shows that it returned:
<unittest.TestSuite
tests=[<unittest.TestSuite
tests=[<internal.tests.master.MyTest testMethod=test_choice>]>,
Basically under the 'test' package, there will be many tests modules. And I want the setUpClass & tearDownClass to work for each test suites. loadTestsFromModule does not satisfy my requirement. Are there other ways to achieve this ?

It works now after I upgraded to use unittest2.

The TestLoader.loadTestsFromModule method is just for loading the test cases from your module into a test suite, you should run your test suite if you want setUpClass & tearDownClass to be executed.

Disabling Python nosetests

When using nosetests for Python it is possible to disable a unit test by setting the test function's __test__ attribute to false. I have implemented this using the following decorator:
def unit_test_disabled():
def wrapper(func):
func.__test__ = False
return func
return wrapper
#unit_test_disabled
def test_my_sample_test()
#code here ...
However, this has the side effect of calling wrapper as the unit test. Wrapper will always pass but it is included in nosetests output. Is there another way of structuring the decorator so that the test will not run and does not appear in nosetests output.

Nose already has a builtin decorator for this:
from nose.tools import nottest
#nottest
def test_my_sample_test()
#code here ...
Also check out the other goodies that nose provides: https://nose.readthedocs.org/en/latest/testing_tools.html

You can also use unittest.skip decorator:
import unittest
#unittest.skip("temporarily disabled")
class MyTestCase(unittest.TestCase):
...

There also is a skiptest plugin for nosetest, which will cause the test show in test output as skipped. Here is a decorator for that:
def skipped(func):
from nose.plugins.skip import SkipTest
def _():
raise SkipTest("Test %s is skipped" % func.__name__)
_.__name__ = func.__name__
return _
Example output:
$ nosetests tests
..........................................................................
..................................S.............
----------------------------------------------------------------------
Ran 122 tests in 2.160s
OK (SKIP=1)

You can just start the class, method or function name with an underscore and nose will ignore it.
#nottest has its uses but I find that it does not work well when classes derive from one another and some base classes must be ignored by nose. This happens often when I have a series of similar Django views to test. They often share characteristics that need testing. For instance, they are accessible only to users with certain permissions. Rather than write the same permission check for all of them, I put such shared test in an initial class from which the other classes derive. The problem though is that the base class is there only to be derived by the later classes and is not meant to be run on its own. Here's an example of the problem:
from unittest import TestCase
class Base(TestCase):
def test_something(self):
print "Testing something in " + self.__class__.__name__
class Derived(Base):
def test_something_else(self):
print "Testing something else in " + self.__class__.__name__
And the output from running nose on it:
$ nosetests test.py -s
Testing something in Base
.Testing something in Derived
.Testing something else in Derived
.
----------------------------------------------------------------------
Ran 3 tests in 0.000s
OK
The Base class is included in the tests.
I cannot just slap #nottest on Base because it will mark the entire hierarchy. Indeed if you just add #nottest to the code above in front of class Base, then nose won't run any tests.
What I do is add an underscore in front of the base class:
from unittest import TestCase
class _Base(TestCase):
def test_something(self):
print "Testing something in " + self.__class__.__name__
class Derived(_Base):
def test_something_else(self):
print "Testing something else in " + self.__class__.__name__
And when running it _Base is ignored:
$ nosetests test3.py -s
Testing something in Derived
.Testing something else in Derived
.
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK
This behavior is not well documented but the code that selects tests explicitly checks for an underscore at the start of class names.
A similar test is performed by nose on function and method names so it is possible to exclude them by adding an underscore at the start of the name.

I think you will also need to rename your decorator to something that has not got test in. The below only fails on the second test for me and the first does not show up in the test suite.
def unit_disabled(func):
def wrapper(func):
func.__test__ = False
return func
return wrapper
#unit_disabled
def test_my_sample_test():
assert 1 <> 1
def test2_my_sample_test():
assert 1 <> 1