We Keep Coding

Filter tests after discover

I'm currently running my tests like this:
tests = unittest.TestLoader().discover('tests')
unittest.TextTestRunner().run(tests)
Now I want to run a specific test knowing his name (like test_valid_user) but not knowing his class. If there is more than one test with such name than I would like to run all such tests. Is there any way to filter tests after discover?
Or maybe there are other solutions to this problem (please note that it shouldn't be done from command line)?

You can use the unittest.loader.TestLoader.testMethodPrefix instance variable to change the test methods filter according to a different prefix than "test".
Say you have a tests directory with this king of unit tests:
import unittest
class MyTest(unittest.TestCase):
def test_suite_1(self):
self.assertFalse("test_suite_1")
def test_suite_2(self):
self.assertFalse("test_suite_2")
def test_other(self):
self.assertFalse("test_other")
You can write your own discover function to discover only test functions starting with "test_suite_", for instance:
import unittest
def run_suite():
loader = unittest.TestLoader()
loader.testMethodPrefix = "test_suite_"
suite = loader.discover("tests")
result = unittest.TestResult()
suite.run(result)
for test, info in result.failures:
print(info)
if __name__ == '__main__':
run_suite()
remark: the argument "tests" in the discover method is a directory path, so you may need to write a fullpath.
As a result, you'll get:
Traceback (most recent call last):
File "/path/to/tests/test_my_module.py", line 8, in test_suite_1
self.assertFalse("test_suite_1")
AssertionError: 'test_suite_1' is not false
Traceback (most recent call last):
File "/path/to/tests/test_my_module.py", line 11, in test_suite_2
self.assertFalse("test_suite_2")
AssertionError: 'test_suite_2' is not false

Another simpler way, would be to use py.test with the -k option which does a test name keyword scan. It will run any tests whose name matches the keyword expression.
Although that is using the command-line which you didn't want, please not that you can call the command-line from your code using subprocess.call to pass any arguments you want dynamically.
E.g.: Assuming you have the following tests:
def test_user_gets_saved(self): pass
def test_user_gets_deleted(self): pass
def test_user_can_cancel(self): pass
You can call py.test from cli:
$ py.test -k "test_user"
Or from code:
return_code = subprocess.call('py.test -k "test_user"', shell=True)

There are two ways to run a single test method:
Command line:
$ python -m unittest test_module.TestClass.test_method
Using Python script:
import unittest
class TestMyCode(unittest.TestCase):
def setUp(self):
pass
def test_1(self):
self.assertTrue(True)
def test_2(self):
self.assertTrue(True)
if __name__ == '__main__':
testSuite = unittest.TestSuite()
testSuite.addTest(TestMyCode('test_1'))
runner=unittest.TextTestRunner()
runner.run(testSuite)
Output:
------------------------------------------------------------
Ran 1 test in 0.000s
OK

Python Unittest and object initialization

My Python version is 3.5.1
I have a simple code (tests.py):
import unittest
class SimpleObject(object):
array = []
class SimpleTestCase(unittest.TestCase):
def test_first(self):
simple_object = SimpleObject()
simple_object.array.append(1)
self.assertEqual(len(simple_object.array), 1)
def test_second(self):
simple_object = SimpleObject()
simple_object.array.append(1)
self.assertEqual(len(simple_object.array), 1)
if __name__ == '__main__':
unittest.main()
If I run it with command 'python tests.py' I will get the results:
.F
======================================================================
FAIL: test_second (__main__.SimpleTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests.py", line 105, in test_second
self.assertEqual(len(simple_object.array), 1)
AssertionError: 2 != 1
----------------------------------------------------------------------
Ran 2 tests in 0.003s
FAILED (failures=1)
Why it is happening? And how to fix it. I expect that each tests run will be independent (each test should pass), but it is not as we can see.

The array is shared by all instances of the class. If you want the array to be unique to an instance you need to put it in the class initializer:
class SimpleObject(object):
def __init__(self):
self.array = []
For more information take a look at this question: class variables is shared across all instances in python?

This can also be accomplished directly in unittest if you prefer to use only 1 class. Implement the setUp class. setUp runs before any of the tests are run as the class is instantiated. It is similar to init but conforms to the unittest library. Note the opposite is tearDown which is executed at the end of the test class if you need to build and deprecate objects. Example:
class SimpleObject(unitest.TestCase):
def setUp(self):
# I run first
self.array = []
def some_test(self):
self.assertTrue(self.array == [])
<some test code here>
def tearDown(self):
# I run last
self.array = []. # or whatever teardown code you need
enjoy!

Python properties and unittest TestCase

Today I wrote test and typoed in one of test methods. My tests failed but I don't understand why. Is it special behaviour of Python properties or something else?
from unittest import TestCase
class FailObject(object):
def __init__(self):
super(FailObject, self).__init__()
self.__action = None
#property
def action(self):
return self.__action
#action.setter
def action(self, value):
self.__action = value
def do_some_work(fcells, fvalues, action, value):
currentFailObject = FailObject()
rects = [currentFailObject]
return rects
class TestModiAction(TestCase):
def testSetFailObjectAction(self):
rect = FailObject # IMPORTANT PART
rect.action = "SOME_ACTION" # No fail!
self.assertEquals("SOME_ACTION", rect.action)
def testSimple(self):
fcells = []
fvalues = []
rects = do_some_work(fcells, fvalues, 'act', 0.56)
rect = rects[0]
self.assertEquals('act', rect.action)
When I run this testcase with nose tests:
.F
======================================================================
FAIL: testSimple (test.ufsim.office.core.ui.cubeeditor.TestProperty.TestModiAction)
----------------------------------------------------------------------
Traceback (most recent call last):
File "TestProperty.py", line 36, in testSimple
self.assertEquals('act', rect.action)
AssertionError: 'act' != 'SOME_ACTION'
----------------------------------------------------------------------
Ran 2 tests in 0.022s
FAILED (failures=1)
If I fix typo with instance creation in testSetFailObjectAction all tests are work as expected. But this example turn me back to question: Is it safe to use properties? What if I will typo again some day?

You can use patch and PropertyMock from mock to this kind of jobs:
#patch(__name__."FailObject.action", new_callable=PropertyMock, return_value="SOME_ACTION")
def testSetFailObjectAction(self, mock_action):
self.assertEquals("SOME_ACTION", FailObject().action)
self.assertTrue(mock_action.called)
#This fail
self.assertEquals("SOME_ACTION", FailObject.action)
By patch you replace the property action just for the test context and you can also check if the property has been used.

Okay, that is Python default behaviour. In testSetFailObjectAction we add new static class variable that hides our properties. There is no way to protect yourself from mistakes like this.
The only suggestation is to use Traits library.

How can I access variables set in the Python nosetests setup function

I think what I'm trying to do is fairly simple. I want to initialize a couple of variables in a test setup function, and then use them in the test functions that are decorated with that setup. The following trivial example illustrates what I mean:
from nose.tools import *
def setup():
foo = 10
def teardown():
foo = None
#with_setup(setup, teardown)
def test_foo_value():
assert_equal(foo, 10)
This results in:
$ nosetests tests/test_foo.py
E
======================================================================
ERROR: test_foo.test_foo_value
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/mtozzi/.virtualenvs/foo/local/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/home/mtozzi/code/foo/tests/test_foo.py", line 12, in test_foo_value
assert_equal(foo, 10)
NameError: global name 'foo' is not defined
----------------------------------------------------------------------
Ran 1 test in 0.006s
FAILED (errors=1)
With the old unittest style, I could set these as instance variables on the test class, but I thought nosetests didn't require you to use classes. I've also considered setting them as package global variables, but that doesn't seem like a terribly good practice. I hope there's something obvious I'm missing for doing this.

As the comments to your question already suggested, simply switch to classes and use instance variables like self.foo. That's the way it should be done.
If you insist on not using classes, try global variables. You didn't hear this from me, though.
from nose.tools import *
foo = None
def setup():
global foo # Ugly.
foo = 10
def teardown():
global foo # Ugly.
foo = None
#with_setup(setup, teardown)
def test_foo_value():
assert_equal(foo, 10)
A third variant might be to use a dictionary for your values. This is slightly less ugly but horribly clumsy:
from nose.tools import *
_globals = {'foo': None}
def setup():
_globals['foo'] = 10
def teardown():
_globals['foo'] = None
#with_setup(setup, teardown)
def test_foo_value():
foo = _globals['foo']
assert_equal(foo, 10)

I use a custom with_setup decorator that uses the poor man's nonlocal: https://gist.github.com/garyvdm/392ae20c673c7ee58d76
def setup():
foo = 10
return [foo], {}
def teardown(foo):
pass
#with_setup_args(setup, teardown)
def test_foo_value(foo):
nose.tools.assert_equal(foo, 10)
For projects that are Python 3 only, I use nonlocal rather than .extend/.update for arguments, kwargs.

Running unittest.main() from a module?

I wrote a little function that dynamically defines unittest.TestCase classes (trivial version below).
When I moved it out of the same source file into its own module, I can't figure out how to get unittest to discover the new classes. Calling unittest.main() from either file doesn't execute any tests.
factory.py:
import unittest
_testnum = 0
def test_factory(a, b):
global _testnum
testname = 'dyntest' + str(_testnum)
globals()[testname] = type(testname, (unittest.TestCase,), {'testme': lambda self: self.assertEqual(a, b)})
_testnum += 1
def finish():
unittest.main()
someotherfile.py:
from factory import test_factory, finish
test_factory(1, 1)
test_factory(1, 2)
if __name__ == '__main__':
finish()
Output:
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
So it doesn't execute any tests.
Note that keeping it all in the same file works as expected:
import unittest
_testnum = 0
def test_factory(a, b):
global _testnum
testname = 'dyntest' + str(_testnum)
globals()[testname] = type(testname, (unittest.TestCase,), {'testme': lambda self: self.assertEqual(a, b)})
_testnum += 1
test_factory(1, 1)
test_factory(1, 2)
if __name__ == '__main__':
unittest.main()
Output (as expected):
.F
======================================================================
FAIL: testme (__main__.dyntest1)
----------------------------------------------------------------------
Traceback (most recent call last):
File "partb.py", line 11, in <lambda>
globals()[testname] = type(testname, (unittest.TestCase,), {'testme': lambda self: self.assertEqual(a, b)})
AssertionError: 1 != 2
----------------------------------------------------------------------
Ran 2 tests in 0.008s
FAILED (failures=1)
How I use my test_factory() function such that I can execute all of the TestCase objects it defines from a separate source file?

The general idea (what unittest.main does for you) is:
suite = unittest.TestLoader().loadTestsFromTestCase(SomeTestCase)
unittest.TextTestRunner(verbosity=2).run(suite)
as per http://docs.python.org/library/unittest.html?highlight=unittest#module-unittest . Your test cases are hidden in globals() by the test_factory function, so just do a dir(), find the globals that are instances of unittest.TestCase (or ones with names starting with 'dyntest', etc), and just build your suite that way and run it.

By default, unittest.main() looks for unit TestCase objects in the main module. The test_factory creates the TestCase objects in its own module. That's why moving it outside of the main module causes the behavior you see.
Try:
def finish():
unittest.main(module=__name__)