My Python version is 3.5.1
I have a simple code (tests.py):
import unittest
class SimpleObject(object):
array = []
class SimpleTestCase(unittest.TestCase):
def test_first(self):
simple_object = SimpleObject()
simple_object.array.append(1)
self.assertEqual(len(simple_object.array), 1)
def test_second(self):
simple_object = SimpleObject()
simple_object.array.append(1)
self.assertEqual(len(simple_object.array), 1)
if __name__ == '__main__':
unittest.main()
If I run it with command 'python tests.py' I will get the results:
.F
======================================================================
FAIL: test_second (__main__.SimpleTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests.py", line 105, in test_second
self.assertEqual(len(simple_object.array), 1)
AssertionError: 2 != 1
----------------------------------------------------------------------
Ran 2 tests in 0.003s
FAILED (failures=1)
Why it is happening? And how to fix it. I expect that each tests run will be independent (each test should pass), but it is not as we can see.
The array is shared by all instances of the class. If you want the array to be unique to an instance you need to put it in the class initializer:
class SimpleObject(object):
def __init__(self):
self.array = []
For more information take a look at this question: class variables is shared across all instances in python?
This can also be accomplished directly in unittest if you prefer to use only 1 class. Implement the setUp class. setUp runs before any of the tests are run as the class is instantiated. It is similar to init but conforms to the unittest library. Note the opposite is tearDown which is executed at the end of the test class if you need to build and deprecate objects. Example:
class SimpleObject(unitest.TestCase):
def setUp(self):
# I run first
self.array = []
def some_test(self):
self.assertTrue(self.array == [])
<some test code here>
def tearDown(self):
# I run last
self.array = []. # or whatever teardown code you need
enjoy!
Related
I want to load the function into the unit test dynamically, passing code as str and use exec() to run them in test.
My current code look like this:
import unittest
class ParametrizedTestCase(unittest.TestCase):
def __init__(self, methodName='runTest', param=None):
super(ParametrizedTestCase, self).__init__(methodName)
exec (param)
self.param = param
#staticmethod
def parametrize(testcase_klass, param=None):
testloader = unittest.TestLoader()
testname = testloader.getTestCaseNames(testcase_klass)
suite = unittest.TestSuite()
for name in testname:
suite.addTest(testcase_klass(name, param=param))
return suite
class TestZero(ParametrizedTestCase):
def test_add(self):
print('param:', self.param)
self.assertEqual(add(1,1), 2)
if __name__ == '__main__':
code = "def add(a, b): return a + b"
suite = ParametrizedTestCase.parametrize(TestZero, param=code)
unittest.TextTestRunner(verbosity=2).run(suite)
and this is current out put:
❯ python test/test0.py
test_add (__main__.TestZero) ... param: def add(a, b): return a + b
ERROR
======================================================================
ERROR: test_add (__main__.TestZero)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test/test0.py", line 21, in test_add
self.assertEqual(add(1,1), 2)
NameError: name 'add' is not defined
----------------------------------------------------------------------
Ran 1 test in 0.000s
FAILED (errors=1)
Is it possible to do that and if so, how?
exec with no additional parameters is defining add in the local scope of ParameterizedTestCase.__init__; once that function complete, add goes out of scope.
One solution is to put add in the global scope, so that it's accessible.
exec(param, globals())
However, in general you aren't going to know what name (or names) the execution of param will make available for future tests.
I'm writing integration tests for an Alexa app.
Our application uses a controller-request-response pattern. The controller receives a request with a specified intent and session variables, routes the request to functions that do some computation with the session variables, and returns a response object with the results of that computation.
We get the right behavior from UnhandledIntentTestCase as far as test_for_smoke is concerned. However, test_returning_reprompt_text
never fires, because returns_reprompt_text is never overwritten.
Can someone explain how I can overwrite it in the parent class and/or
how the correct intent name is passed to the request object in setUpClass?
intent_base_case.py
import unittest
import mycity.intents.intent_constants as intent_constants
import mycity.mycity_controller as mcc
import mycity.mycity_request_data_model as req
import mycity.test.test_constants as test_constants
###############################################################################
# TestCase parent class for all intent TestCases, which are integration tests #
# to see if any changes in codebase have broken response-request model. #
# #
# NOTE: Assumes that address has already been set. #
###############################################################################
class IntentBaseCase(unittest.TestCase):
__test__ = False
intent_to_test = None
returns_reprompt_text = False
#classmethod
def setUpClass(cls):
cls.controller = mcc.MyCityController()
cls.request = req.MyCityRequestDataModel()
key = intent_constants.CURRENT_ADDRESS_KEY
cls.request._session_attributes[key] = "46 Everdean St"
cls.request.intent_name = cls.intent_to_test
cls.response = cls.controller.on_intent(cls.request)
#classmethod
def tearDownClass(cls):
cls.controller = None
cls.request = None
def test_for_smoke(self):
self.assertNotIn("Uh oh", self.response.output_speech)
self.assertNotIn("Error", self.response.output_speech)
def test_correct_intent_card_title(self):
self.assertEqual(self.intent_to_test, self.response.card_title)
#unittest.skipIf(not returns_reprompt_text,
"{} shouldn't return a reprompt text".format(intent_to_test))
def test_returning_reprompt_text(self):
self.assertIsNotNone(self.response.reprompt_text)
#unittest.skipIf(returns_reprompt_text,
"{} should return a reprompt text".format(intent_to_test))
def test_returning_no_reprompt_text(self):
self.assertIsNone(self.response.reprompt_text)
test_unhandled_intent.py
import mycity.test.intent_base_case as base_case
########################################
# TestCase class for unhandled intents #
########################################
class UnhandledIntentTestCase(base_case.IntentBaseCase):
__test__ = True
intent_to_test = "UnhandledIntent"
returns_reprompt_text = True
output
======================================================================
FAIL: test_correct_intent_card_title (mycity.test.test_unhandled_intent.UnhandledIntentTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/wdrew/projects/alexa_311/my_city/mycity/mycity/test/intent_base_case.py", line 44, in test_correct_intent_card_title
self.assertEqual(self.intent_to_test, self.response.card_title)
AssertionError: 'UnhandledIntent' != 'Unhandled intent'
- UnhandledIntent
? ^
+ Unhandled intent
? ^^
======================================================================
FAIL: test_returning_no_reprompt_text (mycity.test.test_unhandled_intent.UnhandledIntentTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/wdrew/projects/alexa_311/my_city/mycity/mycity/test/intent_base_case.py", line 56, in test_returning_no_reprompt_text
self.assertIsNone(self.response.reprompt_text)
AssertionError: 'So, what can I help you with today?' is not None
----------------------------------------------------------------------
This is because of execution order. The SkipIf decorators are executed once during the parsing of the IntentBaseCase class. They aren't re-executed for each class or for each call to the test function.
The decorator pattern for SkipIf is designed for use with fixed global variables such as versions of dependent modules, operating system or some other external resource who's availability can be calculated or known in the global context.
Skipping tests is also something that should be done for external reasons, not for internal ones such as the needs of a sub-class. A skip is still a kind of failing test which is indicated in the report so you can see your test suite isn't exercising the whole of the functional scope of the project.
You should redesign your base class structure so functions are only available to run if the sub-class and skip using Skip for this. My recommendation would be:
class IntentBaseCase(unittest.TestCase):
...
class RepromptBaseCase(IntentBaseCase):
def test_returning_reprompt_text(self):
self.assertIsNotNone(self.response.reprompt_text)
class NoRepromptBaseCase(IntentBaseCase):
def test_returning_no_reprompt_text(self):
self.assertIsNone(self.response.reprompt_text)
You should also consider moving the response portion out of the setUp and put it into a test_ function of it's own and change these test_returning functions into a simpler assertReprompt and assertNoReprompt functions. It's a good idea to set up the tests in setUp, but not a good idea to run the actual code there.
I'm currently running my tests like this:
tests = unittest.TestLoader().discover('tests')
unittest.TextTestRunner().run(tests)
Now I want to run a specific test knowing his name (like test_valid_user) but not knowing his class. If there is more than one test with such name than I would like to run all such tests. Is there any way to filter tests after discover?
Or maybe there are other solutions to this problem (please note that it shouldn't be done from command line)?
You can use the unittest.loader.TestLoader.testMethodPrefix instance variable to change the test methods filter according to a different prefix than "test".
Say you have a tests directory with this king of unit tests:
import unittest
class MyTest(unittest.TestCase):
def test_suite_1(self):
self.assertFalse("test_suite_1")
def test_suite_2(self):
self.assertFalse("test_suite_2")
def test_other(self):
self.assertFalse("test_other")
You can write your own discover function to discover only test functions starting with "test_suite_", for instance:
import unittest
def run_suite():
loader = unittest.TestLoader()
loader.testMethodPrefix = "test_suite_"
suite = loader.discover("tests")
result = unittest.TestResult()
suite.run(result)
for test, info in result.failures:
print(info)
if __name__ == '__main__':
run_suite()
remark: the argument "tests" in the discover method is a directory path, so you may need to write a fullpath.
As a result, you'll get:
Traceback (most recent call last):
File "/path/to/tests/test_my_module.py", line 8, in test_suite_1
self.assertFalse("test_suite_1")
AssertionError: 'test_suite_1' is not false
Traceback (most recent call last):
File "/path/to/tests/test_my_module.py", line 11, in test_suite_2
self.assertFalse("test_suite_2")
AssertionError: 'test_suite_2' is not false
Another simpler way, would be to use py.test with the -k option which does a test name keyword scan. It will run any tests whose name matches the keyword expression.
Although that is using the command-line which you didn't want, please not that you can call the command-line from your code using subprocess.call to pass any arguments you want dynamically.
E.g.: Assuming you have the following tests:
def test_user_gets_saved(self): pass
def test_user_gets_deleted(self): pass
def test_user_can_cancel(self): pass
You can call py.test from cli:
$ py.test -k "test_user"
Or from code:
return_code = subprocess.call('py.test -k "test_user"', shell=True)
There are two ways to run a single test method:
Command line:
$ python -m unittest test_module.TestClass.test_method
Using Python script:
import unittest
class TestMyCode(unittest.TestCase):
def setUp(self):
pass
def test_1(self):
self.assertTrue(True)
def test_2(self):
self.assertTrue(True)
if __name__ == '__main__':
testSuite = unittest.TestSuite()
testSuite.addTest(TestMyCode('test_1'))
runner=unittest.TextTestRunner()
runner.run(testSuite)
Output:
------------------------------------------------------------
Ran 1 test in 0.000s
OK
Today I wrote test and typoed in one of test methods. My tests failed but I don't understand why. Is it special behaviour of Python properties or something else?
from unittest import TestCase
class FailObject(object):
def __init__(self):
super(FailObject, self).__init__()
self.__action = None
#property
def action(self):
return self.__action
#action.setter
def action(self, value):
self.__action = value
def do_some_work(fcells, fvalues, action, value):
currentFailObject = FailObject()
rects = [currentFailObject]
return rects
class TestModiAction(TestCase):
def testSetFailObjectAction(self):
rect = FailObject # IMPORTANT PART
rect.action = "SOME_ACTION" # No fail!
self.assertEquals("SOME_ACTION", rect.action)
def testSimple(self):
fcells = []
fvalues = []
rects = do_some_work(fcells, fvalues, 'act', 0.56)
rect = rects[0]
self.assertEquals('act', rect.action)
When I run this testcase with nose tests:
.F
======================================================================
FAIL: testSimple (test.ufsim.office.core.ui.cubeeditor.TestProperty.TestModiAction)
----------------------------------------------------------------------
Traceback (most recent call last):
File "TestProperty.py", line 36, in testSimple
self.assertEquals('act', rect.action)
AssertionError: 'act' != 'SOME_ACTION'
----------------------------------------------------------------------
Ran 2 tests in 0.022s
FAILED (failures=1)
If I fix typo with instance creation in testSetFailObjectAction all tests are work as expected. But this example turn me back to question: Is it safe to use properties? What if I will typo again some day?
You can use patch and PropertyMock from mock to this kind of jobs:
#patch(__name__."FailObject.action", new_callable=PropertyMock, return_value="SOME_ACTION")
def testSetFailObjectAction(self, mock_action):
self.assertEquals("SOME_ACTION", FailObject().action)
self.assertTrue(mock_action.called)
#This fail
self.assertEquals("SOME_ACTION", FailObject.action)
By patch you replace the property action just for the test context and you can also check if the property has been used.
Okay, that is Python default behaviour. In testSetFailObjectAction we add new static class variable that hides our properties. There is no way to protect yourself from mistakes like this.
The only suggestation is to use Traits library.
I wrote a little function that dynamically defines unittest.TestCase classes (trivial version below).
When I moved it out of the same source file into its own module, I can't figure out how to get unittest to discover the new classes. Calling unittest.main() from either file doesn't execute any tests.
factory.py:
import unittest
_testnum = 0
def test_factory(a, b):
global _testnum
testname = 'dyntest' + str(_testnum)
globals()[testname] = type(testname, (unittest.TestCase,), {'testme': lambda self: self.assertEqual(a, b)})
_testnum += 1
def finish():
unittest.main()
someotherfile.py:
from factory import test_factory, finish
test_factory(1, 1)
test_factory(1, 2)
if __name__ == '__main__':
finish()
Output:
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
So it doesn't execute any tests.
Note that keeping it all in the same file works as expected:
import unittest
_testnum = 0
def test_factory(a, b):
global _testnum
testname = 'dyntest' + str(_testnum)
globals()[testname] = type(testname, (unittest.TestCase,), {'testme': lambda self: self.assertEqual(a, b)})
_testnum += 1
test_factory(1, 1)
test_factory(1, 2)
if __name__ == '__main__':
unittest.main()
Output (as expected):
.F
======================================================================
FAIL: testme (__main__.dyntest1)
----------------------------------------------------------------------
Traceback (most recent call last):
File "partb.py", line 11, in <lambda>
globals()[testname] = type(testname, (unittest.TestCase,), {'testme': lambda self: self.assertEqual(a, b)})
AssertionError: 1 != 2
----------------------------------------------------------------------
Ran 2 tests in 0.008s
FAILED (failures=1)
How I use my test_factory() function such that I can execute all of the TestCase objects it defines from a separate source file?
The general idea (what unittest.main does for you) is:
suite = unittest.TestLoader().loadTestsFromTestCase(SomeTestCase)
unittest.TextTestRunner(verbosity=2).run(suite)
as per http://docs.python.org/library/unittest.html?highlight=unittest#module-unittest . Your test cases are hidden in globals() by the test_factory function, so just do a dir(), find the globals that are instances of unittest.TestCase (or ones with names starting with 'dyntest', etc), and just build your suite that way and run it.
By default, unittest.main() looks for unit TestCase objects in the main module. The test_factory creates the TestCase objects in its own module. That's why moving it outside of the main module causes the behavior you see.
Try:
def finish():
unittest.main(module=__name__)