I'm writing integration tests for an Alexa app.
Our application uses a controller-request-response pattern. The controller receives a request with a specified intent and session variables, routes the request to functions that do some computation with the session variables, and returns a response object with the results of that computation.
We get the right behavior from UnhandledIntentTestCase as far as test_for_smoke is concerned. However, test_returning_reprompt_text
never fires, because returns_reprompt_text is never overwritten.
Can someone explain how I can overwrite it in the parent class and/or
how the correct intent name is passed to the request object in setUpClass?
intent_base_case.py
import unittest
import mycity.intents.intent_constants as intent_constants
import mycity.mycity_controller as mcc
import mycity.mycity_request_data_model as req
import mycity.test.test_constants as test_constants
###############################################################################
# TestCase parent class for all intent TestCases, which are integration tests #
# to see if any changes in codebase have broken response-request model. #
# #
# NOTE: Assumes that address has already been set. #
###############################################################################
class IntentBaseCase(unittest.TestCase):
__test__ = False
intent_to_test = None
returns_reprompt_text = False
#classmethod
def setUpClass(cls):
cls.controller = mcc.MyCityController()
cls.request = req.MyCityRequestDataModel()
key = intent_constants.CURRENT_ADDRESS_KEY
cls.request._session_attributes[key] = "46 Everdean St"
cls.request.intent_name = cls.intent_to_test
cls.response = cls.controller.on_intent(cls.request)
#classmethod
def tearDownClass(cls):
cls.controller = None
cls.request = None
def test_for_smoke(self):
self.assertNotIn("Uh oh", self.response.output_speech)
self.assertNotIn("Error", self.response.output_speech)
def test_correct_intent_card_title(self):
self.assertEqual(self.intent_to_test, self.response.card_title)
#unittest.skipIf(not returns_reprompt_text,
"{} shouldn't return a reprompt text".format(intent_to_test))
def test_returning_reprompt_text(self):
self.assertIsNotNone(self.response.reprompt_text)
#unittest.skipIf(returns_reprompt_text,
"{} should return a reprompt text".format(intent_to_test))
def test_returning_no_reprompt_text(self):
self.assertIsNone(self.response.reprompt_text)
test_unhandled_intent.py
import mycity.test.intent_base_case as base_case
########################################
# TestCase class for unhandled intents #
########################################
class UnhandledIntentTestCase(base_case.IntentBaseCase):
__test__ = True
intent_to_test = "UnhandledIntent"
returns_reprompt_text = True
output
======================================================================
FAIL: test_correct_intent_card_title (mycity.test.test_unhandled_intent.UnhandledIntentTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/wdrew/projects/alexa_311/my_city/mycity/mycity/test/intent_base_case.py", line 44, in test_correct_intent_card_title
self.assertEqual(self.intent_to_test, self.response.card_title)
AssertionError: 'UnhandledIntent' != 'Unhandled intent'
- UnhandledIntent
? ^
+ Unhandled intent
? ^^
======================================================================
FAIL: test_returning_no_reprompt_text (mycity.test.test_unhandled_intent.UnhandledIntentTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/wdrew/projects/alexa_311/my_city/mycity/mycity/test/intent_base_case.py", line 56, in test_returning_no_reprompt_text
self.assertIsNone(self.response.reprompt_text)
AssertionError: 'So, what can I help you with today?' is not None
----------------------------------------------------------------------
This is because of execution order. The SkipIf decorators are executed once during the parsing of the IntentBaseCase class. They aren't re-executed for each class or for each call to the test function.
The decorator pattern for SkipIf is designed for use with fixed global variables such as versions of dependent modules, operating system or some other external resource who's availability can be calculated or known in the global context.
Skipping tests is also something that should be done for external reasons, not for internal ones such as the needs of a sub-class. A skip is still a kind of failing test which is indicated in the report so you can see your test suite isn't exercising the whole of the functional scope of the project.
You should redesign your base class structure so functions are only available to run if the sub-class and skip using Skip for this. My recommendation would be:
class IntentBaseCase(unittest.TestCase):
...
class RepromptBaseCase(IntentBaseCase):
def test_returning_reprompt_text(self):
self.assertIsNotNone(self.response.reprompt_text)
class NoRepromptBaseCase(IntentBaseCase):
def test_returning_no_reprompt_text(self):
self.assertIsNone(self.response.reprompt_text)
You should also consider moving the response portion out of the setUp and put it into a test_ function of it's own and change these test_returning functions into a simpler assertReprompt and assertNoReprompt functions. It's a good idea to set up the tests in setUp, but not a good idea to run the actual code there.
Related
I wonder if it is possible to set variables of a class by a different class using cls?
The story behind it:
I'm writing tests for different purposes but see that one part of the setup is the same as in an already existing class. So I would do the setUp by the already existing one:
The original code:
class TestBase(unittest.TestCase):
def setUp(self):
self.handler = txt.getHandler(hcu.handler3())
self.curves = self.handler.curves()
self.arguments = (getInit())
self.ac = self.connect2DB(self.arguments)
self.au = AutoUtils()
This has worked well so far.
Now in my TestClient I'd like to make use of that:
from .testsDB import TestBase as tb
class TestClient(unittest.TestCase):
def setUp(self):
tb.setUp()
And modified in the TestBase the setUp to the following:
#classmethod
def setUp(cls):
cls.handler = txt.getHandler(hcu.handler3())
cls.graph = cls.handler.curves()
cls.argv = (getInit())
cls.ac = cls.connect2DB(cls.arguments)
cls.au = AutoUtils()
But I'm getting an error as soon as I use one of the values defined in the variables of the TestClient-class:
def test_Duplicates(self):
self.testDB = self.ac.connect(self.ac.client, self.arguments[4])
With the error:
In test_Duplicate (Curves.tests_client.TestClient) :
Traceback (most recent call last):
File "/home/qohelet/Curves/tests_client.py", line 49, in test_Duplicate
self.testDB = self.ac.connect(self.ac.client, self.arguments[4])
AttributeError: 'TestClient' object has no attribute 'ac'
Is it actually possible what I'm trying?
EDIT:
After writing this and seeing the answers I did another review. Yes indeed there is a circular issue I'm having.
TestBase has the function connect2DB which will be executed on setUp.
If it refers to itself (as in the original) it's fine.
If I replace the self with cls it will try to execute TestClient.connect2DB in the setUp - which does not exist. So it would require self again as putting connect2DB into TestClient is not an option.
How to solve that?
Surely your new class should just inherit the setup()?
from .testsDB import TestBase as tb
class TestClient(tb):
def test_Duplicates(self):
self.testDB = self.ac.connect(self.ac.client, self.arguments[4])
The whole point of inheritance is that you don't modify what you inherit from. Your new class should just make use of what is supplied. That is why inheritance is sometimes called programming by difference.
I have a Gateway class that has contains an instance of a Resource class. I've already done my unit testing on Resource, and to simplify testing, Resource is injected into Gateway as a dependency at initialization:
class Gateway:
def __init__(self, resource):
self._resource = resource(Master)
def list_things(self):
return self._resource.list_resource()
Now I'd like write unit test for Gateway to verify that resource.list_resource() gets called as a result of calling gateway.list_things(). My best attempt doesn't work:
class TestGateway(unittest.TestCase):
def test_list_things(self):
mock_resource = Mock()
g = modbus.gateway.Gateway(mock_resource)
g.list_things()
mock_resource.list_resource.assert_called_once()
The result:
AssertionError: Expected 'list_resource' to have been called once. Called 0 times.
What am I missing?
In the actual use case of the mock_resource that you passed into Gateway for your test case, the constructor in Gateway actually makes a further call to the resource argument as a constructor, so that what the test expects is actually emulating the checking of the call against the class method, not the instance method of the mock_resource. Demonstrating this using just the minimum number of statements can be done using the following:
>>> mock_resource = Mock()
>>> self_resource = mock_resource('Master') # emulate Gateway.__init__
>>> self_resource.list_resource() # emulate Gateway.list_things
<Mock name='mock().list_resource()' id='140441464498496'>
>>> mock_resource.list_resource.assert_called_once() # test_list_things
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.6/unittest/mock.py", line 795, in assert_called_once
raise AssertionError(msg)
AssertionError: Expected 'list_resource' to have been called once. Called 0 times.
>>> self_resource.list_resource.assert_called_once() # test the _actual_ call
>>>
Note that I had assigned self_resource to emulate the self._resource = resource(Master) statement in the constructor for Resource, and the mismatching of the test vs. what is actually executed should now be apparent.
To fix this, the test should check the call like so:
def test_list_things(self):
mock_resource = Mock()
g = modbus.gateway.Gateway(mock_resource)
g.list_things()
# mock_resource.list_resource.assert_called_once()
g._resource.list_resource.assert_called_once()
My Python version is 3.5.1
I have a simple code (tests.py):
import unittest
class SimpleObject(object):
array = []
class SimpleTestCase(unittest.TestCase):
def test_first(self):
simple_object = SimpleObject()
simple_object.array.append(1)
self.assertEqual(len(simple_object.array), 1)
def test_second(self):
simple_object = SimpleObject()
simple_object.array.append(1)
self.assertEqual(len(simple_object.array), 1)
if __name__ == '__main__':
unittest.main()
If I run it with command 'python tests.py' I will get the results:
.F
======================================================================
FAIL: test_second (__main__.SimpleTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests.py", line 105, in test_second
self.assertEqual(len(simple_object.array), 1)
AssertionError: 2 != 1
----------------------------------------------------------------------
Ran 2 tests in 0.003s
FAILED (failures=1)
Why it is happening? And how to fix it. I expect that each tests run will be independent (each test should pass), but it is not as we can see.
The array is shared by all instances of the class. If you want the array to be unique to an instance you need to put it in the class initializer:
class SimpleObject(object):
def __init__(self):
self.array = []
For more information take a look at this question: class variables is shared across all instances in python?
This can also be accomplished directly in unittest if you prefer to use only 1 class. Implement the setUp class. setUp runs before any of the tests are run as the class is instantiated. It is similar to init but conforms to the unittest library. Note the opposite is tearDown which is executed at the end of the test class if you need to build and deprecate objects. Example:
class SimpleObject(unitest.TestCase):
def setUp(self):
# I run first
self.array = []
def some_test(self):
self.assertTrue(self.array == [])
<some test code here>
def tearDown(self):
# I run last
self.array = []. # or whatever teardown code you need
enjoy!
Today I wrote test and typoed in one of test methods. My tests failed but I don't understand why. Is it special behaviour of Python properties or something else?
from unittest import TestCase
class FailObject(object):
def __init__(self):
super(FailObject, self).__init__()
self.__action = None
#property
def action(self):
return self.__action
#action.setter
def action(self, value):
self.__action = value
def do_some_work(fcells, fvalues, action, value):
currentFailObject = FailObject()
rects = [currentFailObject]
return rects
class TestModiAction(TestCase):
def testSetFailObjectAction(self):
rect = FailObject # IMPORTANT PART
rect.action = "SOME_ACTION" # No fail!
self.assertEquals("SOME_ACTION", rect.action)
def testSimple(self):
fcells = []
fvalues = []
rects = do_some_work(fcells, fvalues, 'act', 0.56)
rect = rects[0]
self.assertEquals('act', rect.action)
When I run this testcase with nose tests:
.F
======================================================================
FAIL: testSimple (test.ufsim.office.core.ui.cubeeditor.TestProperty.TestModiAction)
----------------------------------------------------------------------
Traceback (most recent call last):
File "TestProperty.py", line 36, in testSimple
self.assertEquals('act', rect.action)
AssertionError: 'act' != 'SOME_ACTION'
----------------------------------------------------------------------
Ran 2 tests in 0.022s
FAILED (failures=1)
If I fix typo with instance creation in testSetFailObjectAction all tests are work as expected. But this example turn me back to question: Is it safe to use properties? What if I will typo again some day?
You can use patch and PropertyMock from mock to this kind of jobs:
#patch(__name__."FailObject.action", new_callable=PropertyMock, return_value="SOME_ACTION")
def testSetFailObjectAction(self, mock_action):
self.assertEquals("SOME_ACTION", FailObject().action)
self.assertTrue(mock_action.called)
#This fail
self.assertEquals("SOME_ACTION", FailObject.action)
By patch you replace the property action just for the test context and you can also check if the property has been used.
Okay, that is Python default behaviour. In testSetFailObjectAction we add new static class variable that hides our properties. There is no way to protect yourself from mistakes like this.
The only suggestation is to use Traits library.
I'm having some difficulty understanding how I would go about changing a unittest report similar to:
======================================================================
FAIL: test_equal (__main__.InequalityTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test_notequal.py", line 7, in test_equal
self.assertNotEqual(1, 3-2, "My Custom Message")
AssertionError: 1 == 1
to a report resembling:
Line 7: My Custom Message
How could I parse these reports?
After further research, my problem can be solved by overriding the default TestResult class as seen here: Turn some print off in python unittest
or by using some third-party customization such as nose, an HTMLTestRunner.
In case you need to create a custom report based on the success/failure of individual test cases, you can create a custom TestRunner, which uses custom TestResult, and in TestResult class, override success and failure methods. This will provide the callback for processing as per requirement.
class CustomTestRunner(TextTestRunner):
def _makeResult(self):
return CustomTestResult(TestResult)
def run(self, test) -> unittest.result.TestResult:
# add implementation as per TextTestRunner run method here
class CustomTestResult:
def addSuccess(self, test):
super(CustomTestResult, self).addSuccess(test)
# your logic to log success cases
def addFailure(self, test):
super(CustomTestResult, self).addFailure(test)
# your logic to log failure cases
For reference, please check python unittest module.