We Keep Coding

In pytest, how to assert if an exception (which is inherited from a parent exception class) is raised?

What I have done :
I have a function def get_holidays(): which raises a Timeout error. My test function test_get_holidays_raises_ioerror(): first sets requests.get.side_effect = IOError and then uses pytest.raises(IOError) to assert if that function raises an IOError.
What the issue is :
Ideally this should fail, since my actual get_holidays() does not raise an IOError. But the test passes.
Possible reason :
This might be because Timeout is inherited from the IOError class.
What I want :
Want to assert specifically if IOError is raised.
Code :
from mock import Mock
import requests
from requests import Timeout
import pytest
requests = Mock()
# Actual function to test
def get_holidays():
try:
r = requests.get('http://localhost/api/holidays')
if r.status_code == 200:
return r.json()
except Timeout:
raise Timeout
return None
# Actual function that tests the above function
def test_get_holidays_raises_ioerror():
requests.get.side_effect = IOError
with pytest.raises(IOError):
get_holidays()

pytest captures the exception in an ExceptionInfo object. You can compare the exact type after the exception.
def test_get_holidays_raises_ioerror():
requests.get.side_effect = IOError
with pytest.raises(IOError) as excinfo:
get_holidays()
assert type(excinfo.value) is IOError

Unit testing tornado applications: How to improve the display of error messages

I am using unittest to test a tornado app having several handlers, one of which raises an exception. If I run the following test code with python test.py:
# test.py
import unittest
import tornado.web
import tornado.testing
class MainHandler(tornado.web.RequestHandler):
def get(self):
self.write('Hello World') # handler works correctly
class HandlerWithError(tornado.web.RequestHandler):
def get(self):
raise Exception('Boom') # handler raises an exception
self.write('Hello World')
def make_app():
return tornado.web.Application([
(r'/main/', MainHandler),
(r'/error/', HandlerWithError),
])
class TornadoTestCase(tornado.testing.AsyncHTTPTestCase):
def get_app(self):
return make_app()
def test_main_handler(self):
response = self.fetch('/main/')
self.assertEqual(response.code, 200) # test should pass
def test_handler_with_error(self):
response = self.fetch('/error/')
self.assertEqual(response.code, 200) # test should fail with error
if __name__ == '__main__':
unittest.main()
the test output looks like:
ERROR:tornado.application:Uncaught exception GET /error/ (127.0.0.1)
HTTPServerRequest(protocol='http', host='localhost:36590', method='GET', uri='/error/', version='HTTP/1.1', remote_ip='127.0.0.1', headers={'Connection': 'close', 'Host': 'localhost:3
6590', 'Accept-Encoding': 'gzip'})
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/tornado/web.py", line 1332, in _execute
result = method(*self.path_args, **self.path_kwargs)
File "test.py", line 13, in get
raise Exception('Boom') # handler raises an exception
Exception: Boom
ERROR:tornado.access:500 GET /error/ (127.0.0.1) 19.16ms
F.
======================================================================
FAIL: test_handler_with_error (__main__.TornadoTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/tornado/testing.py", line 118, in __call__
result = self.orig_method(*args, **kwargs)
File "test.py", line 33, in test_handler_with_error
self.assertEqual(response.code, 200) # test should fail with error
AssertionError: 500 != 200
----------------------------------------------------------------------
Ran 2 tests in 0.034s
FAILED (failures=1)
However, I would expect unittest to report an Error for the second test, instead of a failing assertion. Moreover, the fact that the traceback for the 'Boom' exception appears before the unittest test report and does not include a reference to the failing test function makes it difficult to find the source of the exception.
Any suggestions how to handle this situation?
Thanks in advance!
EDIT
What I find unexpected is the fact that test_handler_with_error actually arrives at making the assertEqual assertion, instead of throwing the error. For example, the following code does not execute the self.assertEqualstatement, and consequently reports an ERROR instead of a FAIL in the test output:
# simple_test.py
import unittest
def foo():
raise Exception('Boom')
return 'bar'
class SimpleTestCase(unittest.TestCase):
def test_failing_function(self):
result = foo()
self.assertEqual(result, 'bar')
if __name__ == '__main__':
unittest.main()

You can disable logging and only the test reports will appear:
logging.disable(logging.CRITICAL)
You can put that for example in
created TestCase subclass
test runner
More info How can I disable logging while running unit tests in Python Django?
Keep in mind that CI/CD systems actually use normalized report e.g. junit and then present it in more readable/elegant way - more info:
Python script to generate JUnit report from another testing result
How to output coverage XML with nosetests?

This is expected behavior. Your test itself asserts that the return code is HTTP 200, and since this is a formal assert that is false, the outcome is a "failure" instead of an "error". You can suppress logs as mentioned in kwaranuk's answer, but then you lose the information about what actually caused the HTTP 500 error.
Why does your code reach the assert, instead of throwing? It's because your test code does not call HandlerWithError.get. Your test code begins an asynchronous HTTP GET operation with an HTTP client provided by the AsyncHTTPTestCase class. (Check the source code of that class for details.) The event loop runs until HandlerWithError.get receives the request over a localhost socket, and responds on that socket with an HTTP 500. When HandlerWithError.get fails, it doesn't raise an exception into your test function, any more than a failure at Google.com would raise an exception: it merely results in an HTTP 500.
Welcome to the world of async! There's no easy way to neatly associate the assertion error and the traceback from HandlerWithError.get().

Unable to pass assertRaises test in Python

So, I have the most trivial in the world example. This is my class to be tested:
# My_Class.py
class My_Class(object):
#staticmethod
def doit(name, params):
try:
raise Exception("This is my error message")
except Exception:
print("Exception: I raised Exception")
And this is the tester itself:
# test.py
import unittest
from My_Class import My_Class
class Test_MyClass(unittest.TestCase):
def setUp(self):
self.my_class = My_Class()
def test_my_class(self):
name = "Abrakadabra"
params = {}
self.assertRaises(Exception, self.my_class.doit, name, params)
And this is what I see in the console, when I'm running my test.py:
$ nosetests test.py
F
======================================================================
FAIL: test_my_class (test.Test_MyClass)
----------------------------------------------------------------------
Traceback (most recent call last):
File ....
nose.proxy.AssertionError: Exception not raised by doit
-------------------- >> begin captured stdout << ---------------------
Exception: I raised Exception
--------------------- >> end captured stdout << ----------------------
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (failures=1)
It is reaaly iteresting, because it is controversial. On the one hand the test says that "Exception not raised by doit", but one line below it clearly prints out a message from the Exception block. So, what I'm doing wrong here??? Thanks!

To directly answer your question, the reason why you are getting that message is because with this assertion:
self.assertRaises(Exception, self.my_class.doit, name, params)
You are testing to make sure an exception was raised. But your try/except suppresses this. If you actually remove your try/except your test will in fact pass, because now your method will raise.
Since you do not want to do this, what you should be doing instead is testing the behaviour of your method when an exception is raised. Ultimately, you want to make sure that your print method is called in your except. I have put together an example below to help understand this.
Keeping in mind what #user2357112 mentioned, which is very important to keep in mind when unittesting, here is an example to help expand on that to provide a practical use for what you are trying to do:
Let us just put together some method:
def some_method():
pass
We will now put this in to your staticmethod you defined as such:
# My_Class.py
class My_Class(object):
#staticmethod
def doit(name, params):
try:
some_method()
except Exception:
print("Exception: I raised Exception")
So now, when it comes to your unittesting, you want to test the behaviour of your method doit. With that in mind, what you will do in this case, is test that some_method will raise an exception and you will validate how your doit method behaves to that exception being raised.
At this point, I suggest taking a look at the documentation behind unittest and mock to get more familiar with what you can do with your testing, but here is an example using mock patching to test the behaviour of your code if an exception is being raised:
#patch('builtins.print')
#patch('__main__.some_method')
def test_my_class(self, m_some_method, m_print):
name = "Abrakadabra"
params = {}
# have the side_effect raise the exception when some_method is called in doit
m_some_method.side_effect = Exception()
self.my_class.doit(name, params)
# check to make sure you caught the exception by checking print was called
self.assertEqual(m_print.call_count, 1)
When you put it all together, the following is functional code that I ran on my end that you can play around with to understand what is happening:
def some_method():
pass
# My_Class.py
class My_Class(object):
#staticmethod
def doit(name, params):
try:
some_method()
except Exception:
print("Exception: I raised Exception")
# test.py
import unittest
from mock import patch
class Test_MyClass(unittest.TestCase):
def setUp(self):
self.my_class = My_Class()
#patch('builtins.print')
#patch('__main__.some_method')
def test_my_class(self, m_some_method, m_print):
name = "Abrakadabra"
params = {}
m_some_method.side_effect = Exception()
self.my_class.doit(name, params)
self.assertEqual(m_print.call_count, 1)
if __name__ == '__main__':
unittest.main()

assertRaises is an assertion about the function's visible behavior, not its internals. It asserts that the stated exception passes out of the function. Any exceptions that are handled inside the function are not assertRaises's concern.

assertRaises failed since there was actually no exception raised. Well, it was raised but handled with except inside the doit() method. The problem is here:
try:
raise Exception("This is my error message")
except Exception:
print("Exception: I raised Exception")
You are raising an exception and then catching it without re-raising. From a caller (assertRaises is the caller in your case) perspective, no errors were thrown during the function call. Re-raising an exception allows a caller to handle an exception as well. Put a raise after the print:
try:
raise Exception("This is my error message")
except Exception:
print("Exception: I raised Exception")
raise # re-raising
Also see Handling Exceptions.

Python mock access "real" objects while mocking a module

Is it possible to access "real" objects while mocking a module? What I'm trying to do is mock some function, but throw "real" exceptions, like this:
#mock.patch('my_project.requests')
def test_foo(self, mock_requests):
mock_requests.post = mock.Mock(side_effect=requests.ConnectionError())
thread = CommandThread("command", 3, 2, 0)
thread.run() #this is were I exercise requests.post
self.assert(thread.was_successful is False)
Inside my CommandThread I have a check like
try:
requests.post(url, data=data)
except (requests.ConnectionError, requests.Timeout):
self.was_successful = False
however, my test fails because the exception is not caught inside the try/except block (when I do like except Exception: it works)
The reason, I think, is simply because I mocked this "namespace" in my test case, so I actually raise my_project.requests.ConnectionError exception rather than proper, requests.ConnectionError from original package.
Is it somehow possible to access/throw "real" exceptions?

This is happening because your mock is actually overwriting the entire requests module in your code. Here is how you can debug this:
In your code, add this:
try:
requests.post('', data='')
except (requests.ConnectionError, requests.Timeout):
was_successful = False
except Exception, err:
import pdb
pdb.set_trace()
When you run the test, you will be dropped into the debugger so that you can take a look at what is happening. If we look at what you are catching, this is what we see:
(Pdb) requests.ConnectionError
<MagicMock name='requests.ConnectionError' id='4562438992'>
You are actually catching a mock ConnectionError because your mock patched out the entire requests module and you were feeding in a real requests error.
You can fix this by making your mock more specific and only overriding the post method on the requests module:
#mock.patch('my_project.requests.post')
def test_foo(self, mock_requests):
mock_requests.side_effect = requests.ConnectionError()
...

Mocking a function to raise an Exception to test an except block

I have a function (foo) which calls another function (bar). If invoking bar() raises an HttpError, I want to handle it specially if the status code is 404, otherwise re-raise.
I am trying to write some unit tests around this foo function, mocking out the call to bar(). Unfortunately, I am unable to get the mocked call to bar() to raise an Exception which is caught by my except block.
Here is my code which illustrates my problem:
import unittest
import mock
from apiclient.errors import HttpError
class FooTests(unittest.TestCase):
#mock.patch('my_tests.bar')
def test_foo_shouldReturnResultOfBar_whenBarSucceeds(self, barMock):
barMock.return_value = True
result = foo()
self.assertTrue(result) # passes
#mock.patch('my_tests.bar')
def test_foo_shouldReturnNone_whenBarRaiseHttpError404(self, barMock):
barMock.side_effect = HttpError(mock.Mock(return_value={'status': 404}), 'not found')
result = foo()
self.assertIsNone(result) # fails, test raises HttpError
#mock.patch('my_tests.bar')
def test_foo_shouldRaiseHttpError_whenBarRaiseHttpErrorNot404(self, barMock):
barMock.side_effect = HttpError(mock.Mock(return_value={'status': 500}), 'error')
with self.assertRaises(HttpError): # passes
foo()
def foo():
try:
result = bar()
return result
except HttpError as error:
if error.resp.status == 404:
print '404 - %s' % error.message
return None
raise
def bar():
raise NotImplementedError()
I followed the Mock docs which say that you should set the side_effect of a Mock instance to an Exception class to have the mocked function raise the error.
I also looked at some other related StackOverflow Q&As, and it looks like I am doing the same thing they are doing to cause and Exception to be raised by their mock.
https://stackoverflow.com/a/10310532/346561
How to use Python Mock to raise an exception - but with Errno set to a given value
Why is setting the side_effect of barMock not causing the expected Exception to be raised? If I am doing something weird, how should I go about testing logic in my except block?

Your mock is raising the exception just fine, but the error.resp.status value is missing. Rather than use return_value, just tell Mock that status is an attribute:
barMock.side_effect = HttpError(mock.Mock(status=404), 'not found')
Additional keyword arguments to Mock() are set as attributes on the resulting object.
I put your foo and bar definitions in a my_tests module, added in the HttpError class so I could use it too, and your test then can be ran to success:
>>> from my_tests import foo, HttpError
>>> import mock
>>> with mock.patch('my_tests.bar') as barMock:
... barMock.side_effect = HttpError(mock.Mock(status=404), 'not found')
... result = my_test.foo()
...
404 -
>>> result is None
True
You can even see the print '404 - %s' % error.message line run, but I think you wanted to use error.content there instead; that's the attribute HttpError() sets from the second argument, at any rate.