We Keep Coding

Unexpected behavior from unittest.mock.patch

What is wrong with my code below?
I'm expecting assert call_func_once_with("b") to throw an error as call_func was passed 'a'. I confimed that the function was indeed called once and with argument 'a'.
from unittest.mock import Mock, patch
def call_func(x):
pass
#patch("__main__.call_func")
def test_call_func(call_func):
call_func("a")
assert call_func.called_once_with("b")
assert call_func.called == 1
print(call_func.call_args)
test_call_func()
Output:
call('a')

You're not the first person to notice strange things with these types of assertions (see Magic mock assert_called_once vs assert_called_once_with weird behaviour)
For what it's worth, I can only advise that you try to create a test class which inherits from unittest.TestCase and then use the assertEqual method to get more consistent test behaviour:
import unittest
from unittest.mock import patch, call
def call_func(x):
pass
class MyTests(unittest.TestCase):
#patch("__main__.call_func")
def test_call_func(self, call_func_mock):
call_func_mock("a")
# assert call_func_mock.called == 1
# assert call_func_mock.called_once_with("b")
self.assertEqual(call_func_mock.call_count, 1)
self.assertEqual(call_func_mock.call_args_list[0], call("b"))
print(call_func_mock.call_args)
unittest.main()
This gives the following (expected) results:
F
======================================================================
FAIL: test_call_func (__main__.MyTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\Python36\lib\unittest\mock.py", line 1179, in patched
return func(*args, **keywargs)
File "C:/scratch.py", line 16, in test_call_func
self.assertEquals(call_func_mock.call_args_list[0], call("b"))
AssertionError: call('a') != call('b')
----------------------------------------------------------------------
Ran 1 test in 0.003s
FAILED (failures=1)
Process finished with exit code 1

Python assertRaises on user-defined exceptions

The following question was triggered by the discussion in this post.
Assume two files (foobar.py and foobar_unittest.py). File foobar.py contains a class (FooBar) with two functions (foo and bar). Function bar raises a built-in exception, function foo a user-defined exception.
# foobar.py
class MyException(Exception):
pass
class FooBar:
def __init__(self):
pass
def bar(self):
raise ValueError('Hello World.')
def foo(self):
raise MyException('Hello World.')
.
# foobar_unittest.py
import unittest
import foobar as fb
class MyException(Exception):
pass
class FooBarTestCases(unittest.TestCase):
def test_bar(self):
with self.assertRaises(ValueError):
fb.FooBar().bar()
def test_foo(self):
with self.assertRaises(MyException):
fb.FooBar().foo()
if __name__ == '__main__':
unittest.main()
When running unit-test on foobar.py, why does the function raising the user-defined exception (foo) fail to pass the test?
>>> python2.7 foobar_unittest.py
.E
======================================================================
ERROR: test_foo (__main__.FooBarTestCases)
----------------------------------------------------------------------
Traceback (most recent call last):
File "foobar_unittest.py", line 11, in test_foo
fb.FooBar().foo()
File "/a_path/foobar.py", line 9, in foo
raise MyException('Hello World.')
MyException: Hello World.
----------------------------------------------------------------------
Ran 2 tests in 0.000s
FAILED (errors=1)

import MyException from foobar, don't redefine it.
import unittest
from foobar import MyException
import foobar as fb
class FooBarTestCases(unittest.TestCase):
def test_bar(self):
with self.assertRaises(ValueError):
fb.FooBar().bar()
def test_foo(self):
with self.assertRaises(MyException):
fb.FooBar().foo()
if __name__ == '__main__':
unittest.main()
This code should work now as
..
----------------------------------------------------------------------
Ran 2 tests in 0.001s
OK

Be aware that the same happens (happened to me) if you use reload to import your exceptions.
In my unittests I have the relevant imports like
from importlib import reload
import foobar
reload(foobar)
from foobar import MyException
This does not work, too, for whatever reason. Writing this just as
from foobar import MyException
will work. Then, of course, you have to reload the modules yourself.
In case you wonder why I am using reload: How do I unload (reload) a Python module?.

Why python mock patch doesn't work?

I have two files
spike.py
class T1(object):
def foo(self, afd):
return "foo"
def get_foo(self):
return self.foo(1)
def bar():
return "bar"
test_spike.py:
from unittest import TestCase
import unittest
from mock import patch, MagicMock
from spike import T1, bar
class TestStuff(TestCase):
#patch('spike.T1.foo', MagicMock(return_value='patched'))
def test_foo(self):
foo = T1().get_foo()
self.assertEqual('patched', foo)
#patch('spike.bar')
def test_bar(self, mock_obj):
mock_obj.return_value = 'patched'
bar = bar()
self.assertEqual('patched', bar)
if __name__ == "__main__":
unittest.main()
When I run python test_spike.py, the first test case would pass, but the second would fail.
and I switch to use nosetests test_spike.py, then both two are failed.
I don't understand how this happened? These cases supposed to pass all.

Access bar using spike.bar. Imported bar is not affected by mock.patch.
from unittest import TestCase
import unittest
from mock import patch, MagicMock
from spike import T1
import spike # <----
class TestShit(TestCase):
#patch('spike.T1.foo', MagicMock(return_value='patched'))
def test_foo(self):
foo = T1().get_foo()
self.assertEqual('patched', foo)
#patch('spike.bar')
def test_bar(self, mock_obj):
mock_obj.return_value = 'patched'
bar = spike.bar() # <-----
self.assertEqual('patched', bar)
if __name__ == "__main__":
unittest.main()

For test_foo you are not using patch correctly. You should be using it like this:
class TestFoo(TestCase):
#patch.object(T1, 'foo', MagicMock(return_value='patched'))
def test_foo(self):
foo = T1().get_foo()
self.assertEqual('patched', foo)
that gives me:
nosetests test_spike.py
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
Now the second example does not work because you import bar function (get a reference to it) and then try to mock it. When you mock something you can't change what your variables hold (reference to original function). To fix this you should use #falsetru suggested method like:
from unittest import TestCase
import unittest
from mock import patch
import spike
class TestFoo(TestCase):
#patch('spike.bar')
def test_bar(self, mock_obj):
mock_obj.return_value = 'patched'
value = spike.bar()
self.assertEqual('patched', value)
if __name__ == "__main__":
unittest.main()
this gives me:
python test_spike.py
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
But when I try to run it with nose I get:
nosetests test_spike.py
F
======================================================================
FAIL: test_bar (src.test_spike.TestFoo)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/zilva/envs/test/local/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
return func(*args, **keywargs)
File "/home/zilva/git/test/src/test_spike.py", line 11, in test_bar
self.assertEqual('patched', value)
AssertionError: 'patched' != 'bar'
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (failures=1)
This happends because I am patching not in the right place. My directory structure is:
test/
└── src/
├── spike.py
├── test_spike.py
└── __init__.py
and I run tests from src directory so I should be patching using path from project root directory like:
#patch('src.spike.bar')
and this would give me:
nosetests test_spike.py
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
or if I am at test directory:
nosetests src/test_spike.py
.
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK

To elaborate on the very helpful top answer, let me paraphrase the official documentation for unittest.mock.
a.py
class SomeClass:
...
b.py
import a
from a import SomeClass
def some_function():
a.SomeClass()
SomeClass()
If you write mock.patch('a.SomeClass'), this will affect the first line of some_function. If you write mock.patch('b.SomeClass'), this will affect the second line.

How to fail a python unittest if the setUpClass throws exception

I am having little trouble using the python setUpClass.
For example consider the following case
class MyTest(unittest.case.TestCase):
#classmethod
def setUpClass(cls):
print "Test setup"
try:
1/0
except:
raise
#classmethod
def tearDownClass(cls):
print "Test teardown"
A couple of questions
Is the above code the right way to handle test setUpClass exceptions (by raising it so that the python unittest can take care of it), there are fail(), skip() methods, but those can only be used by test instances and not the test classes.
When there is a setUpClass exception, how can we ensure that tearDownClass runs (unittest doesn't run it, should we manualy call it).

You can call tearDownClass on an exception as Jeff points it out, but you may also implements the __del__(cls) method :
import unittest
class MyTest(unittest.case.TestCase):
#classmethod
def setUpClass(cls):
print "Test setup"
try:
1/0
except:
raise
#classmethod
def __del__(cls):
print "Test teardown"
def test_hello(cls):
print "Hello"
if __name__ == '__main__':
unittest.main()
Will have the following output :
Test setup
E
======================================================================
ERROR: setUpClass (__main__.MyTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "my_test.py", line 8, in setUpClass
1/0
ZeroDivisionError: integer division or modulo by zero
----------------------------------------------------------------------
Ran 0 tests in 0.000s
FAILED (errors=1)
Test teardown
Note : you should be aware that the __del__ method will be called at the end of the program execution, which is maybe not what you want if you have a more than one test class.
Hope it helps

The best option would be is to add handler for the except which calls tearDownClass and re-raise exception.
#classmethod
def setUpClass(cls):
try:
super(MyTest, cls).setUpClass()
# setup routine...
except Exception: # pylint: disable = W0703
super(MyTest, cls).tearDownClass()
raise

import contextlib
class MyTestCase(unitest.TestCase):
#classmethod
def setUpClass(cls):
with contextlib.ExitStack() as stack:
# ensure teardown is called if error occurs
stack.callback(cls.tearDownClass)
# Do the things here!
# remove callback at the end if no error found
stack.pop_all()

use tearDownModule. It should be called after setUpClass runs.

Running unittest.main() from a module?

I wrote a little function that dynamically defines unittest.TestCase classes (trivial version below).
When I moved it out of the same source file into its own module, I can't figure out how to get unittest to discover the new classes. Calling unittest.main() from either file doesn't execute any tests.
factory.py:
import unittest
_testnum = 0
def test_factory(a, b):
global _testnum
testname = 'dyntest' + str(_testnum)
globals()[testname] = type(testname, (unittest.TestCase,), {'testme': lambda self: self.assertEqual(a, b)})
_testnum += 1
def finish():
unittest.main()
someotherfile.py:
from factory import test_factory, finish
test_factory(1, 1)
test_factory(1, 2)
if __name__ == '__main__':
finish()
Output:
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
So it doesn't execute any tests.
Note that keeping it all in the same file works as expected:
import unittest
_testnum = 0
def test_factory(a, b):
global _testnum
testname = 'dyntest' + str(_testnum)
globals()[testname] = type(testname, (unittest.TestCase,), {'testme': lambda self: self.assertEqual(a, b)})
_testnum += 1
test_factory(1, 1)
test_factory(1, 2)
if __name__ == '__main__':
unittest.main()
Output (as expected):
.F
======================================================================
FAIL: testme (__main__.dyntest1)
----------------------------------------------------------------------
Traceback (most recent call last):
File "partb.py", line 11, in <lambda>
globals()[testname] = type(testname, (unittest.TestCase,), {'testme': lambda self: self.assertEqual(a, b)})
AssertionError: 1 != 2
----------------------------------------------------------------------
Ran 2 tests in 0.008s
FAILED (failures=1)
How I use my test_factory() function such that I can execute all of the TestCase objects it defines from a separate source file?

The general idea (what unittest.main does for you) is:
suite = unittest.TestLoader().loadTestsFromTestCase(SomeTestCase)
unittest.TextTestRunner(verbosity=2).run(suite)
as per http://docs.python.org/library/unittest.html?highlight=unittest#module-unittest . Your test cases are hidden in globals() by the test_factory function, so just do a dir(), find the globals that are instances of unittest.TestCase (or ones with names starting with 'dyntest', etc), and just build your suite that way and run it.

By default, unittest.main() looks for unit TestCase objects in the main module. The test_factory creates the TestCase objects in its own module. That's why moving it outside of the main module causes the behavior you see.
Try:
def finish():
unittest.main(module=__name__)