We Keep Coding

How to mock a class with nested properties and autospec?

I'm wondering if it's possible to mock a class which contains properties by using patch and autospec? The goal in the example below is to mock (recursively) ClassB.
Example:
# file: class_c.py
class ClassC:
def get_default(self) -> list[int]:
return [1, 2, 3]
def delete(self, name: str):
print(f"delete {name}")
# ----------------------
# file: class_b.py
from class_c import ClassC
class ClassB:
def __init__(self) -> None:
self._ds = ClassC()
#property
def class_c(self) -> ClassC:
return self._ds
def from_config(self, cred: str) -> str:
return cred
# ----------------------
# file: class_a.py
from class_b import ClassB
class ClassA:
def __init__(self):
self._client = ClassB()
#property
def class_b(self) -> ClassB:
return self._client
def test(self, cred: str) -> str:
return cred
# ----------------------
# file: test.py
import pytest
from unittest import mock
#mock.patch("class_a.ClassB", autospec=True)
def test_class_a(_):
class_a = ClassA()
with pytest.raises(TypeError):
class_a.class_b.from_config() # ✅ raised - missing 1 required positional argument: 'cred'
with pytest.raises(TypeError):
class_a.class_b.class_c.delete() # <- ❌ Problem - should raise exception since autospec=True
The property class_c of the class ClassB is not mocked properly. I would expect TypeError when trying to call delete() without any argument
I've tried several things but without success. Any idea?
EDIT:
The code is just an example, and the test function was just written to demonstrate the expected behaviour. ClassB can be seen as a third-party service which needs to be mocked.
EDIT2:
Additionally to the accepted answer, I would propose to use PropertyMock for mocking properties:
def test_class_a():
class_b_mock = create_autospec(class_a.ClassB)
class_c_mock = create_autospec(class_b.ClassC)
type(class_b).class_c = PropertyMock(return_value=class_c_mock)
with mock.patch("class_a.ClassB", return_value=class_b_mock):
class_a_instance = ClassA()
with pytest.raises(TypeError):
class_a_instance.class_b.from_config()
with pytest.raises(TypeError):
class_a_instance.class_b.class_c.delete()

Once you patched the target class, anything you try to access under that class will be mocked with MagicMock (also recursively). Therefore, if you want to keep the specification of that class, then yes, you should use the autospec=true flag.
But because you are trying to mock a class within a class accessed by a property,
You have to keep the specification of each class you want to test:
def test_class_a():
class_b_mock = create_autospec(class_a.ClassB)
class_c_mock = create_autospec(class_b.ClassC)
class_b_mock.class_c = class_c_mock
with mock.patch("class_a.ClassB", return_value=class_b_mock):
class_a_instance = ClassA()
with pytest.raises(TypeError):
class_a_instance.class_b.from_config()
with pytest.raises(TypeError):
class_a_instance.class_b.class_c.delete()

Mypy: a generic that helps a decorator annotate a multiple dispatch

I want to put my functions on some registry and pick the required function based on some arguments provided (a version of multiple dispatch).
This is the sample code:
method_registry = {}
def accepts(cls):
""" A decorator that registers functions on some registry"""
def register(func):
method_registry[cls] = func
return func
return register
def handle(obj):
""" Pick a function that corresponds to the provided object class and run it """
handler = method_registry[obj.__class__]
handler(obj)
now, how do i plan to use it:
class Dog:
def bark(self):
print('bark')
class Cat:
def meow(self):
print('meow')
#accepts(Dog)
def handle_dog(obj):
obj.bark()
#accepts(Cat)
def handle_cat(obj):
obj.meow()
# Here comes multiple dispatch
handle(Dog())
handle(Cat())
Now, it all works, but when I try to annotate my functions for mypy, I have to type Cat and Dog twice:
#accepts(Dog)
def handle_dog(obj: Dog) -> None:
obj.bark()
#accepts(Cat)
def handle_cat(obj: Cat) -> None:
obj.meow()
so, I'm guessing there must be a way to create a generic type that could annotate my obj-s for me without duplicating the code.
But I cannot seem to make this work.
My attempt was like this:
V = TypeVar('V')
def accepts(cls: Type[V]) -> Callable[[Callable], Callable[[V], None]]:
def register(func: Callable) -> Callable[[V], None]:
method_registry[cls] = func
return func
return register
but this doesn't help:
#accepts(Dog)
def handle_dog(obj) -> None:
reveal_type(obj) # Revealed type is 'Any'
obj.bark()
Is there a way to make this work?

The code as it stands mandates you specify the type twice: once for MyPy and once for accept. These types could, in theory, be different.
If you'd like to have them both use the same value, one of them needs to change.
It'd me much easier for you to change your accepts method rather than write a plugin for MyPy. It could use the same annotation by inspecting your functions' __annotations__ field.
def accepts():
def register( func ):
cls = func.__annotations__["obj"]
method_registry[cls] = func
return func
return register
...
#accepts()
def handle_dog( obj: Dog ):
obj.bark()
#accepts()
def handle_cat( obj: Cat ):
obj.meow()

pytest discovery misses decorated methods

I want to be able to write a bunch of tests in a format similar to this:
class TestPytest:
#given(3)
#expect(3)
def test_passes(self, g, e):
assert g == e
#given(3)
#expect(4)
def test_fails(self, g, e):
assert g == e
def test_boring(self): # for comparison
pass
(I'm not convinced this is a good idea, but I'll be taking it in further directions, so it's not as strange as it looks.)
To that end, I've attempted to write these decorators:
import functools
class WrappedTest(object):
def __init__(self, f):
self.func = f
self.given = []
self.expects = []
def __get__(self, instance, owner):
#functools.wraps(self.func)
def call(*a, **kw):
return self.func(instance, self.given, self.expects,
*a, **kw)
return call
def given(*objects):
def wrapped(test):
if not isinstance(test, WrappedTest):
test_tmp = WrappedTest(test)
test = functools.update_wrapper(test_tmp, test)
test.given.extend(objects)
return test
return wrapped
def expect(*objects):
def wrapped(test):
if not isinstance(test, WrappedTest):
test_tmp = WrappedTest(test)
test = functools.update_wrapper(test_tmp, test)
test.expects.extend(objects)
return test
return wrapped
But when I try to run this test, pytest doesn't find test_passes or test_fails. It does find test_boring.
My working hypothesis is that I haven't properly wrapped the test methods. They show up as functions rather than methods:
>>> test_pytest.TestPytest().test_fails
<function test_pytest.test_fails>
>>> test_pytest.TestPytest().test_boring
<bound method TestPytest.test_boring of <test_pytest.TestPytest instance at 0x101f3dab8>>
But I'm not sure how to fix this. I've tried changing functools.wraps(self.func) to functools.wraps(self.func.__get__(instance, owner)), under the theory that it would then wrap with a bound method rather than a function. But that was kind of a guess, and it didn't work.
I know pytest is capable of finding decorated functions written properly, so presumably I'm doing something wrong, but I'm not sure what.

It looks like I was wrong about wrapping. Looking through the pytest source, it treats nested classes differently from methods. It accesses members through __dict__, which ignores __get__, so WrappedTest wasn't successfully pretending to be a method.
I've replaced the WrappedTest instance with a function, and it seems to be working fine (even without the #functools.wraps line):
import functools
from collections import namedtuple
def wrap_test_method(meth):
if hasattr(meth, '_storage'):
return meth
Storage = namedtuple('Storage', ['given', 'expects'])
sto = Storage(given=[], expects=[])
#functools.wraps(meth)
def new_meth(self, *a, **kw):
return meth(self, sto.given, sto.expects, *a, **kw)
new_meth._storage = sto
return new_meth
def given(*objects):
def decorator(test_method):
new_test_method = wrap_test_method(test_method)
new_test_method._storage.given.extend(objects)
return new_test_method
return decorator
def expect(*objects):
def decorator(test_method):
new_test_method = wrap_test_method(test_method)
new_test_method._storage.expects.extend(objects)
return new_test_method
return decorator

How to mock asyncio coroutines?

The following code fails with TypeError: 'Mock' object is not iterable in ImBeingTested.i_call_other_coroutines because I've replaced ImGoingToBeMocked by a Mock object.
How can I mock coroutines?
class ImGoingToBeMocked:
#asyncio.coroutine
def yeah_im_not_going_to_run(self):
yield from asyncio.sleep(1)
return "sup"
class ImBeingTested:
def __init__(self, hidude):
self.hidude = hidude
#asyncio.coroutine
def i_call_other_coroutines(self):
return (yield from self.hidude.yeah_im_not_going_to_run())
class TestImBeingTested(unittest.TestCase):
def test_i_call_other_coroutines(self):
mocked = Mock(ImGoingToBeMocked)
ibt = ImBeingTested(mocked)
ret = asyncio.get_event_loop().run_until_complete(ibt.i_call_other_coroutines())

Since mock library doesn't support coroutines I create mocked coroutines manually and assign those to mock object. A bit more verbose but it works.
Your example may look like this:
import asyncio
import unittest
from unittest.mock import Mock
class ImGoingToBeMocked:
#asyncio.coroutine
def yeah_im_not_going_to_run(self):
yield from asyncio.sleep(1)
return "sup"
class ImBeingTested:
def __init__(self, hidude):
self.hidude = hidude
#asyncio.coroutine
def i_call_other_coroutines(self):
return (yield from self.hidude.yeah_im_not_going_to_run())
class TestImBeingTested(unittest.TestCase):
def test_i_call_other_coroutines(self):
mocked = Mock(ImGoingToBeMocked)
ibt = ImBeingTested(mocked)
#asyncio.coroutine
def mock_coro():
return "sup"
mocked.yeah_im_not_going_to_run = mock_coro
ret = asyncio.get_event_loop().run_until_complete(
ibt.i_call_other_coroutines())
self.assertEqual("sup", ret)
if __name__ == '__main__':
unittest.main()

I am writting a wrapper to unittest which aims at cutting the boilerplate when writting tests for asyncio.
The code lives here: https://github.com/Martiusweb/asynctest
You can mock a coroutine with asynctest.CoroutineMock:
>>> mock = CoroutineMock(return_value='a result')
>>> asyncio.iscoroutinefunction(mock)
True
>>> asyncio.iscoroutine(mock())
True
>>> asyncio.run_until_complete(mock())
'a result'
It also works with the side_effect attribute, and an asynctest.Mock with a spec can return CoroutineMock:
>>> asyncio.iscoroutinefunction(Foo().coroutine)
True
>>> asyncio.iscoroutinefunction(Foo().function)
False
>>> asynctest.Mock(spec=Foo()).coroutine
<class 'asynctest.mock.CoroutineMock'>
>>> asynctest.Mock(spec=Foo()).function
<class 'asynctest.mock.Mock'>
All the features of unittest.Mock are expected to work correctly (patch(), etc).

Springing off of Andrew Svetlov's answer, I just wanted to share this helper function:
def get_mock_coro(return_value):
#asyncio.coroutine
def mock_coro(*args, **kwargs):
return return_value
return Mock(wraps=mock_coro)
This lets you use the standard assert_called_with, call_count and other methods and attributes a regular unittest.Mock gives you.
You can use this with code in the question like:
class ImGoingToBeMocked:
#asyncio.coroutine
def yeah_im_not_going_to_run(self):
yield from asyncio.sleep(1)
return "sup"
class ImBeingTested:
def __init__(self, hidude):
self.hidude = hidude
#asyncio.coroutine
def i_call_other_coroutines(self):
return (yield from self.hidude.yeah_im_not_going_to_run())
class TestImBeingTested(unittest.TestCase):
def test_i_call_other_coroutines(self):
mocked = Mock(ImGoingToBeMocked)
mocked.yeah_im_not_going_to_run = get_mock_coro()
ibt = ImBeingTested(mocked)
ret = asyncio.get_event_loop().run_until_complete(ibt.i_call_other_coroutines())
self.assertEqual(mocked.yeah_im_not_going_to_run.call_count, 1)

You can create asynchronous mocks yourself:
import asyncio
from unittest.mock import Mock
class AsyncMock(Mock):
def __call__(self, *args, **kwargs):
sup = super(AsyncMock, self)
async def coro():
return sup.__call__(*args, **kwargs)
return coro()
def __await__(self):
return self().__await__()

A slightly simplified example for python 3.6+ adapted from a few of the answers here:
import unittest
class MyUnittest()
# your standard unittest function
def test_myunittest(self):
# define a local mock async function that does what you want, such as throw an exception. The signature should match the function you're mocking.
async def mock_myasync_function():
raise Exception('I am testing an exception within a coroutine here, do what you want')
# patch the original function `myasync_function` with the one you just defined above, note the usage of `wrap`, which hasn't been used in other answers.
with unittest.mock.patch('mymodule.MyClass.myasync_function', wraps=mock_myasync_function) as mock:
with self.assertRaises(Exception):
# call some complicated code that ultimately schedules your asyncio corotine mymodule.MyClass.myasync_function
do_something_to_call_myasync_function()

You can use asynctest and import CoroutineMock or use asynctest.mock.patch

Dustin's answer is probably the right one in the vast majority of cases. I had a different issue where the coroutine needed to return more than one value, e.g. simulating a read() operation, as briefly described in my comment.
After some more testing, the code below worked for me, by defining an iterator outside the mocking function, effectively remembering the last value returned to send the next one:
def test_some_read_operation(self):
#...
data = iter([b'data', b''])
#asyncio.coroutine
def read(*args):
return next(data)
mocked.read = Mock(wraps=read)
# Here, the business class would use its .read() method which
# would first read 4 bytes of data, and then no data
# on its second read.
So, expanding on Dustin's answer, it would look like:
def get_mock_coro(return_values):
values = iter(return_values)
#asyncio.coroutine
def mock_coro(*args, **kwargs):
return next(values)
return Mock(wraps=mock_coro)
The two immediate downsides I can see in this approach are:
It doesn't allow for raising exceptions easily (e.g. first return some data, then raise an error on second read operation).
I haven't found a way to use the standard Mock .side_effect or .return_value attributes to make it more obvious and readable.

Well, there are a bunch of answers here already, but I'll contribute my expanded version of e-satis's answer. This class mocks an async function and tracks call count and call args, just like the Mock class does for sync functions.
Tested on Python 3.7.0.
class AsyncMock:
''' A mock that acts like an async def function. '''
def __init__(self, return_value=None, return_values=None):
if return_values is not None:
self._return_value = return_values
self._index = 0
else:
self._return_value = return_value
self._index = None
self._call_count = 0
self._call_args = None
self._call_kwargs = None
#property
def call_args(self):
return self._call_args
#property
def call_kwargs(self):
return self._call_kwargs
#property
def called(self):
return self._call_count > 0
#property
def call_count(self):
return self._call_count
async def __call__(self, *args, **kwargs):
self._call_args = args
self._call_kwargs = kwargs
self._call_count += 1
if self._index is not None:
return_index = self._index
self._index += 1
return self._return_value[return_index]
else:
return self._return_value
Example usage:
async def test_async_mock():
foo = AsyncMock(return_values=(1,2,3))
assert await foo() == 1
assert await foo() == 2
assert await foo() == 3

You can subclass Mock to act like a coroutine function:
class CoroMock(Mock):
async def __call__(self, *args, **kwargs):
return super(CoroMock, self).__call__(*args, **kwargs)
def _get_child_mock(self, **kw):
return Mock(**kw)
You can use CoroMock pretty much like a normal mock, with the caveat that calls will not be recorded until the coroutine is executed by an event loop.
If you have a mock object and you want to make a particular method a coroutine, you can use Mock.attach_mock like this:
mock.attach_mock(CoroMock(), 'method_name')

New in Python 3.8 AsyncMock
Setting the spec of a Mock, MagicMock, or AsyncMock to a class with
asynchronous and synchronous functions will automatically detect the
synchronous functions and set them as MagicMock (if the parent mock is
AsyncMock or MagicMock) or Mock (if the parent mock is Mock). All
asynchronous functions will be AsyncMock.
class ExampleClass:
def sync_foo():
pass
async def async_foo():
pass
a_mock = AsyncMock(ExampleClass)
a_mock.sync_foo
>>> <MagicMock name='mock.sync_foo' id='...'>
a_mock.async_foo
>>> <AsyncMock name='mock.async_foo' id='...'>
mock = Mock(ExampleClass)
mock.sync_foo
>>> <Mock name='mock.sync_foo' id='...'>
mock.async_foo
>>> <AsyncMock name='mock.async_foo' id='...'>
Special Attributes for AsyncMock:
assert_awaited
assert_awaited_once
assert_awaited_with
assert_awaited_once_with
assert_any_await
assert_has_awaits
assert_not_awaited
reset_mock
await_count
await_args
await_args_list

You can fake type of AsyncMock
import asyncio
from types import CoroutineType
from unittest.mock import AsyncMock
task = AsyncMock(spec=CoroutineType)
asyncio.iscoroutine(task) # True

How do you mock patch a python class and get a new Mock object for each instantiation?

OK,
I know this is mentioned in the manual, and probably has to do with side_effect and/or return_value, but a simple, direct example will help me immensely.
I have:
class ClassToPatch():
def __init__(self, *args):
_do_some_init_stuff()
def some_func():
_do_stuff()
class UUT():
def __init__(self, *args)
resource_1 = ClassToPatch()
resource_2 = ClassToPatch()
Now, I want to unit test the UUT class, and mock the ClassToPatch. Knowing the UUT class will instantiate exactly two ClassToPatch objects, I want the Mock framework to return a new Mock object for each instantiation, so I can later assert calls on each separately.
How do I achieve this using the #patch decorator in a test case? Namely, how to fix the following code sample?
class TestCase1(unittest.TestCase):
#patch('classToPatch.ClassToPatch',autospec=True)
def test_1(self,mock1,mock2):
_assert_stuff()

Here's a quick'n'dirty example to get you going:
import mock
import unittest
class ClassToPatch():
def __init__(self, *args):
pass
def some_func(self):
return id(self)
class UUT():
def __init__(self, *args):
resource_1 = ClassToPatch()
resource_2 = ClassToPatch()
self.test_property = (resource_1.some_func(), resource_2.some_func())
class TestCase1(unittest.TestCase):
#mock.patch('__main__.ClassToPatch', autospec = True)
def test_1(self, mock1):
ctpMocks = [mock.Mock(), mock.Mock()]
ctpMocks[0].some_func.return_value = "funky"
ctpMocks[1].some_func.return_value = "monkey"
mock1.side_effect = ctpMocks
u = UUT()
self.assertEqual(u.test_property, ("funky", "monkey"))
if __name__ == '__main__':
unittest.main()
I've added test_property to UUT so that the unit test does something useful. Now, without the mock test_property should be a tuple containing the ids of the two ClassToPatch instances. But with the mock it should be the tuple: ("funky", "monkey").
I've used the side_effect property of the mock object so that a different instance of ClassToPatch is returned on each call in the UUT initialiser.
Hope this helps.
Edit: Oh, by the way, when I run the unit test I get:
.
----------------------------------------------------------------------
Ran 1 test in 0.004s
OK

Here is another version which is more generic to handle any number of instances created:
class TestUUT:
def test_init(self, mocker):
class MockedClassToPatchMeta(type):
static_instance = mocker.MagicMock(spec=ClassToPatch)
def __getattr__(cls, key):
return MockedClassToPatchMeta.static_instance.__getattr__(key)
class MockedClassToPatch(metaclass=MockedClassToPatchMeta):
original_cls = ClassToPatch
instances = []
def __new__(cls, *args, **kwargs):
MockedClassToPatch.instances.append(
mocker.MagicMock(spec=MockedClassToPatch.original_cls))
MockedClassToPatch.instances[-1].__class__ = MockedClassToPatch
return MockedClassToPatch.instances[-1]
mocker.patch(__name__ + '.ClassToPatch', new=MockedClassToPatch)
UUT()
# since your original code created two instances
assert 2 == len(MockedClassToPatch.instances)
If you need more thorough validation for each instance you can access MockedClassToPatch.instances[0] or MockedClassToPatch.instances[1].
I've also created a helper library to generate the meta class boilerplate for me. To generate the needed code for your example I wrote:
print(PytestMocker(mocked=ClassToPatch, name=__name__).mock_classes().mock_classes_static().generate())