I'm wondering if there's a way to easily mock an Awaitable object for python tests. I know how to use AsyncMocks to create mocks which mirror coroutines (i.e. will return Awaitable objects from method calls), but am running into a bit of problem when trying to mock an Awaitable directly.
Here's an example:
import unittest
from asyncio import Task
from typing import Any
from unittest import IsolatedAsyncioTestCase
from unittest.mock import AsyncMock
async def function_using_awaitable(task: Task) -> Any:
await task
return task.result()
class TestFunction(IsolatedAsyncioTestCase):
async def test_function_using_awaitable(self):
mock_task = AsyncMock(spec=Task)
result = await function_using_awaitable(mock_task)
mock_task.assert_awaited_once()
assert result == mock_task.result.return_value
if __name__ == '__main__':
unittest.main()
which throws up:
Traceback (most recent call last):
...
File ..., line 9, in function_using_awaitable
await task
TypeError: object AsyncMock can't be used in 'await' expression
Digging into the Python docs it looks like the __await__ magic method is one of the few that isn't supported in mocks so wondering how I can go about doing this?
One solution might be to just extend the AsyncMock to be an awaitable mock object, adding await. In code, that could look like
from typing import Iterator, Any
async def function_using_awaitable(task: Task) -> Any:
await task
return task.result()
class AwaitableMock(AsyncMock):
def __await__(self) -> Iterator[Any]:
self.await_count += 1
return iter([])
class TestFunction(IsolatedAsyncioTestCase):
async def test_function_using_awaitable(self):
mock_task = AwaitableMock(spec=Task)
result = await function_using_awaitable(mock_task)
mock_task.assert_awaited_once()
assert result == mock_task.result.return_value
Related
Im using unittest.mock for building tests for my python code. I have a method that I am trying to test that contains a async call to another function. I want to patch that async call so that I can just have Mock return a testing value for the asset id, and not actually call the async method. I have tried many things I've found online, but none have worked thus far.
Simplified example below:
test.py
import pytest
from app.create.creations import generate_new_asset
from app.fakeapi.utils import create_asset
from unittest.mock import Mock, patch
#patch("app.fakeapi.utils.create_asset")
#pytest.mark.anyio
async def test_generate_new_asset(mock_create):
mock_create.return_value = 12345678
await generate_new_asset()
...
creations.py
from app.fakeapi.utils import create_asset
...
async def generate_new_asset()
...
# When I run tests this does not return the 12345678 value, but actually calls the `create_asset` method.
return await create_asset(...)
Testing async code is bit tricky. If you are using python3.8 or higher AsyncMock is available.
Note: it will work only for Python > 3.8
I think in your case event loop is missing. Here is the code which should work, you may need to do few tweaks. You may also need to install pytest-mock. Having it as fixture will allow you to have mock different values for testing for different scenarios.
import asyncio
from unittest.mock import AsyncMock, Mock
#pytest.fixture(scope="module")
def mock_create_asset(mocker):
async_mock = AsyncMock()
mocker.patch('app.fakeapi.utils.create_asset', side_effect=async_mock)
return async_mock
#pytest.fixture(scope="module")
def event_loop():
return asyncio.get_event_loop()
#pytest.mark.asyncio
async def test_generate_new_asset(mock_create_asset):
mock_create_asset.return_value = 12345678
await generate_new_asset()
The following code works as expected for the synchronous part but gives me a TypeError for the async call (TypeError: object NoneType can't be used in 'await' expression), presumably because the mock constructor can't properly deal with the spec. How do I properly tell Mockito that it needs to set up an asynchronous mock for async_method ?
class MockedClass():
def sync_method(self):
pass
async def async_method(self):
pass
class TestedClass():
def do_something_sync(self, mocked: MockedClass):
mocked.sync_method()
async def do_something_async(self, mocked: MockedClass):
await mocked.async_method()
#pytest.mark.asyncio
async def test():
my_mock = mock(spec=MockedClass)
tested_class = TestedClass()
tested_class.do_something_sync(my_mock)
verify(my_mock).sync_method()
await tested_class.do_something_async(my_mock) # <- Fails here
verify(my_mock).async_method()
Edit:
For reference, this is how it works with the standard mocks (the behavior that I expect):
In mockito my_mock.async_method() would not return anything useful by default and without further configuration. (T.i. it returns None which is not awaitable.)
What I did in the past:
# a helper function
def future(value=None):
f = asyncio.Future()
f.set_result(value)
return f
# your code
#pytest.mark.asyncio
async def test():
my_mock = mock(spec=MockedClass)
when(my_mock).async_method().thenReturn(future(None)) # fill in whatever you expect the method to return
# ....
I need to test the following function
class C:
async def f():
a, b = await self.g() # need to mock g
However, the following test got the error of TypeError: object tuple can't be used in 'await' expression
#pytest.mark.asyncio
async def test_f():
sut = C()
sut.g = MagicMock(return_value=(1,2)) # TypeError: object tuple can't be used in 'await' expression
await sut.f()
sut.g.assert_called_once()
Use AsyncMock instead of the MagicMock:
from unittest.mock import AsyncMock
#pytest.mark.asyncio
async def test_f():
sut = C()
sut.g = AsyncMock(return_value=(1,2))
await sut.f()
sut.g.assert_called_once()
AsyncMock is part of the standard library since Python 3.8; if you need a backport for older versions, install and use the asyncmock package.
I've got a situation where I can't be sure if a function will be sync, or async. In other words, the function has a type signature of Union[Callable[..., Awaitable[Any]], Callable[..., Any]].
I can't seem to find a good (non-deprecated) way of turning the above type signature into a consistent type of Callable[..., Awaitable[Any]].
The only way I was able to find how to do this, is by doing this:
import asyncio
import inspect
from typing import Any, Awaitable, Callable, List, Union
def force_awaitable(function: Union[Callable[..., Awaitable[Any]], Callable[..., Any]]) -> Callable[..., Awaitable[Any]]:
if inspect.isawaitable(function):
# Already awaitable
return function
else:
# Make it awaitable
return asyncio.coroutine(function)
However, asyncio.coroutine (which is normally used as a decorator) is deprecated since Python 3.8. https://docs.python.org/3/library/asyncio-task.html#asyncio.coroutine
The alternative provided does not work for me here, since I don't use asyncio.coroutine as a decorator. Unlike a decorator, the async keyword can't be used as a function.
How do I turn a sync function (not awaitable) into an async function (awaitable)?
Other considered options
Above I've shown that you can detect if something is awaitable. This allows us to change how to call like so:
def my_function():
pass
if inspect.isawaitable(function):
await my_function()
else:
my_function()
However, this feels clunky, can create messy code, and creates unnecessary checks inside big loops. I want to be able to define how to call the function before I'm entering my loop.
In NodeJS I would simply await the sync function:
// Note: Not Python!
function sync() {}
await sync();
When I try to do the same thing in Python, I'm greeted with an error:
def sync():
pass
await sync() # AttributeError: 'method' object has no attribute '__await__'
You can return an async function with the original function object in its scope:
def to_coroutine(f:Callable[..., Any]):
async def wrapper(*args, **kwargs):
return f(*args, **kwargs)
return wrapper
def force_awaitable(function: Union[Callable[..., Awaitable[Any]], Callable[..., Any]]) -> Callable[..., Awaitable[Any]]:
if inspect.iscoroutinefunction(function):
return function
else:
return to_coroutine(function)
Now, if function is not awaitable, force_awaitable will return a coroutine function that contains function.
def test_sync(*args, **kwargs):
pass
async def main():
await force_awaitable(test_sync)('foo', 'bar')
asyncio.run(main())
When I was trying to mock an async function in unittest with MagicMock, I got this exception:
TypeError: object MagicMock can't be used in 'await' expression
With sample code like:
# source code
class Service:
async def compute(self, x):
return x
class App:
def __init__(self):
self.service = Service()
async def handle(self, x):
return await self.service.compute(x)
# test code
import asyncio
import unittest
from unittest.mock import patch
class TestApp(unittest.TestCase):
#patch('__main__.Service')
def test_handle(self, mock):
loop = asyncio.get_event_loop()
app = App()
res = loop.run_until_complete(app.handle('foo'))
app.service.compute.assert_called_with("foo")
if __name__ == '__main__':
unittest.main()
How should I fix it with built-in python3 libraries?
In python 3.8+ you can make use of the AsyncMock
async def test_that_mock_can_be_awaited():
mock = AsyncMock()
mock.x.return_value = 123
result = await mock.x()
assert result == 123
The class AsyncMock object will behave so the object is recognized as an async function, and the result of a call is an awaitable.
>>> mock = AsyncMock()
>>> asyncio.iscoroutinefunction(mock)
True
>>> inspect.isawaitable(mock())
True
I ended up with this hack.
# monkey patch MagicMock
async def async_magic():
pass
MagicMock.__await__ = lambda x: async_magic().__await__()
It only works for MagicMock, not other pre-defined return_value
You can get mocks to return objects that can be awaited by using a Future. The following is a pytest test case, but something similar should be possible with unittest.
async def test_that_mock_can_be_awaited():
mock = MagicMock(return_value=Future())
mock.return_value.set_result(123)
result = await mock()
assert result == 123
In your case, since you're patching Service (which gets passed in as mock), mock.return_value = Future() should do the trick.
shaun shia provided really good universal solution, but i found what in python 3.8 you can use just #patch('__main__.Service', new=AsyncMock)
I found this comment very useful when trying to await a mock object in Python < 3.8. You simply create a child class AsyncMock that inherits from MagicMock and overwrites a __call__ method to be a coroutine:
class AsyncMock(MagicMock):
async def __call__(self, *args, **kwargs):
return super(AsyncMock, self).__call__(*args, **kwargs)
Then, inside your test, do:
#pytest.mark.asyncio
async def test_my_method():
# Test "my_method" coroutine by injecting an async mock
my_mock = AsyncMock()
assert await my_method(my_mock)
you might also want to install pytest-asyncio
To override async classes one needs to tell patch that the return_value needs to be AsyncMock. So use
#patch('__main__.Service', return_value=AsyncMock(Service))
def test_handle(self, mock):
loop = asyncio.get_event_loop()
app = App()
res = loop.run_until_complete(app.handle('foo'))
app.service.compute.assert_called_with("foo")
With that Service will be a MagicMock, but Service() will return an AsyncMock instance of Service.
If you'd like a pytest-mock solution compatible with < py3.8, I did something like this.
class AsyncMock(MagicMock):
async def __call__(self, *args, **kwargs):
return super().__call__(*args, **kwargs)
def test_my_method(mocker):
my_mock = mocker.patch("path.to.mocked.thing", AsyncMock())
my_mock.return_value = [1, 2, 3]
assert my_method()
Definitely borrowed from Tomasz's solution here!