We Keep Coding

Python unit-testing - How to patch an async call internal to the method I am testing

Im using unittest.mock for building tests for my python code. I have a method that I am trying to test that contains a async call to another function. I want to patch that async call so that I can just have Mock return a testing value for the asset id, and not actually call the async method. I have tried many things I've found online, but none have worked thus far.
Simplified example below:
test.py
import pytest
from app.create.creations import generate_new_asset
from app.fakeapi.utils import create_asset
from unittest.mock import Mock, patch
#patch("app.fakeapi.utils.create_asset")
#pytest.mark.anyio
async def test_generate_new_asset(mock_create):
mock_create.return_value = 12345678
await generate_new_asset()
...
creations.py
from app.fakeapi.utils import create_asset
...
async def generate_new_asset()
...
# When I run tests this does not return the 12345678 value, but actually calls the `create_asset` method.
return await create_asset(...)

Testing async code is bit tricky. If you are using python3.8 or higher AsyncMock is available.
Note: it will work only for Python > 3.8
I think in your case event loop is missing. Here is the code which should work, you may need to do few tweaks. You may also need to install pytest-mock. Having it as fixture will allow you to have mock different values for testing for different scenarios.
import asyncio
from unittest.mock import AsyncMock, Mock
#pytest.fixture(scope="module")
def mock_create_asset(mocker):
async_mock = AsyncMock()
mocker.patch('app.fakeapi.utils.create_asset', side_effect=async_mock)
return async_mock
#pytest.fixture(scope="module")
def event_loop():
return asyncio.get_event_loop()
#pytest.mark.asyncio
async def test_generate_new_asset(mock_create_asset):
mock_create_asset.return_value = 12345678
await generate_new_asset()

How to make an unified interface for synchronous and asynchronous code in Python? [duplicate]

When implementing classes that have uses in both synchronous and asynchronous applications, I find myself maintaining virtually identical code for both use cases.
Just as an example, consider:
from time import sleep
import asyncio
class UselessExample:
def __init__(self, delay):
self.delay = delay
async def a_ticker(self, to):
for i in range(to):
yield i
await asyncio.sleep(self.delay)
def ticker(self, to):
for i in range(to):
yield i
sleep(self.delay)
def func(ue):
for value in ue.ticker(5):
print(value)
async def a_func(ue):
async for value in ue.a_ticker(5):
print(value)
def main():
ue = UselessExample(1)
func(ue)
loop = asyncio.get_event_loop()
loop.run_until_complete(a_func(ue))
if __name__ == '__main__':
main()
In this example, it's not too bad, the ticker methods of UselessExample are easy to maintain in tandem, but you can imagine that exception handling and more complicated functionality can quickly grow a method and make it more of an issue, even though both methods can remain virtually identical (only replacing certain elements with their asynchronous counterparts).
Assuming there's no substantial difference that makes it worth having both fully implemented, what is the best (and most Pythonic) way of maintaining a class like this and avoiding needless duplication?

There is no one-size-fits-all road to making an asyncio coroutine-based codebase useable from traditional synchronous codebases. You have to make choices per codepath.
Pick and choose from a series of tools:
Synchronous versions using asyncio.run()
Provide synchronous wrappers around coroutines, which block until the coroutine completes.
Even an async generator function such as ticker() can be handled this way, in a loop:
class UselessExample:
def __init__(self, delay):
self.delay = delay
async def a_ticker(self, to):
for i in range(to):
yield i
await asyncio.sleep(self.delay)
def ticker(self, to):
agen = self.a_ticker(to)
try:
while True:
yield asyncio.run(agen.__anext__())
except StopAsyncIteration:
return
These synchronous wrappers can be generated with helper functions:
from functools import wraps
def sync_agen_method(agen_method):
#wraps(agen_method)
def wrapper(self, *args, **kwargs):
agen = agen_method(self, *args, **kwargs)
try:
while True:
yield asyncio.run(agen.__anext__())
except StopAsyncIteration:
return
if wrapper.__name__[:2] == 'a_':
wrapper.__name__ = wrapper.__name__[2:]
return wrapper
then just use ticker = sync_agen_method(a_ticker) in the class definition.
Straight-up coroutine methods (not generator coroutines) could be wrapped with:
def sync_method(async_method):
#wraps(async_method)
def wrapper(self, *args, **kwargs):
return async.run(async_method(self, *args, **kwargs))
if wrapper.__name__[:2] == 'a_':
wrapper.__name__ = wrapper.__name__[2:]
return wrapper
Factor out common components
Refactor out the synchronous parts, into generators, context managers, utility functions, etc.
For your specific example, pulling out the for loop into a separate generator would minimise the duplicated code to the way the two versions sleep:
class UselessExample:
def __init__(self, delay):
self.delay = delay
def _ticker_gen(self, to):
yield from range(to)
async def a_ticker(self, to):
for i in self._ticker_gen(to):
yield i
await asyncio.sleep(self.delay)
def ticker(self, to):
for i in self._ticker_gen(to):
yield i
sleep(self.delay)
While this doesn't make much of any difference here it can work in other contexts.
Abstract Syntax Tree tranformation
Use AST rewriting and a map to transform coroutines into synchronous code. This can be quite fragile if you are not careful on how you recognise utility functions such as asyncio.sleep() vs time.sleep():
import inspect
import ast
import copy
import textwrap
import time
asynciomap = {
# asyncio function to (additional globals, replacement source) tuples
"sleep": ({"time": time}, "time.sleep")
}
class AsyncToSync(ast.NodeTransformer):
def __init__(self):
self.globals = {}
def visit_AsyncFunctionDef(self, node):
return ast.copy_location(
ast.FunctionDef(
node.name,
self.visit(node.args),
[self.visit(stmt) for stmt in node.body],
[self.visit(stmt) for stmt in node.decorator_list],
node.returns and ast.visit(node.returns),
),
node,
)
def visit_Await(self, node):
return self.visit(node.value)
def visit_Attribute(self, node):
if (
isinstance(node.value, ast.Name)
and isinstance(node.value.ctx, ast.Load)
and node.value.id == "asyncio"
and node.attr in asynciomap
):
g, replacement = asynciomap[node.attr]
self.globals.update(g)
return ast.copy_location(
ast.parse(replacement, mode="eval").body,
node
)
return node
def transform_sync(f):
filename = inspect.getfile(f)
lines, lineno = inspect.getsourcelines(f)
ast_tree = ast.parse(textwrap.dedent(''.join(lines)), filename)
ast.increment_lineno(ast_tree, lineno - 1)
transformer = AsyncToSync()
transformer.visit(ast_tree)
tranformed_globals = {**f.__globals__, **transformer.globals}
exec(compile(ast_tree, filename, 'exec'), tranformed_globals)
return tranformed_globals[f.__name__]
While the above is probably far from complete enough to fit all needs, and transforming AST trees can be daunting, the above would let you maintain just the async version and map that version to synchronous versions directly:
>>> import example
>>> del example.UselessExample.ticker
>>> example.main()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/.../example.py", line 32, in main
func(ue)
File "/.../example.py", line 21, in func
for value in ue.ticker(5):
AttributeError: 'UselessExample' object has no attribute 'ticker'
>>> example.UselessExample.ticker = transform_sync(example.UselessExample.a_ticker)
>>> example.main()
0
1
2
3
4
0
1
2
3
4

async/await is infectious by design.
Accept that your code will have different users — synchronous and asynchronous, and that these users will have different requirements, that over time the implementations will diverge.
Publish separate libraries
For example, compare aiohttp vs. aiohttp-requests vs. requests.
Likewise, compare asyncpg vs. psycopg2.
How to get there
Opt1. (easy) clone implementation, allow them to diverge.
Opt2. (sensible) partial refactor, let e.g. async library depend on and import sync library.
Opt3. (radical) create a "pure" library that can be used both in sync and async program. For example, see https://github.com/python-hyper/hyper-h2 .
On the upside, testing is easier and thorough. Consider how hard (or impossible) it is force the test framework to evaluate all possible concurrent execution orders in an async program. Pure library doesn't need that :)
On the down-side this style of programming requires different thinking, is not always straightforward, and may be suboptimal. For example, instead of await socket.read(2**20) you'd write for event in fsm.push(data): ... and rely on your library user to provide you with data in good-sized chunks.
For context, see the backpressure argument in https://vorpus.org/blog/some-thoughts-on-asynchronous-api-design-in-a-post-asyncawait-world/

How to timeout an async test in pytest with fixture?

I am testing an async function that might get deadlocked. I tried to add a fixture to limit the function to only run for 5 seconds before raising a failure, but it hasn't worked so far.
Setup:
pipenv --python==3.6
pipenv install pytest==4.4.1
pipenv install pytest-asyncio==0.10.0
Code:
import asyncio
import pytest
#pytest.fixture
def my_fixture():
# attempt to start a timer that will stop the test somehow
asyncio.ensure_future(time_limit())
yield 'eggs'
async def time_limit():
await asyncio.sleep(5)
print('time limit reached') # this isn't printed
raise AssertionError
#pytest.mark.asyncio
async def test(my_fixture):
assert my_fixture == 'eggs'
await asyncio.sleep(10)
print('this should not print') # this is printed
assert 0
--
Edit: Mikhail's solution works fine. I can't find a way to incorporate it into a fixture, though.

Convenient way to limit function (or block of code) with timeout is to use async-timeout module. You can use it inside your test function or, for example, create a decorator. Unlike with fixture it'll allow to specify concrete time for each test:
import asyncio
import pytest
from async_timeout import timeout
def with_timeout(t):
def wrapper(corofunc):
async def run(*args, **kwargs):
with timeout(t):
return await corofunc(*args, **kwargs)
return run
return wrapper
#pytest.mark.asyncio
#with_timeout(2)
async def test_sleep_1():
await asyncio.sleep(1)
assert 1 == 1
#pytest.mark.asyncio
#with_timeout(2)
async def test_sleep_3():
await asyncio.sleep(3)
assert 1 == 1
It's not hard to create decorator for concrete time (with_timeout_5 = partial(with_timeout, 5)).
I don't know how to create texture (if you really need fixture), but code above can provide starting point. Also not sure if there's a common way to achieve goal better.

There is a way to use fixtures for timeout, one just needs to add the following hook into conftest.py.
Any fixture prefixed with timeout must return a number of seconds(int, float) the test can run.
The closest fixture w.r.t scope is chosen. autouse fixtures have lesser priority than explicitly chosen ones. Later one is preferred. Unfortunately order in the function argument list does NOT matter.
If there is no such fixture, the test is not restricted and will run indefinitely as usual.
The test must be marked with pytest.mark.asyncio too, but that is needed anyway.
# Add to conftest.py
import asyncio
import pytest
_TIMEOUT_FIXTURE_PREFIX = "timeout"
#pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_setup(item: pytest.Item):
"""Wrap all tests marked with pytest.mark.asyncio with their specified timeout.
Must run as early as possible.
Parameters
----------
item : pytest.Item
Test to wrap
"""
yield
orig_obj = item.obj
timeouts = [n for n in item.funcargs if n.startswith(_TIMEOUT_FIXTURE_PREFIX)]
# Picks the closest timeout fixture if there are multiple
tname = None if len(timeouts) == 0 else timeouts[-1]
# Only pick marked functions
if item.get_closest_marker("asyncio") is not None and tname is not None:
async def new_obj(*args, **kwargs):
"""Timed wrapper around the test function."""
try:
return await asyncio.wait_for(
orig_obj(*args, **kwargs), timeout=item.funcargs[tname]
)
except Exception as e:
pytest.fail(f"Test {item.name} did not finish in time.")
item.obj = new_obj
Example:
#pytest.fixture
def timeout_2s():
return 2
#pytest.fixture(scope="module", autouse=True)
def timeout_5s():
# You can do whatever you need here, just return/yield a number
return 5
async def test_timeout_1():
# Uses timeout_5s fixture by default
await aio.sleep(0) # Passes
return 1
async def test_timeout_2(timeout_2s):
# Uses timeout_2s because it is closest
await aio.sleep(5) # Timeouts
WARNING
Might not work with some other plugins, I have only tested it with pytest-asyncio, it definitely won't work if item is redefined by some hook.

I just loved Quimby's approach of marking tests with timeouts. Here's my attempt to improve it, using pytest marks:
# tests/conftest.py
import asyncio
#pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_pyfunc_call(pyfuncitem: pytest.Function):
"""
Wrap all tests marked with pytest.mark.async_timeout with their specified timeout.
"""
orig_obj = pyfuncitem.obj
if marker := pyfuncitem.get_closest_marker("async_timeout"):
async def new_obj(*args, **kwargs):
"""Timed wrapper around the test function."""
try:
return await asyncio.wait_for(orig_obj(*args, **kwargs), timeout=marker.args[0])
except (asyncio.CancelledError, asyncio.TimeoutError):
pytest.fail(f"Test {pyfuncitem.name} did not finish in time.")
pyfuncitem.obj = new_obj
yield
def pytest_configure(config: pytest.Config):
config.addinivalue_line("markers", "async_timeout(timeout): cancels the test execution after the specified amount of seconds")
Usage:
#pytest.mark.asyncio
#pytest.mark.async_timeout(10)
async def potentially_hanging_function():
await asyncio.sleep(20)
It should not be hard to include this to the asyncio mark on pytest-asyncio, so we can get a syntax like:
#pytest.mark.asyncio(timeout=10)
async def potentially_hanging_function():
await asyncio.sleep(20)
EDIT: looks like there's already a PR for that.

How to use coroutine as a pytest fixture?

Is it possible to write pytest fixtures as tornado coroutines? For example, I want to write a fixture for creating a db, like this:
from tornado import gen
import pytest
#pytest.fixture
#gen.coroutine
def get_db_connection():
# set up
db_name = yield create_db()
connection = yield connect_to_db(db_name)
yield connection
# tear down
yield drop_db(db_name)
#pytest.mark.gen_test
def test_something(get_db_connection):
# some tests
Obviously, this fixture does not work as expected, as it is called as a function, not as coroutine. Is there a way to fix it?

After some research, I came out with this solution:
from functools import partial
from tornado import gen
import pytest
#pytest.fixture
def get_db_connection(request, io_loop): # io_loop is a fixture from pytest-tornado
def fabric():
#gen.coroutine
def set_up():
db_name = yield create_db()
connection = yield connect_to_db(db_name)
raise gen.Return(connection)
#gen.coroutine
def tear_down():
yield drop_db(db_name)
request.addfinalizer(partial(io_loop.run_sync, tear_down))
connection = io_loop.run_sync(set_up)
return connection
return fabric
#pytest.mark.gen_test
def test_something(get_db_connection):
connection = get_db_connection() # note brackets
I'm sure, that it can be done cleaner using some pylint magic.
I found this slides very useful.
EDIT: I found out that above method has a limitation. You can't change scope of get_db_connection fixture, as it uses io_loop fixture with scope "function".

Limit connections for particular view in Tornado

I have view that takes a lot of memory and is asynchronous. Can I limit count of connections simultaneously working inside handler function (like critical section with N max workers inside).
Is this possible in Tornado?
Like:
#tornado.web.asynchronous
def get(self):
with critical_section(count=5):
# some code
Thanks

Toro provides synchronization primitives similar to those found in the threading module for Tornado coroutines. You could use its BoundedSemaphore to gate entry to the handler body:
# global semaphore
sem = toro.BoundedSemaphore(5)
#gen.coroutine
def get(self):
with (yield sem.acquire()):
# do work

Short answer:
As far as I understand Tornado and other frameworks that use Future/Deferred/generator based concurrency, this is not possible. However, it should definitely be possible using higher-order functions, i.e. a critical_section() helper function that takes the body of the with-block as a parameter.
Long answer:
To my best knowledge, Tornado's concurrency works very much like that of Twisted; which means non-blocking calls are limited to using Futures and yield (based on Twisted's #inlineCallbacks or whatever is the equivalent in Tornado).
In order to implement a critical_section context manager, it would have to cooperate with the reactor internally; this can only happen using callbacks or yield. However, neither is composable with context managers.
I'd actually already thrown up some code until I remembered this. This is the code I came up with:
import sys
from contextlib import contextmanager
from collections import defaultdict
from tornado.concurrent import Future
_critical_sections = defaultdict(lambda: (0, []))
#contextmanager
def critical_section(count):
# get the code location of the critical section
frame = sys._getframe()
orig_caller = frame.f_back.f_back
lineno = orig_caller.f_lineno
filename = orig_caller.f_code.co_filename
loc = (filename, lineno)
count, waiters = _critical_sections[loc]
if count > 5:
future = Future()
_critical_sections[loc] = (count + 1, waiters + [future])
# XXX: not possible; either have to set a callback or use yield, but
# then this context manager itself would not work as you'd expect:
future.wait() # <---- not possible in Tornado nor Twisted; only in Gevent/Eventlet
fn(*args, **kwargs)
else:
_critical_sections[loc] = (count + 1, waiters)
try:
yield
finally:
count, waiters = _critical_sections[loc]
_, w_future = waiters[0]
_critical_sections[loc] = (count, waiters[1:])
w_future.set_result(None)
(I have not tested it in anyway, and it's not runnable on Tornado anyway.)
Now, if you're happy with the proposed approach, here's something to get you started (or maybe it even works out of the box):
...
def _critical_section(count, fn, *args, **kwargs):
...
if count > 5:
future = Future()
future.add_done_callback(lambda _: fn(*args, **kwargs))
_critical_sections[loc] = (count + 1, waiters + [future])
# XXX: not possible; either have to set a callback or use yield, but
# then this context manager itself would not work as you'd expect:
return future
else:
_critical_sections[loc] = (count + 1, waiters)
try:
return fn()
finally:
... # same
then you could just turn it into a decorator:
from functools import wraps
def critical_section(count):
def decorate(fn):
#wraps(fn)
def ret(*args, **kwargs):
return _critical_section(count, fn, *args, **kwargs)
return ret
return decorate
Usage:
#tornado.web.asynchronous
def get(self):
#critical_section(count=5)
def some_code():
pass # do stuff
Also, the code is using sys._getframe(), which has (at least) 2 implications:
it will make the code slower when run on PyPy (until/unless PyPy has become able to JIT-compile functions that use sys._getframe), but most of the time, it's an acceptable tradeoff when it comes to web code
I don't think the code will work if you compile it to .pyc and remove the .py—it shouldn't then be able to determine the filename and line number of the calling code, so it will not (probably) be possible to uniquely distinguish the location of the critical section, in which case you'd have to use lock objects.
NOTE: The context manager version would be perfectly feasible on Gevent (http://gevent.org) or Eventlet.