where
This is on Linux, Python 3.5.1.
what
I'm developing a monitor process with asyncio, whose tasks at various places await on asyncio.sleep calls of various durations.
There are points in time when I would like to be able to interrupt all said asyncio.sleep calls and let all tasks proceed normally, but I can't find how to do that. An example is for graceful shutdown of the monitor process.
how (failed assumption)
I thought that I could send an ALRM signal to that effect, but the process dies. I tried catching the ALRM signal with:
def sigalrm_sent(signum, frame):
tse.logger.info("got SIGALRM")
signal.signal(signal.SIGALRM, sigalrm_sent)
Then I get the log line about catching SIGALRM, but the asyncio.sleep calls are not interrupted.
how (kludge)
At this point, I replaced all asyncio.sleep calls with calls to this coroutine:
async def interruptible_sleep(seconds):
while seconds > 0 and not tse.stop_requested:
duration = min(seconds, tse.TIME_QUANTUM)
await asyncio.sleep(duration)
seconds -= duration
So I only have to pick a TIME_QUANTUM that is not too small and not too large either.
but
Is there a way to interrupt all running asyncio.sleep calls and I am missing it?
Interrupting all running calls of asyncio.sleep seems a bit dangerous since it can be used in other parts of the code, for other purposes. Instead I would make a dedicated sleep coroutine that keeps track of its running calls. It is then possible to interrupt them all by canceling the corresponding tasks:
def make_sleep():
async def sleep(delay, result=None, *, loop=None):
coro = asyncio.sleep(delay, result=result, loop=loop)
task = asyncio.ensure_future(coro)
sleep.tasks.add(task)
try:
return await task
except asyncio.CancelledError:
return result
finally:
sleep.tasks.remove(task)
sleep.tasks = set()
sleep.cancel_all = lambda: sum(task.cancel() for task in sleep.tasks)
return sleep
Example:
async def main(sleep, loop):
for i in range(10):
loop.create_task(sleep(i))
await sleep(3)
nb_cancelled = sleep.cancel_all()
await asyncio.wait(sleep.tasks)
return nb_cancelled
sleep = make_sleep()
loop = asyncio.get_event_loop()
result = loop.run_until_complete(main(sleep, loop))
print(result) # Print '6'
For debugging purposes, loop.time = lambda: float('inf') also works.
Based on Vincent's answer, I used the following class (every instance of the class can cancel all its running .sleep tasks, allowing better compartmentalization):
class Sleeper:
"Group sleep calls allowing instant cancellation of all"
def __init__(self, loop):
self.loop = loop
self.tasks = set()
async def sleep(self, delay, result=None):
coro = aio.sleep(delay, result=result, loop=self.loop)
task = aio.ensure_future(coro)
self.tasks.add(task)
try:
return await task
except aio.CancelledError:
return result
finally:
self.tasks.remove(task)
def cancel_all_helper(self):
"Cancel all pending sleep tasks"
cancelled = set()
for task in self.tasks:
if task.cancel():
cancelled.add(task)
return cancelled
async def cancel_all(self):
"Coroutine cancelling tasks"
cancelled = self.cancel_all_helper()
await aio.wait(self.tasks)
self.tasks -= cancelled
return len(cancelled)
Related
I am writing code that has some long-running coroutines that interact with each other. These coroutines can be blocked on await until something external happens. I want to be able to drive these coroutines in a unittest. The regular way of doing await on the coroutine doesn't work, because I want to be able to intercept something in the middle of their operation. I would also prefer not to mess with the coroutine internals either, unless there is something generic/reusable that can be done.
Ideally I would want to run an event loop until all tasks are currently blocked. This should be fairly easy to tell in an event loop implementation. Once everything is blocked, the event loop yields back control, where I can assert some state about the coroutines, and poke them externally. Then I can resume the loop until it gets blocked again. This would allow for deterministic simulation of tasks in an event loop.
Minimal example of the desired API:
import asyncio
from asyncio import Event
# Imagine this is a complicated "main" with many coroutines.
# But event is some external "mockable" event
# that can be used to drive in unit tests
async def wait_on_event(event: Event):
print("Waiting on event")
await event.wait()
print("Done waiting on event")
def test_deterministic():
loop = asyncio.get_event_loop()
event = Event()
task = loop.create_task(wait_on_event(event))
run_until_blocked_or_complete(loop) # define this magic function
# Should print "Waiting on event"
# can make some test assertions here
event.set()
run_until_blocked_or_complete(loop)
# Should print "Done waiting on event"
Anything like that possible? Or would this require writing a custom event loop just for tests?
Additionally, I am currently on Python 3.9 (AWS runtime limitation). If it's not possible to do this in 3.9, what version would support this?
This question has puzzled me since I first read it, because it's almost do-able with standard asyncio functions. The key is Alexander's "magic" is_not_blocked method, which I give verbatim below (except for moving it to the outer indentation level). I also use his wait_on_event method, and his test_deterministic_loop function. I added some extra tests to show how to start and stop other tasks, and how to drive the event loop step-by-step until all tasks are finished.
Instead of his DeterministicLoop class, I use a function run_until_blocked that makes only standard asyncio function calls. The two lines of code:
loop.call_soon(loop.stop)
loop.run_forever()
are a convenient means of advancing the loop by exactly one cycle. And asyncio already provides a method for obtaining all the tasks that run within a given event loop, so there is no need to store them independently.
A comment on the Alexander's "magic" method: if you look at the comments in the asyncio.Task code, the "private" variable _fut_waiter is described as an important invariant. That's very unlikely to change in future versions. So I think it's quite safe in practice.
import asyncio
from typing import Optional, cast
def _is_not_blocked(task: asyncio.Task):
# pylint: disable-next=protected-access
wait_for = cast(Optional[asyncio.Future], task._fut_waiter) # type: ignore
if wait_for is None:
return True
return wait_for.done()
def run_until_blocked():
"""Runs steps of the event loop until all tasks are blocked."""
loop = asyncio.get_event_loop()
# Always run one step.
loop.call_soon(loop.stop)
loop.run_forever()
# Continue running until all tasks are blocked
while any(_is_not_blocked(task) for task in asyncio.all_tasks(loop)):
loop.call_soon(loop.stop)
loop.run_forever()
# This coroutine could spawn many others. Keeping it simple here
async def wait_on_event(event: asyncio.Event) -> int:
print("Waiting")
await event.wait()
print("Done")
return 42
def test_deterministic_loop():
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
event = asyncio.Event()
task = loop.create_task(wait_on_event(event))
assert not task.done()
run_until_blocked()
print("Task done", task.done())
assert not task.done()
print("Tasks running", asyncio.all_tasks(loop))
assert asyncio.all_tasks(loop)
event.set()
# You can start and stop tasks
loop.run_until_complete(asyncio.sleep(2.0))
run_until_blocked()
print("Task done", task.done())
assert task.done()
print("Tasks running", asyncio.all_tasks(loop))
assert task.result() == 42
assert not asyncio.all_tasks(loop)
# If you create a task you must loop run_until_blocked until
# the task is done.
task2 = loop.create_task(asyncio.sleep(2.0))
assert not task2.done()
while not task2.done():
assert asyncio.all_tasks(loop)
run_until_blocked()
assert task2.done()
assert not asyncio.all_tasks(loop)
test_deterministic_loop()
Yes, you can achieve this by creating a custom event loop policy and using a mock event loop in your test. The basic idea is to create a loop that only runs until all the coroutines are blocked, then yield control back to the test code to perform any necessary assertions or external pokes, and then continue running the loop until everything is blocked again, and so on.
import asyncio
class DeterministicEventLoopPolicy(asyncio.DefaultEventLoopPolicy):
def new_event_loop(self):
loop = super().new_event_loop()
loop._blocked = set()
return loop
def get_event_loop(self):
loop = super().get_event_loop()
if not hasattr(loop, "_blocked"):
loop._blocked = set()
return loop
def _enter_task(self, task):
super()._enter_task(task)
if not task._source_traceback:
task._source_traceback = asyncio.Task.current_task().get_stack()
task._loop._blocked.add(task)
def _leave_task(self, task):
super()._leave_task(task)
task._loop._blocked.discard(task)
def run_until_blocked(self, coro):
loop = self.new_event_loop()
asyncio.set_event_loop(loop)
try:
task = loop.create_task(coro)
while loop._blocked:
loop.run_until_complete(asyncio.sleep(0))
finally:
task.cancel()
loop.run_until_complete(task)
asyncio.set_event_loop(None)
This policy creates a new event loop with a _blocked set attribute that tracks the tasks that are currently blocked. When a new task is scheduled on the loop, the _enter_task method is called, and we add it to the _blocked set. When a task is completed or canceled, the _leave_task method is called, and we remove it from the _blocked set.
The run_until_blocked method takes a coroutine and runs the event loop until all the tasks are blocked. It creates a new event loop using the custom policy, schedules the coroutine on the loop, and then repeatedly runs the loop until the _blocked set is empty. This is the point where you can perform any necessary assertions or external pokes.
Here's an example usage of this policy:
async def wait_on_event(event: asyncio.Event):
print("Waiting on event")
await event.wait()
print("Done waiting on event")
def test_deterministic():
asyncio.set_event_loop_policy(DeterministicEventLoopPolicy())
event = asyncio.Event()
asyncio.get_event_loop().run_until_blocked(wait_on_event(event))
assert not event.is_set() # assert that the event has not been set yet
event.set() # set the event
asyncio.get_event_loop().run_until_blocked(wait_on_event(event))
assert event.is_set() # assert that the event has been set
asyncio.get_event_loop().close()
In this test, we create a new Event object and pass it to the wait_on_event coroutine. We use the run_until_blocked method to run the coroutine until it blocks on the event.wait() call. At this point, we can perform any necessary assertions, such as checking that the event has not been set yet. We then set the event, and call run_until_blocked again to resume the coroutine until it completes.
This pattern allows for deterministic simulation of tasks in an event loop and can be used to test coroutines that block on external events.
Hope this helps!
The default event loop simply runs everything that is scheduled in each "pass". If you simply schedule your pause with "loop.call_soon" after getting your tasks running, you should be called at the desired point:
import asyncio
async def worker(n=1):
await asyncio.sleep(n)
def event():
print("blah")
breakpoint()
print("bleh")
async def worker(id):
print(f"starting task {id}")
await asyncio.sleep(0.1)
print(f"ending task {id}")
async def main():
t = []
for id in (1,2,3):
t.append(asyncio.create_task(worker(id)))
loop = asyncio.get_running_loop()
loop.call_soon(event)
await asyncio.sleep(0.2)
And running this on the REPL:
In [8]: asyncio.run(main())
starting task 1
starting task 2
starting task 3
blah
> <ipython-input-3-450374919d79>(4)event()
-> print("bleh")
(Pdb)
Exception in callback event() at <ipython-input-3-450374919d79>:1
[...]
bdb.BdbQuit
ending task 1
ending task 2
ending task 3
After some experimenting I came up with something. Here is the usage first:
# This coroutine could spawn many others. Keeping it simple here
async def wait_on_event(event: asyncio.Event) -> int:
print("Waiting")
await event.wait()
print("Done")
return 42
def test_deterministic_loop():
loop = DeterministicLoop()
event = asyncio.Event()
task = loop.add_coro(wait_on_event(event))
assert not task.done()
loop.step()
# prints Waiting
assert not task.done()
assert not loop.done()
event.set()
loop.step()
# prints Done
assert task.done()
assert task.result() == 42
assert loop.done()
The implementation:
"""Module for testing facilities. Don't use these in production!"""
import asyncio
from enum import IntEnum
from typing import Any, Optional, TypeVar, cast
from collections.abc import Coroutine, Awaitable
def _get_other_tasks(loop: Optional[asyncio.AbstractEventLoop]) -> set[asyncio.Task]:
"""Get a set of currently scheduled tasks in an event loop that are not the current task"""
current = asyncio.current_task(loop)
tasks = asyncio.all_tasks(loop)
if current is not None:
tasks.discard(current)
return tasks
# Works on python 3.9, cannot guarantee on other versions
def _get_unblocked_tasks(tasks: set[asyncio.Task]) -> set[asyncio.Task]:
"""Get the subset of tasks that can make progress. This is the most magic
function, and is heavily dependent on eventloop implementation and python version"""
def is_not_blocked(task: asyncio.Task):
# pylint: disable-next=protected-access
wait_for = cast(Optional[asyncio.Future], task._fut_waiter) # type: ignore
if wait_for is None:
return True
return wait_for.done()
return set(filter(is_not_blocked, tasks))
class TasksState(IntEnum):
RUNNING = 0
BLOCKED = 1
DONE = 2
def _get_tasks_state(
prev_tasks: set[asyncio.Task], cur_tasks: set[asyncio.Task]
) -> TasksState:
"""Given set of tasks for previous and current pass of the event loop,
determine the overall state of the tasks. Are the tasks making progress,
blocked, or done?"""
if not cur_tasks:
return TasksState.DONE
unblocked: set[asyncio.Task] = _get_unblocked_tasks(cur_tasks)
# check if there are tasks that can make progress
if unblocked:
return TasksState.RUNNING
# if no tasks appear to make progress, check if this and last step the state
# has been constant
elif prev_tasks == cur_tasks:
return TasksState.BLOCKED
return TasksState.RUNNING
async def _stop_when_blocked():
"""Schedule this task to stop the event loop when all other tasks are
blocked, or they all complete"""
prev_tasks: set[asyncio.Task] = set()
loop = asyncio.get_running_loop()
while True:
tasks = _get_other_tasks(loop)
state = _get_tasks_state(prev_tasks, tasks)
prev_tasks = tasks
# stop the event loop if all other tasks cannot make progress
if state == TasksState.BLOCKED:
loop.stop()
# finish this task too, if no other tasks exist
if state == TasksState.DONE:
break
# yield back to the event loop
await asyncio.sleep(0.0)
loop.stop()
T = TypeVar("T")
class DeterministicLoop:
"""An event loop for writing deterministic tests."""
def __init__(self):
self.loop = asyncio.get_event_loop_policy().new_event_loop()
asyncio.set_event_loop(self.loop)
self.stepper_task = self.loop.create_task(_stop_when_blocked())
self.tasks: list[asyncio.Task] = []
def add_coro(self, coro: Coroutine[Any, Any, T]) -> asyncio.Task[T]:
"""Add a coroutine to the set of running coroutines, so they can be stepped through"""
if self.done():
raise RuntimeError("No point in adding more tasks. All tasks have finished")
task = self.loop.create_task(coro)
self.tasks.append(task)
return task
def step(self, awaitable: Optional[Awaitable[T]] = None) -> Optional[T]:
if self.done() or not self.tasks:
raise RuntimeError(
"No point in stepping. No tasks to step or all are finished"
)
step_future: Optional[asyncio.Future[T]] = None
if awaitable is not None:
step_future = asyncio.ensure_future(awaitable, loop=self.loop)
# stepper_task should halt us if we're blocked or all tasks are done
self.loop.run_forever()
if step_future is not None:
assert (
step_future.done()
), "Can't step the event loop, where the step function itself might get blocked"
return step_future.result()
return None
def done(self) -> bool:
return self.stepper_task.done()
Here is a simple generalized implementation.
If the loop finds that all tasks are stuck in some async instruction (Event, Semaphore, whatever) for some constant amount of iterations it will exit the loop context until run_until_blocked is called once again.
import asyncio
MAX_LOOP_ITER = 100
def run_until_blocked(loop):
global _tasks
_tasks = {}
while True:
loop.call_soon(loop.stop)
loop.run_forever()
for task in asyncio.all_tasks(loop=loop):
if task.done():
continue
lasti = task.get_stack()[-1].f_lasti
if task in _tasks and _tasks[task]["lasti"] == lasti:
_tasks[task]["iter"] += 1
else:
_tasks[task] = {"iter": 0, "lasti": lasti}
if all(val["iter"] < MAX_LOOP_ITER for val in _tasks.values()):
break
async def wait_on_event(event: asyncio.Event):
print("Waiting on event")
await event.wait()
print("Done waiting on event")
return 42
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
event = asyncio.Event()
coro = wait_on_event(event)
task = loop.create_task(coro)
run_until_blocked(loop)
event.set()
run_until_blocked(loop)
print(task.result())
I use a coroutine to add another coroutine to the event loop multiple times but partway through I cancel the first coroutine. I thought this would mean that any coroutines already added to the event loop would complete successfully and no more would be added, however I find that coroutines that have already been added to the event loop also seem to be cancelled.
I'm running this script in Spyder so I don't need to call run_until_complete, etc. because the event loop is already running in the background on my environment.
I'm sure I'm missing something and the code is behaving exactly as it should - but I can't figure out why. I would also like to know how I might allow cancellation of runTimes but still let slowPrinter complete.
Thank you!
Code below
import asyncio
loop = asyncio.get_event_loop()
async def runTimes(async_func, times):
for i in range(0, times):
task = loop.create_task(async_func())
await task
async def slowPrinter():
await asyncio.sleep(2)
print("slowPrinter done")
async def doStuff():
for i in range(0, 10):
await(asyncio.sleep(1))
print("doStuff done")
async def doLater(delay_ms, method, *args, **kwargs):
try:
print("doLater " + str(delay_ms) + " " + str(method.__name__))
except AttributeError:
print("doLater " + str(delay_ms))
await asyncio.sleep(delay_ms/1000)
method(*args, **kwargs)
print("doLater complete")
task = loop.create_task(runTimes(slowPrinter, 3))
loop.create_task(doLater(3000, task.cancel))
loop.create_task(doStuff())
Output
doLater 3000 cancel
slowPrinter done
doLater complete
doStuff done
Expected Output
doLater 3000 cancel
slowPrinter done
doLater complete
**slowPrinter done**
doStuff done
Edit: Part of the reason I have built the code without using things like run_later is because I need to port the code to micropython later so I am sticking to functions I can use on micropython.
Edit2: Interestingly, task cancellation seems to propagate to tasks created from within the coroutine as well!
async def runTimes(async_func, times):
for i in range(0, times):
task = loop.create_task(async_func())
try:
await task
except asyncio.CancelledError:
print("cancelled as well")
Output
doLater 3000 cancel
slowPrinter done
doLater complete
cancelled as well
slowPrinter done
doStuff done
That's because tour task is waiting on another task:
async def runTimes(async_func, times):
for i in range(0, times):
task = loop.create_task(async_func())
await task ## HERE!
As per asyncio's documentation:
To cancel a running Task use the cancel() method. Calling it will
cause the Task to throw a CancelledError exception into the wrapped
coroutine. If a coroutine is awaiting on a Future object during
cancellation, the Future object will be cancelled.
You may have to look for a way to prevent the task from being cancelled while waiting.
With the second edit where I used this code it became clear the await expression was where the cancellation error was being thrown.
async def runTimes(async_func, times):
for i in range(0, times):
task = loop.create_task(async_func())
try:
await task
except asyncio.CancelledError:
print("cancelled as well")
Changing to awaiting on a timer allowed the main task to complete and the cancellation was thrown on the await asyncio.sleep() method instead which works for my use case.
import asyncio
loop = asyncio.get_event_loop()
async def runTimes(async_func, times):
global current_task
for i in range(0, times):
current_task = loop.create_task(async_func())
while not current_task.done():
await asyncio.sleep(0.05)
async def slowPrinter():
await asyncio.sleep(2)
print("slowPrinter done")
async def doStuff():
for i in range(0, 10):
await(asyncio.sleep(1))
print("doStuff done")
async def doLater(delay_ms, method, *args, **kwargs):
try:
print("doLater " + str(delay_ms) + " " + str(method.__name__))
except AttributeError:
print("doLater " + str(delay_ms))
await asyncio.sleep(delay_ms/1000)
method(*args, **kwargs)
print("doLater complete")
task = loop.create_task(runTimes(slowPrinter, 3))
loop.create_task(doLater(3000, task.cancel))
loop.create_task(doStuff())
With the following snippet, I can't figure why the infiniteTask is not cancelled (it keeps spamming "I'm still standing")
In debug mode, I can see that the Task stored in unfinished is indeed marked as Cancelled but obiously the thread is not cancelled / killed.
Why is the thread not killed when the wrapping task is cancelled ?
What should I do to stop the thread ?
import time
import asyncio
def quickTask():
time.sleep(1)
def infiniteTask():
while True:
time.sleep(1)
print("I'm still standing")
async def main():
finished, unfinished = await asyncio.wait({
asyncio.create_task(asyncio.to_thread(quickTask)),
asyncio.create_task(asyncio.to_thread(infiniteTask))
},
return_when = "FIRST_COMPLETED"
)
for task in unfinished:
task.cancel()
await asyncio.wait(unfinished)
print(" finished : " + str(len(finished))) # print '1'
print("unfinished : " + str(len(unfinished))) # print '1'
asyncio.run(main())
Cause
If we check the definition of asyncio.to_thread():
# python310/Lib/asyncio/threads.py
# ...
async def to_thread(func, /, *args, **kwargs):
"""Asynchronously run function *func* in a separate thread.
Any *args and **kwargs supplied for this function are directly passed
to *func*. Also, the current :class:`contextvars.Context` is propagated,
allowing context variables from the main thread to be accessed in the
separate thread.
Return a coroutine that can be awaited to get the eventual result of *func*.
"""
loop = events.get_running_loop()
ctx = contextvars.copy_context()
func_call = functools.partial(ctx.run, func, *args, **kwargs)
return await loop.run_in_executor(None, func_call)
It's actually a wrapper of loop.run_in_executor.
If we then go into how asyncio's test handle run_in_executor:
# python310/Lib/test/test_asyncio/threads.py
# ...
class EventLoopTestsMixin:
# ...
def test_run_in_executor_cancel(self):
called = False
def patched_call_soon(*args):
nonlocal called
called = True
def run():
time.sleep(0.05)
f2 = self.loop.run_in_executor(None, run)
f2.cancel()
self.loop.run_until_complete(
self.loop.shutdown_default_executor())
self.loop.close()
self.loop.call_soon = patched_call_soon
self.loop.call_soon_threadsafe = patched_call_soon
time.sleep(0.4)
self.assertFalse(called)
You can see it will wait for self.loop.shutdown_default_executor().
Now let's see how it looks like.
# event.pyi
# ...
class BaseEventLoop(events.AbstractEventLoop):
# ...
async def shutdown_default_executor(self):
"""Schedule the shutdown of the default executor."""
self._executor_shutdown_called = True
if self._default_executor is None:
return
future = self.create_future()
thread = threading.Thread(target=self._do_shutdown, args=(future,))
thread.start()
try:
await future
finally:
thread.join()
def _do_shutdown(self, future):
try:
self._default_executor.shutdown(wait=True)
self.call_soon_threadsafe(future.set_result, None)
except Exception as ex:
self.call_soon_threadsafe(future.set_exception, ex)
Here, we can see it creates another thread to wait for _do_shutdown, which then runs self._default_executor.shutdown with wait=True parameter.
Then where the shutdown is implemented:
# Python310/Lib/concurrent/futures/thread.py
# ...
class ThreadPoolExecutor(_base.Executor):
# ...
def shutdown(self, wait=True, *, cancel_futures=False):
with self._shutdown_lock:
self._shutdown = True
if cancel_futures:
# Drain all work items from the queue, and then cancel their
# associated futures.
while True:
try:
work_item = self._work_queue.get_nowait()
except queue.Empty:
break
if work_item is not None:
work_item.future.cancel()
# Send a wake-up to prevent threads calling
# _work_queue.get(block=True) from permanently blocking.
self._work_queue.put(None)
if wait:
for t in self._threads:
t.join()
When wait=True it decides to wait for all thread to be gracefully stops.
From all these we can't see any effort to actually cancel a thread.
To quote from Trio Documentation:
Cancellation is a tricky issue here, because neither Python nor the operating systems it runs on provide any general mechanism for cancelling an arbitrary synchronous function running in a thread. This function will always check for cancellation on entry, before starting the thread. But once the thread is running, there are two ways it can handle being cancelled:
If cancellable=False, the function ignores the cancellation and keeps going, just like if we had called sync_fn synchronously. This is the default behavior.
If cancellable=True, then this function immediately raises Cancelled. In this case the thread keeps running in background – we just abandon it to do whatever it’s going to do, and silently discard any return value or errors that it raises.
So, from these we can learn that there's no way to terminate infinite-loop running in thread.
Workaround
Since now we know we have to design what's going to run in thread with a bit more care, we need a way to signal the thread that we want to stop.
We can utilize Event for such cases.
import time
import asyncio
def blocking_func(event: asyncio.Event):
while not event.is_set():
time.sleep(1)
print("I'm still standing")
async def main():
event = asyncio.Event()
asyncio.create_task(asyncio.to_thread(blocking_func, event))
await asyncio.sleep(5)
# now lets stop
event.set()
asyncio.run(main())
By checking event on every loop we can see program terminating gracefully.
I'm still standing
I'm still standing
I'm still standing
I'm still standing
I'm still standing
Process finished with exit code 0
I have two tasks in a consumer/producer relationship, separated by a asyncio.Queue. If the producer task fails, I'd like the consumer task to also fail as soon as possible, and not wait indefinitely on the queue. The consumer task can be created(spawned) independently from the producer task.
In general terms, I'd like to implement a dependency between two tasks, such that the failure of one is also the failure of the other, while keeping those two tasks concurrent(i.e. one will not await the other directly).
What kind of solutions(e.g. patterns) could be used here?
Basically, I'm thinking of erlang's "links".
I think it may be possible to implement something similar using callbacks, i.e. asyncio.Task.add_done_callback
Thanks!
From the comment:
The behavior I'm trying to avoid is the consumer being oblivious to the producer's death and waiting indefinitely on the queue. I want the consumer to be notified of the producer's death, and have a chance to react. or just fail, and that even while it's also waiting on the queue.
Other than the answer presented by Yigal, another way is to set up a third task that monitors the two and cancels one when the other one finishes. This can be generalized to any two tasks:
async def cancel_when_done(source, target):
assert isinstance(source, asyncio.Task)
assert isinstance(target, asyncio.Task)
try:
await source
except:
# SOURCE is a task which we expect to be awaited by someone else
pass
target.cancel()
Now when setting up the producer and the consumer, you can link them with the above function. For example:
async def producer(q):
for i in itertools.count():
await q.put(i)
await asyncio.sleep(.2)
if i == 7:
1/0
async def consumer(q):
while True:
val = await q.get()
print('got', val)
async def main():
loop = asyncio.get_event_loop()
queue = asyncio.Queue()
p = loop.create_task(producer(queue))
c = loop.create_task(consumer(queue))
loop.create_task(cancel_when_done(p, c))
await asyncio.gather(p, c)
asyncio.get_event_loop().run_until_complete(main())
One way would be to propagate the exception through the queue, combined with delegation of the work handling:
class ValidWorkLoad:
async def do_work(self, handler):
await handler(self)
class HellBrokeLoose:
def __init__(self, exception):
self._exception = exception
async def do_work(self, handler):
raise self._exception
async def worker(name, queue):
async def handler(work_load):
print(f'{name} handled')
while True:
next_work = await queue.get()
try:
await next_work.do_work(handler)
except Exception as e:
print(f'{name} caught exception: {type(e)}: {e}')
break
finally:
queue.task_done()
async def producer(name, queue):
i = 0
while True:
try:
# Produce some work, or fail while trying
new_work = ValidWorkLoad()
i += 1
if i % 3 == 0:
raise ValueError(i)
await queue.put(new_work)
print(f'{name} produced')
await asyncio.sleep(0) # Preempt just for the sake of the example
except Exception as e:
print('Exception occurred')
await queue.put(HellBrokeLoose(e))
break
loop = asyncio.get_event_loop()
queue = asyncio.Queue(loop=loop)
producer_coro = producer('Producer', queue)
consumer_coro = worker('Consumer', queue)
loop.run_until_complete(asyncio.gather(producer_coro, consumer_coro))
loop.close()
Which outputs:
Producer produced
Consumer handled
Producer produced
Consumer handled
Exception occurred
Consumer caught exception: <class 'ValueError'>: 3
Alternatively you could skip the delegation, and designate an item that signals the worker to stop. When catching an exception in the producer you put that designated item in the queue.
Another possible solution:
import asyncio
def link_tasks(t1: Union[asyncio.Task, asyncio.Future], t2: Union[asyncio.Task, asyncio.Future]):
"""
Link the fate of two asyncio tasks,
such that the failure or cancellation of one
triggers the cancellation of the other
"""
def done_callback(other: asyncio.Task, t: asyncio.Task):
# TODO: log cancellation due to link propagation
if t.cancelled():
other.cancel()
elif t.exception():
other.cancel()
t1.add_done_callback(functools.partial(done_callback, t2))
t2.add_done_callback(functools.partial(done_callback, t1))
This uses asyncio.Task.add_done_callback to register callbacks that will cancel the other task if either one fails or is cancelled.
I have a code like the foolowing:
def render():
loop = asyncio.get_event_loop()
async def test():
await asyncio.sleep(2)
print("hi")
return 200
if loop.is_running():
result = asyncio.ensure_future(test())
else:
result = loop.run_until_complete(test())
When the loop is not running is quite easy, just use loop.run_until_complete and it return the coro result but if the loop is already running (my blocking code running in app which is already running the loop) I cannot use loop.run_until_complete since it will raise an exception; when I call asyncio.ensure_future the task gets scheduled and run, but I want to wait there for the result, does anybody knows how to do this? Docs are not very clear how to do this.
I tried passing a concurrent.futures.Future calling set_result inside the coro and then calling Future.result() on my blocking code, but it doesn't work, it blocks there and do not let anything else to run. ANy help would be appreciated.
To implement runner with the proposed design, you would need a way to single-step the event loop from a callback running inside it. Asyncio explicitly forbids recursive event loops, so this approach is a dead end.
Given that constraint, you have two options:
make render() itself a coroutine;
execute render() (and its callers) in a thread different than the thread that runs the asyncio event loop.
Assuming #1 is out of the question, you can implement the #2 variant of render() like this:
def render():
loop = _event_loop # can't call get_event_loop()
async def test():
await asyncio.sleep(2)
print("hi")
return 200
future = asyncio.run_coroutine_threadsafe(test(), loop)
result = future.result()
Note that you cannot use asyncio.get_event_loop() in render because the event loop is not (and should not be) set for that thread. Instead, the code that spawns the runner thread must call asyncio.get_event_loop() and send it to the thread, or just leave it in a global variable or a shared structure.
Waiting Synchronously for an Asynchronous Coroutine
If an asyncio event loop is already running by calling loop.run_forever, it will block the executing thread until loop.stop is called [see the docs]. Therefore, the only way for a synchronous wait is to run the event loop on a dedicated thread, schedule the asynchronous function on the loop and wait for it synchronously from another thread.
For this I have composed my own minimal solution following the answer by user4815162342. I have also added the parts for cleaning up the loop when all work is finished [see loop.close].
The main function in the code below runs the event loop on a dedicated thread, schedules several tasks on the event loop, plus the task the result of which is to be awaited synchronously. The synchronous wait will block until the desired result is ready. Finally, the loop is closed and cleaned up gracefully along with its thread.
The dedicated thread and the functions stop_loop, run_forever_safe, and await_sync can be encapsulated in a module or a class.
For thread-safery considerations, see section “Concurrency and Multithreading” in asyncio docs.
import asyncio
import threading
#----------------------------------------
def stop_loop(loop):
''' stops an event loop '''
loop.stop()
print (".: LOOP STOPPED:", loop.is_running())
def run_forever_safe(loop):
''' run a loop for ever and clean up after being stopped '''
loop.run_forever()
# NOTE: loop.run_forever returns after calling loop.stop
#-- cancell all tasks and close the loop gracefully
print(".: CLOSING LOOP...")
# source: <https://xinhuang.github.io/posts/2017-07-31-common-mistakes-using-python3-asyncio.html>
loop_tasks_all = asyncio.Task.all_tasks(loop=loop)
for task in loop_tasks_all: task.cancel()
# NOTE: `cancel` does not guarantee that the Task will be cancelled
for task in loop_tasks_all:
if not (task.done() or task.cancelled()):
try:
# wait for task cancellations
loop.run_until_complete(task)
except asyncio.CancelledError: pass
#END for
print(".: ALL TASKS CANCELLED.")
loop.close()
print(".: LOOP CLOSED:", loop.is_closed())
def await_sync(task):
''' synchronously waits for a task '''
while not task.done(): pass
print(".: AWAITED TASK DONE")
return task.result()
#----------------------------------------
async def asyncTask(loop, k):
''' asynchronous task '''
print("--start async task %s" % k)
await asyncio.sleep(3, loop=loop)
print("--end async task %s." % k)
key = "KEY#%s" % k
return key
def main():
loop = asyncio.new_event_loop() # construct a new event loop
#-- closures for running and stopping the event-loop
run_loop_forever = lambda: run_forever_safe(loop)
close_loop_safe = lambda: loop.call_soon_threadsafe(stop_loop, loop)
#-- make dedicated thread for running the event loop
thread = threading.Thread(target=run_loop_forever)
#-- add some tasks along with my particular task
myTask = asyncio.run_coroutine_threadsafe(asyncTask(loop, 100200300), loop=loop)
otherTasks = [asyncio.run_coroutine_threadsafe(asyncTask(loop, i), loop=loop)
for i in range(1, 10)]
#-- begin the thread to run the event-loop
print(".: EVENT-LOOP THREAD START")
thread.start()
#-- _synchronously_ wait for the result of my task
result = await_sync(myTask) # blocks until task is done
print("* final result of my task:", result)
#... do lots of work ...
print("*** ALL WORK DONE ***")
#========================================
# close the loop gracefully when everything is finished
close_loop_safe()
thread.join()
#----------------------------------------
main()
here is my case, my whole programe is async, but call some sync lib, then callback to my async func.
follow the answer by user4815162342.
import asyncio
async def asyncTask(k):
''' asynchronous task '''
print("--start async task %s" % k)
# await asyncio.sleep(3, loop=loop)
await asyncio.sleep(3)
print("--end async task %s." % k)
key = "KEY#%s" % k
return key
def my_callback():
print("here i want to call my async func!")
future = asyncio.run_coroutine_threadsafe(asyncTask(1), LOOP)
return future.result()
def sync_third_lib(cb):
print("here will call back to your code...")
cb()
async def main():
print("main start...")
print("call sync third lib ...")
await asyncio.to_thread(sync_third_lib, my_callback)
# await loop.run_in_executor(None, func=sync_third_lib)
print("another work...keep async...")
await asyncio.sleep(2)
print("done!")
LOOP = asyncio.get_event_loop()
LOOP.run_until_complete(main())