I have a function called from an async function without await, and my function needs to call async functions. I can do this with asyncio.get_running_loop().create_task(sleep()) but the run_until_complete at the top level doesn't run until the new task is complete.
How do I get the event loop to run until the new task is complete?
I can't make my function async because it's not called with await.
I can't change future or sleep. I'm only in control of in_control.
import asyncio
def in_control(sleep):
"""
How do I get this to run until complete?
"""
return asyncio.get_running_loop().create_task(sleep())
async def future():
async def sleep():
await asyncio.sleep(10)
print('ok')
in_control(sleep)
asyncio.get_event_loop().run_until_complete(future())
It appears that the package nest_asyncio will help you out here. I've also included in the example fetching the return value of the task.
import asyncio
import nest_asyncio
def in_control(sleep):
print("In control")
nest_asyncio.apply()
loop = asyncio.get_running_loop()
task = loop.create_task(sleep())
loop.run_until_complete(task)
print(task.result())
return
async def future():
async def sleep():
for timer in range(10):
print(timer)
await asyncio.sleep(1)
print("Sleep finished")
return "Sleep return"
in_control(sleep)
print("Out of control")
asyncio.get_event_loop().run_until_complete(future())
Result:
In control
0
1
2
3
4
5
6
7
8
9
Sleep finished
Sleep return
Out of control
[Finished in 10.2s]
Related
I'm trying to run some code asynchronously. My expectation is that the test coroutine should not block the print(running first) statement. This is because I've dispatched it to the event loop, and should be seeing the output of this command logged first.
import asyncio
async def test():
await asyncio.sleep(5)
print("I should run second")
asyncio.run(test())
print('running first')
Does anyone have any tips on how to how this code run so that print('running first') is ran before print("I should run second")? I believe this code should be non-blocking, so I'm confused as to why the order of print messages isn't matching my expectation.
I believe this is what you want:
import asyncio
async def test():
await asyncio.sleep(5)
print("I should run second")
async def main():
task1 = asyncio.create_task(test())
print('running first')
await task1
asyncio.run(main())
A more detail explaination:
asyncio.run() will try to wait all of the task inside it to finish before it continues.
In your code, you are running asyncio.run(test()) first and it will continue ONLY IF test() IS ENDED and you awaited the sleep. so test() will end after the sleep and run the print then the main print.
This is why your code delay so long before running. The solution to it is simple. Fire the task without waiting, which is what asyncio.create_task() is doing, I created a task, fire it but wait it at the end.
btw normally when you are using async, you will have a ton of task in a list. If you want to wait it as a list you should use gather():
import asyncio
async def test():
await asyncio.sleep(5)
print("I should run second")
async def main():
task_list = []
for _ in range(100):
task_list.append(asyncio.create_task(test()))
print('running first')
await asyncio.gather(*task_list)
asyncio.run(main())
I am trying to do something similar like C# ManualResetEvent but in Python.
I have attempted to do it in python but doesn't seem to work.
import asyncio
cond = asyncio.Condition()
async def main():
some_method()
cond.notify()
async def some_method():
print("Starting...")
await cond.acquire()
await cond.wait()
cond.release()
print("Finshed...")
main()
I want the some_method to start then wait until signaled to start again.
This code is not complete, first of all you need to use asyncio.run() to bootstrap the event loop - this is why your code is not running at all.
Secondly, some_method() never actually starts. You need to asynchronously start some_method() using asyncio.create_task(). When you call an "async def function" (the more correct term is coroutinefunction) it returns a coroutine object, this object needs to be driven by the event loop either by you awaiting it or using the before-mentioned function.
Your code should look more like this:
import asyncio
async def main():
cond = asyncio.Condition()
t = asyncio.create_task(some_method(cond))
# The event loop hasn't had any time to start the task
# until you await again. Sleeping for 0 seconds will let
# the event loop start the task before continuing.
await asyncio.sleep(0)
cond.notify()
# You should never really "fire and forget" tasks,
# the same way you never do with threading. Wait for
# it to complete before returning:
await t
async def some_method(cond):
print("Starting...")
await cond.acquire()
await cond.wait()
cond.release()
print("Finshed...")
asyncio.run(main())
I want to make a timer which is started in a normal function, but in the timer function, it should be able to call an async function
I want to do something like this:
startTimer()
while True:
print("e")
def startTimer(waitForSeconds: int):
# Wait for `waitForSeconds`
await myAsyncFunc()
async def myAsyncFunc():
print("in my async func")
Where the while True loop should do its stuff and after waitForSeconds the timer the async function should execute an other async function, but waiting shouldn't block any other actions and doesn't need to be awaited
If something isn't understandable, I'm sorry, I'll try to explain it then
Thanks
If you want to run your synchronous and asynchronous code in parallel, you will need to run one of them in a separate thread. For example:
def sync_code():
while True:
print("e")
async def start_timer(secs):
await asyncio.sleep(secs)
await async_func()
async def main():
asyncio.create_task(start_timer(1))
loop = asyncio.get_event_loop()
# use run_in_executor to run sync code in a separate thread
# while this thread runs the event loop
await loop.run_in_executor(None, sync_code)
asyncio.run(main())
If the above is not acceptable for you (e.g. because it turns the whole program into an asyncio program), you can also run the event loop in a background thread, and submit tasks to it using asyncio.run_coroutine_threadsafe. That approach would allow startTimer to have the signature (and interface) like you wanted it:
def startTimer(waitForSeconds):
loop = asyncio.new_event_loop()
threading.Thread(daemon=True, target=loop.run_forever).start()
async def sleep_and_run():
await asyncio.sleep(waitForSeconds)
await myAsyncFunc()
asyncio.run_coroutine_threadsafe(sleep_and_run(), loop)
async def myAsyncFunc():
print("in my async func")
startTimer(1)
while True:
print("e")
I'm pretty sure that you are familiar with concurent processing, but you didn't show exactly what you want. So if I understand you correctly you want to have 2 processes. First is doing only while True, and the second process is the timer(waits e.g. 5s) and it will call async task. I assume that you are using asyncio according to tags:
import asyncio
async def myAsyncFunc():
print("in my async func")
async def call_after(delay):
await asyncio.sleep(delay)
await myAsyncFunc()
async def while_true():
while True:
await asyncio.sleep(1) # sleep here to avoid to large output
print("e")
async def main():
task1 = asyncio.create_task(
while_true())
task2 = asyncio.create_task(
call_after(5))
# Wait until both tasks are completed (should take
# around 2 seconds.)
await task1
await task2
asyncio.run(main())
I am new to study about asyncio.I don't know how to
describe my question.But here is a minimal example:
import asyncio
async def work():
await asyncio.sleep(3)
async def check_it():
task = asyncio.create_task(work())
await task
while True:
if task.done():
print("Done")
break
print("Trying...")
asyncio.run(check_it())
My idea is very simple:
create a async task in check_it().And await it.
Use a while loop to check whether the task is finished.
If task.done() return True,break the while loop.Then exit the script.
If my question is duplicate, please flag my question.Thanks!
Try asyncio.wait or use asyncio.sleep. Otherwise, your program will output a lot without some pauses.
import asyncio
async def work():
await asyncio.sleep(3)
async def check_it():
task = asyncio.create_task(work())
# "await" block until the task finish. Do not do here.
timeout = 0 # Probably the first timeout is 0
while True:
done, pending = await asyncio.wait({task}, timeout=timeout)
if task in done:
print('Done')
# Do an await here is favourable in case any exception is raised.
await task
break
print('Trying...')
timeout = 1
asyncio.run(check_it())
Sometimes there is some non-critical asynchronous operation that needs to happen but I don't want to wait for it to complete. In Tornado's coroutine implementation you can "fire & forget" an asynchronous function by simply ommitting the yield key-word.
I've been trying to figure out how to "fire & forget" with the new async/await syntax released in Python 3.5. E.g., a simplified code snippet:
async def async_foo():
print("Do some stuff asynchronously here...")
def bar():
async_foo() # fire and forget "async_foo()"
bar()
What happens though is that bar() never executes and instead we get a runtime warning:
RuntimeWarning: coroutine 'async_foo' was never awaited
async_foo() # fire and forget "async_foo()"
Upd:
Replace asyncio.ensure_future with asyncio.create_task everywhere if you're using Python >= 3.7 It's a newer, nicer way to spawn tasks.
asyncio.Task to "fire and forget"
According to python docs for asyncio.Task it is possible to start some coroutine to execute "in the background". The task created by asyncio.ensure_future won't block the execution (therefore the function will return immediately!). This looks like a way to "fire and forget" as you requested.
import asyncio
async def async_foo():
print("async_foo started")
await asyncio.sleep(1)
print("async_foo done")
async def main():
asyncio.ensure_future(async_foo()) # fire and forget async_foo()
# btw, you can also create tasks inside non-async funcs
print('Do some actions 1')
await asyncio.sleep(1)
print('Do some actions 2')
await asyncio.sleep(1)
print('Do some actions 3')
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Output:
Do some actions 1
async_foo started
Do some actions 2
async_foo done
Do some actions 3
What if tasks are executing after the event loop has completed?
Note that asyncio expects tasks to be completed at the moment the event loop completes. So if you'll change main() to:
async def main():
asyncio.ensure_future(async_foo()) # fire and forget
print('Do some actions 1')
await asyncio.sleep(0.1)
print('Do some actions 2')
You'll get this warning after the program finished:
Task was destroyed but it is pending!
task: <Task pending coro=<async_foo() running at [...]
To prevent that you can just await all pending tasks after the event loop has completed:
async def main():
asyncio.ensure_future(async_foo()) # fire and forget
print('Do some actions 1')
await asyncio.sleep(0.1)
print('Do some actions 2')
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
# Let's also finish all running tasks:
pending = asyncio.Task.all_tasks()
loop.run_until_complete(asyncio.gather(*pending))
Kill tasks instead of awaiting them
Sometimes you don't want to await tasks to be done (for example, some tasks may be created to run forever). In that case, you can just cancel() them instead of awaiting them:
import asyncio
from contextlib import suppress
async def echo_forever():
while True:
print("echo")
await asyncio.sleep(1)
async def main():
asyncio.ensure_future(echo_forever()) # fire and forget
print('Do some actions 1')
await asyncio.sleep(1)
print('Do some actions 2')
await asyncio.sleep(1)
print('Do some actions 3')
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
# Let's also cancel all running tasks:
pending = asyncio.Task.all_tasks()
for task in pending:
task.cancel()
# Now we should await task to execute it's cancellation.
# Cancelled task raises asyncio.CancelledError that we can suppress:
with suppress(asyncio.CancelledError):
loop.run_until_complete(task)
Output:
Do some actions 1
echo
Do some actions 2
echo
Do some actions 3
echo
Output:
>>> Hello
>>> foo() started
>>> I didn't wait for foo()
>>> foo() completed
Here is the simple decorator function which pushes the execution to background and line of control moves to next line of the code.
The primary advantage is, you don't have to declare the function as await
import asyncio
import time
def fire_and_forget(f):
def wrapped(*args, **kwargs):
return asyncio.get_event_loop().run_in_executor(None, f, *args, *kwargs)
return wrapped
#fire_and_forget
def foo():
print("foo() started")
time.sleep(1)
print("foo() completed")
print("Hello")
foo()
print("I didn't wait for foo()")
Note: Check my other answer which does the same using plain thread without asyncio.
This is not entirely asynchronous execution, but maybe run_in_executor() is suitable for you.
def fire_and_forget(task, *args, **kwargs):
loop = asyncio.get_event_loop()
if callable(task):
return loop.run_in_executor(None, task, *args, **kwargs)
else:
raise TypeError('Task must be a callable')
def foo():
#asynchronous stuff here
fire_and_forget(foo)
For some reason if you are unable to use asyncio then here is the implementation using plain threads. Check my other answers and Sergey's answer too.
import threading, time
def fire_and_forget(f):
def wrapped():
threading.Thread(target=f).start()
return wrapped
#fire_and_forget
def foo():
print("foo() started")
time.sleep(1)
print("foo() completed")
print("Hello")
foo()
print("I didn't wait for foo()")
produces
>>> Hello
>>> foo() started
>>> I didn't wait for foo()
>>> foo() completed
def fire_and_forget(f):
def wrapped(*args, **kwargs):
threading.Thread(target=functools.partial(f, *args, **kwargs)).start()
return wrapped
is the better version of the above -- does not use asyncio