The current event loop of the coroutine has been opened. Inside a coroutine object, create a new coroutine object and register it in the event loop.
Question: How can I get the return value of the sub-coroutine?
I want to get the return value of func_first
import asyncio
import time
async def func_first(values):
await asyncio.sleep(2)
print('out: func_first')
return values
async def func_second():
s = asyncio.create_task(func_first(100))
await asyncio.sleep(3)
print('out: func_second')
if __name__ == '__main__':
loop = asyncio.get_event_loop()
s = time.time()
loop.run_until_complete(func_second())
print('TIME:', time.time()-s)
In your code, just add await s to your print statement. In fact, it's even easier than that: don't bother with creating a task; just await the function and print the value directly. This code shows both methods:
import asyncio
import time
async def func_first(values):
await asyncio.sleep(2)
print('out: func_first')
return values
async def func_second():
s = asyncio.create_task(func_first(100))
await asyncio.sleep(3)
print('out: func_second', await s)
async def func_third():
print('out: func_third', await func_first(101))
if __name__ == '__main__':
loop = asyncio.get_event_loop()
s = time.time()
loop.run_until_complete(func_second())
print('TIME:', time.time()-s)
loop.run_until_complete(func_third())
print('TIME:', time.time()-s)
Result:
out: func_first
out: func_second 100
TIME: 3.062396764755249
out: func_first
out: func_third 101
TIME: 5.09357476234436
>>>
I found the answer here:
enter link description here
asyncio.Queue can get data, it seems that I still don't know much about this module. lol
I need to suspend a coroutine until a condition is met. Currently, I have:
class Awaiter:
def __init__(self):
self.ready = False
def __await__(self):
while not self.ready:
yield
And the caller code:
await awaiter
This works, but it requires boilerplate code. Is it necessary boilerplate or is there a special syntax to await on a predicate, such as:
await condition
which would yield until condition is false?
At the asyncio package there is a builtin Condition object that you can use.
An asyncio condition primitive can be used by a task to wait for some event to happen and then get exclusive access to a shared resource.
How to use the condition (from the same source):
cond = asyncio.Condition()
# The preferred way to use a Condition is an async with statement
async with cond:
await cond.wait()
# It can also be used as follow
await cond.acquire()
try:
await cond.wait()
finally:
cond.release()
A code example:
import asyncio
cond = asyncio.Condition()
async def func1():
async with cond:
print('It\'s look like I will need to wait')
await cond.wait()
print('Now it\'s my turn')
async def func2():
async with cond:
print('Notifying....')
cond.notify()
print('Let me finish first')
# Main function
async def main(loop):
t1 = loop.create_task(func1())
t2 = loop.create_task(func2())
await asyncio.wait([t1, t2])
if __name__ == '__main__':
l = asyncio.get_event_loop()
l.run_until_complete(main(l))
l.close()
This will results with:
It's look like I will need to wait
Notifying....
Let me finish first
Now it's my turn
An alternative way is to use the asyncio.Event.
import asyncio
event = asyncio.Event()
async def func1():
print('It\'s look like I will need to wait')
await event.wait()
print('Now it\'s my turn')
async def func2():
print('Notifying....')
event.set()
print('Let me finish first')
It will have the same results as the Condition code example.
I need to write a code where i need to to check in real time a status of some variable. I decited to use asyncio to create two async def functions
import asyncio
async def one():
global flag
flag = True
while flag == True:
await asyncio.sleep(0.2)
print("Doing one")
async def two():
await asyncio.sleep(2)
global flag
flag = False
async def main():
tasks = []
tasks.append(one())
tasks.append(two())
await asyncio.gather(*tasks)
loop = asyncio.get_event_loop()
try:
loop.run_until_complete(main())
finally:
loop.close()
print("Loop ended")
When loop starts, all tasks has been lauched and after 2 seconds def two() sets flag=False, which stops def one(). It's good but i want def one() to perform while loop without await asyncio.sleep(0.2) becouse i dont want to have real live update so i set await asyncio.sleep(0.0).
Is it a good practice?
Using a global variable is indeed bad practice. What you are looking for is asyncio's primitives, specifically the asyncio.Event primitive. Here is what you are doing, but with asyncio.Event:
import asyncio
async def one(event):
while event.is_set() == False:
await asyncio.sleep(0.5)
print("Hello World!")
async def two(event):
await asyncio.sleep(2)
event.set()
async def main():
event = asyncio.Event()
await asyncio.gather(*[one(event), two(event)])
asyncio.run(main())
I am trying to set a timer that will interrupt the running process and call a coroutine when it fires. However, I'm not sure what the right way to accomplish this is. I've found AbstractEventLoop.call_later, along with threading.Timer but neither of these seem to work (or I'm using them incorrectly). The code is pretty basic and looks something like this:
def set_timer( time ):
self.timer = Timer( 10.0, timeout )
self.timer.start()
#v2
#self.timer = get_event_loop()
#self.timer.call_later( 10.0, timeout )
return
async def timeout():
await some_func()
return
What is the correct way to set a non-blocking timer, that will call a callback function after some number of seconds? Being able to cancel the timer would be a bonus but is not a requirement. The major things I need are: non-blocking and successfully calling the co-routine. Right now it returns an error that the object can't be await'd (if I toss an await in) or that some_func was never await'd, and the expected output never happens.
Creating Task using ensure_future is a common way to start some job executing without blocking your execution flow. You can also cancel tasks.
I wrote example implementation for you to have something to start from:
import asyncio
class Timer:
def __init__(self, timeout, callback):
self._timeout = timeout
self._callback = callback
self._task = asyncio.ensure_future(self._job())
async def _job(self):
await asyncio.sleep(self._timeout)
await self._callback()
def cancel(self):
self._task.cancel()
async def timeout_callback():
await asyncio.sleep(0.1)
print('echo!')
async def main():
print('\nfirst example:')
timer = Timer(2, timeout_callback) # set timer for two seconds
await asyncio.sleep(2.5) # wait to see timer works
print('\nsecond example:')
timer = Timer(2, timeout_callback) # set timer for two seconds
await asyncio.sleep(1)
timer.cancel() # cancel it
await asyncio.sleep(1.5) # and wait to see it won't call callback
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
try:
loop.run_until_complete(main())
finally:
loop.run_until_complete(loop.shutdown_asyncgens())
loop.close()
Output:
first example:
echo!
second example:
Thanks Mikhail Gerasimov for your answer, it was very useful. Here is an extension to Mikhail’s anwer. This is an interval timer with some twists. Perhaps it is useful for some users.
import asyncio
class Timer:
def __init__(self, interval, first_immediately, timer_name, context, callback):
self._interval = interval
self._first_immediately = first_immediately
self._name = timer_name
self._context = context
self._callback = callback
self._is_first_call = True
self._ok = True
self._task = asyncio.ensure_future(self._job())
print(timer_name + " init done")
async def _job(self):
try:
while self._ok:
if not self._is_first_call or not self._first_immediately:
await asyncio.sleep(self._interval)
await self._callback(self._name, self._context, self)
self._is_first_call = False
except Exception as ex:
print(ex)
def cancel(self):
self._ok = False
self._task.cancel()
async def some_callback(timer_name, context, timer):
context['count'] += 1
print('callback: ' + timer_name + ", count: " + str(context['count']))
if timer_name == 'Timer 2' and context['count'] == 3:
timer.cancel()
print(timer_name + ": goodbye and thanks for all the fish")
timer1 = Timer(interval=1, first_immediately=True, timer_name="Timer 1", context={'count': 0}, callback=some_callback)
timer2 = Timer(interval=5, first_immediately=False, timer_name="Timer 2", context={'count': 0}, callback=some_callback)
try:
loop = asyncio.get_event_loop()
loop.run_forever()
except KeyboardInterrupt:
timer1.cancel()
timer2.cancel()
print("clean up done")
The solution proposed by Mikhail has one drawback. Calling cancel() cancels both: the timer and the actual callback (if cancel() fired after timeout is passed, but actual job is still in progress). Canceling the job itself may be not the desired behavior.
An alternative approach is to use loop.call_later:
async def some_job():
print('Job started')
await asyncio.sleep(5)
print('Job is done')
loop = asyncio.get_event_loop() # or asyncio.get_running_loop()
timeout = 5
timer = loop.call_later(timeout, lambda: asyncio.ensure_future(some_job()))
timer.cancel() # cancels the timer, but not the job, if it's already started
I'm migrating from tornado to asyncio, and I can't find the asyncio equivalent of tornado's PeriodicCallback. (A PeriodicCallback takes two arguments: the function to run and the number of milliseconds between calls.)
Is there such an equivalent in asyncio?
If not, what would be the cleanest way to implement this without running the risk of getting a RecursionError after a while?
For Python versions below 3.5:
import asyncio
#asyncio.coroutine
def periodic():
while True:
print('periodic')
yield from asyncio.sleep(1)
def stop():
task.cancel()
loop = asyncio.get_event_loop()
loop.call_later(5, stop)
task = loop.create_task(periodic())
try:
loop.run_until_complete(task)
except asyncio.CancelledError:
pass
For Python 3.5 and above:
import asyncio
async def periodic():
while True:
print('periodic')
await asyncio.sleep(1)
def stop():
task.cancel()
loop = asyncio.get_event_loop()
loop.call_later(5, stop)
task = loop.create_task(periodic())
try:
loop.run_until_complete(task)
except asyncio.CancelledError:
pass
When you feel that something should happen "in background" of your asyncio program, asyncio.Task might be good way to do it. You can read this post to see how to work with tasks.
Here's possible implementation of class that executes some function periodically:
import asyncio
from contextlib import suppress
class Periodic:
def __init__(self, func, time):
self.func = func
self.time = time
self.is_started = False
self._task = None
async def start(self):
if not self.is_started:
self.is_started = True
# Start task to call func periodically:
self._task = asyncio.ensure_future(self._run())
async def stop(self):
if self.is_started:
self.is_started = False
# Stop task and await it stopped:
self._task.cancel()
with suppress(asyncio.CancelledError):
await self._task
async def _run(self):
while True:
await asyncio.sleep(self.time)
self.func()
Let's test it:
async def main():
p = Periodic(lambda: print('test'), 1)
try:
print('Start')
await p.start()
await asyncio.sleep(3.1)
print('Stop')
await p.stop()
await asyncio.sleep(3.1)
print('Start')
await p.start()
await asyncio.sleep(3.1)
finally:
await p.stop() # we should stop task finally
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Output:
Start
test
test
test
Stop
Start
test
test
test
[Finished in 9.5s]
As you see on start we just start task that calls some functions and sleeps some time in endless loop. On stop we just cancel that task. Note, that task should be stopped at the moment program finished.
One more important thing that your callback shouldn't take much time to be executed (or it'll freeze your event loop). If you're planning to call some long-running func, you possibly would need to run it in executor.
A variant that may be helpful: if you want your recurring call to happen every n seconds instead of n seconds between the end of the last execution and the beginning of the next, and you don't want calls to overlap in time, the following is simpler:
async def repeat(interval, func, *args, **kwargs):
"""Run func every interval seconds.
If func has not finished before *interval*, will run again
immediately when the previous iteration finished.
*args and **kwargs are passed as the arguments to func.
"""
while True:
await asyncio.gather(
func(*args, **kwargs),
asyncio.sleep(interval),
)
And an example of using it to run a couple tasks in the background:
async def f():
await asyncio.sleep(1)
print('Hello')
async def g():
await asyncio.sleep(0.5)
print('Goodbye')
async def main():
t1 = asyncio.ensure_future(repeat(3, f))
t2 = asyncio.ensure_future(repeat(2, g))
await t1
await t2
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
There is no built-in support for periodic calls, no.
Just create your own scheduler loop that sleeps and executes any tasks scheduled:
import math, time
async def scheduler():
while True:
# sleep until the next whole second
now = time.time()
await asyncio.sleep(math.ceil(now) - now)
# execute any scheduled tasks
async for task in scheduled_tasks(time.time()):
await task()
The scheduled_tasks() iterator should produce tasks that are ready to be run at the given time. Note that producing the schedule and kicking off all the tasks could in theory take longer than 1 second; the idea here is that the scheduler yields all tasks that should have started since the last check.
Alternative version with decorator for python 3.7
import asyncio
import time
def periodic(period):
def scheduler(fcn):
async def wrapper(*args, **kwargs):
while True:
asyncio.create_task(fcn(*args, **kwargs))
await asyncio.sleep(period)
return wrapper
return scheduler
#periodic(2)
async def do_something(*args, **kwargs):
await asyncio.sleep(5) # Do some heavy calculation
print(time.time())
if __name__ == '__main__':
asyncio.run(do_something('Maluzinha do papai!', secret=42))
Based on #A. Jesse Jiryu Davis answer (with #Torkel Bjørnson-Langen and #ReWrite comments) this is an improvement which avoids drift.
import time
import asyncio
#asyncio.coroutine
def periodic(period):
def g_tick():
t = time.time()
count = 0
while True:
count += 1
yield max(t + count * period - time.time(), 0)
g = g_tick()
while True:
print('periodic', time.time())
yield from asyncio.sleep(next(g))
loop = asyncio.get_event_loop()
task = loop.create_task(periodic(1))
loop.call_later(5, task.cancel)
try:
loop.run_until_complete(task)
except asyncio.CancelledError:
pass
This solution uses the decoration concept from Fernando José Esteves de Souza, the drifting workaround from Wojciech Migda and a superclass in order to generate most elegant code as possible to deal with asynchronous periodic functions.
Without threading.Thread
The solution is comprised of the following files:
periodic_async_thread.py with the base class for you to subclass
a_periodic_thread.py with an example subclass
run_me.py with an example instantiation and run
The PeriodicAsyncThread class in the file periodic_async_thread.py:
import time
import asyncio
import abc
class PeriodicAsyncThread:
def __init__(self, period):
self.period = period
def periodic(self):
def scheduler(fcn):
async def wrapper(*args, **kwargs):
def g_tick():
t = time.time()
count = 0
while True:
count += 1
yield max(t + count * self.period - time.time(), 0)
g = g_tick()
while True:
# print('periodic', time.time())
asyncio.create_task(fcn(*args, **kwargs))
await asyncio.sleep(next(g))
return wrapper
return scheduler
#abc.abstractmethod
async def run(self, *args, **kwargs):
return
def start(self):
asyncio.run(self.run())
An example of a simple subclass APeriodicThread in the file a_periodic_thread.py:
from periodic_async_thread import PeriodicAsyncThread
import time
import asyncio
class APeriodicThread(PeriodicAsyncThread):
def __init__(self, period):
super().__init__(period)
self.run = self.periodic()(self.run)
async def run(self, *args, **kwargs):
await asyncio.sleep(2)
print(time.time())
Instantiating and running the example class in the file run_me.py:
from a_periodic_thread import APeriodicThread
apt = APeriodicThread(2)
apt.start()
This code represents an elegant solution that also mitigates the time drift problem of other solutions. The output is similar to:
1642711285.3898764
1642711287.390698
1642711289.3924973
1642711291.3920736
With threading.Thread
The solution is comprised of the following files:
async_thread.py with the canopy asynchronous thread class.
periodic_async_thread.py with the base class for you to subclass
a_periodic_thread.py with an example subclass
run_me.py with an example instantiation and run
The AsyncThread class in the file async_thread.py:
from threading import Thread
import asyncio
import abc
class AsyncThread(Thread):
def __init__(self, *args, **kwargs) -> None:
super().__init__(*args, **kwargs)
#abc.abstractmethod
async def async_run(self, *args, **kwargs):
pass
def run(self, *args, **kwargs):
# loop = asyncio.new_event_loop()
# asyncio.set_event_loop(loop)
# loop.run_until_complete(self.async_run(*args, **kwargs))
# loop.close()
asyncio.run(self.async_run(*args, **kwargs))
The PeriodicAsyncThread class in the file periodic_async_thread.py:
import time
import asyncio
from .async_thread import AsyncThread
class PeriodicAsyncThread(AsyncThread):
def __init__(self, period, *args, **kwargs):
self.period = period
super().__init__(*args, **kwargs)
self.async_run = self.periodic()(self.async_run)
def periodic(self):
def scheduler(fcn):
async def wrapper(*args, **kwargs):
def g_tick():
t = time.time()
count = 0
while True:
count += 1
yield max(t + count * self.period - time.time(), 0)
g = g_tick()
while True:
# print('periodic', time.time())
asyncio.create_task(fcn(*args, **kwargs))
await asyncio.sleep(next(g))
return wrapper
return scheduler
An example of a simple subclass APeriodicThread in the file a_periodic_thread.py:
import time
from threading import current_thread
from .periodic_async_thread import PeriodicAsyncThread
import asyncio
class APeriodicAsyncTHread(PeriodicAsyncThread):
async def async_run(self, *args, **kwargs):
print(f"{current_thread().name} {time.time()} Hi!")
await asyncio.sleep(1)
print(f"{current_thread().name} {time.time()} Bye!")
Instantiating and running the example class in the file run_me.py:
from .a_periodic_thread import APeriodicAsyncTHread
a = APeriodicAsyncTHread(2, name = "a periodic async thread")
a.start()
a.join()
This code represents an elegant solution that also mitigates the time drift problem of other solutions. The output is similar to:
a periodic async thread 1643726990.505269 Hi!
a periodic async thread 1643726991.5069854 Bye!
a periodic async thread 1643726992.506919 Hi!
a periodic async thread 1643726993.5089169 Bye!
a periodic async thread 1643726994.5076022 Hi!
a periodic async thread 1643726995.509422 Bye!
a periodic async thread 1643726996.5075526 Hi!
a periodic async thread 1643726997.5093904 Bye!
a periodic async thread 1643726998.5072556 Hi!
a periodic async thread 1643726999.5091035 Bye!
For multiple types of scheduling I'd recommend APSScheduler which has asyncio support.
I use it for a simple python process I can fire up using docker and just runs like a cron executing something weekly, until I kill the docker/process.
This is what I did to test my theory of periodic call backs using asyncio. I don't have experience using Tornado, so I'm not sure exactly how the periodic call backs work with it. I am used to using the after(ms, callback) method in Tkinter though, and this is what I came up with. While True: Just looks ugly to me even if it is asynchronous (more so than globals). The call_later(s, callback, *args) method uses seconds not milliseconds though.
import asyncio
my_var = 0
def update_forever(the_loop):
global my_var
print(my_var)
my_var += 1
# exit logic could be placed here
the_loop.call_later(3, update_forever, the_loop) # the method adds a delayed callback on completion
event_loop = asyncio.get_event_loop()
event_loop.call_soon(update_forever, event_loop)
event_loop.run_forever()