This question already has answers here:
asyncio.sleep() vs time.sleep()
(2 answers)
Closed 19 days ago.
I am confused by to what extent does the following example from the Python documentation is different from a time.sleep. If you replace the asyncio.sleep with the time.sleep below, both versions last for 3 seconds, I see no difference! Can some one explain what the point of this example in the documentation is? Shouldn't the async version actually last 2 seconds instead? I understand it that both calls to say_hello practically start at the same time (that is the whole sense of asyncio.sleep, right?), so that the whole delay should be just the longest one. Help me to understand what I am understanding wrong.
import asyncio
import time
async def say_after(delay, what):
await asyncio.sleep(delay)
print(what)
async def main():
print(f"started at {time.strftime('%X')}")
await say_after(1, 'hello')
await say_after(2, 'world')
print(f"finished at {time.strftime('%X')}")
asyncio.run(main())
With time.sleep, the program would wait for the specified time before continuing to the next line. This means that the program would be blocked for the entire duration of the sleep, and cannot perform any other tasks in the meantime.
On the other hand, asyncio.sleep is a coroutine function that allows the program to continue executing other tasks while the sleep function is still running. This means that the program is not blocked, and can continue to perform other tasks in parallel.
This is important in a real-world scenario where there may be multiple tasks that need to be executed at the same time, and the program should not be blocked waiting for one task to complete. Using asyncio.sleep allows for more efficient use of resources and time.
asyncio.sleep() vs time.sleep() from the above comment does a good job explaining on a example so you might check that as well
Update:
The comments in the docs to your snippet state:
The following snippet of code will print “hello” after waiting for 1 second, and then print “world” after waiting for another 2 seconds
In this example, the second say_after function call should start immediately after the first call, without waiting for it to finish. However, the asyncio.sleep function suspends the execution of the coroutine until the specified delay has passed. This means that both say_after calls run consecutively and take the total sum of the delays, which is 3 seconds in this case but it does not block the execution of other tasks that might be running concurrently.
You can make two calls for say_after so they run simultaneously which is not possible with time.sleep:
import asyncio
import time
async def say_after(delay, what):
await asyncio.sleep(delay)
print(what)
async def main():
print(f"started at {time.strftime('%X')}")
await asyncio.gather(say_after(1, 'hello'), say_after(2, 'world'))
print(f"finished at {time.strftime('%X')}")
asyncio.run(main())
To have the program take only 2 seconds as you were expecting originally you could also use the next function in the documentation asyncio.create_task() like this:
import asyncio
import time
async def say_after(delay, what):
await asyncio.sleep(delay)
print(what)
async def main():
print(f"started at {time.strftime('%X')}")
tasks = [asyncio.create_task(
say_after(1, 'hello')), asyncio.create_task(say_after(2, 'world'))]
await asyncio.gather(*tasks)
print(f"finished at {time.strftime('%X')}")
asyncio.run(main())
Related
I'm trying to understand how does asyncio.create_task actually work. Suppose I have following code:
import asyncio
import time
async def delayer():
await asyncio.sleep(1)
async def messenger():
await asyncio.sleep(1)
return "A Message"
async def main():
message = await messenger()
await delayer()
start_time = time.time()
asyncio.run(main())
end_time = time.time() - start_time
print(end_time)
The code will take about 2 seconds. But if I make some changes to the body of main like this:
import asyncio
import time
async def delayer():
await asyncio.sleep(1)
async def messenger():
await asyncio.sleep(1)
return "A Message"
async def main():
task1 = asyncio.create_task(delayer())
task2 = asyncio.create_task(delayer())
await task1
await task2
start_time = time.time()
asyncio.run(main())
end_time = time.time() - start_time
print(end_time)
Now the code will take about 1 second.
My understanding from what I read is that await is a blocking process as we can see from the first code. In that code we need to wait 1 second for the messenger function to return, then another second for delayer function.
Now the real question come from the second code. We just learnt that await need us to wait for its expression to return. So even if we use async.create_task, shouldn't awaits in one of the function's body block the process and then return whenever it finishes its job, thus should give us 2 seconds for the program to end?
If that wasn't the case, can you help me understand the asyncio.create_task?
What I know:
await is a blocking process
await executes coroutine function and task object
await makes us possible to pause coroutine process (I don't quite understand about this, too)
create_task creates task object and then schedule and execute it as soon as possible
What I am expecting:
I hope I can get a simple but effective answer about how does asyncio.create_task conduct its work using my sample code.
Perhaps it will help to think in the following way.
You cannot understand what await does until you understand what an event loop is. This line:
asyncio.run(main())
creates and executes an event loop, which is basically an infinite loop with some methods for allowing an exit - a "semi-infinite" loop, so to speak. Until that loop exits, it will be entirely responsible for executing the program. (Here I am assuming that your program has only a single thread and a single Process. I'm not talking about concurrent program in any form.) Each unit of code that can run within an event loop is called a "Task." The idea of the loop is that it can run multiple Tasks by switching from one to another, thus giving the illusion that the CPU is doing more than one thing at a time.
The asyncio.run() call does a second thing: it creates a Task, main(). At that moment, it's the only Task. The event loop begins to run the Task at its first line. Initially it runs just like any other function:
async def main():
task1 = asyncio.create_task(delayer())
task2 = asyncio.create_task(delayer())
await task1
await task2
It creates two more tasks, task1 and task2. Now there are 3 Tasks but only one of them can be active. That's still main(). Then you come to this line:
await task1
The await keyword is what allows this whole rigmarole to work. It is an instruction to the event loop to suspend the active task right here, at this point, and possibly allow another Task to become the active one. So to address your first bullet point, await is neither "blocking" nor is it a "process". Its purpose is to mark a point at which the event loop gets control back from the active Task.
There is another thing happening here. The object that follows the await is called, unimaginatively, an "awaitable" object. Its crucial property is whether or not it is "done." The event loop keeps track of this object; as the loop cycles through its Tasks it will keep checking this object. If it's not done, main() doesn't resume. (This isn't exactly how it's implemented because that would be inefficient, but it's conceptually what's happening.) If you want to say that the await is "blocking" main() until task1 is finished, that's sort-of true; but "blocking" has a technical meaning so it's not the best word to use. In any case, the event loop is not "blocked" at all - it can keep running other Tasks until the awaitable task1 is done. After task1 becomes "done" and main() gets its turn to be the active task, execution continues to the next line of code.
Your second bullet point, "await executes coroutine function and task object" is not correct. await doesn't execute anything. As I said, it just marks a point where the Task gets suspended and the event loop gets control back. Its awaitable determines when the Task can be resumed.
You say, "await makes [it] possible to pause coroutine process". Not quite right - it ALWAYS suspends the current Task. Whether or not there is a significant delay in the Task's execution depends on whether there are other Tasks that are ready to take over, and also the state of its awaitable.
"create_task creates task object and then schedule and execute it as soon as possible." Correct. But "as soon as possible" means the next time the current Task hits an await expression. Other Tasks may get a turn to run first, before the new Task gets a chance to start. Those details are up to the implementation of the event loop. But eventually the new Task will get a turn.
In the comments you ask, "Is it safe if I say that plain await, not being involved in any event loop or any kind of it, works in blocking manner?" It's absolutely not safe to say that. First of all, there is no such thing as a "plain await". Your task must wait FOR something, otherwise how would the event loop know when to resume? An await without an event loop is either a syntax error or a runtime error - it makes no sense, because await is a point where the Task and the event loop interact. The main point is that event loops and await expression are intimately related: an await without an event loop is an error; an event loop without any await expressions is useless.
The closest you can come to a plain await is this expression:
await asyncio.sleep(0)
which has the effect of suspending the current Task momentarily, giving the event loop a chance to run other tasks, resuming this Task as soon as possible.
One other point is that the code:
await task1
is an expression which has a value, in this case the returned value from task1. Since your task1 doesn't return anything this will be None. But if your delayer function looked like this:
async def delayer():
await asyncio.sleep(1)
return "Hello"
then in main() you could write:
print(await task1)
and you would see "Hello" on the console.
If I have a coroutine currently sleeping to allow other coroutines to run, is is possible to change the sleep time while sleeping? Or would I have to cancel and restart the coroutine. I think I may have just answered myself there. Looking for help from the more experienced.
The "sleep" coroutine is obviously designed to be simple: it pauses for that amount of time, and it is it.
What you seem to need is a way to synchronize your co-routines, and if no signal gets back in an specified amount of time (the time you are passing to sleep), to move on.
Take a look at the synchronization primitives https://docs.python.org/3.6/library/asyncio-sync.html and asyncio.wait_for
So, you can instead of asyncio.sleep, call a co-routine, with wait_for, where it expects an Event, or a Lock release. The Event or lock-release then is used by whatever part of your code would "cancel sleep" anyway.
I created an example to show both sleeping running to the end, and being canceled.
import asyncio
async def interruptable_sleep(time, event):
try:
await asyncio.wait_for(event.wait(), timeout=time)
except asyncio.TimeoutError:
print("'sleeping' proceeded normaly")
else:
print("'sleeping' canceled")
async def sleeper(m, n, event):
await asyncio.sleep(n)
if n == 3:
event.set()
print(f"cycle {m}, step {n}")
async def main():
event = asyncio.Event()
tasks = []
for cycle in range(3):
event.clear()
# create batch of async tasks to run in parallel
for step in range(6):
tasks.append(asyncio.create_task(sleeper(cycle, step, event), name=f"{cycle}_{step}"))
await interruptable_sleep(2, event)
# 'join' remaining tasks
event.set()
await asyncio.gather(*tasks)
asyncio.run(main())
This pattern sort of "reverses" the idea of a timeout: if a task finishes early, the waiting is canceled . (while timeout means "if a task is too late, cancel it") -
But maybe ou just need the other pattern there: to create a list of all your tasks and call asyncio.gather, rather than calling "sleep" to give "time for the other tasks to run".
Awaiting for multiple async functions is not really working asynchronously,for example,I am expecting below code to run in ~6 seconds, but it is running like synchronous code and executing in ~10 seconds.
But when I tried it in asyncio.gather, it is executing in ~6 seconds.
Can someone explain why is this so?
#Not working concurrently
async def async_sleep(n):
await asyncio.sleep(n+2)
await asyncio.sleep(n)
start_time = time.time()
asyncio.run(async_sleep(4))
end_time = time.time()
print(end_time-start_time)
#Working concurrently
async def async_sleep(n):
await asyncio.gather(asyncio.sleep(n+2),
asyncio.sleep(n))
Can someone explain why [gather is faster than consecutive awaits]?
That is by design: await x means "do not proceed with this coroutine until x is complete." If you place two awaits one after the other, they will naturally execute sequentially. If you want parallel execution, you need to create tasks and wait for them to finish, or use asyncio.gather which will do it for you.
Suppose I have some tasks running asynchronously. They may be totally independent, but I still want to set points where the tasks will pause so they can run concurrently.
What is the correct way to run the tasks concurrently? I am currently using await asyncio.sleep(0), but I feel this is adding a lot of overhead.
import asyncio
async def do(name, amount):
for i in range(amount):
# Do some time-expensive work
print(f'{name}: has done {i}')
await asyncio.sleep(0)
return f'{name}: done'
async def main():
res = await asyncio.gather(do('Task1', 3), do('Task2', 2))
print(*res, sep='\n')
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Output
Task1: has done 0
Task2: has done 0
Task1: has done 1
Task2: has done 1
Task1: has done 2
Task1: done
Task2: done
If we were using simple generators, an empty yield would pause the flow of a task without any overhead, but empty await are not valid.
What is the correct way to set such breakpoints without overhead?
As mentioned in the comments, normally asyncio coroutines suspend automatically on calls that would block or sleep in equivalent synchronous code. In your case the coroutine is CPU-bound, so awaiting blocking calls is not enough, it needs to occasionally relinquish control to the event loop to allow the rest of the system to run.
Explicit yields are not uncommon in cooperative multitasking, and using await asyncio.sleep(0) for that purpose will work as intended, it does carry a risk: sleep too often, and you're slowing down the computation by unnecessary switches; sleep too seldom, and you're hogging the event loop by spending too much time in a single coroutine.
The solution provided by asyncio is to offload CPU-bound code to a thread pool using run_in_executor. Awaiting it will automatically suspend the coroutine until the CPU-intensive task is done, without any intermediate polling. For example:
import asyncio
def do(id, amount):
for i in range(amount):
# Do some time-expensive work
print(f'{id}: has done {i}')
return f'{id}: done'
async def main():
loop = asyncio.get_event_loop()
res = await asyncio.gather(
loop.run_in_executor(None, do, 'Task1', 5),
loop.run_in_executor(None, do, 'Task2', 3))
print(*res, sep='\n')
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
I was trying to explain an example of async programming in python but I failed.
Here is my code.
import asyncio
import time
async def asyncfoo(t):
time.sleep(t)
print("asyncFoo")
loop = asyncio.get_event_loop()
loop.run_until_complete(asyncfoo(10)) # I think Here is the problem
print("Foo")
loop.close()
My expectation is that I would see:
Foo
asyncFoo
With a wait of 10s before asyncFoo was displayed.
But instead I got nothing for 10s, and then they both displayed.
What am I doing wrong, and how can I explain it?
run_until_complete will block until asyncfoo is done. Instead, you would need two coroutines executed in the loop. Use asyncio.gather to easily start more than one coroutine with run_until_complete.
Here is a an example:
import asyncio
async def async_foo():
print("asyncFoo1")
await asyncio.sleep(3)
print("asyncFoo2")
async def async_bar():
print("asyncBar1")
await asyncio.sleep(1)
print("asyncBar2")
loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.gather(async_foo(), async_bar()))
loop.close()
Your expectation would work in contexts where you run your coroutine as a Task independent of the flow of the code. Another situation where it would work is if you are running multiple coroutines side-by-side, in which case the event-loop will juggle the code execution from await to await statement.
Within the context of your example, you can achieve your anticipated behaviour by wrapping your coroutine in a Task object, which will continue-on in the background without holding up the remainder of the code in the code-block from whence it is called.
For example.
import asyncio
async def asyncfoo(t):
await asyncio.sleep(t)
print("asyncFoo")
async def my_app(t):
my_task = asyncio.ensure_future(asyncfoo(t))
print("Foo")
await asyncio.wait([my_task])
loop = asyncio.get_event_loop()
loop.run_until_complete(my_app(10))
loop.close()
Note that you should use asyncio.sleep() instead of the time module.
run_until_complete is blocking. So, even if it'll happen in 10 seconds, it will wait for it. After it's completed, the other print occurs.
You should launch your loop.run_until_complete(asyncfoo(10)) in a thread or a subprocess if you want the "Foo" to be print before.