I have been learning and exploring Python asyncio for a while. Before starting this journey I have read loads of articles to understand the subtle differences between multithreading, multiprocessing, and asyncio. But, as far as I know, I missed something on about a fundamental issue. I'll try to explain what I mean by pseudocodes below.
import asyncio
import time
async def io_bound():
print("Running io_bound...")
await asyncio.sleep(3)
async def main():
start = time.perf_counter()
result_1 = await io_bound()
result_2 = await io_bound()
end = time.perf_counter()
print(f"Finished in {round(end - start, 0)} second(s).")
asyncio.run(main())
For sure, it will take around 6 seconds because we called the io_bound coroutine directly twice and didn't put them to the event loop. This also means that they were not run concurrently. If I would like to run them concurrently I will have to use asyncio.gather(*tasks) feature. I run them concurrently it would only take 3 seconds for sure.
Let's imagine this io_bound coroutine is a coroutine that queries a database to get back some data. This application could be built with FastAPI roughly as follows.
from fastapi import FastAPI
app = FastAPI()
#app.get("/async-example")
async def async_example():
result_1 = await get_user()
result_2 = await get_countries()
if result_1:
return {"result": result_2}
return {"result": None}
Let's say the get_user and get_countries methods take 3 seconds each and have asynchronous queries implemented correctly. My questions are:
Do I need to use asyncio.gather(*tasks) for these two database queries? If necessary, why? If not, why?
What is the difference between io_bound, which I call twice, and get_user and get_countries, which I call back to back, in the above example?
In the io_bound example, if I did the same thing in FastAPI, wouldn't it take only 6 seconds to give a response back? If so, why not 3 seconds?
In the context of FastAPI, when would be the right time to use asyncio.gather(*tasks) in an endpoint?
Do I need to use asyncio.gather(*tasks) for these two database
queries? If necessary, why? If not, why?
Do you need to? Nope, what you have done works. The request will take 6 seconds but will not be blocking so if you had another request coming in, FastAPI can process the two requests at the same time. I.e. two requests coming in at the same time will take 6 seconds still, rather than 12 seconds.
If the two functions get_user() and get_countries() are independant of eachother, then you can get the run the functions concurrently using either asyncio.gather or any of the many other ways of doing it in asyncio, which will mean the request will now take just 3 seconds. For example:
async def main():
start = time.perf_counter()
result_1_task = asyncio.create_task(io_bound())
result_2_task = asyncio.create_task(io_bound())
result_1 = await result_1_task
result_2 = await result_2_task
end = time.perf_counter()
print(f"Finished in {round(end - start, 0)} second(s).")
or
async def main_2():
start = time.perf_counter()
results = await asyncio.gather(io_bound(), io_bound())
end = time.perf_counter()
print(f"Finished in {round(end - start, 0)} second(s).")
What is the difference between io_bound, which I call twice, and
get_user and get_countries, which I call back to back, in the above
example?
assuming get_user and get_countries just call io_bound, nothing.
In the io_bound example, if I did the same thing in FastAPI,
wouldn't it take only 6 seconds to give a response back? If so, why
not 3 seconds?
It will take 6 seconds. FastAPI doesn't do magic to change the way your functions work, it just allows you to create a server that can easily run asynchronous functions.
In the context of FastAPI, when would be the right time to use
asyncio.gather(*tasks) in an endpoint?
When you want run two or more asyncronous functions concurrently. This is the same, regardless of if you are using FastAPI or any other asynchronous code in python.
Related
I have the following code running in an event loop where I'm downloading a large number of files using asyncio and restricting the number of files downloaded using asyncio.queue:
download_tasks = asyncio.Queue()
for file in files:
# download_file() is an async function that downloads a file from Microsoft blob storage
# that is basically await blob.download_blob()
download_tasks.put_nowait(asyncio.create_task(download_file(file=file))
async def worker():
while not download_tasks.empty():
return await download_tasks.get_nowait()
worker_limit = 10
# each call to download_file() returns a pandas dataframe
df_list = await asyncio.gather(*[worker() for _ in range(worker_limit)], return_exceptions=True)
df = pd.concat(df_list)
This code seems to run fine, but I originally had the for loop defined as:
for file in files:
# download_file() is an async function that downloads a file from Microsoft blob storage
# that is basically await blob.download_blob()
download_tasks.put_nowait(download_file(file=file)
With this code, the result is the same but I get the following warning:
RuntimeWarning: coroutine 'download_file' was never awaited
Looking at asyncio examples, sometimes I see create_task() used when creating a list or queue of coroutines to be run in gather and sometimes I don't. Why is it needed in my case and what's the best practice for using it?
Edit: As #user2357112supportsMonica discourteously pointed out, the return statement within worker() doesn't really make sense. The point of this code is to limit concurrency because I may have to download thousands at a time and would like to limit it to 10 at a time using the queue. So my actual question is, how can I use gather to return all my results using this queue implementation?
Edit 2: I seemed to have found an easy solution that works using a semaphore instead of a queue with the following code adapted from this answer https://stackoverflow.com/a/61478547/4844593:
download_tasks = []
for file in files:
download_tasks.append(download_file(file=file))
async def gather_with_concurrency(n, *tasks):
semaphore = asyncio.Semaphore(n)
async def sem_task(task):
async with semaphore:
return await task
return await asyncio.gather(*(sem_task(task) for task in tasks))
df_list = await gather_with_concurrency(10, *download_tasks)
return pd.concat(df_list)
As "user2357112 supports Monica" notes, the original issue probably comes from the workers having a return so each worker will download one file then quit, meaning any coroutines after the first 10 will be ignored and never awaited (you can probably see that if you log information about download_tasks after the supposed completion of your processing).
The create_tasks defeats that because it will immediately schedule the downloading at the same time (defeating the attempted rate limiting / workers pool), then the incorrect worker code will just ignore anything after the first 10 items.
Anyway the difference between coroutines (e.g. bare async functions) and tasks is that tasks are independently scheduled. That is, once you've created a task it lives its life independently and you don't have to await it if you don't want its result. That is similar to Javascript's async functions.
coroutines, however, don't do anything until they are awaited, they will only progress if they are explicitelly polled and that is only done by awaiting them (directly or indirectly e.g. gather or wait will await/poll the objects they wrap).
I'm trying to understand how asyncio works. As for I/O operation i got understand that when await was called, we register Future object in EventLoop, and then calling epoll for get sockets which belongs to Future objects, that ready for give us data. After we run registred callback and resume function execution.
But, the thing that i cant understant, what's happening if we use await not for I/O operation. How eventloop understands that task is complete? Is it create socket for that or use another kind of loop? Is it use epoll? Or doesnt it add to Loop and used it as generator?
There is an example:
import asyncio
async def test():
return 10
async def my_coro(delay):
loop = asyncio.get_running_loop()
end_time = loop.time() + delay
while True:
print("Blocking...")
await test()
if loop.time() > end_time:
print("Done.")
break
async def main():
await my_coro(3.0)
asyncio.run(main())
await doesn't automatically yield to the event loop, that happens only when an async function (anywhere in the chain of awaits) requests suspension, typically due to IO or timeout not being ready.
In your example the event loop is never returned to, which you can easily verify by moving the "Blocking" print before the while loop and changing main to await asyncio.gather(my_coro(3.0), my_coro(3.0)). What you'll observe is that the coroutines are executed in series ("blocking" followed by "done", all repeated twice), not in parallel ("blocking" followed by another "blocking" and then twice "done"). The reason for that was that there was simply no opportunity for a context switch - my_coro executed in one go as if they were an ordinary function because none of its awaits ever chose to suspend.
Awaiting for multiple async functions is not really working asynchronously,for example,I am expecting below code to run in ~6 seconds, but it is running like synchronous code and executing in ~10 seconds.
But when I tried it in asyncio.gather, it is executing in ~6 seconds.
Can someone explain why is this so?
#Not working concurrently
async def async_sleep(n):
await asyncio.sleep(n+2)
await asyncio.sleep(n)
start_time = time.time()
asyncio.run(async_sleep(4))
end_time = time.time()
print(end_time-start_time)
#Working concurrently
async def async_sleep(n):
await asyncio.gather(asyncio.sleep(n+2),
asyncio.sleep(n))
Can someone explain why [gather is faster than consecutive awaits]?
That is by design: await x means "do not proceed with this coroutine until x is complete." If you place two awaits one after the other, they will naturally execute sequentially. If you want parallel execution, you need to create tasks and wait for them to finish, or use asyncio.gather which will do it for you.
I have to send a lot of HTTP requests, once all of them have returned, the program can continue. Sounds like a perfect match for asyncio. A bit naively, I wrapped my calls to requests in an async function and gave them to asyncio. This doesn't work.
After searching online, I found two solutions:
use a library like aiohttp, which is made to work with asyncio
wrap the blocking code in a call to run_in_executor
To understand this better, I wrote a small benchmark. The server-side is a flask program that waits 0.1 seconds before answering a request.
from flask import Flask
import time
app = Flask(__name__)
#app.route('/')
def hello_world():
time.sleep(0.1) // heavy calculations here :)
return 'Hello World!'
if __name__ == '__main__':
app.run()
The client is my benchmark
import requests
from time import perf_counter, sleep
# this is the baseline, sequential calls to requests.get
start = perf_counter()
for i in range(10):
r = requests.get("http://127.0.0.1:5000/")
stop = perf_counter()
print(f"synchronous took {stop-start} seconds") # 1.062 secs
# now the naive asyncio version
import asyncio
loop = asyncio.get_event_loop()
async def get_response():
r = requests.get("http://127.0.0.1:5000/")
start = perf_counter()
loop.run_until_complete(asyncio.gather(*[get_response() for i in range(10)]))
stop = perf_counter()
print(f"asynchronous took {stop-start} seconds") # 1.049 secs
# the fast asyncio version
start = perf_counter()
loop.run_until_complete(asyncio.gather(
*[loop.run_in_executor(None, requests.get, 'http://127.0.0.1:5000/') for i in range(10)]))
stop = perf_counter()
print(f"asynchronous (executor) took {stop-start} seconds") # 0.122 secs
#finally, aiohttp
import aiohttp
async def get_response(session):
async with session.get("http://127.0.0.1:5000/") as response:
return await response.text()
async def main():
async with aiohttp.ClientSession() as session:
await get_response(session)
start = perf_counter()
loop.run_until_complete(asyncio.gather(*[main() for i in range(10)]))
stop = perf_counter()
print(f"aiohttp took {stop-start} seconds") # 0.121 secs
So, an intuitive implementation with asyncio doesn't deal with blocking io code. But if you use asyncio correctly, it is just as fast as the special aiohttp framework. The docs for coroutines and tasks don't really mention this. Only if you read up on the loop.run_in_executor(), it says:
# File operations (such as logging) can block the
# event loop: run them in a thread pool.
I was surprised by this behaviour. The purpose of asyncio is to speed up blocking io calls. Why is an additional wrapper, run_in_executor, necessary to do this?
The whole selling point of aiohttp seems to be support for asyncio. But as far as I can see, the requests module works perfectly - as long as you wrap it in an executor. Is there a reason to avoid wrapping something in an executor ?
But as far as I can see, the requests module works perfectly - as long
as you wrap it in an executor. Is there a reason to avoid wrapping
something in an executor ?
Running code in executor means to run it in OS threads.
aiohttp and similar libraries allow to run non-blocking code without OS threads, using coroutines only.
If you don't have much work, difference between OS threads and coroutines is not significant especially comparing to bottleneck - I/O operations. But once you have much work you can notice that OS threads perform relatively worse due to expensively context switching.
For example, when I change your code to time.sleep(0.001) and range(100), my machine shows:
asynchronous (executor) took 0.21461606299999997 seconds
aiohttp took 0.12484742700000007 seconds
And this difference will only increase according to number of requests.
The purpose of asyncio is to speed up blocking io calls.
Nope, purpose of asyncio is to provide convenient way to control execution flow. asyncio allows you to choose how flow works - based on coroutines and OS threads (when you use executor) or on pure coroutines (like aiohttp does).
It's aiohttp's purpose to speed up things and it copes with the task as shown above :)
I'm getting the flow of using asyncio in Python 3.5 but I haven't seen a description of what things I should be awaiting and things I should not be or where it would be neglible. Do I just have to use my best judgement in terms of "this is an IO operation and thus should be awaited"?
By default all your code is synchronous. You can make it asynchronous defining functions with async def and "calling" these functions with await. A More correct question would be "When should I write asynchronous code instead of synchronous?". Answer is "When you can benefit from it". In cases when you work with I/O operations as you noted you will usually benefit:
# Synchronous way:
download(url1) # takes 5 sec.
download(url2) # takes 5 sec.
# Total time: 10 sec.
# Asynchronous way:
await asyncio.gather(
async_download(url1), # takes 5 sec.
async_download(url2) # takes 5 sec.
)
# Total time: only 5 sec. (+ little overhead for using asyncio)
Of course, if you created a function that uses asynchronous code, this function should be asynchronous too (should be defined as async def). But any asynchronous function can freely use synchronous code. It makes no sense to cast synchronous code to asynchronous without some reason:
# extract_links(url) should be async because it uses async func async_download() inside
async def extract_links(url):
# async_download() was created async to get benefit of I/O
html = await async_download(url)
# parse() doesn't work with I/O, there's no sense to make it async
links = parse(html)
return links
One very important thing is that any long synchronous operation (> 50 ms, for example, it's hard to say exactly) will freeze all your asynchronous operations for that time:
async def extract_links(url):
data = await download(url)
links = parse(data)
# if search_in_very_big_file() takes much time to process,
# all your running async funcs (somewhere else in code) will be frozen
# you need to avoid this situation
links_found = search_in_very_big_file(links)
You can avoid it calling long running synchronous functions in separate process (and awaiting for result):
executor = ProcessPoolExecutor(2)
async def extract_links(url):
data = await download(url)
links = parse(data)
# Now your main process can handle another async functions while separate process running
links_found = await loop.run_in_executor(executor, search_in_very_big_file, links)
One more example: when you need to use requests in asyncio. requests.get is just synchronous long running function, which you shouldn't call inside async code (again, to avoid freezing). But it's running long because of I/O, not because of long calculations. In that case, you can use ThreadPoolExecutor instead of ProcessPoolExecutor to avoid some multiprocessing overhead:
executor = ThreadPoolExecutor(2)
async def download(url):
response = await loop.run_in_executor(executor, requests.get, url)
return response.text
You do not have much freedom. If you need to call a function you need to find out if this is a usual function or a coroutine. You must use the await keyword if and only if the function you are calling is a coroutine.
If async functions are involved there should be an "event loop" which orchestrates these async functions. Strictly speaking it's not necessary, you can "manually" run the async method sending values to it, but probably you don't want to do it. The event loop keeps track of not-yet-finished coroutines and chooses the next one to continue running. asyncio module provides an implementation of event loop, but this is not the only possible implementation.
Consider these two lines of code:
x = get_x()
do_something_else()
and
x = await aget_x()
do_something_else()
Semantic is absolutely the same: call a method which produces some value, when the value is ready assign it to variable x and do something else. In both cases the do_something_else function will be called only after the previous line of code is finished. It doesn't even mean that before or after or during the execution of asynchronous aget_x method the control will be yielded to event loop.
Still there are some differences:
the second snippet can appear only inside another async function
aget_x function is not usual, but coroutine (that is either declared with async keyword or decorated as coroutine)
aget_x is able to "communicate" with the event loop: that is yield some objects to it. The event loop should be able to interpret these objects as requests to do some operations (f.e. to send a network request and wait for response, or just suspend this coroutine for n seconds). Usual get_x function is not able to communicate with event loop.