I am trying to use use grequests to make a single HTTP call asynchronously.
All goes find until I try to call a function (handle_cars) to handle the response.
The problem is that the function is an async function and it I don't know how to await it while passing.
Is this even possible?
I need the function to be async because there is another async function I need to call from it. Another solution would be to be able to call and await the other async function (send_cars) from inside the sync function.
async def handle_cars(res):
print("GOT IT")
await send_cars()
async def get_cars():
req = grequests.get(URL, hooks=dict(response=handle_cars))
job = grequests.send(req, grequests.Pool(1))
How do I set the response argument to await the function? Or how do I await send_cars if I make handle_cars synchronous?
Thanks
According to the OP's comments, he wishes to create a request in the background, and call a callback when it finishes.
The way to do so can be implemented using asyncio.create_task() and using task.add_done_callback().
A simpler way however, will work by creating a new coroutine and running it in the background.
Demonstrated using aiohttp, it will work in the following way:
async def main_loop():
task = asyncio.create_task(handle_cars(URL))
while True:
...
async def handle_cars(url):
async with aiohttp.ClientSession() as session:
async with session.get(url) as response:
await send_cars()
Related
In an API developed using FastAPI framework, I am using asyncio to make calls to solr collections, but when I am calling asyncio.run(man(a,b)) from a python file query.py then I am getting asyncio.run() cannot be called from a running event loop.
in controller.py
#router.api_route()
async def api_call(a,b):
#calling a function
resp = query(a,b)
return resp
in query.py
def query(a,b):
result = asyncio.run(man(a,b))
return result
in database_call.py
async def man(a,b)
async with aiohttp.ClientSession() as session:
url = ''
async with session.get(pokemon_url) as resp:
result = await resp.json()
return result
when I am calling asyncio.run(man(a,b)) from query then I am getting asyncio.run() cannot be called from a running event loop. Kindly help me resolve the issue.
I tried:
in query.py
def query(a,b):
loop = asyncio.get_event_loop
result = loop.create_task(man(a,b))
return result
then I am getting <coroutine object main at 0x0394999ejt>
The docs say that you should have only one call to asyncio.run in a program. "Should" doesn't mean the same thing as "must", so it's not a requirement. But it's a good guideline.
Solution 1: get rid of query entirely, and just await the coroutine man() directly.
#router.api_route()
async def api_call(a,b):
return await man(a, b)
Solution 2: declare query to be an async def function, and await it:
#router.api_route()
async def api_call(a,b):
#calling a function
return await query(a,b)
async def query(a,b):
return await man(a,b)
Solution 3: Do not declare query to be async def, but have it return an awaitable object. This is similar to what you tried in your last listing, but you need to await the result.
#router.api_route()
async def api_call(a,b):
#calling a function
return await query(a,b)
def query(a, b):
return asyncio.create_task(man(a, b))
Solution 4: run query in another thread using the asyncio.to_thread function.
Solution 5: run query in a ThreadPool or ProcessPool.
The whole idea of asyncio (or any other form of parallel processing) is to allow the use of a task or a coroutine that does not finish immediately. It runs at a later time, and its result is not available until enough time has passed. Whether you use threading, multiprocessing or asyncio the situation is the same. If you need the answer from the coroutine/task/function in order for your program to continue, you need to wait for it somehow.
I know about await and async must be pair. But. here is my problem:
I hava a function like
def url_open(req):
#fetch url content synchronize
and now, I want to change to another version like:
def url_open(req):
await self.websocket.send(req)
content = await self.websocket.recv()
the problem is url_open function was exist in many place in the whole project, and it 'is in the bottom of the call tree. it's not possible to make all the caller to be async function.
So, how can I make this work? get some result from another async functions like in python websocket module?
======= updated ======
After I tried as #Viliam Popovec provided
I got this warning
RuntimeWarning: coroutine 'hello' was never awaited
and error
This event loop is already running
The websocket was run like this in my app
async def main():
async with websockets.serve(echo, '0.0.0.0', 8001):
await asyncio.Future()
asyncio.run(main())
It't seems that the new event loop has conflicts with the existing websocket runloop
If you can't modify all instances of url_open function to be ansynchronous, then you could use loops to call some async function (e.g. hello() in code sample below).
async def hello():
await asyncio.sleep(1)
print("Hello World!")
def url_open():
loop = asyncio.get_event_loop()
loop.run_until_complete(hello())
Whenever i try to make a Chatbot
I choose to use endpoint
But i get this error
File "/app/chatbot/plugins/response.py", line 10
print((await get_response('world')))
^
SyntaxError: 'await' outside function
Please help me i would be highly obliged if you help me
Where my code is
import aiohttp
async def get_response(query):
async with aiohttp.ClientSession() as ses:
async with ses.get(
f'https://some-random-api.ml/chatbot?message={query}'
) as resp:
return (await resp.json())['response']
print((await get_response('world')))
Solution:
await is used in an async functions/methods to wait on other asynchronous tasks, but if you are calling an async function/method outside of an async function/method you need to use asyncio.run() method to call an async function/method
Here is the full solution:
import aiohttp
import asyncio #to run async funtions you need to import asyncio
async def get_response(query):
async with aiohttp.ClientSession() as ses:
async with ses.get(
f'https://some-random-api.ml/chatbot?message={query}'
) as resp:
return (await resp.json())['response']
print((asyncio.run( get_response('world')))#run the async function
I'm currently trying to learn asyncio in Python. I know that the await keyword tells the loop that it can switch coroutines. However, when should I actually use it? Why not put it before everything?
Additionally, why is the await before 'response.text()', why not before the session.get(url)?
async def print_preview(url):
# connect to the server
async with aiohttp.ClientSession() as session:
# create get request
async with session.get(url) as response:
# wait for response
response = await response.text()
# print first 3 not empty lines
count = 0
lines = list(filter(lambda x: len(x) > 0, response.split('\n')))
print('-'*80)
for line in lines[:3]:
print(line)
print()
You use await with functions that are marked as coroutine in the documentation. For example, ClientResponse.text is marked as coroutine, while ClientResponse.close is not, which means you must await the former and must not await the latter. If you forget to await a coroutine, it simply won't execute and its return value will be a "coroutine object", which is useless (except for use with await).
session.get() returns an async context manager. When passed to async with, the coroutines it implements are awaited behind the scenes.
Also note that awaiting is not the only thing you can do with coroutines, the other is converting them into tasks, which allows them to run in parallel (without additional cost on the OS level). For more information, consult a tutorial on asyncio.
I am using Python with discord.py. Documentation here
I've got a bot that is running on a Discord server that links the server with a subreddit. Users have various commands that do things like getting the top submissions, getting the latest submissions, and so on.
I want to add some features to the bot, with one of them being a keyword notifier. The bot should search the subreddit for keywords in the title, and then notify users if they are on the list for that keyword. I know how to do this, I've done it plenty of times, but I don't know how to do it with a Discord bot. I have no experience with asynchio or any kind of asynchronous programming.
The way I've tried to do it works, but it is very janky and definitely not good. At the top of the on message() function, I just add a call to the search_submissions() function, so that whenever someone puts sends a new message on the server, the bot will scan the Reddit submissions. The server is busy enough that this would work relatively okay, but I really want to do it the "proper" way.
I don't know how to call the search_submissions() function without putting it inside of on_message().
Edit for extra code:
import discord
TOKEN = "redacted"
client = discord.Client()
#client.event
async def reddit_search():
print("Searching")
#client.event
async def on_message(message):
if message.content.startswith("reddit!hot"):
# Get hot
# Do other things.
#client.event
async def on_ready():
print("Connected to Discord as {}.".format(client.user.name))
client.run(TOKEN)
You can add a function to the bot event loop with Client.loop.create_task(search_submissions()) like this:
async def search_submissions():
pass
client = discord.Client()
client.loop.create_task(search_submissions())
client.run(TOKEN)
Update:
If you want your function to continue working you can put it in a while loop with some sleeping in between:
async def search_submissions():
while(true):
# do your stuff
await asyncio.sleep(1)
The other answers here don't take into account discord.py's helpful tasks.loop decorator.
To make an event occur every 5 seconds, you would use
from discord.ext import tasks, commands
class MyCog(commands.Cog):
def __init__(self):
self.foo.start()
def cog_unload(self):
self.printer.cancel()
#tasks.loop(seconds=5.0)
async def foo(self):
print('bar')
More can be found here: https://discordpy.readthedocs.io/en/latest/ext/tasks/
You want your search_submissions() function to be async so other functions of your bot can still be invoked and your bot stays responsive. Define it to be def async and use aiohttp to send async HTTP requests to reddit -- what this does is send off the request, relinquish control to the event loop, and then take back control once the results have been transmitted back. If you use a standard HTTP library here instead then your whole bot will be blocked until the result comes back. This of course only makes sense if the task is mainly I/O-bound and less CPU-bound.
Then call search_submissions() in on_message(message) -- but call it asynchronously using result = await search_submissions(). This will resume execution of on_message once the result of search_submissions is ready.
If you truly want to do something else in the same context while waiting on search_submissions (which I think is unlikely), dispatch it as task = asyncio.create_task(search_submissions()). This will start the task immediately and allow you to do something else within the same function. Once you need the result you will have to result = await task.
async def search_submissions():
async with aiohttp.ClientSession() as session:
async with session.get(some_reddit_url) as response:
return await response.read()
#client.event
async def on_message(message):
if message.content.startswith("reddit!hot"):
result = await search_submissions()
await message.channel.send(result)