I am trying setup a flag for my async loop. I tried using asyncio.Event but I get a RuntimeError
Here's my code:
import sys
import asyncio
import logging
root = logging.getLogger()
root.setLevel(logging.DEBUG)
ch = logging.StreamHandler(sys.stdout)
ch.setFormatter(
logging.Formatter('%(asctime)s - %(name)s - %(levelname)s %(message)s')
)
root.addHandler(ch)
global event
event = asyncio.Event()
class AsyncTest():
async def bg(self):
root.debug('Waiting for 10 secs')
await asyncio.sleep(10)
event.set()
root.debug('Setted Event')
async def waiter(self):
event.clear()
root.debug('Created task')
asyncio.create_task(self.bg())
await asyncio.sleep(0)
root.debug('waiting')
await event.wait()
root.debug('Finished Waiting.')
return
async def main():
obj = AsyncTest()
await obj.waiter()
asyncio.run(main())
The bg courotine does a periodic task. Therefore it has to be run in asyncio.create_task() .
The waiter coroutine is called at multiple points to wait until the event is set.
So while running this i get this error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\kevin\AppData\Local\Programs\Python\Python37\lib\asyncio\runners.py", line 43, in run
return loop.run_until_complete(main)
File "C:\Users\kevin\AppData\Local\Programs\Python\Python37\lib\asyncio\base_events.py", line 587, in run_until_complete
return future.result()
File "C:\Python Programs\Python\KLibs\NeuralConnect\test\test.py", line 39, in main
await obj.waiter()
File "C:\Python Programs\Python\KLibs\NeuralConnect\test\test.py", line 32, in waiter
await event.wait()
File "C:\Users\kevin\AppData\Local\Programs\Python\Python37\lib\asyncio\locks.py", line 293, in wait
await fut
RuntimeError: Task <Task pending coro=<main() running at C:\Python Programs\Python\KLibs\NeuralConnect\test\test.py:39> cb=[_run_until_complete_cb() at C:\Users\kevin\AppData\Local\Programs\Python\Python37\lib\asyncio\base_events.py:157]> got Future <Future pending> attached to a different loop
can anyone point out what is wrong in my above implementation? Also how should I correct? Should abandon the asyncio.Event() part and then use a bool variable?
The problem using the bool flag variable is that I have to do:
cond = False
while not cond:
await asyncio.sleep(0.3)
I find this un-pythonic. Is there any better implementation of this, incase asyncio.Event isnt suitable?
Finally I recieved an answer to this problem from alex_normane who answered a question linked to this. It seems that the asyncio.Event object has to passed as parameter to the coroutine in create_task(). Plus the asyncio.Event object must be created within a coroutine as also stated by user4815162342.
The answer is here: https://stackoverflow.com/a/65353254/13574717
Related
The main idea is to make a discord integration to the existing telegram bot written with python-telegram-bot library. So here I was thinking about running multiple coroutines with asyncio.gather and asyncio.run (I'm using python 3.10).
Right now I'm not doing anything with my existing code. Instead to avoid any possible blocking code in my main app for now I decided to write this simple program which creates a bot and just push a message to logs if message is received:
import logging as log
import asyncio
import os
import discord
from discord.ext import commands
log.basicConfig(format='%(asctime)s - %(name)s - %(levelname)s - %(message)s', level=log.DEBUG)
client = commands.Bot(command_prefix="/")
#client.event
async def on_ready():
log.info('Connected to Discord')
#client.event
async def on_message(message):
log.info("Got message")
async def start_discord():
await client.start(os.environ.get('DISCORD_TOKEN'))
async def main():
await asyncio.gather(start_discord())
asyncio.run(main())
When I start it everything looks ok, but after a while I started to get exceptions that heartbeat is blocked and WebSocket connection timed out. In addition to that none of both events are triggered no matter how many messages are sent:
2022-07-04 17:50:52,378 - discord.gateway - WARNING - Shard ID None heartbeat blocked for more than 10 seconds.
Loop thread traceback (most recent call last):
File "/mnt/d/Documents/Projects/test.py", line 28, in <module>
asyncio.run(main())
File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/lib/python3.10/asyncio/base_events.py", line 633, in run_until_complete
self.run_forever()
File "/usr/lib/python3.10/asyncio/base_events.py", line 600, in run_forever
self._run_once()
File "/usr/lib/python3.10/asyncio/base_events.py", line 1860, in _run_once
event_list = self._selector.select(timeout)
File "/usr/lib/python3.10/selectors.py", line 469, in select
fd_event_list = self._selector.poll(timeout, max_ev)
But if I replace asyncio.run(main) with asyncio.get_event_loop().run_until_complete(main()) everything starts to work correctly. I see that both events are triggered and message is received. But as I see get_event_loop is deprecated and it's suggested to use asyncio.run() from 3.10:
/mnt/d/Documents/Projects/test.py:29: DeprecationWarning: There is no current event loop
asyncio.get_event_loop().run_until_complete(main())
I am using asyncio and aiohttp to make concurrent requests. I've recently upgraded Python to version 3.8.0 and I'm getting a RuntimeError: Event loop is closed after the program has run.
import asyncio
import aiohttp
async def do_call(name, session):
async with session.get('https://www.google.be') as response:
await response.text()
return 'ok - {}'.format(name)
async def main():
async with aiohttp.ClientSession() as session:
tasks = [do_call(str(i), session) for i in range(0,4)]
results = await asyncio.gather(*tasks)
print(results)
asyncio.run(main())
I do get a valid result from asyncio.gather(), but when exiting the exception is raised.
I'd like to change the code so it doesn't run into exceptions.
The traceback is as follows:
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x000001E9A92079D0>
Traceback (most recent call last):
File "C:\Users\Jonas\AppData\Local\Programs\Python\Python38\lib\asyncio\proactor_events.py", line 116, in __del__
self.close()
File "C:\Users\Jonas\AppData\Local\Programs\Python\Python38\lib\asyncio\proactor_events.py", line 108, in close
self._loop.call_soon(self._call_connection_lost, None)
File "C:\Users\Jonas\AppData\Local\Programs\Python\Python38\lib\asyncio\base_events.py", line 711, in call_soon
self._check_closed()
File "C:\Users\Jonas\AppData\Local\Programs\Python\Python38\lib\asyncio\base_events.py", line 504, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed`
I think this is most likely an aiohttp bug. Specifically, I found this issue on their github: https://github.com/aio-libs/aiohttp/issues/4324
I realise this doesn't necessarily help you, but maybe you can switch back and stop banging your head against a wall. Your code is fine!
I solved this by not calling my_loop.close() after I was done using it. Closing the event loop this way caused the error to be thrown even after I had gotten all the responses I was expecting.
I am using a third party API Wrapper that supports async. I am having a difficult time figuring out how to use it as I am new to asyncio, and I'm not sure if the errors I'm getting are my usage (most likely) or the wrapper itself.
The API can be found here: https://clashroyale.readthedocs.io/en/latest/api.html#royaleapi
The goal of my code is to make three API requests at once (the .get_clan(*lClanGroup)), asynchronously instead of synchronously.
My Code:
import clashroyale
import asyncio
import aiohttp
# Define Tokens
unofficialAPIToken = "secretToken"
# Get Client Objects
session1 = aiohttp.ClientSession()
unofficialClient = clashroyale.royaleapi.Client(unofficialAPIToken, session=session1, is_async=True)
lClanGroups = ['PPCLCJG9', '2LRU2J', 'PGLQ0VQ', 'YU2RQG9', '2LVRQ29'],['PYP8UPJV', 'P9L0CYY0', 'Y2RGQPJ', '8P2GYJ8', '9VQJPL2L'],['RYPRGCJ', '809R8PG8', 'PJY9PP98', '2GCQLC', '2GL2QPPL']
async def x(lClanGroup):
print(*lClanGroup)
a = await unofficialClient.get_clan(*lClanGroup) # Iterates five tags for the API to request at once
return a
async def y(lClanGroups):
result = await asyncio.gather(x(lClanGroups[0]),x(lClanGroups[1]),x(lClanGroups[2]))
return result
async def close_sessions():
await session1.close()
asyncio.run(y(lClanGroups))
asyncio.run(close_sessions())
The error I'm getting is long and difficult to make sense of:
C:\Users\Adam\PycharmProjects\ClashRoyaleMeta\venv2\Scripts\python.exe C:/Users/Adam/PycharmProjects/ClashRoyaleMeta/testme2.py
PPCLCJG9 2LRU2J PGLQ0VQ YU2RQG9 2LVRQ29
PYP8UPJV P9L0CYY0 Y2RGQPJ 8P2GYJ8 9VQJPL2L
RYPRGCJ 809R8PG8 PJY9PP98 2GCQLC 2GL2QPPL
C:/Users/Adam/PycharmProjects/ClashRoyaleMeta/testme2.py:16: DeprecationWarning: The object should be created from async function
session1 = aiohttp.ClientSession()
Traceback (most recent call last):
File "C:/Users/Adam/PycharmProjects/ClashRoyaleMeta/testme2.py", line 33, in <module>
asyncio.run(y(lClanGroups))
File "C:\Program Files\Python38\lib\asyncio\runners.py", line 43, in run
return loop.run_until_complete(main)
File "C:\Program Files\Python38\lib\asyncio\base_events.py", line 612, in run_until_complete
return future.result()
File "C:/Users/Adam/PycharmProjects/ClashRoyaleMeta/testme2.py", line 27, in y
result = await asyncio.gather(x(lClanGroups[0]),x(lClanGroups[1]),x(lClanGroups[2]))
File "C:/Users/Adam/PycharmProjects/ClashRoyaleMeta/testme2.py", line 23, in x
a = await unofficialClient.get_clan(*lClanGroup)
File "C:\Users\Adam\PycharmProjects\ClashRoyaleMeta\venv2\lib\site-packages\clashroyale\royaleapi\client.py", line 203, in _aget_model
raise e
File "C:\Users\Adam\PycharmProjects\ClashRoyaleMeta\venv2\lib\site-packages\clashroyale\royaleapi\client.py", line 196, in _aget_model
data, cached, ts, resp = await self._request(url, **params)
File "C:\Users\Adam\PycharmProjects\ClashRoyaleMeta\venv2\lib\site-packages\clashroyale\royaleapi\client.py", line 150, in _arequest
async with self.session.get(url, timeout=timeout, headers=self.headers, params=params) as resp:
File "C:\Users\Adam\PycharmProjects\ClashRoyaleMeta\venv2\lib\site-packages\aiohttp\client.py", line 1012, in __aenter__
self._resp = await self._coro
File "C:\Users\Adam\PycharmProjects\ClashRoyaleMeta\venv2\lib\site-packages\aiohttp\client.py", line 426, in _request
with timer:
File "C:\Users\Adam\PycharmProjects\ClashRoyaleMeta\venv2\lib\site-packages\aiohttp\helpers.py", line 579, in __enter__
raise RuntimeError('Timeout context manager should be used '
RuntimeError: Timeout context manager should be used inside a task
Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x0000016C5AD50EB0>
Process finished with exit code 1
Thanks for your help!
Each asyncio.run() creates new event loop. Moreover, if you create objects like aiohttp.ClientSession() globally, they may be bind to default event loop.
Using multiple event loops in your program is dangerous and may lead to non-obvious problems. To avoid the situation:
Run asyncio.run() only once
Create all asyncio-related stuff inside main coroutine
I didn't work with royaleapi, but it seems API developers do things differently than you in their tests. Here's how they innit/close stuff and here's how they get clan.
Let's give it a try?
import clashroyale
import asyncio
TOKEN = "secretToken"
clan_groups = [
['PPCLCJG9', '2LRU2J', 'PGLQ0VQ', 'YU2RQG9', '2LVRQ29'],
['PYP8UPJV', 'P9L0CYY0', 'Y2RGQPJ', '8P2GYJ8', '9VQJPL2L'],
['RYPRGCJ', '809R8PG8', 'PJY9PP98', '2GCQLC', '2GL2QPPL']
]
async def get_clans(cr, clan_groups):
return await asyncio.gather(*[
cr.get_clan(tag)
for group in clan_groups
for tag in group
])
async def main():
cr = clashroyale.RoyaleAPI(
TOKEN,
is_async=True,
timeout=30
)
try:
results = await get_clans(cr, clan_groups)
print(results)
finally:
await cr.close()
await asyncio.sleep(2)
asyncio.run(main())
Didn't test it.
I'm trying to make a Discord Bot and it has to check an API every minute or so and then send a message to a channel.
But for some reason, the loop breaks and ends the task.
It's my first time using asyncio and discord.py
Traceback (most recent call last):
File "bot.py", line 207, in <module>
client.loop.run_forever()
File "/usr/local/lib/python3.6/asyncio/base_events.py", line 407, in run_forever
self._check_closed()
File "/usr/local/lib/python3.6/asyncio/base_events.py", line 358, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
Task was destroyed but it is pending!
task: <Task pending coro=<my_background_task() running at bot.py:193> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0xb52dbd10>()]>>
The source code for the script is this one:
bot.py
Just want to have my_background_task() to run forever, or at least restart if it breaks.
Make the bot run the async task
async def bg():
for i in range(3):
print("Background tast")
await asyncio.sleep(1)
#bot.event
async def on_ready():
print("Ready!")
bot.loop.create_task(bg())
I am trying to make an API call from within my service and I am running into event loop issues. Can someone help me understand what I am doing wrong?
Basically I want to make a service that does some calculations based on data pulled from a different service.
I can call this code below from a cli, but not when I start up a web app (i.e.) hitting http://127.0.0.1:8080/add
loop = asyncio.get_event_loop()
data = loop.run_until_complete(run_fetch(loop, 'http://google.com'))
Sample code:
from aiohttp import web
import aiohttp
import asyncio
async def add(request):
loop = asyncio.get_event_loop()
data = loop.run_until_complete(run_fetch(loop, 'http://google.com'))
return web.json_response(data)
async def fetch(client, url):
async with client.get(url) as resp:
assert resp.status == 200
return await resp.text()
async def run_fetch(loop, url):
async with aiohttp.ClientSession(loop=loop) as client:
html = await fetch(client, url)
return html
app = web.Application()
app.router.add_get('/add', add)
web.run_app(app, host='127.0.0.1', port=8080)
Exception:
Error handling request
Traceback (most recent call last):
File ".../aiohttp/web_protocol.py", line 417, in start
resp = yield from self._request_handler(request)
File ".../aiohttp/web.py", line 289, in _handle
resp = yield from handler(request)
File ".../sample.py", line 11, in add data = loop.run_until_complete(run_fetch(loop, 'http://google.com'))
File ".../python3.6/asyncio/base_events.py", line 454, in run_until_complete
self.run_forever()
File ".../python3.6/asyncio/base_events.py", line 408, in run_forever
raise RuntimeError('This event loop is already running')
RuntimeError: This event loop is already running
The run_until_complete is the way to run some async code from the sync context. Under the hood it adds provided future to the given ioloop and calls run_forever then returns result or throws exception (resolved future).
Actually you need to await run_fetch(loop, 'http://google.com'), since the caller function is asynchronous.