I am trying to run a simple asyncio loop that I want to run continuously until Control + C is pressed, which should stop the main function and run a function to shutdown the loop. I am using the following code:
import asyncio
async def run():
while True:
print("Running")
async def shutdown():
print("Starting to shutdown")
await asyncio.sleep(1)
print("Finished sleeping")
try:
loop = asyncio.get_event_loop()
loop.create_task(run())
loop.run_forever()
except KeyboardInterrupt:
loop.stop()
loop.run_until_complete(asyncio.gather(*[shutdown()]))
When running this code and pressing Control + C the script doesn't shutdown gracefully as I expected (also based on this stackoverflow answer but I get the following error:
Traceback (most recent call last):
File "test.py", line 25, in <module>
loop.run_until_complete(asyncio.gather(*[shutdown()]))
File "/usr/lib/python3.7/asyncio/base_events.py", line 585, in run_until_complete
raise RuntimeError('Event loop stopped before Future completed.')
RuntimeError: Event loop stopped before Future completed.
ERROR:asyncio:Task exception was never retrieved
future: <Task finished coro=<run() done, defined at test.py:8> exception=KeyboardInterrupt()>
Traceback (most recent call last):
File "test.py", line 22, in <module>
loop.run_forever()
File "/usr/lib/python3.7/asyncio/base_events.py", line 541, in run_forever
self._run_once()
File "/usr/lib/python3.7/asyncio/base_events.py", line 1786, in _run_once
handle._run()
File "/usr/lib/python3.7/asyncio/events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "test.py", line 10, in run
print("Running")
KeyboardInterrupt
Can anyone tell me where I am going wrong and how I should change my code to solve the issue?
Related
I'm making a discord bot and I want to be able to restart it via a command. I have the command loaded in the main script right below my client events for error, onmessage, etc, as:
#_client.command(name='reboot')
async def reboot(command_context: commands.Context):
~some unimportant code that i have for debugging and user verification
os.execv(sys.executable, ['py']+sys.argv
Upon running this by calling )reboot in the discord chat my bot is monitoring, my program exits with exit code 0 and does not restart.
Here's the place i found some stuff that didn't work for me:
Restart python-script from within itself
Some stuff, not sure how relevant any of it is:
There's async stuff going on, so i was wondering if there might be some voodoo code quirks that come into play because of that
Running the code from pycharm, which has a venv. I don't really know what that means specifically, but I have an idea from my experience, and maybe it's messing something up idk.
Another thing I found, when trying to use exit() to stop the program, it takes awhile and then gives me this error message (apologies for size, no clue what's important and whats not):
Exception in thread Thread-16:
Traceback (most recent call last):
File "C:\Users\lkapp\AppData\Local\Programs\Python\Python310\lib\threading.py", line 1016, in _bootstrap_inner
self.run()
File "C:\Users\lkapp\AppData\Local\Programs\Python\Python310\lib\threading.py", line 1378, in run
self.function(*self.args, **self.kwargs)
File "C:\Users\lkapp\PycharmProjects\Imouto\cog_rcon.py", line 64, in <lambda>
asyncio.run_coroutine_threadsafe(
File "C:\Users\lkapp\AppData\Local\Programs\Python\Python310\lib\asyncio\tasks.py", line 885, in run_coroutine_threadsafe
loop.call_soon_threadsafe(callback)
AttributeError: '_MissingSentinel' object has no attribute 'call_soon_threadsafe'
C:\Users\lkapp\AppData\Local\Programs\Python\Python310\lib\threading.py:1018: RuntimeWarning: coroutine 'Server_Bridge.update_player_count_in_discord_activity' was never awaited
self._invoke_excepthook(self)
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
Task was destroyed but it is pending!
task: <Task pending name='Task-1' coro=<ARC.listenForData() running at C:\Users\lkapp\PycharmProjects\Imouto\bec_rcon.py:518>>
Task was destroyed but it is pending!
task: <Task pending name='Task-2' coro=<ARC.keepAliveLoop() running at C:\Users\lkapp\PycharmProjects\Imouto\bec_rcon.py:554>>
sys:1: RuntimeWarning: coroutine 'ARC.listenForData' was never awaited
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
C:\Users\lkapp\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py:671: RuntimeWarning: coroutine 'ARC.keepAliveLoop' was never awaited
self._ready.clear()
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
Task exception was never retrieved
future: <Task finished name='discord.py: on_message' coro=<Client._run_event() done, defined at C:\Users\lkapp\PycharmProjects\Imouto\venv\lib\site-packages\discord\client.py:401> exception=SystemExit()>
Traceback (most recent call last):
File "C:\Users\lkapp\PycharmProjects\Imouto\venv\lib\site-packages\discord\client.py", line 828, in run
asyncio.run(runner())
File "C:\Users\lkapp\AppData\Local\Programs\Python\Python310\lib\asyncio\runners.py", line 44, in run
return loop.run_until_complete(main)
File "C:\Users\lkapp\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 633, in run_until_complete
self.run_forever()
File "C:\Users\lkapp\AppData\Local\Programs\Python\Python310\lib\asyncio\windows_events.py", line 321, in run_forever
super().run_forever()
File "C:\Users\lkapp\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 600, in run_forever
self._run_once()
File "C:\Users\lkapp\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 1896, in _run_once
handle._run()
File "C:\Users\lkapp\AppData\Local\Programs\Python\Python310\lib\asyncio\events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "C:\Users\lkapp\PycharmProjects\Imouto\venv\lib\site-packages\discord\client.py", line 409, in _run_event
await coro(*args, **kwargs)
File "C:\Users\lkapp\PycharmProjects\Imouto\main.py", line 43, in on_message
await _client.process_commands(message)
File "C:\Users\lkapp\PycharmProjects\Imouto\venv\lib\site-packages\discord\ext\commands\bot.py", line 1389, in process_commands
await self.invoke(ctx) # type: ignore
File "C:\Users\lkapp\PycharmProjects\Imouto\venv\lib\site-packages\discord\ext\commands\bot.py", line 1347, in invoke
await ctx.command.invoke(ctx)
File "C:\Users\lkapp\PycharmProjects\Imouto\venv\lib\site-packages\discord\ext\commands\core.py", line 986, in invoke
await injected(*ctx.args, **ctx.kwargs) # type: ignore
File "C:\Users\lkapp\PycharmProjects\Imouto\venv\lib\site-packages\discord\ext\commands\core.py", line 190, in wrapped
ret = await coro(*args, **kwargs)
File "C:\Users\lkapp\PycharmProjects\Imouto\cog_rcon.py", line 268, in debug
sys.exit()
SystemExit
Task was destroyed but it is pending!
task: <Task pending name='Task-150' coro=<Server_Bridge.cycle_reconnect() done, defined at C:\Users\lkapp\PycharmProjects\Imouto\cog_rcon.py:169> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[_chain_future.<locals>._call_set_state() at C:\Users\lkapp\AppData\Local\Programs\Python\Python310\lib\asyncio\futures.py:392]>
Process finished with exit code 0
The correct way to restart the python script is
os.execv(sys.executable, ['python'] + sys.argv)
not 'py'.
Alternatively, if you might have multiple installations of Python, be more specific by using
os.execv(sys.executable, ['python3'] + sys.argv) # for python3
Or, for example,
os.execv(sys.executable, ['python3.10'] + sys.argv) # for python 3.10
Hope this helped.
I am trying to add a web ui to a discord bot using Quart. From what I've seen the appropriate way to do this is to from a instance of discord.Bot create a task and to run it. I'm currently doing it this way
def start():
bot.loop.create_task(app.run_task("0.0.0.0"))
load_commands()
print(f"loaded commands: {loaded_commands}")
bot.run(TOKEN)
however when this is run I get the following error
Task exception was never retrieved
future: <Task finished name='Task-1' coro=<serve() done, defined at /var/home/nullrequest/.local/share/virtualenvs/lunbot-ldM1Y48e/lib/python3.10/site-packages/hypercorn/asyncio/__init__.py:9> exception=TypeError("BaseEventLoop.create_server() got an unexpected keyword argument 'loop'")>
Traceback (most recent call last):
File "/var/home/nullrequest/.local/share/virtualenvs/lunbot-ldM1Y48e/lib/python3.10/site-packages/hypercorn/asyncio/__init__.py", line 39, in serve
await worker_serve(app, config, shutdown_trigger=shutdown_trigger)
File "/var/home/nullrequest/.local/share/virtualenvs/lunbot-ldM1Y48e/lib/python3.10/site-packages/hypercorn/asyncio/run.py", line 128, in worker_serve
await asyncio.start_server(
File "/usr/lib64/python3.10/asyncio/streams.py", line 84, in start_server
return await loop.create_server(factory, host, port, **kwds)
TypeError: BaseEventLoop.create_server() got an unexpected keyword argument 'loop'
Task was destroyed but it is pending!
task: <Task pending name='Task-3' coro=<Lifespan.handle_lifespan() running at /var/home/nullrequest/.local/share/virtualenvs/lunbot-ldM1Y48e/lib/python3.10/site-packages/hypercorn/asyncio/lifespan.py:30> wait_for=<Future pending cb=[Task.task_wakeup()]>>
I am using python 3.10rc1 if that makes any difference.
I have same error. Python 3.10 released on 04.10.2021. May be however it depends on the version of interpreter.
for message in client.iter_history(chat.id):
if message.from_user.id == user_id:
# My code...
My traceback:
Traceback (most recent call last):
File "D:\freelance\messagesParser\main.py", line 74, in <module>
main(sys.argv)
File "D:\freelance\messagesParser\main.py", line 64, in main
messages[chat] = get_messages(client, chat, **user_data)
File "D:\freelance\messagesParser\main.py", line 22, in get_messages
for message in client.iter_history(chat.id):
File "D:\freelance\messagesParser\venv\lib\site-packages\pyrogram\sync.py", line 59, in async_to_sync_wra
p
return loop.run_until_complete(consume_generator(coroutine))
File "C:\Program Files\Python310\lib\asyncio\base_events.py", line 641, in run_until_complete
return future.result()
File "D:\freelance\messagesParser\venv\lib\site-packages\pyrogram\sync.py", line 34, in consume_generator
return types.List([i async for i in coroutine])
File "D:\freelance\messagesParser\venv\lib\site-packages\pyrogram\sync.py", line 34, in <listcomp>
return types.List([i async for i in coroutine])
File "D:\freelance\messagesParser\venv\lib\site-packages\pyrogram\methods\messages\iter_history.py", line
79, in iter_history
messages = await self.get_history(
File "D:\freelance\messagesParser\venv\lib\site-packages\pyrogram\methods\messages\get_history.py", line
86, in get_history
messages = await utils.parse_messages(
File "D:\freelance\messagesParser\venv\lib\site-packages\pyrogram\utils.py", line 91, in parse_messages
parsed_messages.append(await types.Message._parse(client, message, users, chats, replies=0))
File "D:\freelance\messagesParser\venv\lib\site-packages\pyrogram\types\messages_and_media\message.py", l
ine 619, in _parse
sticker = await types.Sticker._parse(
File "D:\freelance\messagesParser\venv\lib\site-packages\pyrogram\types\messages_and_media\sticker.py", l
ine 135, in _parse
set_name = await Sticker._get_sticker_set_name(client.send, input_sticker_set_id)
File "D:\freelance\messagesParser\venv\lib\site-packages\async_lru.py", line 237, in wrapped
return (yield from asyncio.shield(fut, loop=_loop))
TypeError: shield() got an unexpected keyword argument 'loop'
I have a code that only runs on Linux with asyncio, but I need it to run on Windows, my knowledge is little in multiprocessing, what would be the way to run on Windows? I saw that there is a limitation of asyncio to run on windows: https://docs.python.org/3/library/asyncio-platforms.html#asyncio-windows-subprocess
My code:
import sys
import asyncio
scriptspy = [
'scrp1.py',
'scrp2.py',
'scrp3.py',
'scrp4.py',
'scrp5.py',
'scrp6.py',
'scrp7.py',
'scrp8.py',
'scrp9.py',
'scrp10.py',
'scrp11.py',
'scrp12.py',
]
async def main():
tarefas_rodando = set()
while scriptspy:
# start up to 6 scripts
while len(tarefas_rodando) < 6 and scriptspy:
script = scriptspy.pop()
p = await asyncio.create_subprocess_exec(sys.executable, script)
tarefa = asyncio.create_task(p.wait())
tarefas_rodando.add(tarefa)
# wait for one of the scripts to end
tarefas_rodando, finalizadas = await asyncio.wait(tarefas_rodando, return_when=asyncio.FIRST_COMPLETED)
# finished, wait for the rest to finish
await asyncio.wait(tarefas_rodando, return_when=asyncio.ALL_COMPLETED)
asyncio.run(main())
This code runs fine on linux but not windows. I need to run it on windows. https://docs.python.org/3/library/asyncio-platforms.html#asyncio-windows-subprocess
Exception run in Windows:
Traceback (most recent call last):
File "scripts.py", line 53, in <module>
asyncio.run(main())
File "C:\Python\Python37\lib\asyncio\runners.py", line 43, in run
return loop.run_until_complete(main)
File "C:\Python\Python37\lib\asyncio\base_events.py", line 584, in run_until_complete
return future.result()
File "scripts.py", line 44, in main
p = await asyncio.create_subprocess_exec(sys.executable, script)
File "C:\Python\Python37\lib\asyncio\subprocess.py", line 217, in create_subprocess_exec
stderr=stderr, **kwds)
File "C:\Python\Python37\lib\asyncio\base_events.py", line 1533, in subprocess_exec
bufsize, **kwargs)
File "C:\Python\Python37\lib\asyncio\base_events.py", line 463, in
_make_subprocess_transport
raise NotImplementedError NotImplementedError
According to the documentation, to run subprocesses on Windows, you need to switch to the proactor event loop:
asyncio.set_event_loop_policy(asyncio.WindowsProactorEventLoopPolicy())
I just simply use the tornado application together with threading as the following code:
def MakeApp():
return tornado.web.Application([(r"/websocket", EchoWebSocket), ])
def run_tornado_websocket():
app = MakeApp()
http_server = tornado.httpserver.HTTPServer(app, ssl_options={
"certfile": os.path.join(os.path.abspath("."), "server.crt"),
"keyfile": os.path.join(os.path.abspath("."), "server_no_passwd.key"),
})
http_server.listen(options.port)
tornado.ioloop.IOLoop.current().start()
if __name__ == '__main__':
threads = []
t = threading.Thread(target=run_tornado_websocket, args=())
threads.append(t)
for t in threads:
t.start()
It works fine on python3.5.But it fails on python3.6 and the lasted tornado.It gets the error:
Exception in thread Thread-1:
Traceback (most recent call last):
File "D:\python3\lib\threading.py", line 916, in _bootstrap_inner
self.run()
File "D:\python3\lib\threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "D:\ssl\ws_server.py", line 49, in run_tornado_websocket
http_server.listen(options.port)
File "D:\python3\lib\site-packages\tornado\tcpserver.py", line 145, in listen
self.add_sockets(sockets)
File "D:\python3\lib\site-packages\tornado\tcpserver.py", line 159, in add_sockets
sock, self._handle_connection)
File "D:\python3\lib\site-packages\tornado\netutil.py", line 219, in add_accept_handler
io_loop = IOLoop.current()
File "D:\python3\lib\site-packages\tornado\ioloop.py", line 282, in current
loop = asyncio.get_event_loop()
File "D:\python3\lib\asyncio\events.py", line 694, in get_event_loop
return get_event_loop_policy().get_event_loop()
File "D:\python3\lib\asyncio\events.py", line 602, in get_event_loop
% threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'Thread-1'.
I think there are some changes in IOLOOP in python3.6.But I don't know how to fix this and really want to know the reason.
Starting in Tornado 5.0, the asyncio event loop is used by default. asyncio has some extra restrictions because starting event loops on threads other than the main thread is an uncommon pattern and is often a mistake. You must tell asyncio that you want to use an event loop in your new thread with asyncio.set_event_loop(asyncio.new_event_loop()), or use asyncio.set_event_loop_policy(tornado.platform.asyncio.AnyThreadEventLoopPolicy()) to disable this restriction.
I am using Python 3.6.2, on Fedora 26 Workstation.
Below is some scrapbook code which demonstrates my issue:
EDIT: added Sam Hartman's suggestion to code.
import asyncio, json
from autobahn.asyncio.websocket import WebSocketClientProtocol, WebSocketClientFactory
class MyClientProtocol(WebSocketClientProtocol):
def onConnect(self, response):
print(response.peer)
def onOpen(self):
print("open")
self.sendMessage(json.dumps({'command': 'subscribe', 'channel': "1010"}).encode("utf8"))
def onMessage(self, payload, isBinary):
print("message")
print(json.loads(payload))
factory1 = WebSocketClientFactory("wss://api2.poloniex.com:443")
factory1.protocol = MyClientProtocol
loop1 = asyncio.get_event_loop()
loop1.run_until_complete(loop1.create_connection(factory1, "api2.poloniex.com", 443, ssl=True))
try:
loop1.run_forever()
except KeyboardInterrupt:
pass
loop1.close()
asyncio.set_event_loop(asyncio.new_event_loop())
factory2 = WebSocketClientFactory("wss://api2.poloniex.com:443")
factory2.protocol = MyClientProtocol
loop2 = asyncio.get_event_loop()
loop2.run_until_complete(loop2.create_connection(factory2, "api2.poloniex.com", 443, ssl=True))
try:
loop2.run_forever()
except KeyboardInterrupt:
pass
loop2.close()
After having closed an initial asyncio event loop, creating another and setting it as the global event loop, attempting to use the new event loop yields the following errors:
Fatal write error on socket transport
protocol: <asyncio.sslproto.SSLProtocol object at 0x7f8a84ed4748>
transport: <_SelectorSocketTransport fd=6>
Traceback (most recent call last):
File "/usr/lib64/python3.6/asyncio/selector_events.py", line 762, in write
n = self._sock.send(data)
OSError: [Errno 9] Bad file descriptor
Fatal error on SSL transport
protocol: <asyncio.sslproto.SSLProtocol object at 0x7f8a84ed4748>
transport: <_SelectorSocketTransport closing fd=6>
Traceback (most recent call last):
File "/usr/lib64/python3.6/asyncio/selector_events.py", line 762, in write
n = self._sock.send(data)
OSError: [Errno 9] Bad file descriptor
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.6/asyncio/sslproto.py", line 648, in _process_write_backlog
self._transport.write(chunk)
File "/usr/lib64/python3.6/asyncio/selector_events.py", line 766, in write
self._fatal_error(exc, 'Fatal write error on socket transport')
File "/usr/lib64/python3.6/asyncio/selector_events.py", line 646, in _fatal_error
self._force_close(exc)
File "/usr/lib64/python3.6/asyncio/selector_events.py", line 658, in _force_close
self._loop.call_soon(self._call_connection_lost, exc)
File "/usr/lib64/python3.6/asyncio/base_events.py", line 574, in call_soon
self._check_closed()
File "/usr/lib64/python3.6/asyncio/base_events.py", line 357, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
Exception in callback _SelectorSocketTransport._read_ready()
handle: <Handle _SelectorSocketTransport._read_ready()>
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/txaio/_common.py", line 63, in call_later
self._buckets[real_time][1].append(call)
KeyError: 412835000
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.6/asyncio/events.py", line 127, in _run
self._callback(*self._args)
File "/usr/lib64/python3.6/asyncio/selector_events.py", line 731, in _read_ready
self._protocol.data_received(data)
File "/usr/lib64/python3.6/asyncio/sslproto.py", line 503, in data_received
ssldata, appdata = self._sslpipe.feed_ssldata(data)
File "/usr/lib64/python3.6/asyncio/sslproto.py", line 204, in feed_ssldata
self._handshake_cb(None)
File "/usr/lib64/python3.6/asyncio/sslproto.py", line 619, in _on_handshake_complete
self._app_protocol.connection_made(self._app_transport)
File "/usr/lib/python3.6/site-packages/autobahn/asyncio/websocket.py", line 97, in connection_made
self._connectionMade()
File "/usr/lib/python3.6/site-packages/autobahn/websocket/protocol.py", line 3340, in _connectionMade
WebSocketProtocol._connectionMade(self)
File "/usr/lib/python3.6/site-packages/autobahn/websocket/protocol.py", line 1055, in _connectionMade
self.onOpenHandshakeTimeout,
File "/usr/lib/python3.6/site-packages/txaio/_common.py", line 72, in call_later
self._notify_bucket, real_time,
File "/usr/lib/python3.6/site-packages/txaio/aio.py", line 382, in call_later
return self._config.loop.call_later(delay, real_call)
File "/usr/lib64/python3.6/asyncio/base_events.py", line 543, in call_later
timer = self.call_at(self.time() + delay, callback, *args)
File "/usr/lib64/python3.6/asyncio/base_events.py", line 553, in call_at
self._check_closed()
File "/usr/lib64/python3.6/asyncio/base_events.py", line 357, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
It seems reasonable that one might need to reopen an event loop after having closed an earlier one. Indeed this question even shows how: Asyncio Event Loop is Closed
The code below should achieve this:
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
so I am clearly doing something wrong. Can somebody see something missing?
I have fairly high confidence that your factory object is maintaining a reference to the old event loop presumably that it gets from asyncio.get_event_loop. Asyncio consumers are bad about getting hidden references to loops.
My recommendation is to reconstruct the web socket factory after closing the loop