When trying to run the asyncio hello world code example given in the docs:
import asyncio
async def hello_world():
print("Hello World!")
loop = asyncio.get_event_loop()
# Blocking call which returns when the hello_world() coroutine is done
loop.run_until_complete(hello_world())
loop.close()
I get the error:
RuntimeError: Event loop is closed
I am using python 3.5.3.
On Windows seems to be a problem with EventLoopPolicy, use this snippet to work around it:
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
asyncio.run(main())
You have already called loop.close() before you ran that sample piece of code, on the global event loop:
>>> import asyncio
>>> asyncio.get_event_loop().close()
>>> asyncio.get_event_loop().is_closed()
True
>>> asyncio.get_event_loop().run_until_complete(asyncio.sleep(1))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/.../lib/python3.6/asyncio/base_events.py", line 443, in run_until_complete
self._check_closed()
File "/.../lib/python3.6/asyncio/base_events.py", line 357, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
You need to create a new loop:
loop = asyncio.new_event_loop()
You can set that as the new global loop with:
asyncio.set_event_loop(asyncio.new_event_loop())
and then just use asyncio.get_event_loop() again.
Alternatively, just restart your Python interpreter, the first time you try to get the global event loop you get a fresh new one, unclosed.
As of Python 3.7, the process of creating, managing, then closing the loop (as well as a few other resources) is handled for you when use asyncio.run(). It should be used instead of loop.run_until_complete(), and there is no need any more to first get or set the loop.
...and just in case:
import platform
if platform.system()=='Windows':
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
If you ever deploy your code in the cloud it might avoid painful debug.
Related
There is no problem using asyncio as follows.
import asyncio
async def main():
await asyncio.sleep(1)
aaa = 1
print (aaa)
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
loop.close()
However, I have to use asyncio within Thread as shown in the simple code below, which results in an error as follows
import asyncio
import threading
async def main():
await asyncio.sleep(1)
aaa = 1
print (aaa)
def test():
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
loop.close()
t=threading.Thread(target=test)
t.start()
Error Message (RuntimeError: There is no current event loop in thread 'Thread-1'.)
Exception in thread Thread-1:
Traceback (most recent call last):
File "D:\Anaconda3\Lib\threading.py", line 932, in _bootstrap_inner
self.run()
File "D:\Anaconda3\Lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "D:/Test/Test4/main.py", line 57, in test
loop = asyncio.get_event_loop()
File "D:\Anaconda3\Lib\asyncio\events.py", line 639, in get_event_loop
raise RuntimeError('There is no current event loop in thread %r.'
RuntimeError: There is no current event loop in thread 'Thread-1'.
How can I use asyncio in Thread?
This is very important to me.
I've been struggling with this problem for two days.
I would appreciate your help.
Unless you need to work with a pre-existing event loop (not the case here) or support Python 3.6, use asyncio.run instead of manually handling the loop:
def test():
asyncio.run(main())
This always ensures that there is a valid event loop, and also handles proper cleanup.
This function always creates a new event loop and closes it at the end. It should be used as a main entry point for asyncio programs, and should ideally only be called once.
Note that when combining asyncio with threads, the event loop should generally be run in the main loop (see e.g. bpo#34697). Instead of spawning a thread for asyncio, prefer to spawn one for the synchronous parts of your program.
# launch synchronous work in child threads
t=threading.Thread(target=my_sync_tasks)
t.start()
# launch asynchronous work in main thread
asyncio.run(main())
t.join()
When using a thread other than the main thread you need to set the event loop
def test():
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
loop.run_until_complete(main())
loop.close()
I am trying to start a bunch (one or more) aioserial instances using an for and asyncio.gather without success.
# -*- coding: utf-8 -*-
import asyncio
import aioserial
from protocol import contactid, ademco
def main():
# Loop Asyncio
loop = asyncio.get_event_loop()
centrals = utils.auth()
# List of corotines to be executed in paralel
lst_coro = []
# Unpack centrals
for central in centrals:
protocol = central['protocol']
id = central['id']
name = central['name']
port = central['port']
logger = log.create_logging_system(name)
# Protocols
if protocol == 'contactid':
central = contactid.process
elif protocol == 'ademco':
central = ademco.process
else:
print(f'Unknown protocol: {central["protocol"]}')
# Serial (port ex: ttyUSB0/2400)
dev = ''.join(['/dev/', *filter(str.isalnum, port.split('/')[0])])
bps = int(port.split('/')[-1])
aioserial_instance = aioserial.AioSerial(port=dev, baudrate=bps)
lst_coro.append(central(aioserial_instance, id, name, logger))
asyncio.gather(*lst_coro, loop=loop)
if __name__ == '__main__':
asyncio.run(main())
I based this on the asyncio documentation example and some answers from stack overflow. But when I try to run it, I just got errors:
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/opt/Serial/serial.py", line 39, in <module>
asyncio.run(main())
File "/usr/lib/python3.7/asyncio/runners.py", line 37, in run
raise ValueError("a coroutine was expected, got {!r}".format(main))
ValueError: a coroutine was expected, got None
I also tried to use a set instead of a list, but nothing really changed. Is there a better way to start a bunch of parallels corotines when you need to use a loop? Thanks for the attention.
Your problem isn't with how you call gather. It's with how you define main. The clue is with the error message
ValueError: a coroutine was expected, got None
and the last line of code in the traceback before the except is raised
asyncio.run(main())
asyncio.run wants an awaitable. You pass it the return value of main, but main doesn't return anything. Rather than adding a return value, though, the fix is to change how you define main.
async def main():
This will turn main from a regular function to a coroutine that can be awaited.
Edit
Once you've done this, you'll notice that gather doesn't actually seem to do anything. You'll need to await it in order for main to wait for everything in lst_coro to complete.
await asyncio.gather(*lst_coro)
Unrelated to your error: you shouldn't need to use loop inside main at all. gather's loop argument was deprecated in 3.8 and will be removed in 3.10. Unless you're using an older version of Python, you can remove it and your call to asyncio.get_event_loop.
I am a new python programmer who is trying to write a 'bot' to trade on betfair for myself. (ambitious!!!!)
My problem that has arisen is this - I have the basics of an asyncio event loop running but I have noticed that if one of the coroutines fails in its process ( for instance an API call fails or a mongodb read) then the asyncio event loop just continues running but ignores the one failed coroutine
my question is how could I either restart that one coroutine automatically or handle an error to stop the complete asyncio loop but at the moment everything runs seemingly oblivious to the fact that something is not right and one portion of it has failed. In my case the loop never returned to the 'rungetcompetitionids' function after a database read was not successful and it never returned to the function again even though it is in a while true loop
The usergui is not yet functional but only there to try asyncio
thanks
Clive
import sys
import datetime
from login import sessiontoken as gst
from mongoenginesetups.setupmongo import global_init as initdatabase
from asyncgetcompetitionids import competition_id_pass as gci
from asyncgetcompetitionids import create_comp_id_table_list as ccid
import asyncio
import PySimpleGUI as sg
sg.change_look_and_feel('DarkAmber')
layout = [
[sg.Text('Password'), sg.InputText(password_char='*', key='password')],
[sg.Text('', key='status')],
[sg.Button('Submit'), sg.Button('Cancel')]
]
window = sg.Window('Betfair', layout)
def initialisethedatabase():
initdatabase('xxxx', 'xxxx', xxxx, 'themongo1', True)
async def runsessiontoken():
nextlogontime = datetime.datetime.now()
while True:
returned_login_time = gst(nextlogontime)
nextlogontime = returned_login_time
await asyncio.sleep(15)
async def rungetcompetitionids(compid_from_compid_table_list):
nextcompidtime = datetime.datetime.now()
while True:
returned_time , returned_list = gci(nextcompidtime, compid_from_compid_table_list)
nextcompidtime = returned_time
compid_from_compid_table_list = returned_list
await asyncio.sleep(10)
async def userinterface():
while True:
event, value = window.read(timeout=1)
if event in (None, 'Cancel'):
sys.exit()
if event != "__TIMEOUT__":
print(f"{event} {value}")
await asyncio.sleep(0.0001)
async def wait_list():
await asyncio.wait([runsessiontoken(),
rungetcompetitionids(compid_from_compid_table_list),
userinterface()
])
initialisethedatabase()
compid_from_compid_table_list = ccid()
print(compid_from_compid_table_list)
nextcompidtime = datetime.datetime.now()
print(nextcompidtime)
loop = asyncio.get_event_loop()
loop.run_until_complete(wait_list())
loop.close()
A simple solution would be to use a wrapper function (or "supervisor") that catches Exception and then just blindly retries the function. More elegant solutions would include printing out the exception and stack trace for diagnostic purposes, and querying the application state to see if it makes sense to try and continue. For instance, if betfair tells you your account is not authorised, then continuing makes no sense. And if it's a general network error then retying immediately might be worthwhile. You might also want to stop retrying if the supervisor notices it has restarted quite a lot in a short space of time.
eg.
import asyncio
import traceback
import functools
from collections import deque
from time import monotonic
MAX_INTERVAL = 30
RETRY_HISTORY = 3
# That is, stop after the 3rd failure in a 30 second moving window
def supervise(func, name=None, retry_history=RETRY_HISTORY, max_interval=MAX_INTERVAL):
"""Simple wrapper function that automatically tries to name tasks"""
if name is None:
if hasattr(func, '__name__'): # raw func
name = func.__name__
elif hasattr(func, 'func'): # partial
name = func.func.__name__
return asyncio.create_task(supervisor(func, retry_history, max_interval), name=name)
async def supervisor(func, retry_history=RETRY_HISTORY, max_interval=MAX_INTERVAL):
"""Takes a noargs function that creates a coroutine, and repeatedly tries
to run it. It stops is if it thinks the coroutine is failing too often or
too fast.
"""
start_times = deque([float('-inf')], maxlen=retry_history)
while True:
start_times.append(monotonic())
try:
return await func()
except Exception:
if min(start_times) > monotonic() - max_interval:
print(
f'Failure in task {asyncio.current_task().get_name()!r}.'
' Is it in a restart loop?'
)
# we tried our best, this coroutine really isn't working.
# We should try to shutdown gracefully by setting a global flag
# that other coroutines should periodically check and stop if they
# see that it is set. However, here we just reraise the exception.
raise
else:
print(func.__name__, 'failed, will retry. Failed because:')
traceback.print_exc()
async def a():
await asyncio.sleep(2)
raise ValueError
async def b(greeting):
for i in range(15):
print(greeting, i)
await asyncio.sleep(0.5)
async def main_async():
tasks = [
supervise(a),
# passing repeated argument to coroutine (or use lambda)
supervise(functools.partial(b, 'hello'))
]
await asyncio.wait(
tasks,
# Only stop when all coroutines have completed
# -- this allows for a graceful shutdown
# Alternatively use FIRST_EXCEPTION to stop immediately
return_when=asyncio.ALL_COMPLETED,
)
return tasks
def main():
# we run outside of the event loop, so we can carry out a post-mortem
# without needing the event loop to be running.
done = asyncio.run(main_async())
for task in done:
if task.cancelled():
print(task, 'was cancelled')
elif task.exception():
print(task, 'failed with:')
# we use a try/except here to reconstruct the traceback for logging purposes
try:
task.result()
except:
# we can use a bare-except as we are not trying to block
# the exception -- just record all that may have happened.
traceback.print_exc()
main()
And this will result in output like:
hello 0
hello 1
hello 2
hello 3
a failed, will retry. Failed because:
Traceback (most recent call last):
File "C:\Users\User\Documents\python\src\main.py", line 30, in supervisor
return await func()
File "C:\Users\User\Documents\python\src\main.py", line 49, in a
raise ValueError
ValueError
hello 4
hello 5
hello 6
hello 7
a failed, will retry. Failed because:
Traceback (most recent call last):
File "C:\Users\User\Documents\python\src\main.py", line 30, in supervisor
return await func()
File "C:\Users\User\Documents\python\src\main.py", line 49, in a
raise ValueError
ValueError
hello 8
hello 9
hello 10
hello 11
Failure in task 'a'. Is it in a restart loop?
hello 12
hello 13
hello 14
exception=ValueError()> failed with:
Traceback (most recent call last):
File "C:\Users\User\Documents\python\src\main.py", line 84, in main
task.result()
File "C:\Users\User\Documents\python\src\main.py", line 30, in supervisor
return await func()
File "C:\Users\User\Documents\python\src\main.py", line 49, in a
raise ValueError
ValueError
I have two processes; a main process and a subprocess. The main process is running an asyncio event loop, and starts the subprocess. I want to start another asyncio event loop in the subprocess. I'm using the aioprocessing module to launch the subprocess.
The subprocess function is:
def subprocess_code():
loop = asyncio.get_event_loop()
#asyncio.coroutine
def f():
for i in range(10):
print(i)
yield from asyncio.sleep(1)
loop.run_until_complete(f())
But I get an error:
loop.run_until_complete(f())
File "/usr/lib/python3.4/asyncio/base_events.py", line 271, in run_until_complete
self.run_forever()
File "/usr/lib/python3.4/asyncio/base_events.py", line 239, in run_forever
raise RuntimeError('Event loop is running.')
RuntimeError: Event loop is running.
Is it possible to start a new, or restart the existing, asyncio event loop in the subprocess? If so, how?
Sorry for disturb!
I found a solution!
policy = asyncio.get_event_loop_policy()
policy.set_event_loop(policy.new_event_loop())
loop = asyncio.get_event_loop()
put this code to start new asycnio event loop inside of subprocess started from process with asyncio event loop
Exploring Python 3.4.0's asyncio module, I am attempting to create a class with asyncio.coroutine methods that are called from an event_loop outside of the class.
My working code is below.
import asyncio
class Foo():
#asyncio.coroutine
def async_sleep(self):
print('about to async sleep')
yield from asyncio.sleep(1)
#asyncio.coroutine
def call_as(self):
print('about to call ass')
yield from self.async_sleep()
def run_loop(self):
loop = asyncio.get_event_loop()
loop.run_until_complete(self.call_as())
print('done with loop')
loop.close()
a = Foo()
a.run_loop()
loop = asyncio.get_event_loop()
loop.run_until_complete(a.call_as())
The call to a.run_loop() provides output as expected:
python3 async_class.py
about to call ass
about to async sleep
done with loop
However as soon as the event_loop attempts to process a.call_as() I get the following Traceback:
Traceback (most recent call last):
File "async_class.py", line 26, in <module>
doop.run_until_complete(asyncio.async(a.call_ass()))
File "/opt/boxen/homebrew/opt/pyenv/versions/3.4.0/lib/python3.4/asyncio/base_events.py", line 203, in run_until_complete
self.run_forever()
File "/opt/boxen/homebrew/opt/pyenv/versions/3.4.0/lib/python3.4/asyncio/base_events.py", line 184, in run_forever
self._run_once()
File "/opt/boxen/homebrew/opt/pyenv/versions/3.4.0/lib/python3.4/asyncio/base_events.py", line 778, in _run_once
event_list = self._selector.select(timeout)
AttributeError: 'NoneType' object has no attribute 'select'
I have attempted wrapping a.call_as() in an asyncio.Task(), asyncio.async() and the failure is the same.
As it turns out, the issue was with the context of the event loop.
asyncio magically creates an event loop for a thread at runtime. This event loop's context is set when .get_event_loop() is called.
In the above example, a.run_loop sets the event loop inside the context of Foo.run_loop.
One kicker of event loops is that there may only be one event loop per thread, and a given event loop can only process events in its context.
With that in mind, note that the loop = asyncio.get_event_loop() just after a.run_loop is asking to assign the thread's event loop to the __main__ context. Unfortunately, the event loop was already set to the context of Foo.run_loop, so a None type is set for the __main__ event loop.
Instead, it is necessary to create a new event loop and then set that event loop's context to __main__, i.e.
new_loop = asyncio.new_event_loop()
asyncio.set_event_loop(new_loop)
Only then will an event loop be properly set in the context of __main__, allowing for the proper execution of our now-modified new_loop.run_until_complete(a.call_as())
It is because you close the eventloop at the end of Foo.run_loop()
From BaseEventLoop.close
Close the event loop. The loop must not be running. Pending callbacks will be lost.
This clears the queues and shuts down the executor, but does not wait for the executor to finish.
This is idempotent and irreversible. No other methods should be called after this one.