There is no problem using asyncio as follows.
import asyncio
async def main():
await asyncio.sleep(1)
aaa = 1
print (aaa)
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
loop.close()
However, I have to use asyncio within Thread as shown in the simple code below, which results in an error as follows
import asyncio
import threading
async def main():
await asyncio.sleep(1)
aaa = 1
print (aaa)
def test():
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
loop.close()
t=threading.Thread(target=test)
t.start()
Error Message (RuntimeError: There is no current event loop in thread 'Thread-1'.)
Exception in thread Thread-1:
Traceback (most recent call last):
File "D:\Anaconda3\Lib\threading.py", line 932, in _bootstrap_inner
self.run()
File "D:\Anaconda3\Lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "D:/Test/Test4/main.py", line 57, in test
loop = asyncio.get_event_loop()
File "D:\Anaconda3\Lib\asyncio\events.py", line 639, in get_event_loop
raise RuntimeError('There is no current event loop in thread %r.'
RuntimeError: There is no current event loop in thread 'Thread-1'.
How can I use asyncio in Thread?
This is very important to me.
I've been struggling with this problem for two days.
I would appreciate your help.
Unless you need to work with a pre-existing event loop (not the case here) or support Python 3.6, use asyncio.run instead of manually handling the loop:
def test():
asyncio.run(main())
This always ensures that there is a valid event loop, and also handles proper cleanup.
This function always creates a new event loop and closes it at the end. It should be used as a main entry point for asyncio programs, and should ideally only be called once.
Note that when combining asyncio with threads, the event loop should generally be run in the main loop (see e.g. bpo#34697). Instead of spawning a thread for asyncio, prefer to spawn one for the synchronous parts of your program.
# launch synchronous work in child threads
t=threading.Thread(target=my_sync_tasks)
t.start()
# launch asynchronous work in main thread
asyncio.run(main())
t.join()
When using a thread other than the main thread you need to set the event loop
def test():
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
loop.run_until_complete(main())
loop.close()
Related
When I run this code in Python 3.7:
import asyncio
sem = asyncio.Semaphore(2)
async def work():
async with sem:
print('working')
await asyncio.sleep(1)
async def main():
await asyncio.gather(work(), work(), work())
asyncio.run(main())
It fails with RuntimeError:
$ python3 demo.py
working
working
Traceback (most recent call last):
File "demo.py", line 13, in <module>
asyncio.run(main())
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/runners.py", line 43, in run
return loop.run_until_complete(main)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
return future.result()
File "demo.py", line 11, in main
await asyncio.gather(work(), work(), work())
File "demo.py", line 6, in work
async with sem:
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/locks.py", line 92, in __aenter__
await self.acquire()
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/locks.py", line 474, in acquire
await fut
RuntimeError: Task <Task pending coro=<work() running at demo.py:6> cb=[gather.<locals>._done_callback() at /opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/tasks.py:664]> got Future <Future pending> attached to a different loop
Python 3.10+: This error message should not occur anymore, see answer from #mmdanziger:
(...) the implementation of Semaphore has been changed and no longer grabs the current loop on init
Python 3.9 and older:
It's because Semaphore constructor sets its _loop attribute – in asyncio/locks.py:
class Semaphore(_ContextManagerMixin):
def __init__(self, value=1, *, loop=None):
if value < 0:
raise ValueError("Semaphore initial value must be >= 0")
self._value = value
self._waiters = collections.deque()
if loop is not None:
self._loop = loop
else:
self._loop = events.get_event_loop()
But asyncio.run() starts a completely new loop – in asyncio/runners.py, it's also metioned in the documentation:
def run(main, *, debug=False):
if events._get_running_loop() is not None:
raise RuntimeError(
"asyncio.run() cannot be called from a running event loop")
if not coroutines.iscoroutine(main):
raise ValueError("a coroutine was expected, got {!r}".format(main))
loop = events.new_event_loop()
...
Semaphore initiated outside of asyncio.run() grabs the asyncio "default" loop and so cannot be used with the event loop created with asyncio.run().
Solution
Initiate Semaphore from code called by asyncio.run(). You will have to pass them to the right place, there are more possibilities how to do that, you can for example use contextvars, but I will just give the simplest example:
import asyncio
async def work(sem):
async with sem:
print('working')
await asyncio.sleep(1)
async def main():
sem = asyncio.Semaphore(2)
await asyncio.gather(work(sem), work(sem), work(sem))
asyncio.run(main())
The same issue (and solution) is probably also with asyncio.Lock, asyncio.Event, and asyncio.Condition.
Update: As of Python 3.10 OP's code will run as written. This is because the implementation of Semaphore has been changed and no longer grabs the current loop on init. See this answer for more discussion.
Python 3.10 implementation from GitHub
class Semaphore(_ContextManagerMixin, mixins._LoopBoundMixin):
"""A Semaphore implementation.
A semaphore manages an internal counter which is decremented by each
acquire() call and incremented by each release() call. The counter
can never go below zero; when acquire() finds that it is zero, it blocks,
waiting until some other thread calls release().
Semaphores also support the context management protocol.
The optional argument gives the initial value for the internal
counter; it defaults to 1. If the value given is less than 0,
ValueError is raised.
"""
def __init__(self, value=1, *, loop=mixins._marker):
super().__init__(loop=loop)
if value < 0:
raise ValueError("Semaphore initial value must be >= 0")
self._value = value
self._waiters = collections.deque()
self._wakeup_scheduled = False
Alternative solution for Python 3.9 and older is to instantiate the Event, Lock, Semaphore, etc. as a first step inside the main() task, where possible.
I validated this with an Event case tested on Python 3.10 (Windows) vs Python 3.9 (Raspberry Pi).
What I want to do is, in the Python console:
Create an asyncio future
Attach a callback to this future
Set the future value (and so get at the same time the callback result)
To do this, I tried that, in the Python console:
from threading import Thread
import asyncio
def start_loop(loop):
asyncio.set_event_loop(loop)
loop.run_forever()
new_loop = asyncio.new_event_loop()
t = Thread(target=start_loop, args=(new_loop,))
t.start()
fut = new_loop.create_future()
fut.add_done_callback(lambda _: print("Hey"))
fut.set_result(True)
I expect "Hey" to be printed, but actually I get nothing.
Additional information:
When I do:
import asyncio
loop = asyncio.get_event_loop()
fut = loop.create_future()
fut.add_done_callback(lambda x: print("Hey"))
fut.set_result(True)
loop.run_forever()
I get the expected result, but I had to call run_forever while the future is already set.
My questions are:
Why my first block of code does not do the job as expected ?
Is there a hypothetical syntax like the following existing ?
import asyncio
loop = asyncio.get_event_loop()
fut = loop.create_future()
# Hypothetical part
loop.run_forever_in_background()
# End of hypothetical part
fut.add_done_callback(lambda x: print("Hey"))
fut.set_result(True)
My high level purpose is to play with futures and asyncio directly in the console, to get a better understanding of it.
Pay attention when playing with asyncio loops, futures etc. in multithreading context.
Almost all asyncio objects are not thread safe, which is
typically not a problem unless there is code that works with them from
outside of a Task or a callback.
...
To schedule a coroutine object from a different OS thread, the
run_coroutine_threadsafe() function should be used. It returns a
concurrent.futures.Future to access the result:
from threading import Thread
import asyncio
def start_loop(loop):
asyncio.set_event_loop(loop)
loop.run_forever()
new_loop = asyncio.new_event_loop()
t = Thread(target=start_loop, args=(new_loop,))
t.start()
async def add_future():
fut = new_loop.create_future()
fut.add_done_callback(lambda _: print("Hey"))
fut.set_result(True)
asyncio.run_coroutine_threadsafe(add_future(), loop=new_loop)
Test:
$ python3 test.py
Hey
Another option is using loop.call_soon_threadsafe(callback, *args, context=None).
https://docs.python.org/3/library/asyncio-dev.html#asyncio-multithreading
When trying to run the asyncio hello world code example given in the docs:
import asyncio
async def hello_world():
print("Hello World!")
loop = asyncio.get_event_loop()
# Blocking call which returns when the hello_world() coroutine is done
loop.run_until_complete(hello_world())
loop.close()
I get the error:
RuntimeError: Event loop is closed
I am using python 3.5.3.
On Windows seems to be a problem with EventLoopPolicy, use this snippet to work around it:
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
asyncio.run(main())
You have already called loop.close() before you ran that sample piece of code, on the global event loop:
>>> import asyncio
>>> asyncio.get_event_loop().close()
>>> asyncio.get_event_loop().is_closed()
True
>>> asyncio.get_event_loop().run_until_complete(asyncio.sleep(1))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/.../lib/python3.6/asyncio/base_events.py", line 443, in run_until_complete
self._check_closed()
File "/.../lib/python3.6/asyncio/base_events.py", line 357, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
You need to create a new loop:
loop = asyncio.new_event_loop()
You can set that as the new global loop with:
asyncio.set_event_loop(asyncio.new_event_loop())
and then just use asyncio.get_event_loop() again.
Alternatively, just restart your Python interpreter, the first time you try to get the global event loop you get a fresh new one, unclosed.
As of Python 3.7, the process of creating, managing, then closing the loop (as well as a few other resources) is handled for you when use asyncio.run(). It should be used instead of loop.run_until_complete(), and there is no need any more to first get or set the loop.
...and just in case:
import platform
if platform.system()=='Windows':
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
If you ever deploy your code in the cloud it might avoid painful debug.
I have two processes; a main process and a subprocess. The main process is running an asyncio event loop, and starts the subprocess. I want to start another asyncio event loop in the subprocess. I'm using the aioprocessing module to launch the subprocess.
The subprocess function is:
def subprocess_code():
loop = asyncio.get_event_loop()
#asyncio.coroutine
def f():
for i in range(10):
print(i)
yield from asyncio.sleep(1)
loop.run_until_complete(f())
But I get an error:
loop.run_until_complete(f())
File "/usr/lib/python3.4/asyncio/base_events.py", line 271, in run_until_complete
self.run_forever()
File "/usr/lib/python3.4/asyncio/base_events.py", line 239, in run_forever
raise RuntimeError('Event loop is running.')
RuntimeError: Event loop is running.
Is it possible to start a new, or restart the existing, asyncio event loop in the subprocess? If so, how?
Sorry for disturb!
I found a solution!
policy = asyncio.get_event_loop_policy()
policy.set_event_loop(policy.new_event_loop())
loop = asyncio.get_event_loop()
put this code to start new asycnio event loop inside of subprocess started from process with asyncio event loop
Exploring Python 3.4.0's asyncio module, I am attempting to create a class with asyncio.coroutine methods that are called from an event_loop outside of the class.
My working code is below.
import asyncio
class Foo():
#asyncio.coroutine
def async_sleep(self):
print('about to async sleep')
yield from asyncio.sleep(1)
#asyncio.coroutine
def call_as(self):
print('about to call ass')
yield from self.async_sleep()
def run_loop(self):
loop = asyncio.get_event_loop()
loop.run_until_complete(self.call_as())
print('done with loop')
loop.close()
a = Foo()
a.run_loop()
loop = asyncio.get_event_loop()
loop.run_until_complete(a.call_as())
The call to a.run_loop() provides output as expected:
python3 async_class.py
about to call ass
about to async sleep
done with loop
However as soon as the event_loop attempts to process a.call_as() I get the following Traceback:
Traceback (most recent call last):
File "async_class.py", line 26, in <module>
doop.run_until_complete(asyncio.async(a.call_ass()))
File "/opt/boxen/homebrew/opt/pyenv/versions/3.4.0/lib/python3.4/asyncio/base_events.py", line 203, in run_until_complete
self.run_forever()
File "/opt/boxen/homebrew/opt/pyenv/versions/3.4.0/lib/python3.4/asyncio/base_events.py", line 184, in run_forever
self._run_once()
File "/opt/boxen/homebrew/opt/pyenv/versions/3.4.0/lib/python3.4/asyncio/base_events.py", line 778, in _run_once
event_list = self._selector.select(timeout)
AttributeError: 'NoneType' object has no attribute 'select'
I have attempted wrapping a.call_as() in an asyncio.Task(), asyncio.async() and the failure is the same.
As it turns out, the issue was with the context of the event loop.
asyncio magically creates an event loop for a thread at runtime. This event loop's context is set when .get_event_loop() is called.
In the above example, a.run_loop sets the event loop inside the context of Foo.run_loop.
One kicker of event loops is that there may only be one event loop per thread, and a given event loop can only process events in its context.
With that in mind, note that the loop = asyncio.get_event_loop() just after a.run_loop is asking to assign the thread's event loop to the __main__ context. Unfortunately, the event loop was already set to the context of Foo.run_loop, so a None type is set for the __main__ event loop.
Instead, it is necessary to create a new event loop and then set that event loop's context to __main__, i.e.
new_loop = asyncio.new_event_loop()
asyncio.set_event_loop(new_loop)
Only then will an event loop be properly set in the context of __main__, allowing for the proper execution of our now-modified new_loop.run_until_complete(a.call_as())
It is because you close the eventloop at the end of Foo.run_loop()
From BaseEventLoop.close
Close the event loop. The loop must not be running. Pending callbacks will be lost.
This clears the queues and shuts down the executor, but does not wait for the executor to finish.
This is idempotent and irreversible. No other methods should be called after this one.