Here's a simplified scenario for the problem I'm having.
import asyncio
async def c():
print("yes")
def b():
asyncio.run(c())
async def a():
b()
asyncio.run(a())
I'm expecting the program to print "yes". However, I get this instead:
Traceback (most recent call last):
File [redacted], line 12, in <module>
asyncio.run(a())
File "/usr/lib64/python3.7/asyncio/runners.py", line 43, in run
return loop.run_until_complete(main)
File "/usr/lib64/python3.7/asyncio/base_events.py", line 579, in run_until_complete
return future.result()
File [redacted], line 10, in a
b()
File [redacted], line 7, in b
asyncio.run(c())
File "/usr/lib64/python3.7/asyncio/runners.py", line 34, in run
"asyncio.run() cannot be called from a running event loop")
RuntimeError: asyncio.run() cannot be called from a running event loop
sys:1: RuntimeWarning: coroutine 'c' was never awaited
What would you think would be the solution for this problem?
(Also, can this be done using purely asyncio?)
Don't call asyncio.run() twice in the same program. Instead, once the event loop is running, create a task and await it. A sync function can create a new task but can't await it; it can return it as an awaitable. The async calling function can then perform the await.
import asyncio
async def c():
print("yes")
def b():
return asyncio.create_task(c())
async def a():
task = b()
await task
asyncio.run(a())
>>>yes
Related
I have the following function.
async def f(url):
async with httpx.AsyncClient() as client:
async with client.stream('GET', url) as resp:
async for c in resp.aiter_bytes():
yield c
And I'm writing a test code to test if the function yields.
#pytest.mark.asyncio
async def test():
c = None
async for c in f(URL):
c = chunk
break # there will be no error if removing this line. However, I don't need to loop all the items
assert c
However, the code runs but with the following warnings/errors. What's the error?
lib\site-packages\httpx_client.py:2012: UserWarning: Unclosed <httpx.AsyncClient object at 0x000001AC1E8CF3A0>. See https://www.python-httpx.org/async/#opening-and-closing-clients for details.
Is the "Unclosed" warning caused by break?
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x000001AC1D884DC0>
Traceback (most recent call last):
File "C:\anaconda3\envs\s\lib\asyncio\proactor_events.py", line 116, in __del__
self.close()
File "C:\anaconda3\envs\s\lib\asyncio\proactor_events.py", line 108, in close
self._loop.call_soon(self._call_connection_lost, None)
File "C:\anaconda3\envs\s\lib\asyncio\base_events.py", line 719, in call_soon
self._check_closed()
File "C:\anaconda3\envs\s\lib\asyncio\base_events.py", line 508, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
The error disappears if removing the break statement.
In the answer of Async generator is not an iterator?, the following code will get rid of the error. However, it needs to call a -private- "special" function __anext()__().
#pytest.mark.asyncio
async def test():
it = f(URL):
c = it.__anext__()
assert c
I am trying to make some API calls to firebase using the google.cloud.firestore.AsyncClient module. Fire2 is just a wrapper for this object.
The traceback suggests its a problem with asyncio, but if I make a dummy await it actually runs just fine. What is the problem and why does it occur?
Sample code:
import Fire2
class Test:
def __init__(self):
doc = asyncio.run(self.wait())
async def wait(self):
doc = await Fire2("test1").get(g1) # gives the error
# doc = await asyncio.sleep(1) # runs without error
return doc
def test(self):
x = Test2(p1)
class Test2:
def __init__(self, p):
doc = asyncio.run(self.run(p))
print(doc.to_dict())
async def run(self, p):
doc = await Fire2('test2').get(p)
return doc
p1 = 'foo'
g1 = 'bar'
h = Test()
h.test()
Traceback:
Traceback (most recent call last):
File "<project path>\scratch.py", line 137, in <module>
h.test()
File "<project path>\scratch.py", line 123, in test
x = Test2(p1)
File "<project path>\scratch.py", line 127, in __init__
doc = asyncio.run(self.run(p))
File "<user AppData>\Local\Programs\Python\Python39\lib\asyncio\runners.py", line 44, in run
return loop.run_until_complete(main)
File "<user AppData>\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 642, in run_until_complete
return future.result()
File "<project path>\scratch.py", line 131, in run
doc = await Fire2('test2').get(p)
File "<project path>\<Fire2 file>", line 422, in get
res = await self._collection.document(doc_id).get()
File "<project path>\venv\lib\site-packages\google\cloud\firestore_v1\async_document.py", line 364, in get
response_iter = await self._client._firestore_api.batch_get_documents(
File "<project path>\venv\lib\site-packages\google\api_core\grpc_helpers_async.py", line 171, in error_remapped_callable
call = callable_(*args, **kwargs)
File "<project path>\venv\lib\site-packages\grpc\aio\_channel.py", line 165, in __call__
call = UnaryStreamCall(request, deadline, metadata, credentials,
File "<project path>\venv\lib\site-packages\grpc\aio\_call.py", line 553, in __init__
self._send_unary_request_task = loop.create_task(
File "<user AppData>\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 431, in create_task
self._check_closed()
File "<user AppData>\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 510, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
sys:1: RuntimeWarning: coroutine 'UnaryStreamCall._send_unary_request' was never awaited
Process finished with exit code 1
You are making two separate calls to asyncio.run. We can see from the stack trace that the error is coming from inside the grpc library. I suspect that the same grpc connection is being used both times, and that during its initialization it integrates itself with the async event loop in order to maintain the connection in the background between calls.
The problem is that every time you call asyncio.run, you will create a new event loop, and every time it returns that event loop will be closed. So by the time you call this function again, and the grpc layer tries to do its work, it realizes that the original event loop has been closed and it returns this error.
To fix this, You can have a single call to asyncio.run, typically calling a "main" function, or similar. This would then make calls to async functions which perform the actual work.
If the problem persists then there might be some issues in the “Fire2”class.
For more reference you can check this link which outlines high-level asyncio APIs to work with coroutines and Tasks with the help of various examples.
Use pytest with using fixture with scope="session" in conftest.py
#pytest.fixture(scope="session")
def event_loop():
loop = get_event_loop()
yield loop
loop.close()
I'm new to asyncio. Tried to create an async decorator with that for blocking I/O code.
def singletonAsyncMaker(func):
async def inner(obj, ):
loop = asyncio.get_event_loop()
tasks = (loop.run_in_executor(None, func, i) for i in obj)
return await asyncio.gather(*tasks)
def main(obj):
return asyncio.run(inner(obj))
return main
#singletonAsyncMaker
def sleeper(obj):
sleep(2)
return obj
x = sleeper([8, 5, 6, 6, 4645, 63, 4, 6, 1, 64, 614, 24, 65, ])
print(x)
This worked well normally and also within normal non-async functions, but when called through a route function of a Sanic server it throws this error.
Traceback (most recent call last):
File "/app.py", line 99, in extractImageResponse
result = perform_textract_bytestream(images)
File "async_experiment.py", line 51, in main
return asyncio.run(inner(obj))
File "env/lib/python3.8/asyncio/runners.py", line 33, in run
raise RuntimeError(
RuntimeError: asyncio.run() cannot be called from a running event loop
#app.route("/v0/xxxxxxxxxxxxxxx", methods=['POST'])
def function_name(request):
-----------------
x = sleeper(list)
-----------------
return jsonify(json_op)
Tried adding async to sanic function_name definition and await before sleeper() call to no luck.
I know that sanic works with asyncio, does that contibute to this?
Any fix or work around?
No need to call asyncio.run inside Sanic.
#app.route("/v0/xxxxxxxxxxxxxxx", methods=['POST'])
async def function_name(request):
await call_to_some_coroutine()
I wrote a test program to try out using create_task() that needs to wait until the created task completes.
I tried using loop.run_until_complete() to wait for task completion, but it results in an error with a traceback.
/Users/jason/.virtualenvs/xxx/bin/python3.5 /Users/jason/asyncio/examples/hello_coroutine.py
Traceback (most recent call last):
Test
File "/Users/jason/asyncio/examples/hello_coroutine.py", line 42, in <module>
Hello World, is a task
loop.run_until_complete(test.greet_every_two_seconds())
File "/Users/jason/asyncio/asyncio/base_events.py", line 373, in run_until_complete
return future.result()
File "/Users/jason/asyncio/asyncio/futures.py", line 274, in result
raise self._exception
File "/Users/jason/asyncio/asyncio/tasks.py", line 240, in _step
result = coro.send(None)
File "/Users/jason/asyncio/examples/hello_coroutine.py", line 33, in greet_every_two_seconds
self.a()
File "/Users/jason/asyncio/examples/hello_coroutine.py", line 26, in a
t = loop.run_until_complete(self.greet_every_one_seconds(self.db_presell))
File "/Users/jason/asyncio/asyncio/base_events.py", line 361, in run_until_complete
self.run_forever()
File "/Users/jason/asyncio/asyncio/base_events.py", line 326, in run_forever
raise RuntimeError('Event loop is running.')
RuntimeError: Event loop is running.
The test code is as follows. The function a() must not be a coroutine,
How can I wait for the task until complete?
import asyncio
class Test(object):
def __init__(self):
print(self.__class__.__name__)
pass
#asyncio.coroutine
def greet_every_one_seconds(self, value):
print('Hello World, one second.')
fut = asyncio.sleep(1,result=value)
a = yield from fut
print(a)
def a(self):
loop = asyncio.get_event_loop()
task=loop.run_until_complete(self.greet_every_one_seconds(4))
#How can i wait for the task until complete?
#asyncio.coroutine
def greet_every_two_seconds(self):
while True:
self.a()
print('Hello World, two seconds.')
yield from asyncio.sleep(2)
if __name__ == '__main__':
test = Test()
loop = asyncio.get_event_loop()
try:
loop.run_until_complete(test.greet_every_two_seconds())
finally:
loop.close()
How can i wait for the task until complete?
loop.run_until_complete(task) in an ordinary function. Or await task in a coroutine.
Here's a complete code example that demonstrates both cases:
#!/usr/bin/env python3
import asyncio
async def some_coroutine(loop):
task = loop.create_task(asyncio.sleep(1)) # just some task
await task # wait for it (inside a coroutine)
loop = asyncio.get_event_loop()
task = loop.create_task(asyncio.sleep(1)) # just some task
loop.run_until_complete(task) # wait for it (outside of a coroutine)
loop.run_until_complete(some_coroutine(loop))
I'm trying to experiment with the new await syntax for coroutines in Python 3.5.
I had a simple example like this:
#! /usr/bin/env python
import asyncio
#asyncio.coroutine
def adder(*args, delay):
while True:
yield from asyncio.sleep(delay)
print(sum(args))
def main():
asyncio.Task(adder(1, 2, 3, delay=5))
asyncio.Task(adder(10, 20, delay=3))
loop = asyncio.get_event_loop()
loop.run_forever()
if __name__ == "__main__":
main()
I changed the yield from line to use the await keyword:
await asyncio.sleep(delay)
And I get SyntaxError:
File "./test.py", line 8
await asyncio.sleep(delay)
^
SyntaxError: invalid syntax
So I try await (asyncio.sleep(delay)) just to see what happens:
Task exception was never retrieved
future: <Task finished coro=<adder() done, defined at c:\python35\Lib\asyncio\coroutines.py:198> exception=NameError("name 'await' is not defined",)>
Traceback (most recent call last):
File "c:\python35\Lib\asyncio\tasks.py", line 239, in _step
result = coro.send(value)
File "c:\python35\Lib\asyncio\coroutines.py", line 200, in coro
res = func(*args, **kw)
File "./test.py", line 8, in adder
await (asyncio.sleep(delay))
NameError: name 'await' is not defined
Task exception was never retrieved
future: <Task finished coro=<adder() done, defined at c:\python35\Lib\asyncio\coroutines.py:198> exception=NameError("name 'await' is not defined",)>
Traceback (most recent call last):
File "c:\python35\Lib\asyncio\tasks.py", line 239, in _step
result = coro.send(value)
File "c:\python35\Lib\asyncio\coroutines.py", line 200, in coro
res = func(*args, **kw)
File "./test.py", line 8, in adder
await (asyncio.sleep(delay))
NameError: name 'await' is not defined
Am I using the keyword wrong? Why is await not defined? I got my await syntax from this post.
Just to cover all my bases:
$ /usr/bin/env python --version
Python 3.5.0
Edit:
I suppose adding the parens to the await line is trying to call a function await() - so that's why that doesn't work and gives me a NameError. But why doesn't the keyword get recognized in either example?
You must declare your function as async to use await. Replacing yield from is not sufficient.
#! /usr/bin/env python
import asyncio
#asyncio.coroutine
async def adder(*args, delay):
while True:
await asyncio.sleep(delay)
print(sum(args))
def main():
asyncio.Task(adder(1, 2, 3, delay=5))
asyncio.Task(adder(10, 20, delay=3))
loop = asyncio.get_event_loop()
loop.run_forever()
if __name__ == "__main__":
main()
You need to declare your function as async to make this work. The await keyword will only be enabled if you include an a function declared with async def in your file – see PEP 492 for details.
Per PEP 492, the await keyword is only recognized in functions defined with the new async def syntax, which removes the need for a __future__ import.