I come from the land of Twisted/Klein. I come in peace and to ask for Tornado help. I'm investigating Tornado and how its take on async differs from Twisted. Twisted has something similar to gen.coroutine which is defer.inlineCallbacks and I'm able to write async code like this:
kleinsample.py
#app.route('/endpoint/<int:n>')
#defer.inlineCallbacks
def myRoute(request, n):
jsonlist = []
for i in range(n):
yield jsonlist.append({'id': i})
return json.dumps(jsonlist)
curl cmd:
curl localhost:9000/json/2000
This endpoint will create a JSON string with n number of elements. n can be small or very big. I'm able to break it up in Twisted such that the event loop won't block using yield. Now here's how I tried to convert this into Tornado:
tornadosample.py
async def get(self, n):
jsonlist = []
for i in range(n):
await gen.Task(jsonlist.append, {'id': i}) # exception here
self.write(json.dumps(jsonlist))
The traceback:
TypeError: append() takes no keyword arguments
I'm confused about what I'm supposed to do to properly iterate each element in the loop so that the event loop doesn't get blocked. Does anyone know the "Tornado" way of doing this?
You cannot and must not await append, since it isn't a coroutine and doesn't return a Future. If you want to occasionally yield to allow other coroutines to proceed using Tornado's event loop, await gen.moment.
from tornado import gen
async def get(self, n):
jsonlist = []
for i in range(n):
jsonlist.append({'id': i})
if not i % 1000: # Yield control for a moment every 1k ops
await gen.moment
return json.dumps(jsonlist)
That said, unless this function is extremely CPU-intensive and requires hundreds of milliseconds or more to complete, you're probably better off just doing all your computation at once instead of taking multiple trips through the event loop before your function returns.
list.append() returns None, so it's a little misleading that your Klein sample looks like it's yielding some object. This is equivalent to jsonlist.append(...); yield as two separate statements. The tornado equivalent would be to do await gen.moment in place of the bare yield.
Also note that in Tornado, handlers produce their responses by calling self.write(), not by returning values, so the return statement should be self.write(json.dumps(jsonlist)).
Let's have a look at gen.Task docs:
Adapts a callback-based asynchronous function for use in coroutines.
Takes a function (and optional additional arguments) and runs it with those arguments plus a callback keyword argument. The argument passed to the callback is returned as the result of the yield expression.
Since append doesn't accept a keyword argument it doesn't know what to do with that callback kwarg and spits that exception.
What you could do is wrap append with your own function that does accept a callback kwarg or the approach showed in this answer.
Related
I am reading the python asyncio library. The gather function confused me. Usually, we use it in the following way.
async def func(n):
await asyncio.sleep(n)
return n
result = await asyncio.gather(func(1), func(2)) # should be [1, 2]
It looks so naturally. There are two coroutines and result will be the list of their returns. However, I am confused by the implementation of asyncio.gather. See code. It returns a _GatheringFuture.
outer = _GatheringFuture(children, loop=loop)
return outer
The _GetheringFuture class is not like Task. It does not contain logic to call Future.set_result() or Future.set_execption(), so when we write await asyncio.gather(...), who sets the result/exception?
Ah. I see where set_result is called. It is inside def _done_callback(fut). So each coroutine in the input will execute this callback. The last finished one will set result of the outer future, i.e., _GatheringFuture.
I have a Python script that generates [str, float] tuples which are then indexed into ElasticSearch using a custom function which eventually calls helper.streaming_bulk().
This is how the generator is implemented:
doc_ids: List[str] = [...]
docs = ((doc_id, get_value(doc_id) for doc_id in doc_ids)
get_value() calls a remote service that computes a float value per document id.
Next, these tuples are passed on to update_page_quality_bulk():
for success, item in update_page_quality_bulk(
islice(doc_qualities, size)
):
total_success += success
if not success:
logging.error(item)
Internally, update_page_quality_bulk() creates the ElasticSearch requests.
One of the advantages of using a generator here is that the first size elements can be fed into update_page_quality_bulk() through islice().
In order to make the entire process faster, I would like to parallelize the get_value() calls. As mentioned, these are remote calls so the local compute cost in negligible, but the duration is significant.
The order of the tuples does not matter, neither which elements are passed into update_page_quality_bulk(). On a high level, I would like to make the get_value() calls (up to x in parallel) for any n tuples and pass on whichever ones are finished first.
My naive attempt was to define get_value() as asynchronous:
async def get_value():
...
and await the call in the generator:
docs = ((doc_id, await get_value(doc_id) for doc_id in doc_ids)
However, this raises an error in the subsequent islice() call:
TypeError: 'async_generator' object is not iterable
Removing the islice call and passing the unmodified docs generator to update_page_quality_bulk() causes the same error to be raised when looping over the tuples to convert them into ElasticSearch requests.
I am aware that the ElasticSearch client provides asynchronous helpers, but they don't seem applicable here because I need to generate the actions first.
According to this answer, it seems like I have to change the implementation to using a queue.
This answer implies that it cannot be done without using multiprocessing due to Python GIL, but that answer is not marked as correct and is quite old too.
Generally, I am looking for a way to change the current logic as little as possible while parallelizing the get_value() calls.
So, you want to pass an "synchronous looking" generator to a call that expects a normal lazy generator such as islice, and keep getting the results for this in parallel.
It sounds like a work for asyncio.as_completed: you use your plain generator to create tasks - these are run in parallel by the asyncio machinery, and the results are made available as the tasks are completed (d'oh!).
However since update_page_quality_bulk is not asynco aware, it will never yield the control to the asyncio loop, so that it can complete the tasks which got their results. This would likely block.
Calling update_page_quality_bulk in another thread probably won't work as well. I did not try it here, but I'd say you can't just iterate over doc in a different thread than the one it (and its tasks) where created.
So, first things first - the "generator expression" syntax does not work when you want some terms of the generator to be calculated asynchronously, as you found out - we refactor that so that the tuples are created in an coroutine-function - and we wrap all calls for those in tasks (some of the asyncio functions do the wrapping in a task automatically)
Then we can us the asyncio machinery to schedule all the calls and call update_page_quality_bulk as these results arrive. The problem is that as_completed, as stated above, can't be passed directly to a non-async function: the asyncio loop would never get control back. Instead, we keep picking the results of tasks in the main thread, and call the sync function in another thread - using a Queue to pass the fetched results. And finally, so that the results can be consumed as made available inside update_page_quality_bulk, we create a small wrapper class to the threading.Queue, so that it can be consumed as in iterator - this is transparent for the code consuming the iterator.
# example code: untested
async def get_doc_values(doc_id):
loop = asyncio.get_running_loop()
# Run_in_executor runs the synchronous function in parallel in a thread-pool
# check the docs - you might want to pass a custom executor with more than
# the default number of workers, instead of None:
return doc_id, await asyncio.run_in_executor(None, get_value, doc_id)
def update_es(iterator):
# this function runs in a separate thread -
for success, item in update_page_quality_bulk(iterator):
total_success += success
if not success:
logging.error(item)
sentinel = Ellipsis # ... : python ellipsis - a nice sentinel that also worker for multiprocessing
class Iterator:
"""This allows the queue, fed in the main thread by the tasks as they are as they are completed
to behave like an ordinary iterator, which can be consumed by "update_page_quality_bulk" in another thread
"""
def __init__(self, source_queue):
self.source = source_queue
def __next__(self):
value= self.source.get()
if value is sentinel:
raise StopIteration()
return value
queue = threading.Queue()
iterator = Iterator(queue)
es_worker = threading.Thread(target=update_es, args=(iterator,))
es_worker.start()
for doc_value_task in asyncio.as_completed(get_doc_values(doc_id) for doc_id in doc_ids):
doc_value = await doc_value_task
queue.put(doc_value)
es_worker.join()
Environment: cooperative RTOS in C and micropython virtual machine is one of the tasks.
To make the VM not block the other RTOS tasks, I insert RTOS_sleep() in vm.c:DISPATCH() so that after every bytecode is executed, the VM relinquishes control to the next RTOS task.
I created a uPy interface to asynchronously obtain data from a physical data bus - could be CAN, SPI, ethernet - using producer-consumer design pattern.
Usage in uPy:
can_q = CANbus.queue()
message = can_q.get()
The implementation in C is such that can_q.get() does NOT block the RTOS: it polls a C-queue and if message is not received, it calls RTOS_sleep() to give another task the chance to fill the queue. Things are synchronized because the C-queue is only updated by another RTOS task and RTOS tasks only switch when RTOS_sleep() is called i.e. cooperative
The C-implementation is basically:
// gives chance for c-queue to be filled by other RTOS task
while(c_queue_empty() == true) RTOS_sleep();
return c_queue_get_message();
Although the Python statement can_q.get() does not block the RTOS, it does block the uPy script.
I'd like to rewrite it so I can use it with async def i.e. coroutine and have it not block the uPy script.
Not sure of the syntax but something like this:
can_q = CANbus.queue()
message = await can_q.get()
QUESTION
How do I write a C-function so I can await on it?
I would prefer a CPython and micropython answer but I would accept a CPython-only answer.
Note: this answer covers CPython and the asyncio framework. The concepts, however, should apply to other Python implementations as well as other async frameworks.
How do I write a C-function so I can await on it?
The simplest way to write a C function whose result can be awaited is by having it return an already made awaitable object, such as an asyncio.Future. Before returning the Future, the code must arrange for the future's result to be set by some asynchronous mechanism. All of these coroutine-based approaches assume that your program is running under some event loop that knows how to schedule the coroutines.
But returning a future isn't always enough - maybe we'd like to define an object with an arbitrary number of suspension points. Returning a future suspends only once (if the returned future is not complete), resumes once the future is completed, and that's it. An awaitable object equivalent to an async def that contains more than one await cannot be implemented by returning a future, it has to implement a protocol that coroutines normally implement. This is somewhat like an iterator implementing a custom __next__ and be used instead of a generator.
Defining a custom awaitable
To define our own awaitable type, we can turn to PEP 492, which specifies exactly which objects can be passed to await. Other than Python functions defined with async def, user-defined types can make objects awaitable by defining the __await__ special method, which Python/C maps to the tp_as_async.am_await part of the PyTypeObject struct.
What this means is that in Python/C, you must do the following:
specify a non-NULL value for the tp_as_async field of your extension type.
have its am_await member point to a C function that accepts an instance of your type and returns an instance of another extension type that implements the iterator protocol, i.e. defines tp_iter (trivially defined as PyIter_Self) and tp_iternext.
the iterator's tp_iternext must advance the coroutine's state machine. Each non-exceptional return from tp_iternext corresponds to a suspension, and the final StopIteration exception signifies the final return from the coroutine. The return value is stored in the value property of StopIteration.
For the coroutine to be useful, it must also be able to communicate with the event loop that drives it, so that it can specify when it is to be resumed after it has suspended. Most of coroutines defined by asyncio expect to be running under the asyncio event loop, and internally use asyncio.get_event_loop() (and/or accept an explicit loop argument) to obtain its services.
Example coroutine
To illustrate what the Python/C code needs to implement, let's consider simple coroutine expressed as a Python async def, such as this equivalent of asyncio.sleep():
async def my_sleep(n):
loop = asyncio.get_event_loop()
future = loop.create_future()
loop.call_later(n, future.set_result, None)
await future
# we get back here after the timeout has elapsed, and
# immediately return
my_sleep creates a Future, arranges for it to complete (its result to become set) in n seconds, and suspends itself until the future completes. The last part uses await, where await x means "allow x to decide whether we will now suspend or keep executing". An incomplete future always decides to suspend, and the asyncio Task coroutine driver special-cases yielded futures to suspend them indefinitely and connects their completion to resuming the task. Suspension mechanisms of other event loops (curio etc) can differ in details, but the underlying idea is the same: await is an optional suspension of execution.
__await__() that returns a generator
To translate this to C, we have to get rid of the magic async def function definition, as well as of the await suspension point. Removing the async def is fairly simple: the equivalent ordinary function simply needs to return an object that implements __await__:
def my_sleep(n):
return _MySleep(n)
class _MySleep:
def __init__(self, n):
self.n = n
def __await__(self):
return _MySleepIter(self.n)
The __await__ method of the _MySleep object returned by my_sleep() will be automatically called by the await operator to convert an awaitable object (anything passed to await) to an iterator. This iterator will be used to ask the awaited object whether it chooses to suspend or to provide a value. This is much like how the for o in x statement calls x.__iter__() to convert the iterable x to a concrete iterator.
When the returned iterator chooses to suspend, it simply needs to produce a value. The meaning of the value, if any, will be interpreted by the coroutine driver, typically part of an event loop. When the iterator chooses to stop executing and return from await, it needs to stop iterating. Using a generator as a convenience iterator implementation, _MySleepIter would look like this:
def _MySleepIter(n):
loop = asyncio.get_event_loop()
future = loop.create_future()
loop.call_later(n, future.set_result, None)
# yield from future.__await__()
for x in future.__await__():
yield x
As await x maps to yield from x.__await__(), our generator must exhaust the iterator returned by future.__await__(). The iterator returned by Future.__await__ will yield if the future is incomplete, and return the future's result (which we here ignore, but yield from actually provides) otherwise.
__await__() that returns a custom iterator
The final obstacle for a C implementation of my_sleep in C is the use of generator for _MySleepIter. Fortunately, any generator can be translated to a stateful iterator whose __next__ executes the piece of code up to the next await or return. __next__ implements a state machine version of the generator code, where yield is expressed by returning a value, and return by raising StopIteration. For example:
class _MySleepIter:
def __init__(self, n):
self.n = n
self.state = 0
def __iter__(self): # an iterator has to define __iter__
return self
def __next__(self):
if self.state == 0:
loop = asyncio.get_event_loop()
self.future = loop.create_future()
loop.call_later(self.n, self.future.set_result, None)
self.state = 1
if self.state == 1:
if not self.future.done():
return next(iter(self.future))
self.state = 2
if self.state == 2:
raise StopIteration
raise AssertionError("invalid state")
Translation to C
The above is quite some typing, but it works, and only uses constructs that can be defined with native Python/C functions.
Actually translating the two classes to C quite straightforward, but beyond the scope of this answer.
I am working on a Python3 tornado web server with asynchronous coroutines for GET requests, using the #gen.coroutine decorator. I want to use this function from a library:
#gen.coroutine
def foo(x):
yield do_something(x)
which is simple enough:
#gen.coroutine
def get(self):
x = self.some_parameter
yield response(foo(x))
Now assume there are multiple functions foo1, foo2, etc. of the same type. I want to do something like ...foo3(foo2(foo1(x).result()).result())... and yield that instead of just response(foo(x)) in the get method.
I thought this would be easy with reduce and the result method. However, because of how tornado works, I cannot force the foos to return something with the result method. This means that yield reduce(...) gives an error: "DummyFuture does not support blocking for results". From other answers on SO and elsewhere, I know I will have to use IOLoop or something, which I didn't really understand, and...
...my question is, how can I avoid evaluating all the foos and yield that unevaluated chunk from the get method?
Edit: This is not a duplicate of this question because I want to: 1. nest a lot of functions and 2. try not to evaluate immediately.
In Tornado, you must yield a Future inside a coroutine in order to get a result. Review Tornado's coroutine guide.
You could write a reducer that is a coroutine. It runs each coroutine to get a Future, calls yield with the Future to get a result, then runs the next coroutine on that result:
from tornado.ioloop import IOLoop
from tornado import gen
#gen.coroutine
def f(x):
# Just to prove we're really a coroutine.
yield gen.sleep(1)
return x * 2
#gen.coroutine
def g(x):
return x + 1
#gen.coroutine
def h():
return 10
#gen.coroutine
def coreduce(*funcs):
# Start by calling last function in list.
result = yield funcs[-1]()
# Call remaining functions.
for func in reversed(funcs[:-1]):
result = yield func(result)
return result
# Wrap in lambda to satisfy your requirement, to
# NOT evaluate immediately.
latent_result = lambda: coreduce(f, g, h)
final_result = IOLoop.current().run_sync(latent_result)
print(final_result)
I am working on tornado and motor in python 3.4.3.
I got three files. Lets name it like main.py, model.py, core.py
I have three functions, one in each...
main.py
def getLoggedIn(request_handler):
# request_handler = tornado.web.RequestHandler()
db = request_handler.settings["db"]
uid = request_handler.get_secure_cookie("uid")
result = model.Session.get(db, uid=uid)
return result.get("_id", None) if result else None
model.py
#classmethod
def get(cls, db, user_id=None, **kwargs):
session = core.Session(db)
return session.get(user_id, **kwargs)
core.py
#gen.coroutine
def get(self, user_id, **kwargs):
params = kwargs
if user_id:
params.update({"_id": ObjectId(user_id)}) #This does not exist in DB
future = self.collection.find_one(params)
print(future) #prints <tornado.concurrent.Future object at 0x04152A90>
result = yield future
print(result) #prints None
return result
The calls look like getLoggedIn => model.get => core.get
core.get is decorated with #gen.coroutine and I call yield self.collection.find_one(params)
The print(result) prints None but if I return result and try to print the return value in getLoggedIn function it prints .
I believe this is related to asynchronous nature of tornado and the print gets called before yield but I am not sure. It would be a great help if someone could explain about coroutine/generators principles and behavior in different possible cases.
PEP 255 covers the original specification for generators.
However, tornado uses yield inside of coroutines in a very specific way: http://www.tornadoweb.org/en/stable/guide/coroutines.html#how-it-works
Your code doesn't really look or smell like an ordinary generator because the Python notion of generators is being co-opted by tornado to define coroutines.
I would say that you don't really want the principles of generator writing, but the principles of tornado generators -- a wholly different beast.
Assigning the value of the yield is a way for the wrapping #gen.coroutine decorator to pass the result of the future back into core.get.
That way, result is not assigned the future object, but future.result().
yield future essentially suspends your function and turns it into a callback that the future will invoke, resuming execution at the location of the yield.
The asynchronous nature of tornado does not allow the yield to run before the print, as you worried.
Most likely, your Future is not returning anything, or is returning None (semantically equivalent, I know).
It might be best to think of result = yield future as a specialized version of result = future.result()
Every call to a coroutine must be yielded, and the caller must also be a coroutine. So getLoggedIn must be a coroutine that calls:
result = yield model.Session.get(db, uid=uid)
And so on. See my article on refactoring Tornado coroutines for a detailed example and explanation.