I want to write a simple async http server with Tornado.
It is not clear to me how to set the callback in order to free the server for additional requests while the current request is processed.
The code I wrote is:
import tornado.web
from tornado.ioloop import IOLoop
from tornado import gen
import time
class TestHandler(tornado.web.RequestHandler):
#gen.coroutine
def post(self, *args, **kwargs):
json_input = tornado.escape.json_decode(self.request.body)
print ('Now in POST. body: {}'.format(json_input))
self.perform_long_task(*args, **json_input)
#gen.coroutine
def perform_long_task(self, **params):
time.sleep(10)
self.write(str(params))
self.finish()
application = tornado.web.Application([
(r"/test", TestHandler),
])
application.listen(9999)
IOLoop.instance().start()
To test I tried to send few POST requests in parallel:
curl -v http://localhost:9999/test -X POST -H "Content-Type:appication/json" -d '{"key1": "val1", "key2": "val2"}' &
Currently the server is blocked while perform_long_task() is processed.
I need help getting the server to be a non-blocking.
Never use time.sleep in Tornado code!
http://www.tornadoweb.org/en/latest/faq.html#why-isn-t-this-example-with-time-sleep-running-in-parallel
Do this in your code instead:
class TestHandler(tornado.web.RequestHandler):
#gen.coroutine
def post(self, *args, **kwargs):
json_input = tornado.escape.json_decode(self.request.body)
print ('Now in POST. body: {}'.format(json_input))
# NOTE: yield here
yield self.perform_long_task(*args, **json_input)
#gen.coroutine
def perform_long_task(self, **params):
yield gen.sleep(10)
self.write(str(params))
# NOTE: no need for self.finish()
You don't need to call self.finish - when the "post" coroutine finishes, Tornado automatically finishes the request.
You must yield self.perform_long_task(), though, otherwise Tornado will end your request early, before you've called "self.write()".
Once you make these changes, two "curl" commands will show that you're doing concurrent processing in Tornado.
I'm still using time.sleep() as my code calls other code that I can't control how is written.
The FAQ http://www.tornadoweb.org/en/latest/faq.html#why-isn-t-this-example-with-time-sleep-running-in-parallel describes three methods. The third one is what I needed.
The only change I needed in my code is to replace:
yield self.perform_long_task(*args, **json_input)
which works only for a class that is written for async,
with:
yield executor.submit(self.perform_long_task,*args, **json_input)
All replies and comments were helpful. Many thanks!
Related
I created batch delayed http (async) client which allows to trigger multiple async http requests and most importantly it allows to delay the start of requests so for example 100 requests are not triggered at a time.
But it has an issue. The http .fetch() method has a handleMethod parameter which handles the response, but I found out that if the delay (sleep) after the fetch isn't long enough the handle method is not even triggered. (maybe the request is killed or what meanwhile).
It is probably related to .run_sync method. How to fix that? I want to put delays but dont want this issue happen.
I need to parse the response regardless how long the request takes, regardless the following sleep call (that call has another reason as i said, and should not be related to response handling at all)
class BatchDelayedHttpClient:
def __init__(self, requestList):
# class members
self.httpClient = httpclient.AsyncHTTPClient()
self.requestList = requestList
ioloop.IOLoop.current().run_sync(self.execute)
#gen.coroutine
def execute(self):
print("exec start")
for request in self.requestList:
print("requesting " + request["url"])
self.httpClient.fetch(request["url"], request["handleMethod"], method=request["method"], headers=request["headers"], body=request["body"])
yield gen.sleep(request["sleep"])
print("exec end")
This page describes how to set an errback that fires when the connection to
the client is lost before the response is generated.
Is there a way to do something similar using klein?
Code from referenced page is below, which works with twisted.web. I would like something like:
request.notifyFinish().addErrback(self._responseFailed, call)
which is code to fire an errback when request does not finish, that works with klein.
from twisted.web.resource import Resource
from twisted.web.server import NOT_DONE_YET
from twisted.internet import reactor
class DelayedResource(Resource):
def _delayedRender(self, request):
request.write("<html><body>Sorry to keep you waiting.</body></html>")
request.finish()
def _responseFailed(self, err, call):
call.cancel()
def render_GET(self, request):
call = reactor.callLater(5, self._delayedRender, request)
request.notifyFinish().addErrback(self._responseFailed, call)
return NOT_DONE_YET
resource = DelayedResource()
Klein handlers are passed a regular Twisted Web Request object. You can use the same notifyFinish method on it to get interrupted response notification.
here is my code:
class AsyncTestHandler(BaseHandler):
def testTimeOut(self, callback):
time.sleep(20)
callback("ok")
#tornado.web.asynchronous
def post(self):
user = self.get_current_user()
self.testTimeOut(callback=self.respones)
def respones(self,msg):
self.finish(msg)
i have used "#tornado.web.asynchronous" with callback,but the request is not asynchronous,how can id?
Tornado uses only one process and one thread. All the IO operations in it are asynchronous, which doesn't mean that they are processed concurrently. So, if you call time.sleep(xx) in your code anywhere, your Tornado process will totally 'stopped' for that time!
The correct way to sleep in Tornado is to call ioloop.add_timeout.
See tornado equivalent of delay.
See http://caisong.com/Tornado%20don't%20use%20time.sleep%20.html.
The problem is, time.sleep isn't asynchronous, so the main loop is blocked, while sleeping. For running synchronous code asynchronously you can use a seperate worker thread.
class HugeQueryHandler(BaseHandler):
executor = tornado.concurrent.futures.ThreadPoolExecutor(5)
#tornado.concurrent.run_on_executor
def sleep_async(self):
time.sleep(20)
return
#tornado.web.asynchronous
#gen.engine
def get(self):
r = yield self.sleep_async()
self.finish()
Is there a way in Flask to send the response to the client and then continue doing some processing? I have a few book-keeping tasks which are to be done, but I don't want to keep the client waiting.
Note that these are actually really fast things I wish to do, thus creating a new thread, or using a queue, isn't really appropriate here. (One of these fast things is actually adding something to a job queue.)
QUICK and EASY method.
We will use pythons Thread Library to acheive this.
Your API consumer has sent something to process and which is processed by my_task() function which takes 10 seconds to execute.
But the consumer of the API wants a response as soon as they hit your API which is return_status() function.
You tie the my_task to a thread and then return the quick response to the API consumer, while in the background the big process gets compelete.
Below is a simple POC.
import os
from flask import Flask,jsonify
import time
from threading import Thread
app = Flask(__name__)
#app.route("/")
def main():
return "Welcome!"
#app.route('/add_')
def return_status():
"""Return first the response and tie the my_task to a thread"""
Thread(target = my_task).start()
return jsonify('Response asynchronosly')
def my_task():
"""Big function doing some job here I just put pandas dataframe to csv conversion"""
time.sleep(10)
import pandas as pd
pd.DataFrame(['sameple data']).to_csv('./success.csv')
return print('large function completed')
if __name__ == "__main__":
app.run(host="0.0.0.0", port=8080)
Sadly teardown callbacks do not execute after the response has been returned to the client:
import flask
import time
app = flask.Flask("after_response")
#app.teardown_request
def teardown(request):
time.sleep(2)
print("teardown_request")
#app.route("/")
def home():
return "Success!\n"
if __name__ == "__main__":
app.run()
When curling this you'll note a 2s delay before the response displays, rather than the curl ending immediately and then a log 2s later. This is further confirmed by the logs:
teardown_request
127.0.0.1 - - [25/Jun/2018 15:41:51] "GET / HTTP/1.1" 200 -
The correct way to execute after a response is returned is to use WSGI middleware that adds a hook to the close method of the response iterator. This is not quite as simple as the teardown_request decorator, but it's still pretty straight-forward:
import traceback
from werkzeug.wsgi import ClosingIterator
class AfterResponse:
def __init__(self, app=None):
self.callbacks = []
if app:
self.init_app(app)
def __call__(self, callback):
self.callbacks.append(callback)
return callback
def init_app(self, app):
# install extension
app.after_response = self
# install middleware
app.wsgi_app = AfterResponseMiddleware(app.wsgi_app, self)
def flush(self):
for fn in self.callbacks:
try:
fn()
except Exception:
traceback.print_exc()
class AfterResponseMiddleware:
def __init__(self, application, after_response_ext):
self.application = application
self.after_response_ext = after_response_ext
def __call__(self, environ, start_response):
iterator = self.application(environ, start_response)
try:
return ClosingIterator(iterator, [self.after_response_ext.flush])
except Exception:
traceback.print_exc()
return iterator
Which you can then use like this:
#app.after_response
def after():
time.sleep(2)
print("after_response")
From the shell you will see the response return immediately and then 2 seconds later the after_response will hit the logs:
127.0.0.1 - - [25/Jun/2018 15:41:51] "GET / HTTP/1.1" 200 -
after_response
This is a summary of a previous answer provided here.
I had a similar problem with my blog. I wanted to send notification emails to those subscribed to comments when a new comment was posted, but I did not want to have the person posting the comment waiting for all the emails to be sent before he gets his response.
I used a multiprocessing.Pool for this. I started a pool of one worker (that was enough, low traffic site) and then each time I need to send an email I prepare everything in the Flask view function, but pass the final send_email call to the pool via apply_async.
You can find an example on how to use celery from within Flask
here https://gist.github.com/jzempel/3201722
The gist of the idea (pun intended) is to define the long, book-keeping tasks as #celery.task and use apply_async1 or delay to from within the view to start the task
Sounds like Teardown Callbacks would support what you want. And you might want to combine it with the pattern from Per-Request After-Request Callbacks to help with organizing the code.
You can do this with WSGI's close protocol, exposed from the Werkzeug Response object's call_on_close decorator. Explained in this other answer here: https://stackoverflow.com/a/63080968/78903
I just thought about the non-blocking infrastructure of tornado and event-driven programming. Actually I'm writing a simple webapp which is accessing a HTTP-API of an external webservice. I understand why I should call this API non-blocking. But are there any disadvantages if I do just the first call non-blocking, so that the IOLoop can loop further?
For Example:
#tornado.web.asynchronous
def get(self):
nonblocking_call1(self._callback)
def _callback(self, response):
self.write(str(response))
self.write(str(blocking_call2()))
self.write(str(blocking_call3()))
self.finish()
vs.
#tornado.web.asynchronous
def get(self):
nonblocking_call1(self._nonblocking_callback1)
def _callback1(self, response):
self.write(str(response))
nonblocking_call2(self._nonblocking_callback2)
def _callback2(self, response):
self.write(str(response))
nonblocking_call3(self._nonblocking_callback3)
def _callback3(self, response):
self.write(str(response))
self.finish()
If you use blocking code inside tornado, the same tornado process can not process any other requests while any blocking code is waiting. Your app will not support more than one simultaneous user, and even if the blocking call only takes something like 100ms, it will still be a HUGE performance killer.
If writing this way is exhausting for you (it is for me), you can use tornado's gen module:
class GenAsyncHandler(RequestHandler):
#asynchronous
#gen.engine
def get(self):
http_client = AsyncHTTPClient()
response = yield gen.Task(http_client.fetch, "http://example.com")
do_something_with_response(response)
self.render("template.html")