Below tornado APP has 2 end points. One(/) is slow because it waits for an IO operation and other(/hello) is fast.
My requirement is to make a request to both end points simultaneously.I observed it takes 2nd request only after it finishes the 1st one. Even though It is asynchronous why it is not able to handle both requests at same time ?
How to make it to handle simultaneously?
Edit : I am using windows 7, Eclipse IDE
****************Module*****************
import tornado.ioloop
import tornado.web
class MainHandler(tornado.web.RequestHandler):
#tornado.web.asynchronous
def get(self):
self.do_something()
self.write("FINISHED")
self.finish()
def do_something(self):
inp = input("enter to continue")
print (inp)
class HelloHandler(tornado.web.RequestHandler):
def get(self):
print ("say hello")
self.write("Hello bro")
self.finish(
def make_app():
return tornado.web.Application([
(r"/", MainHandler),
(r"/hello", HelloHandler)
])
if __name__ == "__main__":
app = make_app()
app.listen(8888)
tornado.ioloop.IOLoop.current().start()
It is asynchronous only if you make it so. A Tornado server runs in a single thread. If that thread is blocked by a synchronous function call, nothing else can happen on that thread in the meantime. What #tornado.web.asynchronous enables is the use of generators:
#tornado.web.asynchronous
def get(self):
yield from self.do_something()
^^^^^^^^^^
This yield/yield from (in current Python versions await) feature suspends the function and lets other code run on the same thread while the asynchronous call completes elsewhere (e.g. waiting for data from the database, waiting for a network request to return a response). I.e., if Python doesn't actively have to do something but is waiting for external processes to complete, it can yield processing power to other tasks. But since your function is very much running in the foreground and blocking the thread, nothing else will happen.
See http://www.tornadoweb.org/en/stable/guide/async.html and https://docs.python.org/3/library/asyncio.html.
Related
I am new to tornado and have an API that makes a blocking database call. Because of this blocking call, tornado isn't able to serve all the requests if multiple requests come at the same time.
I looked about it and found out that it could be solved using two approaches: Making the code asynchronous and/or using Process Pool Executors. My assumption here is that having multiple process pool executors is like having multiple processes on tornado to serve multiple requests. Every single example I looked at about implementing Process Pool Executor also makes the code asynchronous.
I cannot make the code asynchronous for the time being because it would require more code changes and so I was looking at simple fix using Process Pool Executors.
What I have currently
import tornado.ioloop
import tornado.web
def blocking_call():
import time
time.sleep(60)
return "Done"
class MainHandler(tornado.web.RequestHandler):
def get(self):
val = blocking_call()
self.write(val)
if __name__ == "__main__":
app = tornado.web.Application([(r"/", MainHandler)])
app.listen(8888)
tornado.ioloop.IOLoop.current().start()
What I tried
import tornado.ioloop
import tornado.web
from concurrent.futures import ProcessPoolExecutor
def blocking_call():
import time
time.sleep(60)
return "Done"
class MainHandler(tornado.web.RequestHandler):
def initialize(self, executor):
self.executor = executor
def get(self):
val = self.executor.submit(blocking_call)
self.write(val)
if __name__ == "__main__":
executor = ProcessPoolExecutor(5)
app = tornado.web.Application(
[(r"/", MainHandler, dict(executor=executor))])
app.listen(8888)
tornado.ioloop.IOLoop.current().start()
My problem with this approach is that now I am getting a future object instead of actual response. How do I make the Get request wait for self.executor to complete before sending back the response?
The executor.submit() returns a concurrent.futures.Future which is not awaitable.
I suggest you use Tornado's run_in_executor method to execute the blocking task.
async def get(self):
loop = tornado.ioloop.IOLoop.current()
val = await loop.run_in_executor(self.executor, blocking_call)
self.write(val)
I have written a Tornado HTTP server and trying to start a thread as soon as the API request http://localhost:8889/img is hit. I am getting a RuntimeError exception after some time or immediately. Not able to understand this behavior. Please explain me.
import tornado.ioloop
import tornado.web
import threading
import time
class MainHandler(tornado.web.RequestHandler):
def get(self):
self.write("Hello, world")
class img(tornado.web.RequestHandler):
def post(self):
t_img = threading.Thread(target=self.sendimg,)
t_img.start()
def sendimg(self):
while True:
self.write({"name":"vinay"})
print("sent")
time.sleep(0.5)
def make_app():
return tornado.web.Application([
(r"/", MainHandler),
(r"/img", img),
])
if __name__ == "__main__":
app = make_app()
app = tornado.httpserver.HTTPServer(app)
app.listen(8889)
tornado.ioloop.IOLoop.current().start()
Error:
self.write({"name":"vinay"})
File "/Users/vinaykp/anaconda3/lib/python3.6/site-
packages/tornado/web.py", line 708, in write
raise RuntimeError("Cannot write() after finish()")
RuntimeError: Cannot write() after finish()
I'm not an expert in concurrency and Tornado, but it seems that main problem is with lifecycle of request object. See, when you're passing img class to tornado.web.Application:
def make_app():
return tornado.web.Application([
(r"/", MainHandler),
(r"/img", img),
])
Tornado receives reference to the class, create its object and executes proper method. As request is done and client get response from the server - it's nothing more to do. You cannot write to client once request is done and in "finished" state, because it's completely pointless, as at the moment client doesn't expect nothing from you.
If you think of HTTP design, it's how this protocol works. It's not ongoing connection, but rather stateless, request-response communication that has its start and end, which from the other hand determines request lifecycle.
You can of course start new Thread when your server was hit with request, like:
def get(self):
self.write("Hello, world!")
tf = threading.Thread(target=self.thread_func)
tf.start()
def thread_func(self):
while True:
print("ok\n")
time.sleep(0.5)
And new Thread will be created every time you hit proper endpoint with request, but you cannot really write to self anymore after request is done.
I'm trying to write a asynchronous method for use in a Tornado application. My method needs to manage a connection that can and should be shared among other calls to the function The connection is created by awaiting. To manage this, I was using asyncio.Lock. However, every call to my method would hang waiting for the lock.
After a few hours of experimenting, I found out a few things,
If nothing awaits in the lock block, everything works as expected
tornado.ioloop.IOLoop.configure('tornado.platform.asyncio.AsyncIOLoop') does not help
tornado.platform.asyncio.AsyncIOMainLoop().install() allows it to work, regardless if the event loop is started with tornado.ioloop.IOLoop.current().start() or asyncio.get_event_loop().run_forever()
Here is some sample code that wont work until unless you uncomment AsyncIOMainLoop().install():
import tornado.ioloop
import tornado.web
import tornado.gen
import tornado.httpclient
from tornado.platform.asyncio import AsyncIOMainLoop
import asyncio
import tornado.locks
class MainHandler(tornado.web.RequestHandler):
_lock = asyncio.Lock()
#_lock = tornado.locks.Lock()
async def get(self):
print("in get")
r = await tornado.gen.multi([self.foo(str(i)) for i in range(2)])
self.write('\n'.join(r))
async def foo(self, i):
print("Getting first lock on " + i)
async with self._lock:
print("Got first lock on " + i)
# Do something sensitive that awaits
await asyncio.sleep(0)
print("Unlocked on " + i)
# Do some work
print("Work on " + i)
await asyncio.sleep(0)
print("Getting second lock on " + i)
async with self._lock:
print("Got second lock on " + i)
# Do something sensitive that doesnt await
pass
print("Unlocked on " + i)
return "done"
def make_app():
return tornado.web.Application([
(r"/", MainHandler),
])
if __name__ == "__main__":
#AsyncIOMainLoop().install() # This will make it work
#tornado.ioloop.IOLoop.configure('tornado.platform.asyncio.AsyncIOLoop') # Does not help
app = make_app()
app.listen(8888)
print('starting app')
tornado.ioloop.IOLoop.current().start()
I now know that tornado.locks.Lock() exists and works, but I'm curious why the asyncio.Lock does not work.
Both Tornado and asyncio have a global singleton event loop which everything else depends on (for advanced use cases you can avoid the singleton, but using it is idiomatic). To use both libraries together, the two singletons need to be aware of each other.
AsyncIOMainLoop().install() makes a Tornado event loop that points to the asyncio singleton, then sets it as the tornado singleton. This works.
IOLoop.configure('AsyncIOLoop') tells Tornado "whenever you need an IOLoop, create a new (non-singleton!) asyncio event loop and use that. The asyncio loop becomes the singleton when the IOLoop is started. This almost works, but when the MainHandler class is defined (and creates its class-scoped asyncio.Lock, the asyncio singleton is still pointing to the default (which will be replaced by the one created by AsyncIOLoop).
TL;DR: Use AsyncIOMainLoop, not AsyncIOLoop, unless you're attempting to use the more advanced non-singleton use patterns. This will get simpler in Tornado 5.0 as asyncio integration will be enabled by default.
here is my code:
class AsyncTestHandler(BaseHandler):
def testTimeOut(self, callback):
time.sleep(20)
callback("ok")
#tornado.web.asynchronous
def post(self):
user = self.get_current_user()
self.testTimeOut(callback=self.respones)
def respones(self,msg):
self.finish(msg)
i have used "#tornado.web.asynchronous" with callback,but the request is not asynchronous,how can id?
Tornado uses only one process and one thread. All the IO operations in it are asynchronous, which doesn't mean that they are processed concurrently. So, if you call time.sleep(xx) in your code anywhere, your Tornado process will totally 'stopped' for that time!
The correct way to sleep in Tornado is to call ioloop.add_timeout.
See tornado equivalent of delay.
See http://caisong.com/Tornado%20don't%20use%20time.sleep%20.html.
The problem is, time.sleep isn't asynchronous, so the main loop is blocked, while sleeping. For running synchronous code asynchronously you can use a seperate worker thread.
class HugeQueryHandler(BaseHandler):
executor = tornado.concurrent.futures.ThreadPoolExecutor(5)
#tornado.concurrent.run_on_executor
def sleep_async(self):
time.sleep(20)
return
#tornado.web.asynchronous
#gen.engine
def get(self):
r = yield self.sleep_async()
self.finish()
In Tornado's chat demo, it has a method like this:
#tornado.web.asynchronous
def post(self):
cursor = self.get_argument("cursor", None)
global_message_buffer.wait_for_messages(self.on_new_messages,
cursor=cursor)
I'm fairly new to this long polling thing, and I don't really understand exactly how the threading stuff works, though it states:
By using non-blocking network I/O, Tornado can scale to tens of thousands of open connections...
My theory was that by making a simple app:
import tornado.ioloop
import tornado.web
import time
class MainHandler(tornado.web.RequestHandler):
#tornado.web.asynchronous
def get(self):
print("Start request")
time.sleep(4)
print("Okay done now")
self.write("Howdy howdy howdy")
self.finish()
application = tornado.web.Application([
(r'/', MainHandler),
])
That if I made two requests in a row (i.e. I opened two browser windows and quickly refreshed both) I would see this:
Start request
Start request
Okay done now
Okay done now
Instead, I see
Start request
Okay done now
Start request
Okay done now
Which leads me to believe that it is, in fact, blocking in this case. Why is it that my code is blocking, and how do I get some code to do what I expect? I get the same output on Windows 7 with a core i7, and a linux Mint 13 box with I think two cores.
Edit:
I found one method - if someone can provide a method that works cross-platform (I'm not too worried about performance, only that it's non-blocking), I'll accept that answer.
The right way to convert your test app into a form that won't block the IOLoop is like this:
from tornado.ioloop import IOLoop
import tornado.web
from tornado import gen
import time
#gen.coroutine
def async_sleep(timeout):
""" Sleep without blocking the IOLoop. """
yield gen.Task(IOLoop.instance().add_timeout, time.time() + timeout)
class MainHandler(tornado.web.RequestHandler):
#gen.coroutine
def get(self):
print("Start request")
yield async_sleep(4)
print("Okay done now")
self.write("Howdy howdy howdy")
self.finish()
if __name__ == "__main__":
application = tornado.web.Application([
(r'/', MainHandler),
])
application.listen(8888)
IOLoop.instance().start()
The difference is replacing the call to time.sleep with one which won't block the IOLoop. Tornado is designed to handle lots of concurrent I/O without needing multiple threads/subprocesses, but it will still block if you use synchronous APIs. In order for your long-polling solution to handle concurrency the way you'd like, you have to make sure that no long-running calls block.
The problem with the code in original question is that when you call time.sleep(4) you are effectively blocking the execution of event loop for 4 seconds. And accepted answer doesn't solve the problem either (IMHO).
Asynchronous serving in Tornado works on trust. Tornado will call your functions whenever something happens, but it trusts you that you will return control to it as soon as possible. If you block with time.sleep() then this trust is breached - Tornado can't handle new connections.
Using multiple threads only hides the mistake; running Tornado with thousands of threads (so you can serve 1000s of connections simultaneously) would be very inefficient. The appropriate way is running a single thread which only blocks inside Tornado (on select or whatever Tornado's way of listening for events is) - not on your code (to be exact: never on your code).
The proper solution is to just return from get(self) right before time.sleep() (without calling self.finish()), like this:
class MainHandler(tornado.web.RequestHandler):
#tornado.web.asynchronous
def get(self):
print("Starting")
You must of course remember that this request is still open and call write() and finish() on it later.
I suggest you take a look at chat demo. Once you strip out the authentication you get a very nice example of async long polling server.
Since Tornado 5.0, asyncio is enabled automatically, so pretty much just changing time.sleep(4) to await asyncio.sleep(4) and #tornado.web.asynchronous def get(self): to async def get(self): solves the problem.
Example:
import tornado.ioloop
import tornado.web
import asyncio
class MainHandler(tornado.web.RequestHandler):
async def get(self):
print("Start request")
await asyncio.sleep(4)
print("Okay done now")
self.write("Howdy howdy howdy")
self.finish()
app = tornado.web.Application([
(r'/', MainHandler),
])
app.listen(8888)
tornado.ioloop.IOLoop.current().start()
Output:
Start request
Start request
Okay done now
Okay done now
Sources:
Tornado on asyncio
asyncio usage example