In Tornado's chat demo, it has a method like this:
#tornado.web.asynchronous
def post(self):
cursor = self.get_argument("cursor", None)
global_message_buffer.wait_for_messages(self.on_new_messages,
cursor=cursor)
I'm fairly new to this long polling thing, and I don't really understand exactly how the threading stuff works, though it states:
By using non-blocking network I/O, Tornado can scale to tens of thousands of open connections...
My theory was that by making a simple app:
import tornado.ioloop
import tornado.web
import time
class MainHandler(tornado.web.RequestHandler):
#tornado.web.asynchronous
def get(self):
print("Start request")
time.sleep(4)
print("Okay done now")
self.write("Howdy howdy howdy")
self.finish()
application = tornado.web.Application([
(r'/', MainHandler),
])
That if I made two requests in a row (i.e. I opened two browser windows and quickly refreshed both) I would see this:
Start request
Start request
Okay done now
Okay done now
Instead, I see
Start request
Okay done now
Start request
Okay done now
Which leads me to believe that it is, in fact, blocking in this case. Why is it that my code is blocking, and how do I get some code to do what I expect? I get the same output on Windows 7 with a core i7, and a linux Mint 13 box with I think two cores.
Edit:
I found one method - if someone can provide a method that works cross-platform (I'm not too worried about performance, only that it's non-blocking), I'll accept that answer.
The right way to convert your test app into a form that won't block the IOLoop is like this:
from tornado.ioloop import IOLoop
import tornado.web
from tornado import gen
import time
#gen.coroutine
def async_sleep(timeout):
""" Sleep without blocking the IOLoop. """
yield gen.Task(IOLoop.instance().add_timeout, time.time() + timeout)
class MainHandler(tornado.web.RequestHandler):
#gen.coroutine
def get(self):
print("Start request")
yield async_sleep(4)
print("Okay done now")
self.write("Howdy howdy howdy")
self.finish()
if __name__ == "__main__":
application = tornado.web.Application([
(r'/', MainHandler),
])
application.listen(8888)
IOLoop.instance().start()
The difference is replacing the call to time.sleep with one which won't block the IOLoop. Tornado is designed to handle lots of concurrent I/O without needing multiple threads/subprocesses, but it will still block if you use synchronous APIs. In order for your long-polling solution to handle concurrency the way you'd like, you have to make sure that no long-running calls block.
The problem with the code in original question is that when you call time.sleep(4) you are effectively blocking the execution of event loop for 4 seconds. And accepted answer doesn't solve the problem either (IMHO).
Asynchronous serving in Tornado works on trust. Tornado will call your functions whenever something happens, but it trusts you that you will return control to it as soon as possible. If you block with time.sleep() then this trust is breached - Tornado can't handle new connections.
Using multiple threads only hides the mistake; running Tornado with thousands of threads (so you can serve 1000s of connections simultaneously) would be very inefficient. The appropriate way is running a single thread which only blocks inside Tornado (on select or whatever Tornado's way of listening for events is) - not on your code (to be exact: never on your code).
The proper solution is to just return from get(self) right before time.sleep() (without calling self.finish()), like this:
class MainHandler(tornado.web.RequestHandler):
#tornado.web.asynchronous
def get(self):
print("Starting")
You must of course remember that this request is still open and call write() and finish() on it later.
I suggest you take a look at chat demo. Once you strip out the authentication you get a very nice example of async long polling server.
Since Tornado 5.0, asyncio is enabled automatically, so pretty much just changing time.sleep(4) to await asyncio.sleep(4) and #tornado.web.asynchronous def get(self): to async def get(self): solves the problem.
Example:
import tornado.ioloop
import tornado.web
import asyncio
class MainHandler(tornado.web.RequestHandler):
async def get(self):
print("Start request")
await asyncio.sleep(4)
print("Okay done now")
self.write("Howdy howdy howdy")
self.finish()
app = tornado.web.Application([
(r'/', MainHandler),
])
app.listen(8888)
tornado.ioloop.IOLoop.current().start()
Output:
Start request
Start request
Okay done now
Okay done now
Sources:
Tornado on asyncio
asyncio usage example
Related
How can I make a POST method to accept multiple requests and run in parallel?
I have this code and when I make two POST requests to localhost:5050/, the second request won't start until the first one gave a response.
from tornado import ioloop, web
class MyHandler(web.RequestHandler):
def post(self):
print("1")
print("2")
print("3")
print(self.request.body.rstrip())
app = web.Application([
web.URLSpec('/', MyHandler)
])
app.listen(port=5000)
print("Listening...")
ioloop.IOLoop.current().start()
I've read gen.coroutine but it doesn't work. What is the best way to handle parallel requests? Thanks in advance.
Tornado is built with assumption that large part of webserver time is spent handling IO. In non-ideal situation you end up with significant amount of compute required in webservice, it should be removed from tornado IOLoop. Given you have IO heavy app, tornado will provide needed concurrency. Here is an example simulating an IO heavy app.
from tornado import ioloop, web
from time import sleep
from random import randint
class MyHandler(web.RequestHandler):
async def post(self):
wait_val = randint(1, 10)
await ioloop.IOLoop.current().run_in_executor(None, sleep, wait_val)
print(self.request.body.rstrip())
app = web.Application([
web.URLSpec('/', MyHandler)
])
app.listen(port=5000)
print("Listening...")
ioloop.IOLoop.current().start()
To test the app you can use Siege
siege "http://127.0.0.1:5000/ POST"
Taking the standard Tornado demonstration and pushing the IOLoop into a background thread allows querying of the server within a single script. This is useful when the Tornado server is an interactive object (see Dask or similar).
import asyncio
import requests
import tornado.ioloop
import tornado.web
from concurrent.futures import ThreadPoolExecutor
class MainHandler(tornado.web.RequestHandler):
def get(self):
self.write("Hello, world")
def make_app():
return tornado.web.Application([
(r"/", MainHandler),
])
pool = ThreadPoolExecutor(max_workers=2)
loop = tornado.ioloop.IOLoop()
app = make_app()
app.listen(8888)
fut = pool.submit(loop.start)
print(requests.get("https://localhost:8888"))
The above works just fine in a standard python script (though it is missing safe shutdown). Jupyter notebook are optimal environment for these interactive Tornado server environments. However, when it comes to Jupyter this idea breaks down as there is already a active running loop:
>>> import asyncio
>>> asyncio.get_event_loop()
<_UnixSelectorEventLoop running=True closed=False debug=False>
This is seen when running the above script in a Jupyter notebook, both the server and the request client are trying to open a connection in the same thread and the code hangs. Building a new Asyncio loop and/or Tornado IOLoop does not seem to help and I suspect I am missing something in Jupyter itself.
The question: Is it possible to have a live Tornado server running in the background within a Jupyter notebook so that standard python requests or similar can connect to it from the primary thread? I would prefer to avoid Asyncio in the code presented to users if possible due to its relatively complexity for novice users.
Based on my recent PR to streamz, here is something that works, similar to your idea:
class InNotebookServer(object):
def __init__(self, port):
self.port = port
self.loop = get_ioloop()
self.start()
def _start_server(self):
from tornado.web import Application, RequestHandler
from tornado.httpserver import HTTPServer
from tornado import gen
class Handler(RequestHandler):
source = self
#gen.coroutine
def get(self):
self.write('Hello World')
application = Application([
('/', Handler),
])
self.server = HTTPServer(application)
self.server.listen(self.port)
def start(self):
"""Start HTTP server and listen"""
self.loop.add_callback(self._start_server)
_io_loops = []
def get_ioloop():
from tornado.ioloop import IOLoop
import threading
if not _io_loops:
loop = IOLoop()
thread = threading.Thread(target=loop.start)
thread.daemon = True
thread.start()
_io_loops.append(loop)
return _io_loops[0]
To call in the notebook
In [2]: server = InNotebookServer(9005)
In [3]: import requests
requests.get('http://localhost:9005')
Out[3]: <Response [200]>
Part 1: Let get nested tornado(s)
To find the information you need you would have had to follow the following crumbtrails, start by looking at what is described in the release notes of IPython 7
It particular it will point you to more informations on the async and await sections in the documentation, and to this discussion,
which suggest the use of nest_asyncio.
The Crux is the following:
A) either you trick python into running two nested event loops. (what nest_asyncio does)
B) You schedule coroutines on already existing eventloop. (I'm not sure how to do that with tornado)
I'm pretty sure you know all that, but I'm sure other reader will appreciate.
There are unfortunately no ways to make it totally transparent to users – well unless you control the deployment like on a jupyterhub, and can add these lines to the IPython startups scripts that are automatically loaded. But I think the following is simple enough.
import nest_asyncio
nest_asyncio.apply()
# rest of your tornado setup and start code.
Part 2: Gotcha Synchronous code block eventloop.
Previous section takes only care of being able to run the tornado app. But note that any synchronous code will block the eventloop; thus when running print(requests.get("http://localhost:8000")) the server will appear to not work as you are blocking the eventloop, which will restart only when the code finish execution which is waiting for the eventloop to restart...(understanding this is an exercise left to the reader). You need to either issue print(requests.get("http://localhost:8000")) from another kernel, or, use aiohttp.
Here is how to use aiohttp in a similar way as requests.
import aiohttp
session = aiohttp.ClientSession()
await session.get('http://localhost:8889')
In this case as aiohttp is non-blocking things will appear to work properly. You here can see some extra IPython magic where we autodetect async code and run it on the current eventloop.
A cool exercise could be to run a request.get in a loop in another kernel, and run sleep(5) in the kernel where tornado is running, and see that we stop processing requests...
Part 3: Disclaimer and other routes:
This is quite tricky and I would advise to not use in production, and warn your users this is not the recommended way of doing things.
That does not completely solve your case, you will need to run things not in the main thread which I'm not sure is possible.
You can also try to play with other loop runners like trio and curio; they might allow you to do stuff you can't with asyncio by default like nesting, but here be dragoons. I highly recommend trio and the multiple blog posts around its creation, especially if you are teaching async.
Enjoy, hope that helped, and please report bugs, as well as things that did work.
You can make the tornado server run in background using the %%script --bg magic command. The option --bg tells jupyter to run the code of the current cell in background.
Just create a tornado server in one cell alongwith the magic command and run that cell.
Example:
%%script python --bg
import tornado.ioloop
import tornado.web
class MainHandler(tornado.web.RequestHandler):
def get(self):
self.write("Hello, world")
def make_app():
return tornado.web.Application([
(r"/", MainHandler),
])
loop = tornado.ioloop.IOLoop.current()
app = make_app()
app.listen(8000) # 8888 was being used by jupyter in my case
loop.start()
And then you can use requests in a separate cell to connect to the server:
import requests
print(requests.get("http://localhost:8000"))
# prints <Response [200]>
One thing to note here is that if you stop/interrupt the kernel on any cell, the background script will also stop. So you'll have to run this cell again to start the server.
Below tornado APP has 2 end points. One(/) is slow because it waits for an IO operation and other(/hello) is fast.
My requirement is to make a request to both end points simultaneously.I observed it takes 2nd request only after it finishes the 1st one. Even though It is asynchronous why it is not able to handle both requests at same time ?
How to make it to handle simultaneously?
Edit : I am using windows 7, Eclipse IDE
****************Module*****************
import tornado.ioloop
import tornado.web
class MainHandler(tornado.web.RequestHandler):
#tornado.web.asynchronous
def get(self):
self.do_something()
self.write("FINISHED")
self.finish()
def do_something(self):
inp = input("enter to continue")
print (inp)
class HelloHandler(tornado.web.RequestHandler):
def get(self):
print ("say hello")
self.write("Hello bro")
self.finish(
def make_app():
return tornado.web.Application([
(r"/", MainHandler),
(r"/hello", HelloHandler)
])
if __name__ == "__main__":
app = make_app()
app.listen(8888)
tornado.ioloop.IOLoop.current().start()
It is asynchronous only if you make it so. A Tornado server runs in a single thread. If that thread is blocked by a synchronous function call, nothing else can happen on that thread in the meantime. What #tornado.web.asynchronous enables is the use of generators:
#tornado.web.asynchronous
def get(self):
yield from self.do_something()
^^^^^^^^^^
This yield/yield from (in current Python versions await) feature suspends the function and lets other code run on the same thread while the asynchronous call completes elsewhere (e.g. waiting for data from the database, waiting for a network request to return a response). I.e., if Python doesn't actively have to do something but is waiting for external processes to complete, it can yield processing power to other tasks. But since your function is very much running in the foreground and blocking the thread, nothing else will happen.
See http://www.tornadoweb.org/en/stable/guide/async.html and https://docs.python.org/3/library/asyncio.html.
I'm new to tornado.
What I want is to write some functions to fetch webpages asynchronously. Since no requesthandlers, apps, or servers involved here, I think I can use tornado.httpclient.AsyncHTTPClient alone.
But all the sample codes seem to be in a tornado server or requesthandler. When I tried to use it alone, it never works.
For example:
def handle(self,response):
print response
print response.body
#tornado.web.asynchronous
def fetch(self,url):
client=tornado.httpclient.AsyncHTTPClient()
client.fetch(url,self.handle)
fetch('http://www.baidu.com')
It says "'str' object has no attribute 'application'", but I'm trying to use it alone?
or :
#tornado.gen.coroutine
def fetch_with_coroutine(url):
client=tornado.httpclient.AsyncHTTPClient()
response=yield http_client.fetch(url)
print response
print response.body
raise gen.Return(response.body)
fetch_with_coroutine('http://www.baidu.com')
doesn't work either.
Earlier, I tried pass a callback to AsyncHTTPHandler.fetch, then start the IOLoop, It works and the webpage source code is printed. But I can't figure out what to do with the ioloop.
#tornado.web.asynchronous can only be applied to certain methods in RequestHandler subclasses; it is not appropriate for this usage.
Your second example is the correct structure, but you need to actually run the IOLoop. The best way to do this in a batch-style program is IOLoop.current().run_sync(fetch_with_coroutine). This starts the IOLoop, runs your callback, then stops the IOLoop. You should run a single function within run_sync(), and then use yield within that function to call any other coroutines.
For a more complete example, see https://github.com/tornadoweb/tornado/blob/master/demos/webspider/webspider.py
Here's an example I've used in the past...
from tornado.httpclient import AsyncHTTPClient
from tornado.ioloop import IOLoop
AsyncHTTPClient.configure(None, defaults=dict(user_agent="MyUserAgent"))
http_client = AsyncHTTPClient()
def handle_response(response):
if response.error:
print("Error: %s" % response.error)
else:
print(response.body)
async def get_content():
await http_client.fetch("https://www.integralist.co.uk/", handle_response)
async def main():
await get_content()
print("I won't wait for get_content to finish. I'll show immediately.")
if __name__ == "__main__":
io_loop = IOLoop.current()
io_loop.run_sync(main)
I've also detailed how to use Pipenv with tox.ini and Flake8 with this tornado example so others should be able to get up and running much more quickly https://gist.github.com/fd603239cacbb3d3d317950905b76096
Basically we can call xmlrpc handlers following way:
import xmlrpclib
s = xmlrpclib.ServerProxy('http://remote_host/rpc/')
print s.system.listmethods()
In tornado we can integrate it like this:
import xmlrpclib
import tornado.web
s = xmlrpclib.ServerProxy('http://remote_host/rpc/')
class MyHandler(tornado.web.RequestHandler):
def get(self):
result = s.system.listmethods()
I have following, a little bit newbie, questions:
Will result = s.system.listmethods() block tornado?
Are there any non-blocking xmlrpc clients around?
How can we achieve result = yield gen.Task(s.system.listmethods)?
1.Yes it will block tornado, since xmlrpclib uses blocking python sockets (as it is)
2.Not that I'm aware of, but I'll provide a solution where you can keep xmlrpclib but have it async
3.My solution doesn't use tornado gen.
Ok, so one useful library to have at mind whenever you're doing networking and need to write async code is gevent, it's a really good high quality library that I would recommend to everyone.
Why is it good and easy to use ?
You can write asynchronous code in a synchronous manner (so that makes it easy)
All you have to do, to do so is monkey patch with one simple line :
from gevent import monkey; monkey.patch_all()
When using tornado you need to know two things (that you may already know) :
Tornado only supports asynchronous views when acting as a HTTPServer (WSGI isn't supported for async views)
Async views need to terminate the responses by themselves you do by using either self.finish() or self.render() (which calls self.finish())
Ok so here's an example illustrating what you would need with the necessary gevent integration with tornado :
# Python immports
import functools
# Tornado imports
import tornado.ioloop
import tornado.web
import tornado.httpserver
# XMLRpc imports
import xmlrpclib
# Asynchronous gevent decorator
def gasync(func):
#tornado.web.asynchronous
#functools.wraps(func)
def f(self, *args, **kwargs):
return gevent.spawn(func, self, *args, **kwargs)
return f
# Our XML RPC service
xml_service = xmlrpclib.ServerProxy('http://remote_host/rpc/')
class MyHandler(tornado.web.RequestHandler):
#gasync
def get(self):
# This doesn't block tornado thanks to gevent
# Which patches all of xmlrpclib's socket calls
# So they no longer are blocking
result = xml_service.system.listmethods()
# Do something here
# Write response to client
self.write('hello')
self.finish()
# Our URL Mappings
handlers = [
(r"/", MyHandler),
]
def main():
# Setup app and HTTP server
application = tornado.web.Application(handlers)
http_server = tornado.httpserver.HTTPServer(application)
http_server.listen(8000)
# Start ioloop
tornado.ioloop.IOLoop.instance().start()
if __name__ == "__main__":
main()
So give the example a try (adapt it to your needs obviously) and you should be good to go.
No need to write any extra code, gevent does all the work of patching up python sockets so they can be used asynchronously while still writing code in a synchronous fashion (which is a real bonus).
Hope this helps :)
I do not think so.
Because Tornado has it's own ioloop, but gevent's ioloop is libevent.
So gevent will block Tornado's ioloop.