I have a server running on tornado. I have a page that opens a websocket to the same server. Now I have observed that opening multiple instances of this page makes all of them wait except one. Only after that one has finished its websocket, does another one start. Is this normal tornado behaviour of I'm doing something wrong?
Earlier my server was running with django but I migrated to tornado for the websocket support. For that I use fallback server as django.
#!/usr/bin/env python
# Run this with
# PYTHONPATH=. DJANGO_SETTINGS_MODULE=testsite.settings testsite/tornado_main.py
# Serves by default at
# http://localhost:8080/hello-tornado and
# http://localhost:8080/hello-django
from tornado.options import options, define, parse_command_line
import django.core.handlers.wsgi
import tornado.httpserver
import tornado.ioloop
import tornado.web
import tornado.wsgi
define('port', type=int, default=8000)
class HelloHandler(tornado.web.RequestHandler):
def get(self):
self.write('Hello from tornado')
def main():
wsgi_app = tornado.wsgi.WSGIContainer(
django.core.handlers.wsgi.WSGIHandler())
tornado_app = tornado.web.Application(
[
('/hello-tornado', HelloHandler),
('.*', tornado.web.FallbackHandler, dict(fallback=wsgi_app)),
])
server = tornado.httpserver.HTTPServer(tornado_app)
server.listen(options.port)
tornado.ioloop.IOLoop.instance().start()
if __name__ == '__main__':
main()
Can I do something that can allow me to make multiple connections?
You need to look into the Asych facilities in tornado to get this to work properly. Tornado in it's normal state is a single threaded stack and thus you can only handle one connection at a time.
You can use the normal #asynchronous decorator or use their gen library to allow your code to handle multiple connections.
Decorator: http://www.tornadoweb.org/documentation/web.html#decorators
Gen: http://www.tornadoweb.org/documentation/gen.html
Read the documentation carefully if you choose to use the #asynchronous decorator as you need to close the connection when you are done with it.
Yes, this is normal Tornado behaviour in case when you trying run heavy blocking applications like Django in it.
You, definitely, should run django and tornado in separate OS processes. Especially if you use Django ORM.
Need I describe why?
Related
How can I make a POST method to accept multiple requests and run in parallel?
I have this code and when I make two POST requests to localhost:5050/, the second request won't start until the first one gave a response.
from tornado import ioloop, web
class MyHandler(web.RequestHandler):
def post(self):
print("1")
print("2")
print("3")
print(self.request.body.rstrip())
app = web.Application([
web.URLSpec('/', MyHandler)
])
app.listen(port=5000)
print("Listening...")
ioloop.IOLoop.current().start()
I've read gen.coroutine but it doesn't work. What is the best way to handle parallel requests? Thanks in advance.
Tornado is built with assumption that large part of webserver time is spent handling IO. In non-ideal situation you end up with significant amount of compute required in webservice, it should be removed from tornado IOLoop. Given you have IO heavy app, tornado will provide needed concurrency. Here is an example simulating an IO heavy app.
from tornado import ioloop, web
from time import sleep
from random import randint
class MyHandler(web.RequestHandler):
async def post(self):
wait_val = randint(1, 10)
await ioloop.IOLoop.current().run_in_executor(None, sleep, wait_val)
print(self.request.body.rstrip())
app = web.Application([
web.URLSpec('/', MyHandler)
])
app.listen(port=5000)
print("Listening...")
ioloop.IOLoop.current().start()
To test the app you can use Siege
siege "http://127.0.0.1:5000/ POST"
Taking the standard Tornado demonstration and pushing the IOLoop into a background thread allows querying of the server within a single script. This is useful when the Tornado server is an interactive object (see Dask or similar).
import asyncio
import requests
import tornado.ioloop
import tornado.web
from concurrent.futures import ThreadPoolExecutor
class MainHandler(tornado.web.RequestHandler):
def get(self):
self.write("Hello, world")
def make_app():
return tornado.web.Application([
(r"/", MainHandler),
])
pool = ThreadPoolExecutor(max_workers=2)
loop = tornado.ioloop.IOLoop()
app = make_app()
app.listen(8888)
fut = pool.submit(loop.start)
print(requests.get("https://localhost:8888"))
The above works just fine in a standard python script (though it is missing safe shutdown). Jupyter notebook are optimal environment for these interactive Tornado server environments. However, when it comes to Jupyter this idea breaks down as there is already a active running loop:
>>> import asyncio
>>> asyncio.get_event_loop()
<_UnixSelectorEventLoop running=True closed=False debug=False>
This is seen when running the above script in a Jupyter notebook, both the server and the request client are trying to open a connection in the same thread and the code hangs. Building a new Asyncio loop and/or Tornado IOLoop does not seem to help and I suspect I am missing something in Jupyter itself.
The question: Is it possible to have a live Tornado server running in the background within a Jupyter notebook so that standard python requests or similar can connect to it from the primary thread? I would prefer to avoid Asyncio in the code presented to users if possible due to its relatively complexity for novice users.
Based on my recent PR to streamz, here is something that works, similar to your idea:
class InNotebookServer(object):
def __init__(self, port):
self.port = port
self.loop = get_ioloop()
self.start()
def _start_server(self):
from tornado.web import Application, RequestHandler
from tornado.httpserver import HTTPServer
from tornado import gen
class Handler(RequestHandler):
source = self
#gen.coroutine
def get(self):
self.write('Hello World')
application = Application([
('/', Handler),
])
self.server = HTTPServer(application)
self.server.listen(self.port)
def start(self):
"""Start HTTP server and listen"""
self.loop.add_callback(self._start_server)
_io_loops = []
def get_ioloop():
from tornado.ioloop import IOLoop
import threading
if not _io_loops:
loop = IOLoop()
thread = threading.Thread(target=loop.start)
thread.daemon = True
thread.start()
_io_loops.append(loop)
return _io_loops[0]
To call in the notebook
In [2]: server = InNotebookServer(9005)
In [3]: import requests
requests.get('http://localhost:9005')
Out[3]: <Response [200]>
Part 1: Let get nested tornado(s)
To find the information you need you would have had to follow the following crumbtrails, start by looking at what is described in the release notes of IPython 7
It particular it will point you to more informations on the async and await sections in the documentation, and to this discussion,
which suggest the use of nest_asyncio.
The Crux is the following:
A) either you trick python into running two nested event loops. (what nest_asyncio does)
B) You schedule coroutines on already existing eventloop. (I'm not sure how to do that with tornado)
I'm pretty sure you know all that, but I'm sure other reader will appreciate.
There are unfortunately no ways to make it totally transparent to users – well unless you control the deployment like on a jupyterhub, and can add these lines to the IPython startups scripts that are automatically loaded. But I think the following is simple enough.
import nest_asyncio
nest_asyncio.apply()
# rest of your tornado setup and start code.
Part 2: Gotcha Synchronous code block eventloop.
Previous section takes only care of being able to run the tornado app. But note that any synchronous code will block the eventloop; thus when running print(requests.get("http://localhost:8000")) the server will appear to not work as you are blocking the eventloop, which will restart only when the code finish execution which is waiting for the eventloop to restart...(understanding this is an exercise left to the reader). You need to either issue print(requests.get("http://localhost:8000")) from another kernel, or, use aiohttp.
Here is how to use aiohttp in a similar way as requests.
import aiohttp
session = aiohttp.ClientSession()
await session.get('http://localhost:8889')
In this case as aiohttp is non-blocking things will appear to work properly. You here can see some extra IPython magic where we autodetect async code and run it on the current eventloop.
A cool exercise could be to run a request.get in a loop in another kernel, and run sleep(5) in the kernel where tornado is running, and see that we stop processing requests...
Part 3: Disclaimer and other routes:
This is quite tricky and I would advise to not use in production, and warn your users this is not the recommended way of doing things.
That does not completely solve your case, you will need to run things not in the main thread which I'm not sure is possible.
You can also try to play with other loop runners like trio and curio; they might allow you to do stuff you can't with asyncio by default like nesting, but here be dragoons. I highly recommend trio and the multiple blog posts around its creation, especially if you are teaching async.
Enjoy, hope that helped, and please report bugs, as well as things that did work.
You can make the tornado server run in background using the %%script --bg magic command. The option --bg tells jupyter to run the code of the current cell in background.
Just create a tornado server in one cell alongwith the magic command and run that cell.
Example:
%%script python --bg
import tornado.ioloop
import tornado.web
class MainHandler(tornado.web.RequestHandler):
def get(self):
self.write("Hello, world")
def make_app():
return tornado.web.Application([
(r"/", MainHandler),
])
loop = tornado.ioloop.IOLoop.current()
app = make_app()
app.listen(8000) # 8888 was being used by jupyter in my case
loop.start()
And then you can use requests in a separate cell to connect to the server:
import requests
print(requests.get("http://localhost:8000"))
# prints <Response [200]>
One thing to note here is that if you stop/interrupt the kernel on any cell, the background script will also stop. So you'll have to run this cell again to start the server.
In Tornado's chat demo, it has a method like this:
#tornado.web.asynchronous
def post(self):
cursor = self.get_argument("cursor", None)
global_message_buffer.wait_for_messages(self.on_new_messages,
cursor=cursor)
I'm fairly new to this long polling thing, and I don't really understand exactly how the threading stuff works, though it states:
By using non-blocking network I/O, Tornado can scale to tens of thousands of open connections...
My theory was that by making a simple app:
import tornado.ioloop
import tornado.web
import time
class MainHandler(tornado.web.RequestHandler):
#tornado.web.asynchronous
def get(self):
print("Start request")
time.sleep(4)
print("Okay done now")
self.write("Howdy howdy howdy")
self.finish()
application = tornado.web.Application([
(r'/', MainHandler),
])
That if I made two requests in a row (i.e. I opened two browser windows and quickly refreshed both) I would see this:
Start request
Start request
Okay done now
Okay done now
Instead, I see
Start request
Okay done now
Start request
Okay done now
Which leads me to believe that it is, in fact, blocking in this case. Why is it that my code is blocking, and how do I get some code to do what I expect? I get the same output on Windows 7 with a core i7, and a linux Mint 13 box with I think two cores.
Edit:
I found one method - if someone can provide a method that works cross-platform (I'm not too worried about performance, only that it's non-blocking), I'll accept that answer.
The right way to convert your test app into a form that won't block the IOLoop is like this:
from tornado.ioloop import IOLoop
import tornado.web
from tornado import gen
import time
#gen.coroutine
def async_sleep(timeout):
""" Sleep without blocking the IOLoop. """
yield gen.Task(IOLoop.instance().add_timeout, time.time() + timeout)
class MainHandler(tornado.web.RequestHandler):
#gen.coroutine
def get(self):
print("Start request")
yield async_sleep(4)
print("Okay done now")
self.write("Howdy howdy howdy")
self.finish()
if __name__ == "__main__":
application = tornado.web.Application([
(r'/', MainHandler),
])
application.listen(8888)
IOLoop.instance().start()
The difference is replacing the call to time.sleep with one which won't block the IOLoop. Tornado is designed to handle lots of concurrent I/O without needing multiple threads/subprocesses, but it will still block if you use synchronous APIs. In order for your long-polling solution to handle concurrency the way you'd like, you have to make sure that no long-running calls block.
The problem with the code in original question is that when you call time.sleep(4) you are effectively blocking the execution of event loop for 4 seconds. And accepted answer doesn't solve the problem either (IMHO).
Asynchronous serving in Tornado works on trust. Tornado will call your functions whenever something happens, but it trusts you that you will return control to it as soon as possible. If you block with time.sleep() then this trust is breached - Tornado can't handle new connections.
Using multiple threads only hides the mistake; running Tornado with thousands of threads (so you can serve 1000s of connections simultaneously) would be very inefficient. The appropriate way is running a single thread which only blocks inside Tornado (on select or whatever Tornado's way of listening for events is) - not on your code (to be exact: never on your code).
The proper solution is to just return from get(self) right before time.sleep() (without calling self.finish()), like this:
class MainHandler(tornado.web.RequestHandler):
#tornado.web.asynchronous
def get(self):
print("Starting")
You must of course remember that this request is still open and call write() and finish() on it later.
I suggest you take a look at chat demo. Once you strip out the authentication you get a very nice example of async long polling server.
Since Tornado 5.0, asyncio is enabled automatically, so pretty much just changing time.sleep(4) to await asyncio.sleep(4) and #tornado.web.asynchronous def get(self): to async def get(self): solves the problem.
Example:
import tornado.ioloop
import tornado.web
import asyncio
class MainHandler(tornado.web.RequestHandler):
async def get(self):
print("Start request")
await asyncio.sleep(4)
print("Okay done now")
self.write("Howdy howdy howdy")
self.finish()
app = tornado.web.Application([
(r'/', MainHandler),
])
app.listen(8888)
tornado.ioloop.IOLoop.current().start()
Output:
Start request
Start request
Okay done now
Okay done now
Sources:
Tornado on asyncio
asyncio usage example
Basically we can call xmlrpc handlers following way:
import xmlrpclib
s = xmlrpclib.ServerProxy('http://remote_host/rpc/')
print s.system.listmethods()
In tornado we can integrate it like this:
import xmlrpclib
import tornado.web
s = xmlrpclib.ServerProxy('http://remote_host/rpc/')
class MyHandler(tornado.web.RequestHandler):
def get(self):
result = s.system.listmethods()
I have following, a little bit newbie, questions:
Will result = s.system.listmethods() block tornado?
Are there any non-blocking xmlrpc clients around?
How can we achieve result = yield gen.Task(s.system.listmethods)?
1.Yes it will block tornado, since xmlrpclib uses blocking python sockets (as it is)
2.Not that I'm aware of, but I'll provide a solution where you can keep xmlrpclib but have it async
3.My solution doesn't use tornado gen.
Ok, so one useful library to have at mind whenever you're doing networking and need to write async code is gevent, it's a really good high quality library that I would recommend to everyone.
Why is it good and easy to use ?
You can write asynchronous code in a synchronous manner (so that makes it easy)
All you have to do, to do so is monkey patch with one simple line :
from gevent import monkey; monkey.patch_all()
When using tornado you need to know two things (that you may already know) :
Tornado only supports asynchronous views when acting as a HTTPServer (WSGI isn't supported for async views)
Async views need to terminate the responses by themselves you do by using either self.finish() or self.render() (which calls self.finish())
Ok so here's an example illustrating what you would need with the necessary gevent integration with tornado :
# Python immports
import functools
# Tornado imports
import tornado.ioloop
import tornado.web
import tornado.httpserver
# XMLRpc imports
import xmlrpclib
# Asynchronous gevent decorator
def gasync(func):
#tornado.web.asynchronous
#functools.wraps(func)
def f(self, *args, **kwargs):
return gevent.spawn(func, self, *args, **kwargs)
return f
# Our XML RPC service
xml_service = xmlrpclib.ServerProxy('http://remote_host/rpc/')
class MyHandler(tornado.web.RequestHandler):
#gasync
def get(self):
# This doesn't block tornado thanks to gevent
# Which patches all of xmlrpclib's socket calls
# So they no longer are blocking
result = xml_service.system.listmethods()
# Do something here
# Write response to client
self.write('hello')
self.finish()
# Our URL Mappings
handlers = [
(r"/", MyHandler),
]
def main():
# Setup app and HTTP server
application = tornado.web.Application(handlers)
http_server = tornado.httpserver.HTTPServer(application)
http_server.listen(8000)
# Start ioloop
tornado.ioloop.IOLoop.instance().start()
if __name__ == "__main__":
main()
So give the example a try (adapt it to your needs obviously) and you should be good to go.
No need to write any extra code, gevent does all the work of patching up python sockets so they can be used asynchronously while still writing code in a synchronous fashion (which is a real bonus).
Hope this helps :)
I do not think so.
Because Tornado has it's own ioloop, but gevent's ioloop is libevent.
So gevent will block Tornado's ioloop.
I am working my way through learning Twisted, and have stumbled across something I'm not sure I'm terribly fond of - the "Twisted Command Prompt". I am fiddling around with Twisted on my Windows machine, and tried running the "Chat" example:
from twisted.protocols import basic
class MyChat(basic.LineReceiver):
def connectionMade(self):
print "Got new client!"
self.factory.clients.append(self)
def connectionLost(self, reason):
print "Lost a client!"
self.factory.clients.remove(self)
def lineReceived(self, line):
print "received", repr(line)
for c in self.factory.clients:
c.message(line)
def message(self, message):
self.transport.write(message + '\n')
from twisted.internet import protocol
from twisted.application import service, internet
factory = protocol.ServerFactory()
factory.protocol = MyChat
factory.clients = []
application = service.Application("chatserver")
internet.TCPServer(1025, factory).setServiceParent(application)
However, to run this application as a Twisted server, I have to run it via the "Twisted Command Prompt", with the command:
twistd -y chatserver.py
Is there any way to change the code (set Twisted configuration settings, etc) so that I can simply run it via:
python chatserver.py
I've Googled, but the search terms seem to be too vague to return any meaningful responses.
Thanks.
I don't know if it's the best way to do this but what I do is instead of:
application = service.Application("chatserver")
internet.TCPServer(1025, factory).setServiceParent(application)
you can do:
from twisted.internet import reactor
reactor.listenTCP(1025, factory)
reactor.run()
Sumarized if you want to have the two options (twistd and python):
if __name__ == '__main__':
from twisted.internet import reactor
reactor.listenTCP(1025, factory)
reactor.run()
else:
application = service.Application("chatserver")
internet.TCPServer(1025, factory).setServiceParent(application)
Hope it helps!
Don't confuse "Twisted" with "twistd". When you use "twistd", you are running the program with Python. "twistd" is a Python program that, among other things, can load an application from a .tac file (as you're doing here).
The "Twisted Command Prompt" is a Twisted installer-provided convenience to help out people on Windows. All it is doing is setting %PATH% to include the directory containing the "twistd" program. You could run twistd from a normal command prompt if you set your %PATH% properly or invoke it with the full path.
If you're not satisfied with this, perhaps you can expand your question to include a description of the problems you're having when using "twistd".
On windows you can create .bat file with your command in it, use full paths, then just click on it to start up.
For example I use:
runfileserver.bat:
C:\program_files\python26\Scripts\twistd.py -y C:\source\python\twisted\fileserver.tac
Maybe one of run or runApp in twisted.scripts.twistd modules will work for you. Please let me know if it does, it will be nice to know!
I haven't used twisted myself. However, you may try seeing if the twistd is a python file itself. I would take a guess that it is simply managing loading the appropriate twisted libraries from the correct path.
I am successfully using the simple Twisted Web server on Windows for Flask web sites.
Are others also successfully using Twisted on Windows, to validate that configuration?
new_app.py
if __name__ == "__main__":
reactor_args = {}
def run_twisted_wsgi():
from twisted.internet import reactor
from twisted.web.server import Site
from twisted.web.wsgi import WSGIResource
resource = WSGIResource(reactor, reactor.getThreadPool(), app)
site = Site(resource)
reactor.listenTCP(5000, site)
reactor.run(**reactor_args)
if app.debug:
# Disable twisted signal handlers in development only.
reactor_args['installSignalHandlers'] = 0
# Turn on auto reload.
import werkzeug.serving
run_twisted_wsgi = werkzeug.serving.run_with_reloader(run_twisted_wsgi)
run_twisted_wsgi()
old_app.py
if __name__ == "__main__":
app.run()