We Keep Coding

Do ping-pong at regular interval using Tornado websocket

I am new to Tornado server in python and trying to a ping-pong at regular interval to the clients connected.
I saw something called websocket_ping_interval in tornado documentation but couldn't find any example as to how/when to use it.
I did the below using ioloop.PeriodicCallback but doesn't seem to be doing any ping.
import tornado.web
from tornado import ioloop
from terminado import TermSocket, SingleTermManager
from tornado import websocket
# BaseWebSocketHandler removed, because we need to track all opened
# sockets in the class. You could change this later.
class MeterInfo(websocket.WebSocketHandler):
"""Establish an websocket connection and send meter readings."""
opened_sockets = []
previous_meter_reading = 0
def open(self):
self.write_message('Connection Established.')
MeterInfo.opened_sockets.append(self)
def on_close(self):
"""Close the connection."""
self.write_message('bye')
MeterInfo.opened_sockets.remove(self)
#classmethod
def try_send_new_reading(cls):
"""Send new reading to all connected clients"""
new_reading = "text"
if new_reading == cls.previous_meter_reading:
return
cls.previous_meter_reading = new_reading
for socket in cls.opened_sockets:
socket.write_message({'A': new_reading})
if __name__ == '__main__':
term_manager = SingleTermManager(shell_command=['bash'])
handlers = [
(r"/websocket", TermSocket, {'term_manager': term_manager}),
(r"/()", tornado.web.StaticFileHandler, {'path': 'index.html'}),
(r"/(.*)", tornado.web.StaticFileHandler, {'path': '.'}),
]
app = tornado.web.Application(handlers)
app.listen(8010)
METER_CHECK_INTERVAL = 100 # ms
ioloop.PeriodicCallback(MeterInfo.try_send_new_reading,METER_CHECK_INTERVAL).start()
ioloop.IOLoop.instance().start()
All I need to do is to keep pinging the clients connected with something.

websocket_ping_interval is an application setting, so you pass it to the Application constructor:
app = tornado.web.Application(handlers, websocket_ping_interval=10)

tornado - WebSocketClientConnection - How to catch and handle connection failures?

I am a Tornado and also Websocket newbie. There are many resources how to implement a websocket server application with Tornado. However, I haven't found a complete example that contains a websocket client application built on top of Tornado.
Server Application (server.py)
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import logging
import tornado.web
import tornado.websocket
import tornado.ioloop
import tornado.options
from tornado.options import define, options
define("port", default=3000, help="run on the given port", type=int)
class Application(tornado.web.Application):
def __init__(self):
handlers = [(r"/", MainHandler)]
settings = dict(debug=True)
tornado.web.Application.__init__(self, handlers, **settings)
class MainHandler(tornado.websocket.WebSocketHandler):
def check_origin(self, origin):
return True
def open(self):
logging.info("A client connected.")
def on_close(self):
logging.info("A device disconnected")
def main():
tornado.options.parse_command_line()
app = Application()
app.listen(options.port)
tornado.ioloop.IOLoop.instance().start()
if __name__ == "__main__":
main()
Client Application (client.py):
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from tornado.ioloop import IOLoop
from tornado.websocket import websocket_connect
if __name__ == "__main__":
ws = websocket_connect("ws://localhost:3000")
IOLoop.instance().start()
Problems & Questions
I don't have any problems with server application. However, I can't say same thing for client application. Why?
If server goes down, client application doesn't react. It does not say "Hey, server is down buddy.". I write my own WebSocketClientConnection class (inherited by the original) and also websocket_connect function (same function, but this one uses my WebSocketClientConnection class). I override on_connection_close method in my WebSocketClientConnection class, so I can catch "server down on middle of the connection" failure. However, I cannot make it reconnect. How do I make it reconnect to server?
If server is already down before the connection, client application does not raise anything and seems perfectly. I don't know how to catch that failure. How do I catch this connection failure?
My WebSocketClientConnection class:
class MyWebSocketClientConnection(tornado.websocket.WebSocketClientConnection):
def on_connection_close(self):
super(MyWebSocketClientConnection, self).on_connection_close()
print "connection closed"
Thanks.

By the way, after I figured out, I did write an example Tornado WebSocket client/server pair to demonstrate how to do it.
https://github.com/ilkerkesen/tornado-websocket-client-example
I hope it helps someone.

websocket_connect returns a Future; to see whether it succeeded or failed you must examine the Future (probably by yielding it in a coroutine). Furthermore, after establishing the connection you should go into a read_message loop. Even if you don't expect the client to send any messages, you should still call read_message: this is how you will connections that are closed after being established (read_message will return None).
#gen.coroutine
def my_websocket_client(url):
ws = yield websocket_connect(url)
while True:
msg = yield ws.read_message()
if msg is None: break
# do stuff with msg

Implement SSL in tornadoweb websockets server

Good Day to all,
I have a websocket server written in python that serves clients changes done to a postgresql database table. Presently it uses the standard ws protocol. I would Like to implement SSL wss but there seems to be no documentation on the tornadoweb site. Any help appreciated!
Code as follows:
import tornado.web
import tornado.websocket
import tornado.ioloop
import tornado.httpserver
import threading
import select
import psycopg2
import psycopg2.extensions
# This is a global variable to store all connected clients
websockets = []
# Connect to DB to listen for notifications
conn = psycopg2.connect("host=xxx.xxx.xxx.xxx dbname=mydb user=xxx port=yyy")
conn.set_isolation_level(psycopg2.extensions.ISOLATION_LEVEL_AUTOCOMMIT)
curs = conn.cursor()
curs.execute("LISTEN notifier;")
# Create the websocket
class WebSocketHandler(tornado.websocket.WebSocketHandler):
#tornado.web.asynchronous
# Add to global variable the connected client
def open(self):
self.set_nodelay(True)
# on disconnect remove client
def on_close(self):
# Search for it in the list. If it matches with us (the client) then remove it as he has quit the con
try:
for websocket in websockets:
if websocket[0] == self:
websockets.remove(websocket)
except ValueError:
print ValueError
def on_message(self, message):
if self not in websockets:
websockets.append([self,message])
def on_pong(self, data):
print data
# This is the function that polls the database for changes, then it sends change to clients through websockets
def db_listener():
while 1:
if select.select([conn],[],[],5) == ([],[],[]):
for websocket in websockets:
# ping the client regularly to avoid disconnect
websocket[0].ping("ping")
else:
conn.poll()
while conn.notifies:
notify = conn.notifies.pop()
details = notify.payload.split(",")
if len(details) > 1:
for websocket in websockets:
if details[34] in websocket[1]:
websocket[0].write_message(notify.payload)
application = tornado.web.Application([
(r"/websocket", WebSocketHandler),
])
if __name__ == "__main__":
# Start a separate thread for every client so that we do not block the main websockets program!
threading.Thread(target=db_listener).start()
application.listen(5252)
tornado.ioloop.IOLoop.instance().start()
Any suggestions?
Thanks to all!!

try this:
server = tornado.httpserver.HTTPServer(application, ssl_options = {
"certfile": "path-to-crt-file",
"keyfile": "path-to-key-file",
})
server.listen(5252)

Attaching ZMQStream with existing tornado ioloop

I have an application where every websocket connection (within tornado open callback) creates a zmq.SUB socket to an existing zmq.FORWARDER device. Idea is to receive data from zmq as callbacks, which can then be relayed to frontend clients over websocket connection.
https://gist.github.com/abhinavsingh/6378134
ws.py
import zmq
from zmq.eventloop import ioloop
from zmq.eventloop.zmqstream import ZMQStream
ioloop.install()
from tornado.websocket import WebSocketHandler
from tornado.web import Application
from tornado.ioloop import IOLoop
ioloop = IOLoop.instance()
class ZMQPubSub(object):
def __init__(self, callback):
self.callback = callback
def connect(self):
self.context = zmq.Context()
self.socket = self.context.socket(zmq.SUB)
self.socket.connect('tcp://127.0.0.1:5560')
self.stream = ZMQStream(self.socket)
self.stream.on_recv(self.callback)
def subscribe(self, channel_id):
self.socket.setsockopt(zmq.SUBSCRIBE, channel_id)
class MyWebSocket(WebSocketHandler):
def open(self):
self.pubsub = ZMQPubSub(self.on_data)
self.pubsub.connect()
self.pubsub.subscribe("session_id")
print 'ws opened'
def on_message(self, message):
print message
def on_close(self):
print 'ws closed'
def on_data(self, data):
print data
def main():
application = Application([(r'/channel', MyWebSocket)])
application.listen(10001)
print 'starting ws on port 10001'
ioloop.start()
if __name__ == '__main__':
main()
forwarder.py
import zmq
def main():
try:
context = zmq.Context(1)
frontend = context.socket(zmq.SUB)
frontend.bind('tcp://*:5559')
frontend.setsockopt(zmq.SUBSCRIBE, '')
backend = context.socket(zmq.PUB)
backend.bind('tcp://*:5560')
print 'starting zmq forwarder'
zmq.device(zmq.FORWARDER, frontend, backend)
except KeyboardInterrupt:
pass
except Exception as e:
logger.exception(e)
finally:
frontend.close()
backend.close()
context.term()
if __name__ == '__main__':
main()
publish.py
import zmq
if __name__ == '__main__':
context = zmq.Context()
socket = context.socket(zmq.PUB)
socket.connect('tcp://127.0.0.1:5559')
socket.send('session_id helloworld')
print 'sent data for channel session_id'
However, my ZMQPubSub class doesn't seem like is receiving any data at all.
I further experimented and realized that I need to call ioloop.IOLoop.instance().start() after registering on_recv callback within ZMQPubSub. But, that will just block the execution.
I also tried passing main.ioloop instance to ZMQStream constructor but doesn't help either.
Is there a way by which I can bind ZMQStream to existing main.ioloop instance without blocking flow within MyWebSocket.open?

In your now complete example, simply change frontend in your forwarder to a PULL socket and your publisher socket to PUSH, and it should behave as you expect.
The general principles of socket choice that are relevant here:
use PUB/SUB when you want to send a message to everyone who is ready to receive it (may be no one)
use PUSH/PULL when you want to send a message to exactly one peer, waiting for them to be ready
it may appear initially that you just want PUB-SUB, but once you start looking at each socket pair, you realize that they are very different. The frontend-websocket connection is definitely PUB-SUB - you may have zero-to-many receivers, and you just want to send messages to everyone who happens to be available when a message comes through. But the backend side is different - there is only one receiver, and it definitely wants every message from the publishers.
So there you have it - backend should be PULL and frontend PUB. All your sockets:
PUSH -> [PULL-PUB] -> SUB
publisher.py: socket is PUSH, connected to backend in device.py
forwarder.py: backend is PULL, frontend is PUB
ws.py: SUB connects and subscribes to forwarder.frontend.
The relevant behavior that makes PUB/SUB fail on the backend in your case is the slow joiner syndrome, which is described in The Guide. Essentially, subscribers take a finite time to tell publishers about there subscriptions, so if you send a message immediately after opening a PUB socket, the odds are it hasn't been told that it has any subscribers yet, so it's just discarding messages.

ZeroMq subscribers have to subscribe on what messages they wish to receive; I don't see that in your code. I believe the Python way is this:
self.socket.setsockopt(zmq.SUBSCRIBE, "")

Tornado server stuck in loop, not accepting clients

I've written a code in tornado that connects to a server that is pushing an infinite data stream, processes the data stream and sends it out on a websocket server.
The problem is that the way I implemented it the server became blocked on a particular function and doesn't accept any more clients since it never exits the function serving the data to the websocket. I want the connection to the server and the data retrieved from it processed only once but send the processed data to all the clients that connect to my tornado server. Could someone please help me, I can't figure out a way to do it. Here's my code with processing of data removed:
import socket
import ssl
import tornado.httpserver
import tornado.websocket
import tornado.ioloop
import tornado.web
websockets = []
class WSHandler(tornado.websocket.WebSocketHandler):
def readData(self):
while True:
line = self.ssl_sock.read()
#PROCESS THE READ LINE AND CONVERT INTO RESULTING DATA
if(toSend):
self.write_message(result)
def makeConnection(self):
self.ssl_sock.connect(self.address)
self.readData()
def open(self):
print 'New connection was opened'
self.s=socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.ssl_sock=ssl.wrap_socket(self.s, cert_reqs=ssl.CERT_NONE)
self.address=('SERVER_ADDRESS',5000)
self.nodes=[]
self.edges=[]
if self not in websockets:
print ('added')
websockets.append(self)
if(len(websockets)==1):
print('executing make conn')
self.makeConnection()
else:
self.readData()
print('executing read data')
def on_message(self, message):
print 'Incoming message:', message
self.write_message("You said: " + message)
def on_close(self):
print 'Connection was closed...'
application = tornado.web.Application([
(r'/ws', WSHandler),
])
if __name__ == "__main__":
http_server = tornado.httpserver.HTTPServer(application)
http_server.listen(8888)
tornado.ioloop.IOLoop.instance().start()

Tornado is an asynchronous framework, that is, all your IO must be run within its event loop, otherwise the whole server gets stuck.
Try having a look at Tornado Async Client.