We Keep Coding

How to send message when file changes detected? Twisted and Web Sockets

I am currently trying to create a small demo where I connect a web socket between my computer and my localhost ws://localhost:8080/ws. I want the web socket to monitor a file on my computer for changes. If there are changes, send a message. The connection and output is being monitored using Advanced Rest Client.
Is there a specific method I can use on the class to constantly check the contents of this file?
EDIT
I have implemented an observer using watchdog that detects any event for files in a specified directory. However, my message is not being sent in the sendFSEvent method and I also realized that my client isn't being registered when I connect to the web socket.
Here is my code in server.py
import sys
import os
from watchdog.observers import Observer
from twisted.web.static import File
from twisted.python import log
from twisted.web.server import Site
from twisted.internet import reactor, defer
from autobahn.twisted.websocket import WebSocketServerFactory, \
WebSocketServerProtocol, listenWS
from MessangerEventHandler import MessangerEventHandler
class WsProtocol(WebSocketServerProtocol):
def connectionMade(self):
print("Connection made")
WebSocketServerProtocol.connectionMade(self)
def onOpen(self):
WebSocketServerProtocol.onOpen(self)
print("WebSocket connection open")
def onMessage(self, payload, isBinary):
print("Message was: {}".format(payload))
self.sendMessage("message received")
def sendFSEvent(self, json):
WebSocketProtocol.sendMessage(self, json)
print('Sent FS event')
def onClose(self, wasClean, code, reason):
print("Connection closed: {}".format(reason))
WebSocketServerProtocol.onClose(self, wasClean, code, reason)
class WsServerFactory(WebSocketServerFactory):
protocol = WsProtocol
def __init__(self, url='ws://localhost', port=8080):
addr = url + ':' + str(port)
print("Listening on: {}".format(addr))
WebSocketServerFactory.__init__(self, addr)
self.clients = []
def register(self, client):
if not client in self.clients:
print("Registered client: {}".format(client))
self.clients.append(client)
def unregister(self, client):
if client in self.clients:
print("Unregistered client: {}".format(client))
self.clients.remove(client)
self._printConnected()
def _printConnected(self):
print("Connected clients:[")
def notify_clients(self, message):
print("Broadcasting: {}".format(message))
for c in self.clients:
c.sendFSEvent(message)
print("\nSent messages")
if __name__ == '__main__':
if len(sys.argv) < 2:
print("Usage: python server_defer.py <dirs>")
sys.exit(1)
log.startLogging(sys.stdout)
ffactory = WsServerFactory("ws://localhost", 8080)
ffactory.protocol = WsProtocol
listenWS(ffactory)
observers = []
for arg in sys.argv[1:]:
dir_path = os.path.abspath(arg)
if not os.path.exists(dir_path):
print('{} does not exist.'.format(dir_path))
sys.exit(1)
if not os.path.isdir(dir_path):
print('{} is not a directory.'.format(dir_path))
sys.exit(1)
# Check for and handle events
event_handler = MessangerEventHandler(ffactory, reactor, os.getcwd())
observer = Observer()
observer.schedule(event_handler, path=dir_path, recursive=True)
observer.start()
observers.append(observer)
try:
reactor.run()
except KeyboardInterrupt:
for obs in observers:
obs.stop()
reactor.stop()
print("\nGoodbye")
sys.exit(1)
Any help would be greatly appreciated.
Thank you,
Brian

Most enterprise distros come with inotify which is really well suited for monitoring files and directories. The basic idea is to capture a list of connected web socket clients as they connect. Then create a callback that will execute when a change occurs on the files you're monitoring. Within this callback, you can iterate the clients and send them a message like 'file: "blah/blah.txt" has changed'. It's a little wonky, but the code snippet should clear things up for you.
from functools import partial
from twisted.internet import inotify
from twisted.python import filepath
# the rest of your imports ...
class SomeServerProtocol(WebSocketServerProtocol):
def onConnect(self, request):
self.factory.append(self) # <== append this client to the list in the factory
def notification_callback(ignored, filepath, mask, ws_clients):
"""
function that will execute when files are modified
"""
payload = "event on {0}".format(filepath)
for client in ws_clients:
client.sendMessage(
payload.encode('utf8'), # <== don't forget to encode the str to bytes before sending!
isBinary = False)
if __name__ == '__main__':
root = File(".")
factory = WebSocketServerFactory(u"ws://127.0.01:8080")
factory.protocol = SomeServerProtocol
factory.clients = [] # <== create a container for the clients that connect
# inotify stuff
notify = partial(notification_callback, ws_clients=factory.clients) # <== use functools.partial to pass extra params
notifier = inotify.INotify()
notifier.startReading()
notifier.watch(filepath.FilePath("/some/directory"), callbacks=[notify])
# the rest of your code ...

Python import async tcp server as a module: data exchange not working, Multithreading with modules

I am working on the following project: I created a async tcp tcp ip client/ server connection. The client (c# wpf GUI) writes messages to the server (Raspberry pi, Python) and waits for sensor data. The server receive the message from the client and sends back the current sensor data in a loop.
The server/ client application standalone works fine. In the next step I want to use modules in Python. Here are the main parts shown that I want to implement in a seperate Python file:
Main.py
start the async tcp server and run it in the background (thread)
print the received data from the client
read sensor data (thread)
evaluation (optional)
hand over the sensor data to the async server
Here is the code in python that I have so fare (start the async tcp server):
Main.py
import thread
import logging
import socket
import asyncore
#Import async server
from AsyncServerPi_Test import Server
if __name__=='__main__':
logging.basicConfig(level=logging.DEBUG,format='%(name)s: %(message)s',)
log = logging.getLogger('main')
#parameter
ip = socket.gethostname()
port = 12345
address = (ip, port) # port 0 means the kernel gives port
dataFromPi= 'SensorData'
server = Server(address,dataFromPi) #issue with parameter dataFromPI
#dataFromClient = server.getDataFromClient() Code is not working!
#log.debug(dataFromGUI)
asyncore.loop(timeout=1) # checks all client's readable/writable every second
AsyncServerPi_Test.py
import asyncore
import logging
import time
import socket
# Create one or more network channels -- instances of class asyncore.dispatcher.
# These channels are automatically added to a global map, used by the loop() function.
class Server(asyncore.dispatcher):
"""Receives connections and establishes handlers for each client."""
def __init__(self, address, dataFromPi):
self.dataFromPi = dataFromPi
self.logger = logging.getLogger('Server')
asyncore.dispatcher.__init__(self)
self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
self.bind(address)
self.address = self.socket.getsockname()
self.logger.debug('binding to %s', self.address)
self.listen(1)
def timein(self):
self.start_time = time.time()
self.logger.debug('init start_time -> %e', self.start_time)
def handle_accept(self):
# Called when a client connects to our socket
client_info = self.accept()
if client_info is not None:
# start the timeout clock right away
self.timein()
self.logger.debug('handle_accept() -> %s', client_info[1])
Handler(client_info[0], self.start_time, self.dataFromPi)
def handle_close(self):
self.logger.debug('server_handle_close()')
self.close()
class Handler(asyncore.dispatcher):
"""Handles echoing messages from a single client. """
def __init__(self, sock, start_time, dataFromPi, chunk_size=1024):
self.dataFromPi = dataFromPi
self.start_time = start_time
# send data per 5 seconds
self.timeout = 5
self.chunk_size = chunk_size
self.logger = logging.getLogger('Handler%s' % str(sock.getsockname()))
asyncore.dispatcher.__init__(self, sock=sock)
self.data_to_write = []
def timeout_check(self):
#Time event
delta_t = time.time() - self.start_time
if delta_t > self.timeout:
self.logger.debug('timeout! -> %e %e', delta_t, self.timeout)
return True
else:
self.logger.debug('no timeout -> %e %e', delta_t, self.timeout)
return False
def trigger_close(self):
return self.timeout_check()
def writable(self):
"""We want to write if we have received data and when sendSensor data is on."""
# the trigger_close here is a hack
response = bool(self.data_to_write) or self.trigger_close()
self.logger.debug('writable() -> %s', response)
return response
def handle_write(self):
"""Write as much as possible of the reversed recent message we received."""
self.logger.debug('write data to GUI')
if self.trigger_close(): # hack to timeout socket
self.start_time = time.time()
#data = self.data_to_write.pop()
sent = self.send(dataFromPi)
return
data = self.data_to_write.pop()
sent = self.send(dataFromPi)
self.logger.debug('Send data to GUI:'+ str(sent))
#if sent < len(data)
#remaining = data[sent:]
#self.data_to_write.append(remaining)
#self.logger.debug('handle_write() -> (%d) "%s"', sent, data[:sent])
#if not self.writable(): # empty recv
#self.handle_close()
def reverse(self, s):
s = list(s)
s.reverse()
s = "".join(s)
return s
def handle_read(self):
"""Read an incoming message from the client and put it into our outgoing queue."""
data = self.recv(self.chunk_size)
self.logger.debug('handle_read() -> (%d) "%s"', len(data), data)
data = self.reverse(data)
self.data_to_write.insert(0, data)
#return self.data --> didnt work!
def handle_close(self):
"""server close only gets called if client decides or after timeout"""
self.logger.debug('handle_close()')
self.close()
I have some issues with the parameter dataFromPi and dataFromClient. dataFromPi is a input parameter that I handover the instance of the server (server = Server(address,dataFromPi)). I get the following error when I execute the main.py file:
error: uncaptured python exception, closing channel <AsyncServerPi_Test.Handler connected 192.168.0.108:51028 at 0x31ba7d8> (<type 'exceptions.NameError'>:global name 'dataFromPi' is not defined [C:\Python27\lib\asyncore.py|write|91] [C:\Python27\lib\asyncore.py|handle_write_event|468] [C:\Python27\lib\AsyncServerPi_Test.py|handle_write|75])
Handler('192.168.0.108', 12345): handle_close()
I checked the handover of dataFromPi in the AsyncServerPi_Test.py file but I couldnt find a solution.
The next issue I have is that I am not able to print the dataFromClient in the main.py file. Within the AsyncServerPi_Test.py file I can read the dataFromClient (handle_read). I tried to return the received data in handle_read, but I think the return of the data must happen in the class server(). How can I archieve that the server class returns the data from the client so I can print the received data in main.py?
Furthermore the server should run in the background so that I can run other operations parallel (read sensor data from ultrasonic, evaluation...). As an example I want that the main.py should look similar like that:
import thread
import logging
import socket
import asyncore
#Import US
import Ultrasonic
#Import async tcp ip server
from AsyncServerPi_Test import Server
def main():
#Logging
logging.basicConfig(level=logging.DEBUG,format='%(name)s: %(message)s',)
log = logging.getLogger('main')
#Parameter for server
ip = socket.gethostname()
port = 12345
address = (ip, port) # port 0 means the kernel gives port
ultrasonicTest = Ultrasonic() # run in a seperate thread?
dataFromSensor = ultrasonicTest.readSensorData
serverTest = Server(address,dataFromSensor) # run in a seperate thread?
dataFromClient = serverTest.receivedData()
asyncore.loop(timeout=1) # checks all client's readable/writable every second
if __name__=='__main__':
main()
In this context I have the following questions:
The instance of the server (serverTest = Server(address,dataFromSensor)) must run in a seperate thread?
Create a new thread for the sensor (ultrasonicTest = Ultrasonic()):
Is it better to run threads within a module (e.g. start threading within the ultrasonic.py and not in the main.py)
Synchronize the threads?
I am relatie new in python and I dont have much experience in multithreading. It would be helpful if someone has done some similar projects in python and could give me some tips or reference to similar projects.

Python SSH server (twisted.conch) command filtering and port forwarding

I need to create an SSH server (twisted.conch has been chosen for the job) which would do the following:
Perform port forwarding (the attached code does not do that and I do
not know what to modify)
Filter commands BEFORE they are executed (or at least log them before or after).
The code attached below creates a perfect SSH and SFTP server, BUT it is missing one main component - port forwarding (and command filtering, but that is not as important as port forwarding)
I searched where I possibly could, but could not find these two.. Please help me out, - it is the last peace of the puzzle.
#!/usr/bin/env python
from twisted.conch.unix import UnixSSHRealm
from twisted.cred.portal import Portal
from twisted.cred.credentials import IUsernamePassword
from twisted.cred.checkers import ICredentialsChecker
from twisted.cred.error import UnauthorizedLogin
from twisted.conch.ssh.factory import SSHFactory
from twisted.internet import reactor, defer
from twisted.conch.ssh.transport import SSHServerTransport
from twisted.conch.ssh.userauth import SSHUserAuthServer
from twisted.conch.ssh.connection import SSHConnection
from twisted.conch.ssh.keys import Key
from zope.interface import implements
from subprocess import Popen,PIPE
from crypt import crypt
publicKey = 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAGEArzJx8OYOnJmzf4tfBEvLi8DVPrJ3/c9k2I/Az64fxjHf9imyRJbixtQhlH9lfNjUIx+4LmrJH5QNRsFporcHDKOTwTTYLh5KmRpslkYHRivcJSkbh/C+BR3utDS555mV'
privateKey = """-----BEGIN RSA PRIVATE KEY-----
MIIByAIBAAJhAK8ycfDmDpyZs3+LXwRLy4vA1T6yd/3PZNiPwM+uH8Yx3/YpskSW
4sbUIZR/ZXzY1CMfuC5qyR+UDUbBaaK3Bwyjk8E02C4eSpkabJZGB0Yr3CUpG4fw
vgUd7rQ0ueeZlQIBIwJgbh+1VZfr7WftK5lu7MHtqE1S1vPWZQYE3+VUn8yJADyb
Z4fsZaCrzW9lkIqXkE3GIY+ojdhZhkO1gbG0118sIgphwSWKRxK0mvh6ERxKqIt1
xJEJO74EykXZV4oNJ8sjAjEA3J9r2ZghVhGN6V8DnQrTk24Td0E8hU8AcP0FVP+8
PQm/g/aXf2QQkQT+omdHVEJrAjEAy0pL0EBH6EVS98evDCBtQw22OZT52qXlAwZ2
gyTriKFVoqjeEjt3SZKKqXHSApP/AjBLpF99zcJJZRq2abgYlf9lv1chkrWqDHUu
DZttmYJeEfiFBBavVYIF1dOlZT0G8jMCMBc7sOSZodFnAiryP+Qg9otSBjJ3bQML
pSTqy7c3a2AScC/YyOwkDaICHnnD3XyjMwIxALRzl0tQEKMXs6hH8ToUdlLROCrP
EhQ0wahUTCk1gKA4uPD6TMTChavbh4K63OvbKg==
-----END RSA PRIVATE KEY-----"""
# check if username/password is valid
def checkPassword(username,password):
try:
ret=False
if username and password:
output=Popen(["grep",username,"/etc/shadow"],stdout=PIPE,stderr=PIPE).communicate()[0]
hash=""
if output:
tmp=output.split(":")
if tmp>=2:
hash=tmp[1]
del tmp
ret=crypt(password,hash)==hash
del output,hash
except Exception,e:
ret=False
return ret
# authorization methods
class XSSHAuth(object):
credentialInterfaces=IUsernamePassword,implements(ICredentialsChecker)
def requestAvatarId(self, credentials):
#print "Credentials:",credentials.username,credentials.password
if credentials.username=="root" and credentials.password and checkPassword(credentials.username,credentials.password):
# successful authorization
return defer.succeed(credentials.username)
# failed authorization
return defer.fail(UnauthorizedLogin("invalid password"))
class XSSHUserAuthServer(SSHUserAuthServer):
def _ebPassword(self, reason):
addr = self.transport.getPeer().address
if addr.host!="3.22.116.85" and addr.host!="127.0.0.1":
p1 = Popen(["iptables","-I","INPUT","-s",addr.host,"-j","DROP"], stdout=PIPE, stderr=PIPE)
p1.communicate()
print(addr.host, addr.port, self.user, self.method)
self.transport.loseConnection()
return defer.fail(UnauthorizedLogin("invalid password"))
# the transport class - we use it to log MOST OF THE ACTIONS executed thru the server
class XSSHTransport(SSHServerTransport):
ourVersionString="SSH-2.0-X"
logCommand=""
def connectionMade(self):
print "Connection made",self.getPeer()
SSHServerTransport.connectionMade(self)
#self.transport.loseConnection()
def connectionLost(self,reason):
print "Connection closed",self.getPeer()
SSHServerTransport.connectionLost(self,reason)
def dataReceived(self, data):
SSHServerTransport.dataReceived(self,data)
def dispatchMessage(self, messageNum, payload):
SSHServerTransport.dispatchMessage(self,messageNum,payload)
# start the server
class XSSHFactory(SSHFactory):
protocol=XSSHTransport
factory = XSSHFactory()
factory.publicKeys = {'ssh-rsa': Key.fromString(data=publicKey)}
factory.privateKeys = {'ssh-rsa': Key.fromString(data=privateKey)}
factory.services = {
'ssh-userauth': XSSHUserAuthServer,
'ssh-connection': SSHConnection
}
portal=Portal(UnixSSHRealm())
portal.registerChecker(XSSHAuth())
factory.portal=portal
reactor.listenTCP(22, factory)
reactor.run()

Since you are using UnixConchUser which implements global_tcpip_forward, it does in fact work. When I run your example and connect to it with ssh -L4321:remote.host:1234 root#localhost -p 2222 and then telnet localhost 4321, I get tunneled through to remote.host 1234. You will have to state your problem in more detail.

commands log may be can done in dataReceived(self, data):
def dataReceived(self, data):
SSHServerTransport.dataReceived(self,data)
self.buf += data
if data == '\r':
cmd = self.buf
self.buf = ''
But it can't handle the delete key, tab, arrow-up, arrow-down, and other special characters well. I want to know how you get the command last.

How to detect the server closing a unix domain socket?

I'm messing around with the python twisted library, and I can't seem to figure out how to get my client to detect a server closing its socket. My client continues to let me send data to the non existent server. Here is my server:
test_server.py
from twisted.internet import protocol, reactor, endpoints, stdio
from twisted.protocols.basic import LineReceiver
class ConnectionProtocol(LineReceiver):
from os import linesep as delimiter
def lineReceived(self, line):
print 'got line: %s' % line
self.sendLine(line)
class ConnectionFactory(protocol.Factory):
def buildProtocol(self, addr):
return ConnectionProtocol()
def main():
endpoint = endpoints.UNIXServerEndpoint(reactor, './test.sock', 5, 0777, False)
endpoint.listen(ConnectionFactory())
print 'starting the reactor'
reactor.run()
main()
All it does is send each line it gets back to the connecting client.
Here is the client:
test_client.py
import os
from twisted.internet import protocol, reactor, endpoints, stdio
from twisted.protocols.basic import LineReceiver
class CommandProtocol(protocol.Protocol):
def dataReceived(self, data):
print data,
class StdinProtocol(LineReceiver):
from os import linesep as delimiter
def __init__(self, client):
self._client = client
def connectionMade(self):
self.transport.write('>>> ')
def lineReceived(self, line):
print 'writing line: %s' % line
self._client.transport.write(line + os.linesep)
def printError(failure):
print str(failure)
def main():
point = endpoints.UNIXClientEndpoint(reactor, './test.sock')
proto = CommandProtocol()
d = endpoints.connectProtocol(point, proto)
d.addErrback(printError)
stdio.StandardIO(StdinProtocol(proto))
reactor.run()
main()
If I run the server and then the client, and then kill the server, the client still writes to the CommandProtocol's transport like nothing happened. I thought the errback function would at least report something. In the case where the client is run before the server, the errback function is called with a ConnectError, but I'm specifically looking to detect the situation where the client has already connected to a server, and the server exits.
How can I detect that the server has shutdown?

ITransport.write is a silent no-op if called on a transport that has been disconnected.
If you want to learn that a connection was lost, override the connectionLost method. Once you know that the connection has been lost you can arrange for your program to stop accepting input or do something else with the input it receives.

i guess os.stat method and stat module can help you. Can you change your client codes to like below:
import os
import stat
from twisted.internet import protocol, reactor, endpoints, stdio
from twisted.protocols.basic import LineReceiver
class CommandProtocol(protocol.Protocol):
def dataReceived(self, data):
print data,
class StdinProtocol(LineReceiver):
from os import linesep as delimiter
def __init__(self, client):
self._client = client
def connectionMade(self):
self.transport.write('>>> ')
def lineReceived(self, line):
print 'writing line: %s' % line
self._client.transport.write(line + os.linesep)
def printError(failure):
print str(failure)
def main():
if os.stat('./test.sock').st_mode & (stat.S_IRGRP | stat.S_IRUSR | stat.S_IROTH):
print "1"
print "using control socket"
point = endpoints.UNIXClientEndpoint(reactor, './test.sock')
proto = CommandProtocol()
d = endpoints.connectProtocol(point, proto)
d.addErrback(printError)
stdio.StandardIO(StdinProtocol(proto))
reactor.run()
else:
print "not ready"
main()

Using asyncore to create interactive sessions with client/server model

I am attempting to create a program that allows many clients to connect to 1 server simultaneously. These connections should be interactive on the server side, meaning that I can send requests from the server to the client, after the client has connected.
The following asyncore example code simply replies back with an echo, I need instead of an echo a way to interactively access each session. Somehow background each connection until I decided to interact with it. If I have 100 sessions I would like to chose a particular one or choose all of them or a subset of them to send a command to. Also I am not 100% sure that the asyncore lib is the way to go here, any help is appreciated.
import asyncore
import socket
class EchoHandler(asyncore.dispatcher_with_send):
def handle_read(self):
data = self.recv(8192)
if data:
self.send(data)
class EchoServer(asyncore.dispatcher):
def __init__(self, host, port):
asyncore.dispatcher.__init__(self)
self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
self.set_reuse_addr()
self.bind((host, port))
self.listen(5)
def handle_accept(self):
pair = self.accept()
if pair is not None:
sock, addr = pair
print 'Incoming connection from %s' % repr(addr)
handler = EchoHandler(sock)
server = EchoServer('localhost', 8080)
asyncore.loop()

Here's a Twisted server:
import sys
from twisted.internet.task import react
from twisted.internet.endpoints import serverFromString
from twisted.internet.defer import Deferred
from twisted.internet.protocol import Factory
from twisted.protocols.basic import LineReceiver
class HubConnection(LineReceiver, object):
def __init__(self, hub):
self.name = b'unknown'
self.hub = hub
def connectionMade(self):
self.hub.append(self)
def lineReceived(self, line):
words = line.split(" ", 1)
if words[0] == b'identify':
self.name = words[1]
else:
for connection in self.hub:
connection.sendLine("<{}> {}".format(
self.name, line
).encode("utf-8"))
def connectionLost(self, reason):
self.hub.remove(self)
def main(reactor, listen="tcp:4321"):
hub = []
endpoint = serverFromString(reactor, listen)
endpoint.listen(Factory.forProtocol(lambda: HubConnection(hub)))
return Deferred()
react(main, sys.argv[1:])
and command-line client:
import sys
from twisted.internet.task import react
from twisted.internet.endpoints import clientFromString
from twisted.internet.defer import Deferred, inlineCallbacks
from twisted.internet.protocol import Factory
from twisted.internet.stdio import StandardIO
from twisted.protocols.basic import LineReceiver
from twisted.internet.fdesc import setBlocking
class HubClient(LineReceiver):
def __init__(self, name, output):
self.name = name
self.output = output
def lineReceived(self, line):
self.output.transport.write(line + b"\n")
def connectionMade(self):
self.sendLine("identify {}".format(self.name).encode("utf-8"))
def say(self, words):
self.sendLine("say {}".format(words).encode("utf-8"))
class TerminalInput(LineReceiver, object):
delimiter = "\n"
hubClient = None
def lineReceived(self, line):
if self.hubClient is None:
self.output.transport.write("Connecting, please wait...\n")
else:
self.hubClient.sendLine(line)
#inlineCallbacks
def main(reactor, name, connect="tcp:localhost:4321"):
endpoint = clientFromString(reactor, connect)
terminalInput = TerminalInput()
StandardIO(terminalInput)
setBlocking(0)
hubClient = yield endpoint.connect(
Factory.forProtocol(lambda: HubClient(name, terminalInput))
)
terminalInput.transport.write("Connecting...\n")
terminalInput.hubClient = hubClient
terminalInput.transport.write("Connected.\n")
yield Deferred()
react(main, sys.argv[1:])
which implement a basic chat server. Hopefully the code is fairly self-explanatory; you can run it to test with python hub_server.py in one terminal, python hub_client.py alice in a second and python hub_client.py bob in a third; then type into alice and bob's sessions and you can see what it does.

Review Requirements
You want
remote calls in client/server manner
probably using TCP communication
using sessions in the call
It is not very clear, how you really want to use sessions, so I will consider, that session is just one of call parameters, which has some meaning on server as well client side and will skip implementing it.
zmq as easy and reliable remote messaging platform
ZeroMQ is lightweight messaging platform, which does not require complex server infrastructure. It can handle many messaging patterns, following example showing request/reply pattern using multipart messages.
There are many alternatives, you can use simple messages encoded to some format like JSON and skip using multipart messages.
server.py
import zmq
class ZmqServer(object):
def __init__(self, url="tcp://*:5555"):
context = zmq.Context()
self.sock = context.socket(zmq.REP)
self.sock.bind(url)
self.go_on = False
def echo(self, message, priority=None):
priority = priority or "not urgent"
msg = "Echo your {priority} message: '{message}'"
return msg.format(**locals())
def run(self):
self.go_on = True
while self.go_on:
args = self.sock.recv_multipart()
if 1 <= len(args) <= 2:
code = "200"
resp = self.echo(*args)
else:
code = "401"
resp = "Bad request, 1-2 arguments expected."
self.sock.send_multipart([code, resp])
def stop(self):
self.go_on = False
if __name__ == "__main__":
ZmqServer().run()
client.py
import zmq
import time
class ZmqClient(object):
def __init__(self, url="tcp://localhost:5555"):
context = zmq.Context()
self.socket = context.socket(zmq.REQ)
self.socket.connect(url)
def call_echo(self, message, priority=None):
args = [message]
if priority:
args.append(priority)
self.socket.send_multipart(args)
code, resp = self.socket.recv_multipart()
assert code == "200"
return resp
def too_long_call(self, message, priority, extrapriority):
args = [message, priority, extrapriority]
self.socket.send_multipart(args)
code, resp = self.socket.recv_multipart()
assert code == "401"
return resp
def test_server(self):
print "------------------"
rqmsg = "Hi There"
print "rqmsg", rqmsg
print "response", self.call_echo(rqmsg)
print "------------------"
time.sleep(2)
rqmsg = ["Hi There", "very URGENT"]
print "rqmsg", rqmsg
print "response", self.call_echo(*rqmsg)
print "------------------"
time.sleep(2)
time.sleep(2)
rqmsg = []
print "too_short_call"
print "response", self.too_long_call("STOP", "VERY URGENT", "TOO URGENT")
print "------------------"
if __name__ == "__main__":
ZmqClient().test_server()
Play with the toy
Start the server:
$ python server.py
Now it runs and awaits requests.
Now start the client:
$ python client.py
------------------
rqmsg Hi There
response Echo your not urgent message: 'Hi There'
------------------
rqmsg ['Hi There', 'very URGENT']
response Echo your very URGENT message: 'Hi There'
------------------
too_short_call
response Bad request, 1-2 arguments expected.
------------------
Now experiment a bit:
start client first, then server
stop the server during processing, restart later on
start multiple clients
All these scenarios shall be handled by zmq without adding extra lines of Python code.
Conclusions
ZeroMQ provides very convenient remote messaging solution, try counting lines of messaging related code and compare with any other solution, providing the same level of stability.
Sessions (which were part of OP) can be considered just extra parameter of the call. As we saw, multiple parameters are not a problem.
Maintaining sessions, different backends can be used, they could live in memory (for single server instance), in database, or in memcache or Redis. This answer does not elaborate further on sessions, as it is not much clear, what use is expected.