We Keep Coding

Python-twisted: Trying to make UDP and websockets work together?

I have a websocket server written using twisted and autobahn. It is an echo server, I want to add the functionality of forwarding messages received from an multicast UDP port to the clients of the websocket server.
I tried what I did for the exact same functionality in a tcp server here but that doesn't seem to work.
class SomeServerProtocol(WebSocketServerProtocol):
def onOpen(self):
self.factory.register(self)
# Adding this line in TCP protocol's on connection method worked.
self.port = reactor.listenMulticast(6027, Listener(self), listenMultiple=True)
def connectionLost(self, reason):
self.factory.unregister(self)
def onMessage(self, payload, isBinary):
self.sendMessage(payload)
class PriceListener(DatagramProtocol):
def __init__(self, stream):
self.stream = stream
def startProtocol(self):
self.transport.setTTL(5)
self.transport.joinGroup("0.0.0.0")
def datagramReceived(self, datagram, address):
# Do some processing
# Send the data
class SomeServerFactory(WebSocketServerFactory):
def __init__(self, *args, **kwargs):
super(SomeServerFactory, self).__init__(*args, **kwargs)
self.clients = {}
def register(self, client):
self.clients[client.peer] = {"object": client, "id": k}
def unregister(self, client):
self.clients.pop(client.peer)
if __name__ == "__main__":
log.startLogging(sys.stdout)
# static file server seving index.html as root
root = File(".")
factory = SomeServerFactory(u"ws://127.0.0.1:8080")
factory.protocol = SomeServerProtocol
resource = WebSocketResource(factory)
# websockets resource on "/ws" path
root.putChild(u"ws", resource)
site = Site(root)
reactor.listenTCP(8080, site)
reactor.run()
I have marked in the SomeServerProtocol the line I added to have it listen on the UDP line. On removing this line everything works fine. I am getting data on UDP line, I want to push whatever data comes on the UDP line to all the clients connected to the websocket server.
I have already checked that the server works and clients are able to connect.
How do I do this? Also it would be great if one could clarify why does the TCP solution wouldn't work here.
PS
I am getting the following error on the client side.
WebSocket connection to 'ws://localhost:8080/ws' failed: One or more reserved bits are on: reserved1 = 1, reserved2 = 0, reserved3 = 1
I understood something about this issue here. So is the receiving of data from multicast inside the websocket protocol causing this?

Twisted: How to send messages by twisted client on single port?

I may send messages to server from twisted client by calling connector.connect(). But clients will be made on different ports. Following code is demonstrated this case:
SERVER_HOST = 'localhost'
SERVER_PORT = '5000'
class EchoClient(protocol.Protocol):
def connectionMade(self):
self.transport.write("message")
self.transport.loseConnection()
class EchoFactory(protocol.ClientFactory):
def buildProtocol(self, addr):
print('Connected.')
return EchoClient()
def clientConnectionLost(self, connector, reason):
print('Lost connection. Reason:', reason)
connector.connect()
def main():
reactor.connectTCP(SERVER_HOST, SERVER_PORT, EchoFactory())
reactor.run()
And my twisted server say me:
Packet received, client 127.0.0.1:41930, size: 7
Connection lost
Packet received, client 127.0.0.1:41931, size: 7
Connection lost
Packet received, client 127.0.0.1:41932, size: 7
Connection lost
Packet received, client 127.0.0.1:41933, size: 7
Clients has different ports - 41930, 41931, etc. How send messages from twisted client with single port?

You can use the bindAddress parameter in either connectTCP, clientFromString, TCP4ClientEndpoint, or TCP6ClientEndpoint. Using your example, your code would look like:
reactor.connectTCP(SERVER_HOST, SERVER_PORT, EchoFactory(), bindAddress=('127.0.0.1',9999))
I would advise you to avoid this if it's not absolutely necessary because the port may be in use by another process and will cause an exception. It's better for the OS to chose the ip:port for your app to bind to.

How do I wirelessly connect to a computer using python

I am wondering if there is any way to wirelessly connect to a computer/server using python's socket library. The dir(socket) brought up a lot of stuff and I wanted help sorting it out.

but one question. Is the socket server specific to python, or can
another language host and python connect or vise-versa?
As long as you are using sockets - you can connect to any socket-based server (made with any language). And vice-versa: any socket-based client will be able to connect to your server. Moreover it's cross-platform: socket-based client from any OS can connect to any socket-based server (from any OS).

It is unclear of what you mean by "Connect to a computer" so I gave you a TCP socket server and client.
Create a socket server on the computer you wish to "connect to" with:
import SocketServer
class MyTCPHandler(SocketServer.BaseRequestHandler):
def handle(self):
self.data = self.request.recv(1024).strip()
print "{} wrote:".format(self.client_address[0])
print self.data
self.request.sendall(self.data.upper())
if __name__ == "__main__":
HOST, PORT = "localhost", 9999
# Create the server, binding to localhost on port 9999
server = SocketServer.TCPServer((HOST, PORT), MyTCPHandler)
server.serve_forever()
Now create the client:
import socket
import sys
HOST, PORT = "localhost", 9999
data = " ".join(sys.argv[1:])
(SOCK_STREAM means a TCP socket)
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
sock.connect((HOST, PORT))
sock.sendall(data + "\n")
received = sock.recv(1024)
finally:
sock.close()
print "Sent: {}".format(data)
print "Received: {}".format(received)
You run the server and then the client and the server should receive the client's connection and send it whatever you have as the data variable on the server. Source: https://docs.python.org/2/library/socketserver.html

Socket disconnected after a message, (Connection refused by remote host), using python

I have uploaded my python socket, (cloud service project), on azure ,and when ever I connected to Hercules client side socket ....after a message or two, connection closed by remote host... forcefully...?
Server Code
from twisted.internet.protocol import Factory, Protocol
from twisted.internet import reactor
import SocketServer
class IphoneChat(Protocol):
def connectionMade(self):
self.factory.clients.append(self)
print('clients are'), self.factory.clients
def connectionLost(self, reason):
self.factory.clients.remove(self)
def dataReceived(self, data):
msg = ""
msg = data.strip()
for c in self.factory.clients:
c.message(msg)
def message(self, message):
self.transport.write(message + '\n')
print ('Iphone Chat server startedd')
factory = Factory()
factory.protocol = IphoneChat
factory.clients = []
reactor.listenTCP(9073, factory)
reactor.run()

I tried to reproduce your issue on my environment, but failed. Base on my experience, we often pay attention to 3 points if we use socket in azure worker role.
Open input TCP port. If we open a port as listener port, we'd better to add this port into the endpoints setting.
Considered the worker role status, I suggest we can code logic into the while True loop function, as following,
while True:
reactor.listenTCP(65534, f)
print "server run"
reactor.run()
sleep(1.0)
According to the error message, I guess the reason is that azure load balancer will kill the idle connection in 240s. I recommend you can refer to this blog to configure your idleTimeoutInMinutes value.
Please try to check your project and configuration. Any further findings, please let me know.

Python TCP socket doesn't close?

Maybe someone here will have a response for this thing which is just driving me insane.
To make it simple, I'm making a kind of proxy. Whenever it receives something, it forwards everything to a server, and sends back the response. So there is one socket always listening on port 4557 for clients, and for each incoming connection, there is a new socket created on a random port to connect to the server port 4556.
Clients <==> Proxy <==> Server
Also, there another socket which is instantiated and listening for requests coming from the server and to be forwarded to the corresponding client.
Here is an example:
Client A connects to proxy on port 4557
Proxy creates a socket to Server on port 4556
Along with that, it creates a socket listening on port 40100
Client sends stuff, forwarded to Server
Client disconnects. Close client connection and socket to server
Some time later, Server sends stuff to proxy on port 40100
Everything's forwarded to Client A (port 40100 corresponding to Client A)
And so on..
So far in my tests, I use a simple python script for sending a unique tcp packet to the proxy, along with a dump server showing received data and echoing back.
So the issue is that when a connection to the proxy is closed, the connection to the Server should also be closed with "sock.close()". However it just seems to be completely ignored. The socket remains as ESTABLISHED.
About the code now.
A few notes.
DTN and Node are respectively Server and Clients.
runCallback is called in a loop until thread dies.
finalCallback is called when the thread is dying.
Associations between remote hosts (Client), proxy ports (to Server) and proxies are kept in the dictionaries: TCPProxyHostRegister (RemoteHost => Proxy), TCPProxyPortRegister (Port => Proxy), TCPPortToHost (Port => RemoteHost).
The first class is TCPListenerThread.
It just listen on a specific port and instantiate proxies (one for each Client=>Server couple and Server=>Client couple) and forward them connections.
class TCPListenerThread(StoppableThread):
def __init__(self, tcp_port):
StoppableThread.__init__(self)
self.tcp_port = tcp_port
self.sock = socket.socket( socket.AF_INET, # Internet
socket.SOCK_STREAM ) # tcp
self.sock.bind( (LOCAL_ADDRESS, self.tcp_port) )
self.sock.listen(1)
def runCallback(self):
print "Listen on "+str(self.tcp_port)+".."
conn, addr = self.sock.accept()
if isFromDTN(addr):
tcpProxy = getProxyFromPort(tcp_port)
if not tcpProxy:
tcpProxy = TCPProxy(host, True)
else:
host = addr[0]
tcpProxy = getProxyFromHost(host)
if not tcpProxy:
tcpProxy = TCPProxy(host, False)
tcpProxy.handle(conn)
def finalCallback(self):
self.sock.close()
Now comes the TCP Proxy:
It associates a remote host (Client) with a port connecting to Server.
If it's a connection coming from a new Client, it will create a new listener (see above) for the Server and create a socket ready to forward everything to Server.
class TCPProxy():
def __init__(self, remote, isFromDTN):
#remote = port for Server or Remote host for Client
self.isFromDTN = isFromDTN
self.conn = None
#add itself to proxy registries
#If listening from a node
if not isFromDTN:
#Set node remote host
self.remoteHost = remote
TCPProxyHostRegister[self.remoteHost] = self
#Set port to DTN interface + listener
self.portToDTN = getNewTCPPort()
TCPPortToHost[self.portToDTN] = self.remoteHost
newTCPListenerThread(self.portToDTN)
#Or from DTN
else:
self.portToDTN = remote
TCPProxyPortRegister[self.portToDTN] = self
self.remoteHost = getRemoteHostFromPortTCP(self.portToDTN)
def handle(self, conn):
print "New connection!"
#shouldn't happen, but eh
if self.conn != None:
self.closeConnections()
self.conn = conn
#init socket with remote
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
#self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
if self.isFromDTN:
self.sock.connect((self.remoteHost, 4556)) #TODO: handle dynamic port..
else:
self.sock.connect((DTN_Address, DTN_TCPPort))
#handle connection in a thread
self.handlerThread = newTCPHandlerThread(self)
#handle reply in a therad
self.replyThread = newTCPReplyThread(self)
def closeConnections(self):
try:
if self.conn != None:
print "Close connections!"
self.sock.close()
self.conn.close()
self.conn = None
self.handlerThread.kill()
self.replyThread.kill()
except Exception, err:
print str(err)
#pass
def forward(self, data):
print "TCP forwarding data: "+data
self.sock.send(data)
def forwardBack(self, data):
print "TCP forwarding data back: "+data
self.conn.send(data)
In this proxy class, I instantiate two classes, TCPHandlerThread and TCPReplyThread. They are responsible for forwarding to Server, and forwarding back to Client, respectively.
class TCPHandlerThread(StoppableThread):
def __init__(self, proxy):
StoppableThread.__init__(self)
self.proxy = proxy
def runCallback(self):
test = False
while 1:
data = self.proxy.conn.recv(BUFFER_SIZE)
if test:
self.proxy.sock.close()
test = True
if not data:
break
print "TCP received data:", data
self.proxy.forward(data)
self.kill()
def finalCallback(self):
self.proxy.closeConnections()
class TCPReplyThread(StoppableThread):
def __init__(self, proxy):
StoppableThread.__init__(self)
self.proxy = proxy
def runCallback(self):
while 1:
data = self.proxy.sock.recv(BUFFER_SIZE)
if not data:
break
print "TCP received back data: "+data
self.proxy.forwardBack(data)
self.kill()
def finalCallback(self):
self.proxy.closeConnections()
You see that whenever a connection is closed, the thread dies and the other connection (Client/Server to proxy or Proxy to Server/Client) should be closed in Proxy.closeConnections()
I noticed that when closeConnections() is "data = self.proxy.conn.recv(BUFFER_SIZE)", it goes well, but when it's called even right after the latter statement, it goes wrong.
I wiresharked TCP, and the proxy doesn't send any "bye signal". The socket state doesn't go to TIME_WAIT or whatever, it just remains ESTABLISHED.
Also, I tested it on Windows and Ubuntu.
On Windows it goes exactly as I explained
On Ubuntu, it works well for usually (not always), 2 connections, and the third time I connect with the same client in exactly the same way to the proxy, it goes wrong again exactly as explained.
Here are the three files i'm using so that you can have a look at the whole code. I'm sorry the proxy file might not be really easy to read. Was SUPPOSED to be a quick dev.
http://hognerud.net/stackoverflow/
Thanks in advance..
It's surely something stupid. Please don't hit me too hard when you see it :(

First I'm sorry that I currently have not the time to actually run and test your code.
But the idea came to my mind, that your problem might actually have something todo with using blocking mode vs. non-blocking mode on the socket. In that case you should checkout the "socket" module help in the python documentation, especially socket.setblocking().
My guess is, that the proxy.conn.recv() function only returns, when actually BUFFER_SIZE bytes where received by the socket. Because of this the thread is blocked until enough data was received and therefore the socket doesn't get closed.
As I said first, this is currently just a guess, so please don't vote me down if it doesn't solve the problem...