We Keep Coding

How can I let my code send messages simultaneously?

I'm trying to do UDP socket programming in Python, and I want both the client and the server to be able to send messages without the need to wait for the other party to send a message first.
Here is my server code:
import socket
sock=socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
sock.bind(('127.0.0.1',12345))
while True:
data,addr=sock.recvfrom(4096) #byte size
print("Client says: ")
print(str(data))
message = bytes(input("Enter message here: ").encode('utf_8'))
sock.sendto(message,addr)
and here is my client code:
import socket
client_socket=socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
while True:
msg=input("Enter your message here: ")
client_socket.sendto(msg.encode('utf_8'),('127.0.0.1',12345))
data,addr=client_socket.recvfrom(4096) #byte size
print("Server says")
print(data)
What should I edit in my code to make this work?

You need to create two functions to handle listening and sending.
Then you start them as threads so they run in parallel.
Each function has its own loop. The receive thread waits for messages and the send thread waits for user input
def client_receive():
while True:
data,addr=client_socket.recvfrom(4096) #byte size
print("Server says")
print(data)
def send_chat_message():
while True:
msg=input("Enter your message here: ")
client_socket.sendto(msg.encode('utf_8'),('127.0.0.1',12345))
receive_thread = threading.Thread(target=client_receive)
receive_thread.start()
send_thread = threading.Thread(target=send_chat_message)
send_thread.start()
The above code will not really work at this point. But there are good tutorials out there. Search for python socket chat threaded tutorial
Often they are not based on a server handling inputs, but multiple clients connecting to a server. If you understand the concept, you'll have no problem adjusting the server to allow inputs there aswell.

Can't make a Python socket blocking

I'm trying to write a fairly simple client-server Python application using socket and SocketServer. To allow for two-way communication between client and server, the client maintains one connected socket with the server so it can listen for messages in a separate thread, while the main thread creates one-time-use sockets to send messages to the server. I want my "listening" socket to be blocking, as it is running in a separate thread whose only purpose is to wait for data without blocking the main program. Here is the function where I create this socket:
def connect(self, alias, serverIP):
if not alias or not isinstance(alias, str):
print "ERROR: Must specify an alias"
return
self.serverIP = serverIP
self.downConnection = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.downConnection.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.downConnection.setblocking(1)
self.downConnection.connect((self.serverIP, 11100))
self.downConnection.send("SENDSERVER CONNECT %s" % alias)
Here is the loop where the persistent socket listens for messages from the server (with some debugging code thrown in):
i = 0
while True:
print "LOOP", i,
if self.closed:
break
try:
data = self.downConnection.recv(1024)
except socket.timeout, e:
print "Timeout"
pass
else:
print "Received %d" % len(data)
if data:
self.received(data)
i += 1
I would expect to see "Received ##" messages only when the server sends data, and maybe periodic "Timeout" messages otherwise. Instead, the output grows very rapidly and looks entirely like this:
LOOP 33858 Received 0
LOOP 33859 Received 0
LOOP 33860 Received 0
LOOP 33861 Received 0
LOOP 33862 Received 0
LOOP 33863 Received 0
LOOP 33864 Received 0
LOOP 33865 Received 0
So it seems that self.downConnection.recv() is immediately returning an empty string each time it is called, rather than blocking until it receives substantive data like it's supposed to. This is puzzling, as I'm explicitly setting the socket to be blocking (which I think is also the default setting). Constantly executing this loop instead of the thread spending most of its time waiting for data is wasting a good deal of CPU time. What am I doing wrong in setting up the blocking socket?
Here is the full server code. The Comms class is also the superclass of the client class, to allow for some basic common functionality.
Something does seem to be wrong with the connection from the server's end. The server can receive data from the client, but trying to send data to the client gives a socket.error: [Errno 9] Bad file descriptor exception.

How do I add timeout, in order to move to next client if current client has not sent any data?

How do I add timeout, in order to move to next client if current client has not sent any data in Python? I have my all the connected clients stored in the conn_clients list.
Here's my code for receive function:
def receive(connection):
curr_con = connection
while True:
message = connection.recv(BUFFER_SIZE)
if not message:
print "Closing connection"
conn_clients.remove(connection) #removing socket from list
return
send_all(curr_con, message) #sending message to all cleints

You did not mention the operating system, and I'm not sure if this is applicable for Windows.
GLib provides timeout functions which you can start and tell to execute a function after a certain interval. I found this article which shows how to connect g_io_add_watch to socket events (much cleaner that waiting for events) and simultaneously start a timer (from the same library). If the socket doesn't show activity in the time set, the function can abort the socket process.

Multi-client UDP server python

I would like to implement an UDP server with Python.
I want to be able to wait for some clients to connect and chat with others at the same time.
I tried to use an SocketServer implementation
import SocketServer
class MyUDPHandler(SocketServer.BaseRequestHandler):
def handle(self):
data = self.request[0].strip()
socket = self.request[1]
print("{} wrote:".format(self.client_address))
print("data -> ", data)
socket.sendto(data.upper(), self.client_address)
if __name__ == "__main__":
HOST, PORT = "localhost", 9999
server = SocketServer.UDPServer((HOST, PORT), MyUDPHandler)
server.serve_forever()
With this implementation, I can send data from different clients to this server.
To be clear, what I want to do is to go in another function when a client sent UDP data to the server to be able to communicate with him. But at the same time, I still want other clients be able to send UDP data. I guess multithreading will be a solution ?
I'm not sure to be clear...

UDP is connectionless. So you can receive messages from multiple clients with just the single SocketServer that you have, and distinguish clients from each other using client_address. You don't need threads or multiple processes.
Since it's a chat server, outgoing messages are probably always in response to incoming ones, but if you want to be able to send unsolicited messages as well, you should replace serve_forever() with handle_request() and set self.timeout in __init__(). This way you can check whether extra actions need to be performed periodically, e.g. once a minute you could send heartbeats or whatever.

ZeroMQ: HWM on PUSH does not work

I am trying to write a server/client script with a server that vents the tasks, and multiple workers that execute it.
The problem is that my ventilator has so many tasks that it would fill up the memory in a heartbeat.
I tried to set the HWM before it binds, but with no success. It just keeps on sending messages as soon as a worker connects, completely disregarding the HWM that was set. I also have a sink that keeps record of the tasks that were done.
server.py
import zmq
def ventilate():
context = zmq.Context()
# Socket to send messages on
sender = context.socket(zmq.PUSH)
sender.setsockopt(zmq.SNDHWM, 30) #Big messages, so I don't want to keep too many in queue
sender.bind("tcp://*:5557")
# Socket with direct access to the sink: used to syncronize start of batch
sink = context.socket(zmq.PUSH)
sink.connect("tcp://localhost:5558")
print "Sending tasks to workers…"
# The first message is "0" and signals start of batch
sink.send('0')
print "Sent starting signal"
while True:
sender.send("Message")
if __name__=="__main__":
ventilate()
worker.py
import zmq
from multiprocessing import Process
def work():
context = zmq.Context()
# Socket to receive messages on
receiver = context.socket(zmq.PULL)
receiver.connect("tcp://localhost:5557")
# Socket to send messages to
sender = context.socket(zmq.PUSH)
sender.connect("tcp://localhost:5558")
# Process t asks forever
while True:
msg = receiver.recv_msg()
print "Doing sth with msg %s"%(msg)
sender.send("Message %s done"%(msg))
if __name__ == "__main__":
for worker in range(10):
Process(target=work).start()
sink.py
import zmq
def sink():
context = zmq.Context()
# Socket to receive messages on
receiver = context.socket(zmq.PULL)
receiver.bind("tcp://*:5558")
# Wait for start of batch
s = receiver.recv()
print "Received start signal"
while True:
msg = receiver.recv_msg()
print msg
if __name__=="__main__":
sink()

Ok, I had a play around, I don't think the issue is with the PUSH HWM, but rather that you can't set a HWM for PULL. If you look at this documentation, you can see there it says N/A for action on HWM.
The PULL sockets seem to be taking hundreds of messages each (and I did try setting a HWM just in case it did anything on the PULL socket. It didn't.). I evidenced this by changing the ventilator to send messages with an incrementing integer, and changing each worker in the pool to wait 2 seconds between calls to recv(). The workers print out that they are processing messages with vastly different integers. For instance, one worker will be working on message 10, while the next is working on message 400. As time goes on, you see the worker who was processing message 10, is now processing message 11, 12, 13, etc. while the other is processing 401, 402, etc.
This indicates to me that the ZMQ_PULL socket is buffering the messages somewhere. So while the ZMQ_PUSH socket does have a HWM, the PULL socket is requesting messages quickly, despite them not actually being accessed by a call to recv(). So that results in the PUSH HWM effectively being ignored if a PULL socket is connected. As far as I can see, you can't control the length of the buffer of the PULL socket (I would expect the RCVHWM socket option to control this but it doesn't appear to).
This behaviour of course begs the question what is the point of the ZMQ_PULL HWM option, which only makes sense to have if you can also control the receiving sockets HWM.
At this point, I'd start asking the 0MQ people whether you are missing something obvious, or if this is considered a bug.
Sorry I couldn't be more help!

ZeroMQ has buffers on both sending and receiving ends of a socket, hence you need to set high water marks on both the PUSH and the PULL socket in your code (and indeed before a bind() or connect()).
In the Python bindings this is now conveniently done via socket.hwm = 1 which will set both ZMQ_SNDHWM and ZMQ_RCVHWM in one go.