I am creating a file server and need to have several clients send images to a server. In the client send method, I am shutting down the socket after the image has been sent to tell the server to stop receiving. Is it possible to keep the same socket connection for the next time that client sends an image rather than reconnecting with a new socket?
No. A shutdown is a definitive operation at the underlying socket library level. It is not intended to be used as a transfert acknowledgment, but only as part or a graceful shutdown.
If you want to re-use the connection, you must use a different protocol to signal the end of transmission. Common usages are size + data (binary protocol) or commands and encoded data (text protocol).
Related
I have a basic implementation of a TCP client using python sockets, all the client does is connect to a server and send heartbeats every X seconds. The problem is that I don't want to send the server a heartbeat if the connection is closed, but I'm not sure how to detect this situation without actually sending a heartbeat and catch an exception. When I turn off the server, in the traffic capture I see FIN/ACK arriving and the client sends an ACK back, this is when I want my code to do something (or at least change some internal state of the connection). Currently, what happens is after the server went down and X seconds passed since last heartbeat the client will try to send another heartbeat, only then I see RST packet in the traffic capture and get an exception of broken pipe (errno 32). Clearly python socket handles the transport layer and the heartbeats are part of application layer, the problem I want to solve is not to send the redundant heartbeat after FIN/ACK arrived from server, any simple way to know the connection state with python socket?
I use a TCP server in python, that implements this class:
class ThreadedTCPServer(SocketServer.ThreadingTCPServer):
pass
The normal use of it works perfect (initiating the server, handling requests and so on).
Now- I need to send a message to the clients, outside of the handle function in the TcpRequestHandler(SocketServer.BaseRequestHandler) class.
I tried to use the following trick, of using the internal socket of the server (it works for UDP)-
tcp_server.client_socket.send(message)
But I get this error message-
socket.error: [Errno 10057] A request to send or receive data was disallowed because the socket is not connected and (when sending on a datagram socket using a sendto call) no address was supplied
So I assume it is not possible for TCP.
Is there any other way to do it?
I assume some servers need to send messages to their client sometimes (that are not just responses to requests), but I couldn't find a good way.
Thanks!
You have two general options with TCP:
Send a message to the client out of band (OOB). In this, the server connects separately to the client and the roles are reversed. The client has to listen on a port for OOB messages and acts as a server in this regard. From your problem description you don’t want to do this, which is fine.
Implement a protocol where the server can send messages to the client in response to incoming messages. You will need a way to multiplex the extra messages along with any expected return value to the initiating message. You could implement this with a shared queue on your server. You put messages into this queue outside of your handler and then when the handler is responding to messages you consume from the queue and insert them into the response.
If that sounds like something you are interested in I could write some example code later.
There are pros & cons between both approaches:
In (1) you have more socket connections to manage and you expose the client host to connections which you might not desire. The protocols are simpler because they are not multiplexed.
In (2) you only have a single TCP stream but you have to multiplex your OOB message. You also have increased latency if the client is not regularly contacting the server.
Hope that helps.
I have a python program that reads in values from an ADC then writes them to a file as well as send them via TCP if a connection is available. I can send the data fine however as data is constantly being read I would like to be able to keep the connection open. How do I get the client to check that the Server has more data to send and thus to keep the connection open?
This scenario seems very similar to one of our applications. We use ZeroMQ for this.
Using ZeromMQ with PUB/SUB
On PyZMQ doc is example for using Pub/Sub.
Data provider creates PUB socket and sends messages to it.
Data consumer sets up SUB socket and reads messages from.
Typically, PUB socket is fixed part of infrastructure, so it binds to some port, and SUB connects. But if you like, you can switch it and it works too.
Advantages are:
provider sends messages to PUB socket and does not block on it
reconnects are handled automatically
if there is no consumer or connection, PUB socket silently drops the messages.
the code to implement this is very short
Other messaging patterns with ZeroMQ
PUB/SUB is just one option, there are other combinations like PUSH/PULL, REQ/REP etc.
I have some services running for years with PUSH/PULL and it stays "quite well" (typically there are 6 weeks before there is a need to do some restart, but this is rather due to problems on hardware than in ZeroMQ library.)
I am developing a group chat application to learn how to use sockets, threads (maybe), and asycore module(maybe).
What my thought was have a client-server architecture so that when a client connects to the server the server sends the client a list of other connects (other client 'user name', ip addres) and then a person can connect to one or more people at a time and the server would set up a P2P connection between the client(s). I have the socket part working, but the server can only handle one client connection at a time.
What would be the best, most common, practical way to go about handling multiple connections?
Do I create a new process/thread whenever I new connection comes into the server and then connect the different client connections together, or use the asycore module which from what I understand makes the server send the same data to multiple sockets(connection) and I just have to regulate where the data goes.
Any help/thoughts/advice would be appreciated.
For a group chat application, the general approach will be:
Server side (accept process):
Create the socket, bind it to a well known port (and on appropriate interface) and listen
While (app_running)
Client_socket = accept (using serverSocket)
Spawn a new thread and pass this socket to the thread. That thread handles the client that just connected.
Continue, so that server can continue to accept more connections.
Server-side client mgmt Thread:
while app_running:
read the incoming message, and store to a queue or something.
continue
Server side (group chat processing):
For all connected clients:
check their queues. If any message present, send that to ALL the connected clients (including the client that sent this message -- serves as ACK sort of)
Client side:
create a socket
connect to server via IP-address, and port
do send/receive.
There can be lots of improvement on the above. Like the server could poll the sockets or use "select" operation on a group of sockets. That would make it efficient in the sense that having a separate thread for each connected client will be an overdose when there are many. (Think ~1MB per thread for stack).
PS: I haven't really used asyncore module. But I am just guessing that you would notice some performance improvement when you have lots of connected clients and very less processing.
I have implemented a Modbus over TCP as server software in Python. App is multithreaded and relies heavily on standard libs. I have problems managing the connection on the server side.
Meanwhile my implementation as Modbus over TCP as client works just fine.
Implementation description
The server is multithreaded, one thread manages the SOCK_STREAM socket for receiving
frames
select is used out of efficiency reasons
A semaphore is used for preventing concurrent access on socket resource while sending or receiving
Encapsulation of Modbus upper layer is done transparently through send and receive methods, it is only a matter of building a frame with the right header and payload anyway...
Another threads runs, inside it, Modbus send and receive methods are invoked.
TCP Context
TCP is up and running, bound to a port, max client set and listening.
Traces under wireshark show:
Client: SYN
My app Server: SYN, ACK
Client: ACK
On the server side a brand new socket has been created as expected and bound to the client socket.
So far, all is good.
Modbus Context
Client: Send Modbus frame, TCP flags = 0x18 which is ACK + PUSH
My app Server: Does not wait and send a single empty TCP ack frame.
Client: Waits for a modbus frame with tcp ack flag. Therefore, takes it as an error and asks to closes the connection.
Hence, my server software cannot send any actual response afterwards as the socket on the client side is being closed or is already closed.
My problem
I receive a modbus frame that the main thread need to process (server side)
Processing takes a few ms, in the meantime a TCP ACK frame is sent through my server socket, whereas I would like it not to send anything !
Do you have any idea on how to manage the ACK behavior ? I have read stuff about the naggle algorithm, but it does not seem to be in the scope of the problem here...
I'm not sure that any option of the setsockopt method would solve my problem also, but I may be mistaken.
If you have any suggestion I am very interested...
I hope I am clear enough.
It seems like a strange requirement that all TCP packets must contain a payload as this is very difficult to control unless you are integrated with the TCP stack. If it really is the case that the client crashes because the ACK has no Modbus payload, I think the only thing you can do from python is try disabling the TCP_QUICKACK socket option so that TCP waits 500ms before sending an ACK. This obviously won't work in all cases (or may not at all if it takes your code > 500ms to create a response), but I don't know of any other options using the socket API from python.
This SO answer tells you how to disable it: Disable TCP Delayed ACKs. Should be easy to figure out how to enable it from that. Note, you need to constantly re-enable it after receiving data.