I use a TCP server in python, that implements this class:
class ThreadedTCPServer(SocketServer.ThreadingTCPServer):
pass
The normal use of it works perfect (initiating the server, handling requests and so on).
Now- I need to send a message to the clients, outside of the handle function in the TcpRequestHandler(SocketServer.BaseRequestHandler) class.
I tried to use the following trick, of using the internal socket of the server (it works for UDP)-
tcp_server.client_socket.send(message)
But I get this error message-
socket.error: [Errno 10057] A request to send or receive data was disallowed because the socket is not connected and (when sending on a datagram socket using a sendto call) no address was supplied
So I assume it is not possible for TCP.
Is there any other way to do it?
I assume some servers need to send messages to their client sometimes (that are not just responses to requests), but I couldn't find a good way.
Thanks!
You have two general options with TCP:
Send a message to the client out of band (OOB). In this, the server connects separately to the client and the roles are reversed. The client has to listen on a port for OOB messages and acts as a server in this regard. From your problem description you don’t want to do this, which is fine.
Implement a protocol where the server can send messages to the client in response to incoming messages. You will need a way to multiplex the extra messages along with any expected return value to the initiating message. You could implement this with a shared queue on your server. You put messages into this queue outside of your handler and then when the handler is responding to messages you consume from the queue and insert them into the response.
If that sounds like something you are interested in I could write some example code later.
There are pros & cons between both approaches:
In (1) you have more socket connections to manage and you expose the client host to connections which you might not desire. The protocols are simpler because they are not multiplexed.
In (2) you only have a single TCP stream but you have to multiplex your OOB message. You also have increased latency if the client is not regularly contacting the server.
Hope that helps.
Related
I have a basic implementation of a TCP client using python sockets, all the client does is connect to a server and send heartbeats every X seconds. The problem is that I don't want to send the server a heartbeat if the connection is closed, but I'm not sure how to detect this situation without actually sending a heartbeat and catch an exception. When I turn off the server, in the traffic capture I see FIN/ACK arriving and the client sends an ACK back, this is when I want my code to do something (or at least change some internal state of the connection). Currently, what happens is after the server went down and X seconds passed since last heartbeat the client will try to send another heartbeat, only then I see RST packet in the traffic capture and get an exception of broken pipe (errno 32). Clearly python socket handles the transport layer and the heartbeats are part of application layer, the problem I want to solve is not to send the redundant heartbeat after FIN/ACK arrived from server, any simple way to know the connection state with python socket?
I am creating a file server and need to have several clients send images to a server. In the client send method, I am shutting down the socket after the image has been sent to tell the server to stop receiving. Is it possible to keep the same socket connection for the next time that client sends an image rather than reconnecting with a new socket?
No. A shutdown is a definitive operation at the underlying socket library level. It is not intended to be used as a transfert acknowledgment, but only as part or a graceful shutdown.
If you want to re-use the connection, you must use a different protocol to signal the end of transmission. Common usages are size + data (binary protocol) or commands and encoded data (text protocol).
I am creating a colloabrative note-making app in python.
Here, one guy on computer running the app can create the server subseuqently the changes on the screen([color, pixel], where pixel=[x,y]) will be transmitted to others connected to the server.
I am using kivy for creating the app. My question is with respect to transmitting the data over the server.
I can create server using this:
import socket
ip_address=socket.gethostbyname(socket.gethostname())
execfile( "manage.py runserver "+ip_address+":8000" )
Now, how do others connect to the server and request the data(assuming the above code is correct). Also, how to send the data in django.
Well, Django is a framework that allows creating a site or API that is reachable through HTTP protocol. This has several consequences for you:
Server cannot send a message to client unless the client asks. HTTP is a "request-response" protocol. Client sends a request (for example, http://server.com/getUpdates?id=100500) and gets a response from server.
Creating clients that ask the server to give them updates all the time is a bad practice, probably leading to server DoS.
Although you can use WebSockets, using Django for such a task is really an overkill.
Summarizing, you need a reliable duplex channel for sending data in both directions. I'd start with TCP server, rather than HTTP. Fortunately, Python stdlib has a module you can start with - socketserver.
Additional reading
TCP
UDP (you will probably want this for broadcasting)
Berkeley sockets (a socket standard underlying socketserver module)
TCP vs. UDP
When deciding what protocol to use, following aspects should be considered:
TCP is reliable. Messages never disappear implicitly. If there was a network error, message will be resent. If there's no connection, explicit error will be raised. TCP uses several algorithms to fit into the network channel. It is an intelligent protocol.
UDP is unreliable. It possesses no feature TCP has. Packets can disappear, get reordered. But UDP messages are lightweight and in experienced hands they summon to life such systems as network action games and streaming video (lost and reordered messages aren't crucial here and TCP becomes too slow).
So I'd recommend to start with TCP. It's way more easier to get working fast and correct than UDP. Switch to UDP if you have some experience with TCP and there are a lot of people using you app and wanting to get the lowest latency possible.
I am developing a group chat application to learn how to use sockets, threads (maybe), and asycore module(maybe).
What my thought was have a client-server architecture so that when a client connects to the server the server sends the client a list of other connects (other client 'user name', ip addres) and then a person can connect to one or more people at a time and the server would set up a P2P connection between the client(s). I have the socket part working, but the server can only handle one client connection at a time.
What would be the best, most common, practical way to go about handling multiple connections?
Do I create a new process/thread whenever I new connection comes into the server and then connect the different client connections together, or use the asycore module which from what I understand makes the server send the same data to multiple sockets(connection) and I just have to regulate where the data goes.
Any help/thoughts/advice would be appreciated.
For a group chat application, the general approach will be:
Server side (accept process):
Create the socket, bind it to a well known port (and on appropriate interface) and listen
While (app_running)
Client_socket = accept (using serverSocket)
Spawn a new thread and pass this socket to the thread. That thread handles the client that just connected.
Continue, so that server can continue to accept more connections.
Server-side client mgmt Thread:
while app_running:
read the incoming message, and store to a queue or something.
continue
Server side (group chat processing):
For all connected clients:
check their queues. If any message present, send that to ALL the connected clients (including the client that sent this message -- serves as ACK sort of)
Client side:
create a socket
connect to server via IP-address, and port
do send/receive.
There can be lots of improvement on the above. Like the server could poll the sockets or use "select" operation on a group of sockets. That would make it efficient in the sense that having a separate thread for each connected client will be an overdose when there are many. (Think ~1MB per thread for stack).
PS: I haven't really used asyncore module. But I am just guessing that you would notice some performance improvement when you have lots of connected clients and very less processing.
I have implemented a Modbus over TCP as server software in Python. App is multithreaded and relies heavily on standard libs. I have problems managing the connection on the server side.
Meanwhile my implementation as Modbus over TCP as client works just fine.
Implementation description
The server is multithreaded, one thread manages the SOCK_STREAM socket for receiving
frames
select is used out of efficiency reasons
A semaphore is used for preventing concurrent access on socket resource while sending or receiving
Encapsulation of Modbus upper layer is done transparently through send and receive methods, it is only a matter of building a frame with the right header and payload anyway...
Another threads runs, inside it, Modbus send and receive methods are invoked.
TCP Context
TCP is up and running, bound to a port, max client set and listening.
Traces under wireshark show:
Client: SYN
My app Server: SYN, ACK
Client: ACK
On the server side a brand new socket has been created as expected and bound to the client socket.
So far, all is good.
Modbus Context
Client: Send Modbus frame, TCP flags = 0x18 which is ACK + PUSH
My app Server: Does not wait and send a single empty TCP ack frame.
Client: Waits for a modbus frame with tcp ack flag. Therefore, takes it as an error and asks to closes the connection.
Hence, my server software cannot send any actual response afterwards as the socket on the client side is being closed or is already closed.
My problem
I receive a modbus frame that the main thread need to process (server side)
Processing takes a few ms, in the meantime a TCP ACK frame is sent through my server socket, whereas I would like it not to send anything !
Do you have any idea on how to manage the ACK behavior ? I have read stuff about the naggle algorithm, but it does not seem to be in the scope of the problem here...
I'm not sure that any option of the setsockopt method would solve my problem also, but I may be mistaken.
If you have any suggestion I am very interested...
I hope I am clear enough.
It seems like a strange requirement that all TCP packets must contain a payload as this is very difficult to control unless you are integrated with the TCP stack. If it really is the case that the client crashes because the ACK has no Modbus payload, I think the only thing you can do from python is try disabling the TCP_QUICKACK socket option so that TCP waits 500ms before sending an ACK. This obviously won't work in all cases (or may not at all if it takes your code > 500ms to create a response), but I don't know of any other options using the socket API from python.
This SO answer tells you how to disable it: Disable TCP Delayed ACKs. Should be easy to figure out how to enable it from that. Note, you need to constantly re-enable it after receiving data.