I am building a socket server with Python.
This server
receives data from client
does something here (approximately it takes about 10 sec in maximum depending on input data)
sends back data after some works done above
This system works fine unless client doesn't send data simultaneously in a row. For example, say server takes 5 seconds to process data and client sends data every 10 seconds. The problem, however, is that client send multiple requests at a time, thus causing a delay. Currently, client cannot send data to server unless server is ready to receive data which means that server is not doing any work. Below are what I want to build.
a) build a queue at socket server whose main task is to make a queue of input data so that client can send data to server even when server is busy
b) make a thread(here, I'm bit confused with concurrency and parallelism. Work in socket focused on computation rather than system call) at socket server so that server can do work 'simultaneously'.
c) send back data to client socket
My questions are as follows.
Is it Queue that I need to use in order to achieve a) ?
Is it thread or something else that I need to use in order to achieve b)?
Thanks in advance
Best
Gee
Yeah something like this could work.
First, you'll need a thread to receive and send data. If you have limited amount of clients, you can create a thread per client, but it's not an option for a more or less robust system. In order to be able to serve multiple clients in a single thread, the sockets should be nonblocking. Otherwise one long transmission would block other transmissions. Nonblocking code has more sophisticated structure that uses select, so I would advice to spend some time reading about it.
Then you'll need a thread to do the math. Or several threads/processes if "the math" is taking long to execute.
Last but not least, these socket threads and a "math" thread should use two queues to exchange data. Simple lists's are enough, but make sure they are synchronized. Guard them with mutexes, or locks. This is another vast topic that is worth reading about.
Related
I am building a simple star-like client-server topology.
The idea is that clients connect to the server, can send messages, and the server can send messages to them, when the server decides to. There will be a relatively small number of clients, about 30, but so many that it is not sensible to send all outgoing data to all. I'm sure I'm just boneheaded, but this seems to be completely impossible with ZeroMQ.
The last part is the reason this question does not provide answer.
The catch is this :
I can use a ROUTER socket to receive messages from clients. This also carries identification. However, I cannot use the same socket for sending, since ZeroMQ sockets are not threadsafe. I.e. I can't have one thread waiting for incoming messages, and another sending outgoing from the server itself. I am not aware of any way I could wait in blocking for both - socket.recv(), and for example .get() on a queue - at the same time on a single thread in python. Maybe there is a way to do that.
Using two sockets - one incoming one outgoing - doesn't work either. The identification is not shared between sockets, and so the sending socket would still have to be polled to obtain client id mapping, if even for once. We obviously can't use own port for each client. There seems to be no way for the server to send a message to a single client out of it's own volition.
(subscription topics are a dead idea too: message filtering is performed on client-side, and the server would just flood all client networks)
In the end TCP sockets can handle this sort of asynchronous situation easily, but effective message framing on python is a nightmare to build. All I'm essentially after is a reliable socket that handles messages, and has well defined failure modes.
I don't know Python but for C/C++ I would use zmq_poll(). There are several options, depending on your requirements.
Use zmq_poll() to wait for messages from clients. If a message arrives, process it. Also use a time-out. When the time-out expires, check if you need to send messages to clients and send them.
zmq_poll() can also wait on general file descriptors. You can use some type of file descriptor and trigger it (write to it) from another process or thread when you have a message to send to a client. If this file descriptor is triggered, send messages to clients.
Use ZeroMQ sockets internally inside your server. Use zmq_poll() to wait both on messages from clients and internal processes or threads. If the internal sockets are triggered, send messages to clients.
You can use the file descriptor or internal ZeroMQ sockets just for triggering but you can also send the message content through the file descriptor or ZeroMQ socket.
Q : "ZeroMQ: How to construct simple asynchronous broker?"
The concept builds on a few assumptions that are not supported or do not hold :
a)Python threads actually never execute concurrently, they are re-[SERIAL]-ised into a sequence of soloists execution blocks & for any foreseeable future will remain such, since ever & forever (as Guido van ROSSUM has explained this feature to be a pyramidal reason for collision prevention - details on GIL-lock, serving this purpose, are countless )
b)ZeroMQ thread-safeness has nothing to do with using a blocking-mode for operations.
c)ZeroMQ PUB/SUB archetype does perform a topic-filtering, yet in different versions on different sides of the "ocean" :
Until v3.1, subscription mechanics ( a.k.a. a TOPIC-filter ) was handled on the SUB-side, so this part of the processing got distributed among all SUB-s ( at a cost of uniformly wide data-traffic across all transport-classes involved ) and there was no penalty, except for a sourcing such data-flow related workload ... on the PUB-side.
Since v3.1, the TOPIC-filter is processed on the PUB-side, at a cost of such a processing overhead & memory allocations, but saving all the previously wasted transport-capacities, consumed just to later realise at the SUB-side the message is not matching the TOPIC-filter and will be disposed off.
Using a .poll()-based & zmq.NOBLOCK-modes of .recv()- & .send()-methods in the code design will never leave one in ambiguous, the less in an unsalvagable deadlock waiting-state and adds the capability to design even a lightweight priority-driven soft-scheduler for doing so with different relative priority levels.
Given your strong exposure in realtime systems, you might like to have a read into this to review the ZeroMQ Framework properties.
I'm currently working on a Benchmark project, where I'm trying to stress the server out with zmq requests.
I was wondering what would be the best way to approach this, I was thinking of having a context to create a socket and push it into a thread, in which I would send request and wait for responses in each thread respectively, but I'm not too sure this is possible with python's limitations.
More over, would it be the same socket for all threads, that is, if I'm waiting for a response on one thread (With it's own socket), would it be possible for another thread to catch that response?
Thanks.
EDIT:
Test flow logic would be like this:
Client socket would use zmq.REQ.
Client sends message.
Client waits for a response.
If no response, client reconnects and tries again until limit.
I'd like to scale this operation up to any number of clients, preferring not to deal with Processes unless performance wise the difference is significant..
How would you do this?
Q : "...can I have one context and use several sockets?"
Oh sure you can.
Moreover, you can have several Context()-instances, each one managing ... almost... any number of Socket()-instances, each Socket()-instance's methods may get called from one and only one python-thread ( a Zen-of-Zero rule: zero-sharing ).
Due to known GIL-lock re-[SERIAL]-isation of all the thread-based code-execution flow, this still has to and will wait for acquiring the GIL-lock ownership, which in turn permits a GIL-lock owner ( and nobody else ) to execute a fixed amount of python instructions, before it re-releases the GIL-lock to some other thread...
First, of, I've read around a fair amount of time including many threads on this site, however I still need some clarification on Sockets, TCP and Networking in Python, as I feel like I don't fully understand what's happening in my program.
I'm sending data from a server to a client via an Unix Domain Socket (AF_UNIX) using TCP (SOCK_STREAM).
On the server side, a process is continuously putting items on a Queue.Queue and another process is sending items to the client by running
while True:
conn.sendall(queue.get())
On the client side, data is read by running
while True:
conn.recv(1024)
# time.sleep(10)
Now, I emulate a slow client by sending the client process to sleep after every call on recv(). What I expect is that the queue on the server side is filled up, since send() should block because the client can't read off data fast enough.
I monitor the number of items send to the client as well as the queue size. What I notice is that several dozen messages (roughly depending on the size of the messages, but slightly different message sizes might behave the same) are sent to the client (which are received by the client with delay, due to time.seep()) before the queue starts to fill up.
What is happening here? Why is send() not blocking immediately?
I suspect that some sort of network or file buffer is involved, which queues the send items and fills up before my implemented queue.
There are a number of buffers in various places in the system, on both the sender and the receiver. Your call to a sending function won't block until all those buffers are filled up. When the receiver drains some of the buffers, data will flow again and eventually it will unblock the send call.
Typically there's a buffer in the sender holding data waiting to be put on the wire, a buffer "in flight" allowing a certain number of bytes to be send before having to wait for the receiver to acknowledge, and lastly receive buffers holding data that has been acknowledged but not yet delivered to the receiving application.
Were this not so, forward progress would be extremely limited. The sender would be stuck waiting to send until the receiver called receive. Then, whichever one finishes first would have to wait for the other one. Even if the sender was finished first, it couldn't make any forward progress at all until the receiver finished processing the previous chunk of data. That would be quite sub-optimal for most applications.
Consider the following scenario: A process on the server is used to handle data from a network connection. Twisted makes this very easy with spawnProcess and you can easily connect the ProcessTransport with your protocol on the network side.
However, I was unable to determine how Twisted handles a situation where the data from the network is available faster than the process performs reads on its standard input. As far as I can see, Twisted code mostly uses an internal buffer (self._buffer or similar) to store unconsumed data. Doesn't this mean that concurrent requests from a fast connection (eg. over local gigabit LAN) could fill up main memory and induce heavy swapping, making the situation even worse? How can this be prevented?
Ideally, the internal buffer would have an upper bound. As I understand it, the OS's networking code would automatically stall the connection/start dropping packets if the OS's buffers are full, which would slow down the client. (Yes I know, DoS on the network level is still possible, but this is a different problem). This is also the approach I would take if implementing it myself: just don't read from the socket if the internal buffer is full.
Restricting the maximum request size is also not an option in my case, as the service should be able to process files of arbitrary size.
The solution has two parts.
One part is called producers. Producers are objects that data comes out of. A TCP transport is a producer. Producers have a couple useful methods: pauseProducing and resumeProducing. pauseProducing causes the transport to stop reading data from the network. resumeProducing causes it to start reading again. This gives you a way to avoid building up an unbounded amount of data in memory that you haven't processed yet. When you start to fall behind, just pause the transport. When you catch up, resume it.
The other part is called consumers. Consumers are objects that data goes in to. A TCP transport is also a consumer. More importantly for your case, though, a child process transport is also a consumer. Consumers have a few methods, one in particular is useful to you: registerProducer. This tells the consumer which producer data is coming to it from. The consumer can them call pauseProducing and resumeProducing according to its ability to process the data. When a transport (TCP or process) cannot send data as fast as a producer is asking it to send data, it will pause the producer. When it catches up, it will resume it again.
You can read more about producers and consumers in the Twisted documentation.
I've been struggling along with sockets, making OK progress, but I keep running into problems, and feeling like I must be doing something wrong for things to be this hard.
There are plenty of tutorials out there that implement a TCP client and server, usually where:
The server runs in an infinite loop, listening for and echoing back data to clients.
The client connects to the server, sends a message, receives the same thing back, and then quits.
That I can handle. However, no one seems to go into the details of what you should and shouldn't be doing with sequential communication between the same two machines/processes.
I'm after the general sequence of function calls for doing multiple messages, but for the sake of asking a real question, here are some constraints:
Each event will be a single message client->server, and a single string response.
The messages are pretty short, say 100 characters max.
The events occur relatively slowly, max of say, 1 every 5 seconds, but usually less than half that speed.
and some specific questions:
Should the server be closing the connection after its response, or trying to hang on to the connection until the next communication?
Likewise, should the client close the connection after it receives the response, or try to reuse the connection?
Does a closed connection (either through close() or through some error) mean the end of the communication, or the end of the life of the entire object?
Can I reuse the object by connecting again?
Can I do so on the same port of the server?
Or do I have reinstantiate another socket object with a fresh call to socket.socket()?
What should I be doing to avoid getting 'address in use' errors?
If a recv() times out, is the socket reusable, or should I throw it away? Again, can I start a new connection with the same socket object, or do I need a whole new socket?
If you know that you will communicate between the two processes soon again, there is no need for closing the connection. If your server has to deal with other connections as well, you want to make it multithreaded, though.
The same. You know that both have to do the same thing, right?
You have to create a new socket on the client and you can also not reuse the socket on the server side: you have to use the new socket returned by the next (clientsocket, address) = serversocket.accept() call. You can use the same port. (Think of webservers, they always accept connections to the same port, from thousands of clients)
In both cases (closing or not closing), you should however have a message termination sign, for example a \n. Then you have to read from the socket until you have reached the sign. This usage is so common, that python has a construct for that: socket.makefile and file.readline
UPDATE:
Post the code. Probably you have not closed the connection correctly.
You can call recv() again.
UPDATE 2:
You should never assume that the connection is reliable, but include mechanisms to reconnect in case of errors. Therefore it is ok to try to use the same connection even if there are longer gaps.
As for errors you get: if you need specific help for your code, you should post small (but complete) examples.