I am trying to implement a multi-threading server which can handle with simultaneously read/write from client.
The server method:
The client connects to the server, when each message starts with the name of the user they want to send the message to, followed by '|'. It looks something like that: "USER_NAME|DATA".
After receiving the data, the server knows by a dictionary of {socket:username} where to send the data. Everything works great, except the fact that the client can't handle with simultaneously reading and writing. I searched for a method to handle that and i found the select() function, but with a lack of examples- i couldn't integrate that function in my code.
therefore I have 2 questions:
Is the select() function should be on the server side? will it be more efficient?
Is someone can demonstrate with a simple example how the select() method should look in the client side?
Thanks in advance!!!
Though select() will work, you have to use threads if you want to do other things while the system is blocked on the select.
Have a look at glib's GIO library. There you can connect callbacks to the actions you want to monitor or act on, for example the 'connect's from clients.
Just open a socket, and use its file descriptor to hang a gio.add_watch on. Here's a mini-tutorial on using giochannels.
Related
I need to have a tcp socket client connected to a server to send data and receive.
But this socket must be always on and i cannot open another socket.
I have always some data to send over the time and then later process the answer to the data sent previously.
If i could open many sockets, i think it was more easy. But in my case i have to send everything on the same socket asynchronously.
So the question is, what do you recommend to use within the Python ecosystem? (twisted, tornado, etc)
Should i consider node.js or another option?
I highly recommend Twisted for this:
It comes with out-of-the-box support for many TCP protocols.
It is easy to maintain a single connection, there is a ReconnectingClientFactory that will deal with disconnections and use exponential backoff, and LoopingCall makes it easy to implement a heartbeat.
Stateful protocols are also easy to implement and intermingle with complex business logic.
It's fun.
I have a service that is exactly like the one you mention (single login, stays on all the time, processes data). It's been on for months working like a champ.
Twisted is possibly hard to get your head around, but the tutorials here are a great start. Knowing Twisted will get you far in the long run!
"i have to send everything on the same socket asynchronously"
Add your data to a queue, have a separate thread taking items out of the queue and sending via socket.send()
I have an application built with Python / Twisted Matrix which uses methods from a SOAP client in order to send some messages. Problem is that sometimes i want to send a lot of messages and when that happens i would like to do it in multiple threads. For example if i have to send 100 messages i would like this broken into groups of 20 messages and to create 5 threads to send the messages in parallel.
What should i look for ? Any ideas ? I would also like the threads to be able to report back with the gathered data
P.S. given the fact that probably working with SOAP clients is more of a problem of waiting around ... do you think that threading is not the best approach to solve this ? Can the callbacks of the soap client be used to create some sort of "pool" of senders and have the senders somehow as for new stuff to send as soon as they are free ? Ideas ?
The best approach is probably defined by the distribution of the SOAP services and methods your accessing.
My first suggestion would be to not use OS Threading, but use micro-threading generator-coroutines with inlineCallbacks and deferredSemaphores.
But you might want to tune things to reuse connections for the same server and/or retain server cookies.
I'm working on a program using the Twisted IRCClient module, and am having a bit of a problem. There are several methods in the class that can be overloaded, for say when the client signs on to a server, or when the client receives a MOTD from a server. However, there don't seem to be any methods to deal with messages from the server itself, or to respond to ping queries that have a random number that needs to be sent back to the server.
Ideally I could implement these methods myself with the raw data from the server, using the lineReceived method of the class. However, it seems that when the lineReceived method is called by an incoming line, it gobbles up the line and the other class callbacks never fire. Is there a way around this problem? Thanks.
First, see 'METHODNAME' as Client method versus irc_'METHODNAME' in twisted for an explanation of how IRCClient dispatches messages. Then, take a look at irc_PING (which is already implemented, and already does the right thing).
Other server messages are handled via other similar callback methods.
Which way is better?
Creating a while loop and then using the select module OR using ThreadedTCPServer with a custom class.
Im having problems with the Threaded TCP Server, although it could just be my coding.
My personal recommendation is to use Twisted. It's a Python-based framework intended primarily for writing event-driven network software. The documentation has a lot of great examples of how to create various types of servers and clients, as well.
I am sure there is no such a thing like the "correct" way.
If you want not, must not or cannot use any of the existing server implementations the general idea is (in pseudo code):
ss = serversocket()
ss.bind ()
while (True):
cs = ss.accept ()
spawnCommThread (cs)
In the CommThread for each client you take care of reading from the socket returned by accept, communicate with your client and die, when the client closes the connection or another criterion is given.
I'm writing a python server/client app. If I serialize a function on the client and pass it to the server, can the server use it as a callback? I'm assuming there must be something extra I'd have to do as the client and server are communicating via packets, I just don't know what.
What I actually need is for the server to change one of the client's attributes (when the server is ready to accept another command), and I want an alternative to having the client continuously poll the server. Thanks for any help.
Take a look at Twisted JSON RPC.
A recent SO post: Python Twisted JSON RPC