We Keep Coding

select() function for simultaneously I/O

I am trying to implement a multi-threading server which can handle with simultaneously read/write from client.
The server method:
The client connects to the server, when each message starts with the name of the user they want to send the message to, followed by '|'. It looks something like that: "USER_NAME|DATA".
After receiving the data, the server knows by a dictionary of {socket:username} where to send the data. Everything works great, except the fact that the client can't handle with simultaneously reading and writing. I searched for a method to handle that and i found the select() function, but with a lack of examples- i couldn't integrate that function in my code.
therefore I have 2 questions:
Is the select() function should be on the server side? will it be more efficient?
Is someone can demonstrate with a simple example how the select() method should look in the client side?
Thanks in advance!!!

Though select() will work, you have to use threads if you want to do other things while the system is blocked on the select.
Have a look at glib's GIO library. There you can connect callbacks to the actions you want to monitor or act on, for example the 'connect's from clients.
Just open a socket, and use its file descriptor to hang a gio.add_watch on. Here's a mini-tutorial on using giochannels.

Client to Server Remote Function Calls in Python. How to implement?

I'm trying to set up a simple client to server interface for calling functions/programs on the server. A client will send a simple command to the server listening for such commands. Once the server receives a command from the client it will execute the following function or program on the server. I have looked into a simple TCP server receiving a text string and parsing that string then executing the a function or external program. I have read into using XML-RPC implemented with a twisted server as well.
What I'm asking is which would be the easiest to set up or are there any other ways to easily do this task?
Thanks.

There is a great tutorial for twisted that will do just fine as a teaching tool (and guide you by hand in writing a basic server/client services). Have a go at it http://twistedmatrix.com/documents/current/core/howto/tutorial/ what you will probably want to do is parse received info and act accordingly.

If it is appliable in your case, maybe you can use full-featured system for async/remote job execution like Celery?

There are more than one way to achieve your requirement ach with some pros and cons:
Python Low Level Sockets
Using Standard python socket libraries and cliet server architecture
Connecting to Server via protocols like Telnet/SSh and then triggering some code.
Using Python libraries like Telnet/ssh or Subprocess.
XML-RPC
Sending a XMP RPC request as described here http://docs.python.org/2/library/xmlrpclib.html
In my opinion easiest method to achieve remote method triggering is via Python Subprocess Module. I generally use following kind of syntax for my general purposes.
import subprocess
ret = subprocess.call(["ssh", "user#host", "program"]);
# or, with stderr:
prog = subprocess.Popen(["ssh", "user#host", "program"], stderr=subprocess.PIPE)
errdata = prog.communicate()[1]
Hope it helps

How would I handle multiple sockets and send data between them in Python 2.7.3?

I am trying to create a server in Python 2.7.3 which sends data to all client connections whenever one client connection sends data to the server. For instance, if client c3 sent "Hello, world!" to my server, I would like to then have my server send "Hello, world!" to client connections c1 and c2. By client connections, I mean the communications sockets returned by socket.accept(). Note that I have tried using the asyncore and twisted modules, but AFAIK they do not support this. Does anybody know any way to accomplish this?
EDIT: I have seen Twisted, but I would much rather use the socket module. Is there a way (possibly multithreading, possibly using select) that I can do this using the socket module?

You can absolutely do this using Twisted Python. You just accept the connections and set up your own handling logic (of course the library does not including built-in support for your particular communication pattern exactly, but you can't expect that).

Python JSON-RPC_2.0 TCP Server Client Explained

I'm having a difficult time fully understanding the nature of a TCP server/client relationship when a JSON string is sent to the server. The information I need may be out there, but I'm perhpas not using the correct search paramaters as I'm looking.
I've built a Python TCP, JSON-RPC Server from the following examples:
https://github.com/joshmarshall/jsonrpclib
http://code.activestate.com/recipes/552751-json-rpc-server-and-client/
In both cases, I can communicate with the Python server from a Python console on a different computer, sending commands from one (the client) to the other (server). In all of the examples, I've had to install the libraries mentioned above on both the client and the server machines in order to facilitate the TCP communication.
So the background to my situation and question is, when does JSON enter the mix? This is what I want to do:
Setup a Python TCP server that accepts a JSON string from a remote client inside (or outside) the network. The server parses the JSON string, fetches the method and parameters from the objectified string, and executes the method. The server then sends a JSON string result to the calling client. In this case, the client is a mobile application (iPad, Android, etc) with a JavaScript library that I'll use to send the requests to the server.
Why would I need a Python client? From what I can gather, the client just needs to open a connection to the server and then send the JSON string, right? Why do all the code samples include Python client examples? Are they assuming a server machine is going to talk to a server machine, so they have included client code to help generate the JSON string that will be sent to the server?
If I assume that a Python client isn't really needed for anything, I've been sending JSON strings to the Python server from the iPad, but in each case the server is reporting a "Bad request syntax" error. I'll pen a new question on that issue if I'm understanding the current question correctly.
Insight is appreciated.

The JSON encoding is the lingua franca of your RPC protocol, so you can indeed use any client you like. The implementations you found for JSON-RPC use the HTTP protocol, a very specific communication protocol built on top of TCP/IP, but you can implement the same protocol over raw TCP-IP sockets if so required.
The examples include both the Python client and the server because they illustrate how to implement the JSON-RPC standard in Python, not in JavaScript or C or Lisp. They focus on the implementation in one language. The JSON-RPC standard however, is language agnostic. It doesn't matter what language you write either the server or the client in, as long as they use the same standard.

Making moves w/ websockets and python / django ( / twisted? )

The fun part of websockets is sending essentially unsolicited content from the server to the browser right?
Well, I'm using django-websocket by Gregor Müllegger. It's a really wonderful early crack at making websockets work in Django.
I have accomplished "hello world." The way this works is: when a request is a websocket, an object, websocket, is appended to the request object. Thus, I can, in the view interpreting the websocket, do something like:
request.websocket.send('We are the knights who say ni!')
That works fine. I get the message back in the browser like a charm.
But what if I want to do that without issuing a request from the browser at all?
OK, so first I save the websocket in the session dictionary:
request.session['websocket'] = request.websocket
Then, in a shell, I go and grab the session by session key. Sure enough, there's a websocket object in the session dictionary. Happy!
However, when I try to do:
>>> session.get_decoded()['websocket'].send('With a herring!')
I get:
Traceback (most recent call last):
File "<console>", line 1, in <module>
error: [Errno 9] Bad file descriptor
Sad. :-(
OK, so I don't know much of anything about sockets, but I know enough to sniff around in a debugger, and lo and behold, I see that the socket in my debugger (which is tied to the genuine websocket from the request) has fd=6, while the one that I grabbed from the session-saved websocket has fd=-1.
Can a socket-oriented person help me sort this stuff out?

I'm the author of django-websocket. I'm not a real expert in the topic of websockets and networking, however I think I have a decent understanding of whats going on. Sorry for going into great detail. Even if most of the answer isn't specific to your question it might help you at some other point. :-)
How websockets work
Let me explain shortly what a websocket is. A websocket starts as something that really looks like a plain HTTP request, established from the browser. It indicates through a HTTP header that it wants to "upgrade" the protocol to be a websocket instead of a HTTP request. If the server supports websockets, it agrees on the handshake and both - server and client - now know that they will use the established tcp socket formerly used for the HTTP request as a connection to interchange websocket messages.
Beside sending and waiting for messages, they have also of course the ability to close the connection at any time.
How django-websocket abuses the python's wsgi request environment to hijack the socket
Now lets get into the details of how django-websocket implements the "upgrading" of the HTTP request in a django request-response cylce.
Django usually uses the WSGI specification to talk to the webserver like apache or gunicorn etc. This specification was designed just with the very limited communication model of HTTP in mind. It assumes that it gets a HTTP request (only incoming data) and returns the response (only outgoing data). This makes it tricky to force django into the concept of a websocket where bidirectional communication is allowed.
What I'm doing in django-websocket to achieve this is that I dig very deeply into the internals of WSGI and django's request object to retrieve the underlaying socket. This tcp socket is then used to handle the upgrade the HTTP request to a websocket instance directly.
Now to your original question ...
I hope the above makes it obvious that when a websocket is established, there is no point in returning a HttpResponse. This is why you usually don't return anything in a view that is handled by django-websocket.
However I wanted to stick close to the concept of a view that holds the logic and returns data based on the input. This is why you should only use the code in your view to handle the websocket.
After you return from the view, the websocket is automatically closed. This is done for a reason: We don't want to keep the socket open for an undefined amount of time and relying on the client (the browser) to close it.
This is why you cannot access a websocket with django-websocket outside of your view. The file descriptor is then of course set to -1 indicating that its already closed.
Disclaimer
I explained above that I'm digging in the surrounding environment of django to get somehow -- in a very hackish way -- access to the underlaying socket. This is very fragile and also not supposed to work since WSGI is not designed for this! I also explained above that the websocket is closed after the view ends - however after the websocket closed down (AND closed the tcp socket), django's WSGI implementation tries to send a HTTP response - it doesn't know about websockets and thinks it is in a normal HTTP request-response cycle. But the socket is already closed an the sending will fail. This usually causes an exception in django.
This didn't affected my testings with the development server. The browser will never notice (you know .. the socket is already closed ;-) - but raising an unhandled error in every request is not a very good concept and may leak memory, doesn't handle database connection shutdown correctly and many athor things that will break at some point if you use django-websocket for more than experimenting.
This is why I would really advise you not to use websockets with django yet. It doesn't work by design. Django and especially WSGI would need a total overhaul to solve these problems (see this discussion for websockets and WSGI). Since then I would suggest using something like eventlet. Eventlet has a working websocket implementation (I borrowed some code from eventlet for the initial version of django-websocket) and since its just plain python code you can import your models and everything else from django. The only drawback is that you need a second webserver running just to handle websockets.

As Gregor Müllegger pointed out, Websockets can't be properly handled by WSGI, because that protocol never was designed to handle such a feature.
uWSGI, since version 1.9.11, can handle Websockets out of the box. Here uWSGI communicates with the application server using raw HTTP rather than the WSGI protocol. A server written that way, can therefore handle the protocol internals and keep the connection open over a long period. Having long living connections handled by a Django view is not a good idea either, because they then would block a worker thread, which is a limited resource.
The main purpose of Websockets, is to have the server push messages to the client in an asynchronous way. This can be a Django view triggered by other browsers (ex.: chat clients, multiplayer games), or an event triggered by, say django-celery (ex.: sport results). It therefore is fundamental for these Django services, to use a message queue for pushing messages to the client.
To handle this in a scalable way, I wrote django-websocket-redis, a Django module which can keep open all those long living Websocket connections in one single thread/process using Redis as the backend message queue.

You could give stargate a bash: http://boothead.github.com/stargate/ and http://pypi.python.org/pypi/stargate/.
It's built on top of pyramid and eventlet (I also contributed a fair bit of the websocket support and tests to eventlet). The big advantage of pyramid for this sort of thing is that it's got the concept of a resource which the url maps to, rather than just the result of a callable. So you end up with a graph of persistent resources that maps to your url structure and websocket connections are simply routed and connected to those resources.
So you end up only needing to do two things:
class YourView(WebSocketView):
def handler(self, websocket):
self.request.context.add_listener(websocket)
while True:
msg = websocket.wait()
# Do something with message
To receive messages
and
resource.send(some_other_message)
Here resource is an instance of a stargate.resource.WebSocketAwareContext (as is self.request.context) above and the send method sends the message to all clients connected with the add_listener method.
To publish a message to all of the connected clients you just call node.send(message)
I'm hopefully going to write up a little example app in the next week or two to demonstrate this a little better.
Feel free to ping me on github if you want some help with it.

request.websocket is probably get closed when you return from the request handler (view). The simple solution is to keep the handler alive (by not returning from the view). If your server is not multi-threaded you won't be able to accept any other simultaneous requests.