We Keep Coding

How to process data stream from multiple client sockets without closing any of those connections?

I have a server which accepts a stream of JSON data from a client socket. The server needs to keep all the connections open forever and process the data that comes from them, but the current version of my code can handle a single connection only because the second while loop never ends once the first connection has been established.
import socket
IP = '127.0.0.1'
PORT = 1235
BUFFER = 10
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server_socket.bind((IP, PORT))
server_socket.listen(10)
print(f"Listening for incoming connections # {IP}:{PORT}")
while True:
client_socket, client_address = server_socket.accept()
print(f"Established connection with {client_address}")
while True:
received_message = client_socket.recv(BUFFER).decode('utf-8')
while '}' not in received_message:
received_message += client_socket.recv(BUFFER).decode('utf-8')
print(client_address, received_message)
client_socket.send("Message received".encode('utf-8'))

Was able to solve the problem by implementing threading. Here's the working code if anyone wants to use it.
import socket
from _thread import *
IP = '127.0.0.1'
PORT = 1235
BUFFER = 100
thread_count = 0
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server_socket.bind((IP, PORT))
server_socket.listen(10)
def threaded_client(client_socket, address):
while True:
received_message = client_socket.recv(BUFFER).decode('utf-8')
while '}' not in received_message:
received_message += client_socket.recv(BUFFER).decode('utf-8')
print(client_address, received_message)
client_socket.send("Message received.".encode('utf-8'))
while True:
client_socket, client_address = server_socket.accept()
print(f"Established connection with {client_address}")
start_new_thread(threaded_client, (client_socket, client_address))
thread_count += 1
print(thread_count)

The primary problem you're facing is that almost all of python's IO is blocking. That means that the execution of the program stops when you want to read or write to a file, or to a socket in this case. The standard solution is to use threads through the threading-library, but there are many other options.
I'd recommend watching David Beazley's video on concurrency, which covers this topic quite well! It can get a little bit complicated, but that's because it's covering a lot of ground fairly quickly!

Socket server running other code and sending updates to clients

This may have been already answered but did not find anything or dont know what to search for. I have a socket server threaded for multiple clients (code below) and want the server to run code (ie. check the status of something) and then send a message to the clients. How do I go about doing this?
To clarify, I need to learn how to add a asynchronous task to this code so it can do checking and send a message to the clients if it needs to
Example: while there are clients connected I want the server to continually check a log file and if it changes and if so I want it to send a message to the clients
Server.py(working code)
from socket import *
import thread
BUFF = 1024
HOST = '127.0.0.1'# must be input parameter #TODO
PORT = 9999 # must be input parameter #TODO
def response(key):
return 'Server response: ' + key
def handler(clientsock,addr):
while 1:
data = clientsock.recv(BUFF)
if not data: break
print repr(addr) + ' recv:' + repr(data)
clientsock.send(response(data))
print repr(addr) + ' sent:' + repr(response(data))
if "close" == data.rstrip(): break # type 'close' on client console to close connection from the server side
clientsock.close()
print addr, "- closed connection" #log on console
if __name__=='__main__':
ADDR = (HOST, PORT)
serversock = socket(AF_INET, SOCK_STREAM)
serversock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
serversock.bind(ADDR)
serversock.listen(5)
while 1:
print 'waiting for connection... listening on port', PORT
clientsock, addr = serversock.accept()
print '...connected from:', addr
thread.start_new_thread(handler, (clientsock, addr))

I have added below code server.py and this works for sending a status change message when log file changes.
import socket
import sys
import time
from thread import *
HOST = 'localhost' # Symbolic name meaning all available interfaces
PORT = 9999 # Arbitrary non-privileged port
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print 'Socket created'
#Bind socket to local host and port
try:
s.bind((HOST, PORT))
except socket.error as msg:
print 'Bind failed. Error Code : ' + str(msg[0]) + ' Message ' + msg[1]
sys.exit()
print 'Socket bind complete'
#Start listening on socket
s.listen(10)
print 'Socket now listening'
#Function for handling connections. This will be used to create threads
def clientthread(conn):
#Sending message to connected client
conn.send('Welcome to the server. Type something and hit enter\n') #send only takes string
data = conn.recv(1024)
#infinite loop so that function do not terminate and thread do not end.
while True:
logfile = open("serverlog.txt","r")
#Receiving from client
logfile.seek(0,2)
while True:
line = logfile.readline()
if not line:
time.sleep(0.1) # Sleep briefly
continue
reply = 'File Changed...Your Data' + data
break
conn.sendall(reply)
#came out of loop
conn.close()
#now keep talking with the client
while 1:
#wait to accept a connection - blocking call
conn, addr = s.accept()
print 'Connected with ' + addr[0] + ':' + str(addr[1])
#start new thread takes 1st argument as a function name to be run, second is the tuple of arguments to the function.
start_new_thread(clientthread ,(conn,))
s.close()
To test it run this server.py in one command prompt and keep it open. something like this
> python server.py
Socket created
Socket bind complete
Socket now listening
And run a simple telnet from other cmd prompt to verify the connection
telnet localhost 9999
Replace localhost with IP. Type anything on these telnet connection and you should get response properly.
Also you can check on server.py cmd prompt for the connections made.
And as I mentioned, check this link.

Python server programming

I am trying a little client server project to get me into network programming but I seem to have got stuck at the first hurdle. I cant seem to get past getting the first line of data only even if its a new connection.
#!/usr/bin/python
import socket
s = socket.socket()
host = '192.168.0.233' # Test Server
port = 7777
s.bind((host, port))
s.listen(5)
while True:
c, addr = s.accept()
print 'Got connection from', addr
data = c.recv(2048)
print(data)
If I telnet to the host running the server, the connection opens fine and I see on the server Got connection from addr, but I also only see the first line of data when I sent 4 lines of data,
I thought because its in a loop it should now always be looking for data?
I know im doing something wrong but unsure what.
Im using Python 2.6.6

recv needs to be in a loop too, at the moment your code is receiving some data and then waiting for a new connection.
https://docs.python.org/2/library/socket.html#example has an example of socket.recv in a loop.

Try this:
#!/usr/bin/python
import socket
import threading
def listenForClients(sock):
while True:
client, address = sock.accept()
client.settimeout(5)
threading.Thread( target = listenToClient, args = (client,address) ).start()
def listenToClient(client, address):
size = 2048
while True:
try:
data = client.recv(size)
if data:
response = "Got connection"
client.send(response)
else:
raise error('Client disconnected')
except:
client.close()
return False
def main(host, port):
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind((host, port))
sock.listen(5)
listenForClients(sock)
if __name__ == "__main__":
main('192.168.0.233',7777)
Here I use a thread for each client. The problem that you have with having Socket.accept() in the loop is that it blocks meaning that concurrent access won't work and you'll only be able to talk to one client at a time.
Try running it in the background and sending it messages with:
#!/usr/bin/python
import socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect(('192.168.0.233',7777))
vwhile True:
data = raw_input("enter a message: ")
sock.send(data)
print sock.recv(2048)

Sending string via socket (python)

I have two scripts, Server.py and Client.py.
I have two objectives in mind:
To be able to send data again and again to server from client.
To be able to send data from Server to client.
here is my Server.py :
import socket
serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host = "192.168.1.3"
port = 8000
print (host)
print (port)
serversocket.bind((host, port))
serversocket.listen(5)
print ('server started and listening')
while 1:
(clientsocket, address) = serversocket.accept()
print ("connection found!")
data = clientsocket.recv(1024).decode()
print (data)
r='REceieve'
clientsocket.send(r.encode())
and here is my client :
#! /usr/bin/python3
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host ="192.168.1.3"
port =8000
s.connect((host,port))
def ts(str):
s.send('e'.encode())
data = ''
data = s.recv(1024).decode()
print (data)
while 2:
r = input('enter')
ts(s)
s.close ()
The function works for the first time ('e' goes to the server and I get return message back), but how do I make it happen over and over again (something like a chat application) ?
The problem starts after the first time. The messages don't go after the first time.
what am I doing wrong?
I am new with python, so please be a little elaborate, and if you can, please give the source code of the whole thing.

import socket
from threading import *
serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host = "192.168.1.3"
port = 8000
print (host)
print (port)
serversocket.bind((host, port))
class client(Thread):
def __init__(self, socket, address):
Thread.__init__(self)
self.sock = socket
self.addr = address
self.start()
def run(self):
while 1:
print('Client sent:', self.sock.recv(1024).decode())
self.sock.send(b'Oi you sent something to me')
serversocket.listen(5)
print ('server started and listening')
while 1:
clientsocket, address = serversocket.accept()
client(clientsocket, address)
This is a very VERY simple design for how you could solve it.
First of all, you need to either accept the client (server side) before going into your while 1 loop because in every loop you accept a new client, or you do as i describe, you toss the client into a separate thread which you handle on his own from now on.

client.py
import socket
s = socket.socket()
s.connect(('127.0.0.1',12345))
while True:
str = raw_input("S: ")
s.send(str.encode());
if(str == "Bye" or str == "bye"):
break
print "N:",s.recv(1024).decode()
s.close()
server.py
import socket
s = socket.socket()
port = 12345
s.bind(('', port))
s.listen(5)
c, addr = s.accept()
print "Socket Up and running with a connection from",addr
while True:
rcvdData = c.recv(1024).decode()
print "S:",rcvdData
sendData = raw_input("N: ")
c.send(sendData.encode())
if(sendData == "Bye" or sendData == "bye"):
break
c.close()
This should be the code for a small prototype for the chatting app you wanted.
Run both of them in separate terminals but then just check for the ports.

This piece of code is incorrect.
while 1:
(clientsocket, address) = serversocket.accept()
print ("connection found!")
data = clientsocket.recv(1024).decode()
print (data)
r='REceieve'
clientsocket.send(r.encode())
The call on accept() on the serversocket blocks until there's a client connection. When you first connect to the server from the client, it accepts the connection and receives data. However, when it enters the loop again, it is waiting for another connection and thus blocks as there are no other clients that are trying to connect.
That's the reason the recv works correct only the first time. What you should do is find out how you can handle the communication with a client that has been accepted - maybe by creating a new Thread to handle communication with that client and continue accepting new clients in the loop, handling them in the same way.
Tip: If you want to work on creating your own chat application, you should look at a networking engine like Twisted. It will help you understand the whole concept better too.

Python - Server and client problems

I'm trying to create a basic server and client script. The idea is that the client can connect to the server and execute commands. Kinda like SSH but very simple. Heres my server code:
import sys, os, socket
host = ''
port = 50103
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((host, port))
print("Server started on port: ", port)
s.listen(1)
while (1):
conn, addr = s.accept()
print 'New connection from ', addr
try:
while True:
rc = conn.recv(2)
pipe = os.popen(rc)
rl = pipe.readlines()
fl = conn.makefile('w', 0)
fl.writelines(rl[:-1])
fl.close()
except IOError:
conn.close()
And here is my client:
import sys, socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host = 'localhost'
port = input('Port: ')
s.connect((host, port))
while (1):
cmd = raw_input('$ ')
s.send(cmd)
file = s.makefile('r', 0)
sys.stdout.writelines(file.readlines())
file.close()
Here is my problem. I start the server and then run the client on the same machine. I enter the port and connect. Then I get the raw_input which is the '$'. If I type a command like 'ls' it just hangs on the client side. I have to exit the server for the client to receive the output of ls. By the way I am running Ubuntu Linux. Not sure if that matters.

When you makefile() on the socket and then use readlines() on it, it will continue until you reach an end of file, which in the socket case is that it closed from the other end.
Using makefile() in this case makes no sense to me, especially since you create it and close it after each command. Just use send() and recv() on both ends.
You probably also want to have some sort of actual "protocol" so the server tells the client "HERE COMES A RESPONSE" and "THIS IS THE END OF THE RESPONSE" so that the client knows. Otherwise it gets hard to know when to stop waiting for more response. :)
Update with an example that works:
server.py:
import sys, os, socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(('', 50500))
print("Server started")
s.listen(1)
while True:
print "Accepting"
conn, addr = s.accept()
print 'New connection from ', addr
while True:
try:
rc = conn.recv(1024)
print "Command", rc
if not rc.strip():
continue
if rc.strip() == 'END':
print "Close"
conn.send("**END**")
conn.close()
break
else:
conn.send("This is the result of command %s\n" % rc)
except Exception:
conn.close()
sys.exit()
client.py
import sys, os, socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('localhost', 50500))
while True:
cmd = raw_input('$ ')
s.send(cmd)
result = s.recv(1024)
print result
if result == "**END**":
print "Ending"
break

Well for one thing you're only connecting on the client once and on the server you're closing the socket after every read.
You should take a look at this example.
http://ilab.cs.byu.edu/python/socket/echoserver.html
You're doing quite a few things incorrectly.