We Keep Coding

How to handle multithreading with sockets in Python?

I am currently working on a problem where I need to have a server handle multiple clients at the same time. I have a server.py file and a client.py file. Here is a snippet from server.py:
connections = []
addresses = []
for c in connections:
c.close()
del connections[:]
del addresses[:] #clears anything pre-existing
while True:
try:
csock, address = s.accept() #takes in connection and stores info
s.setblocking(1) #prevents timeout
connections.append(csock)
addresses.append(address)
print(f"Connection from {address} has been established!")
except:
print("Error has occurred")
When I run my server in terminal and then connect to it with a client. It behaves as I would expect, with it printing out Connection from ('192.168.1.84', 50824) has been established!.
When I open up another terminal and run client.py to make an additional connection, nothing happens. That is, until I close out of my first client process and then the server prints out
Error occurred
Connection from ('192.168.1.84', 50826) has been established!
I can kind of see what is happening here, but I'm very new to networking and I'm not super great at multithreading, so could anyone give me some insight as to what's going on and what I can do to make these processes run simultaneously as I would expect?

After s.accept() you should use threading to run code in separated thread - and this thread should continue connection with client. At the same time main thread may go back to s.accept() to wait for next client.
Minimal working code with some extra settings.
Server:
import socket
import threading
import time
# --- functions ---
def handle_client(conn, addr):
print("[thread] starting")
# recv message
message = conn.recv(1024)
message = message.decode()
print("[thread] client:", addr, 'recv:', message)
# simulate longer work
time.sleep(5)
# send answer
message = "Bye!"
message = message.encode()
conn.send(message)
print("[thread] client:", addr, 'send:', message)
conn.close()
print("[thread] ending")
# --- main ---
host = '0.0.0.0'
port = 8080
s = socket.socket()
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) # solution for "[Error 89] Address already in use". Use before bind()
s.bind((host, port))
s.listen(1)
all_threads = []
try:
while True:
print("Waiting for client")
conn, addr = s.accept()
print("Client:", addr)
t = threading.Thread(target=handle_client, args=(conn, addr))
t.start()
all_threads.append(t)
except KeyboardInterrupt:
print("Stopped by Ctrl+C")
finally:
if s:
s.close()
for t in all_threads:
t.join()
Client (for test)
import socket
# --- main ---
host = '0.0.0.0'
port = 8080
s = socket.socket()
s.connect((host, port))
print("Connected to the server")
message = "Hello"
print('send:', message)
message = message.encode()
s.send(message)
message = s.recv(1024)
message = message.decode()
print('recv:', message)

Python socket : Error receive data

I write a network application. The server has ability to find client base on given subnet. If the client receive authentication message from server, it will respond to server. Everything working good but server, it can't receiver from client.
Client :
def ListenServer():
# Listen init signal from Server to send data
HOST = '' # Symbolic name meaning all available interfaces
PORT = 50007 # Arbitrary non-privileged port
# UDP Socket
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.bind((HOST, PORT))
data, addr = s.recvfrom(1024)
if data == 'Authen':
SocketConnect(addr[0])
def SocketConnect(HOST):
# Connect to Server to send data
print HOST
PORT = 50008 # The same port as used by the server
# Create Socket
print "Create Socket"
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except socket.error, e:
print "Error creating socket: %s" %e
sys.exit(1)
# Connect
print "Connect"
try:
s.connect((HOST, PORT))
except socket.error, e:
print "Connection error: %s" %e
sys.exit(1)
# Send Data
print "Send Data"
try:
s.sendall('Hello, world')
except socket.error, e:
print "Error sending data: %s" % e
sys.exit(1)
# Close Socket
s.close()
print "Close Socket"
ListenServer()
Server :
from netaddr import IPAddress
import socket
import sys
import ipaddress
import time
def FindAgent():
PORT = 50007 # Port use to find Agent
#Find broadcast address
"""IPAddress("255.255.255.0").netmask_bits() #Convert Subnet Mask to Prefix Length, Result is 24"""
try :
HOST = str(ipaddress.ip_network(u'192.168.10.0/24')[-1])
except ValueError as e :
"""e = sys.exc_info()[0] # Find Exception you need"""
print e
# UDP client
MESSAGE = "Authen"
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
for x in range(0,2):
sock.sendto(MESSAGE, (HOST, PORT))
def ListenClient():
# Listen Client sent data
HOST = socket.gethostbyname(socket.gethostname())
PORT = 50008
# TCP socket
# Create Socket
print "Create Socket"
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except socket.error, e:
print "Error creating socket: %s" %e
sys.exit(1)
# Bind
print "Bind"
try:
s.bind((HOST, PORT))
except socket.error, e:
print "Error bind: %s" %e
sys.exit(1)
# Listen
print "Listen"
try:
s.listen(10)
except socket.error, e:
print "Error listen: %s" %e
sys.exit(1)
# Accept data from client
print "Accept data from client"
try:
conn, addr = s.accept()
data = s.recv(1024)
except socket.error, e:
print "Error listen: %s" %e
sys.exit(1)
print data
s.close()
FindAgent()
ListenClient()
Error on Server :
Create Socket
Bind
Listen
Accept data from client
Error listen: [Errno 10057] A request to send or receive data was disallowed because the socket is not connected and (when sending on a datagram socket using a sendto call) no address was supplied
[Finished in 0.8s with exit code 1]
[shell_cmd: python -u "C:\Users\Win7_Lab\Desktop\Server.py"]
[dir: C:\Users\Win7_Lab\Desktop]
[path: C:\Python27\;C:\Python27\Scripts;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\]
Without the line data = s.recv(1024) on Server, it working fine. But with it, the error show up. Can anybody please tell me why it happen ?

The crash come from s.recv(1024) as you said it's because the recieve (.recv()) methode on your server need to be called on the client connection.
Follow this example : here
Server file :
conn, addr = s.accept()
data = conn.recv(1024)
Client file :
s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
s.connect((HOST, PORT))
data = s.recv(1024)
As you see the client recieve data from socket (server).
And server recieve data from client connection.
Hope that was usefull.
Edit add some examples links.
You can find what you want in these tutorial:
How to use socket
How to create server with socket and select
Example client server giving time
Example chat client server

Errno 98: Address already in use - Python Socket

This question has been asked before but none of the answers was helpful in my case. The problem seems very simple. I am running a TCP server on an raspberry pi and try to connect to it from another machine. I have a custom class receiver that pipes sensor data to this script.
When I close the program running on the other machine (the socket is 'shutdown(2)'d and then 'close()'d), I cannot reconnect to that same port anymore. I tried to alternate between two sockets (1180 and 1181) but this did not work. When I connect over a port once, it is gone forever until I restart the TCP server. I tried restarting the script (with executl()) but that did not resolve my problem. I am telling the socket that it should re-use addresses but to no avail.
What I could do is use more ports but that would require opening more ports on the RPi which I would like to avoid (there must be another way to solve this).
import socket
from receiver import receiver
import pickle
import time
import os
import sys
TCP_IP = ''
TCP_PORT = 1180
BUFFER_SIZE = 1024
print 'Script started'
while(1):
try:
print 'While begin'
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
print 'Socket created'
s.settimeout(5)
print 'Trying to bind'
s.bind((TCP_IP, TCP_PORT))
print 'bound to', (TCP_IP, TCP_PORT)
s.listen(1)
print 'listening for connection'
conn, addr = s.accept()
print 'accepted incoming connection'
s.settimeout(5)
r = receiver()
print 'Connection address:', addr
for cur in r:
#print "sending data:", cur
print len(cur.tostring())
conn.send(cur.tostring()) # echo
except Exception as e:
r.running = False
print e
if TCP_PORT == 1181:
TCP_PORT = 1180
else:
TCP_PORT = 1181
time.sleep(1)
print 'sleeping 1sec'

Your server socket is still in use, so you cannot open more than one server socket for each port. But why should one. Just reuse the same socket for all connections (that's what server sockets made for):
import socket
from receiver import receiver
import logging
TCP_IP = ''
TCP_PORT = 1180
BUFFER_SIZE = 1024
print 'Script started'
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print 'Socket created'
server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
print 'Trying to bind'
server.bind((TCP_IP, TCP_PORT))
print 'bound to', (TCP_IP, TCP_PORT)
server.listen(1)
print 'listening for connection'
while True:
try:
conn, addr = server.accept()
print 'accepted incoming connection'
print 'Connection address:', addr
for cur in receiver():
data = cur.tostring()
#print "sending data:", cur
print len(data)
conn.sendall(data) # echo
except Exception:
logging.exception("processing request")

python3 - broken pipe error when using socket.send()

I am programming a client-server instant message program. I created a similar program in Python 2, and am trying to program it in Python 3. The problem is when the server takes the message and tries to send it to the other client, it gives me "[Errno 32] Broken Pipe" and exits.
I have done some research, and found that this occurs when the client disconnects, so I did some more testing but could not find when the client disconnects. (I am using Ubuntu 14.04 and Python 3.4)
Here is the server code:
import socket, select, sys
def broadcast(sock, messaged):
for socket in connection_list:
if socket != s and socket != sock:
# Here is where it gives me the broken pipe error
try:
s.send(messaged.encode("utf-8"))
except BrokenPipeError as e:
print(e)
sys.exit()
connection_list = []
host = ''
port = 5558
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((host,port))
s.listen(5)
connection_list.append(s)
read_sockets,write_sockets,error_sockets = select.select(connection_list,[],[])
while True:
for sock in read_sockets:
if sock == s:
conn, addr = s.accept()
connection_list.append(conn)
client = "Client (%s,%s) connected" % addr
print(client)
broadcast(sock,client)
else:
try:
data = sock.recv(2048)
decodeddata = data.decode("utf-8")
if data:
broadcast(sock, decodeddata)
except:
offline = "Client " + addr + "is offline"
broadcast(sock, offline)
print(offline)
connection_list.remove(sock)
sock.close()
continue
And the client code:
import socket, select, string, sys, time
def prompt(data) :
print("<You> " + data)
def Person(data) :
print("<Receiver> " + data)
if __name__ == "__main__":
host = "localhost"
port = 5558
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.settimeout(2)
try:
s.connect((host,port))
except:
print('Unable to connect')
sys.exit()
print('Connected.')
socket_list = [s]
read_sockets,write_sockets,error_sockets = select.select(socket_list,[],[])
while 1:
for sock in read_sockets:
if sock == s:
try:
time.sleep(1)
data = sock.recv(1024)
Person(data.decode("utf-8"))
except:
msg = input("Send a message: ")
try:
s.send(str.encode(msg))
except:
print("Server is offline")
sys.exit()
else:
print("Server is offline")
sys.exit()

There are two problems that you have to fix to make this work.
First, on both the client side and the server side, you have to put the select inside the loop, not outside. Otherwise, if there was something to read before you got to the loop, you'll recv over and over, and if there wasn't, you'll never recv. Once you fix this, you can get rid of the time.sleep(1). (You should never need a sleep to solve a problem like this; at best it masks the problem, and usually introduces new ones.)
Meanwhile, on the server side, inside broadcast, you're doing s.send. But s is your listener socket, not a connected client socket. You want socket.send here, because socket is each socket in connection_list.
There are a number of unrelated problems in your code as well. For example:
I'm not sure what the except: in the client is supposed to be catching. What it mainly seems to catch is that, about 50% of the time, hitting ^C to end the program triggers the send prompt. But of course, like any bare except:, it also masks any other problems with your code.
There's no way to send any data back and forth other than the "connected" message except for that except: clause.
addr is a tuple of host and port, so when someone goes offline, the server raises a TypeError from trying to format the offline message.
addr is always the last client who connected, not the one who's disconnecting.
You're not setting your sockets to nonblocking mode.
You're not checking for EOF on the recv. This means that you don't actually detect that a client has gone offline until you get an error. Which normally happens only after you try to send them a message (e.g., because someone else has connected or disconnected).

Python - Server and client problems

I'm trying to create a basic server and client script. The idea is that the client can connect to the server and execute commands. Kinda like SSH but very simple. Heres my server code:
import sys, os, socket
host = ''
port = 50103
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((host, port))
print("Server started on port: ", port)
s.listen(1)
while (1):
conn, addr = s.accept()
print 'New connection from ', addr
try:
while True:
rc = conn.recv(2)
pipe = os.popen(rc)
rl = pipe.readlines()
fl = conn.makefile('w', 0)
fl.writelines(rl[:-1])
fl.close()
except IOError:
conn.close()
And here is my client:
import sys, socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host = 'localhost'
port = input('Port: ')
s.connect((host, port))
while (1):
cmd = raw_input('$ ')
s.send(cmd)
file = s.makefile('r', 0)
sys.stdout.writelines(file.readlines())
file.close()
Here is my problem. I start the server and then run the client on the same machine. I enter the port and connect. Then I get the raw_input which is the '$'. If I type a command like 'ls' it just hangs on the client side. I have to exit the server for the client to receive the output of ls. By the way I am running Ubuntu Linux. Not sure if that matters.

When you makefile() on the socket and then use readlines() on it, it will continue until you reach an end of file, which in the socket case is that it closed from the other end.
Using makefile() in this case makes no sense to me, especially since you create it and close it after each command. Just use send() and recv() on both ends.
You probably also want to have some sort of actual "protocol" so the server tells the client "HERE COMES A RESPONSE" and "THIS IS THE END OF THE RESPONSE" so that the client knows. Otherwise it gets hard to know when to stop waiting for more response. :)
Update with an example that works:
server.py:
import sys, os, socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(('', 50500))
print("Server started")
s.listen(1)
while True:
print "Accepting"
conn, addr = s.accept()
print 'New connection from ', addr
while True:
try:
rc = conn.recv(1024)
print "Command", rc
if not rc.strip():
continue
if rc.strip() == 'END':
print "Close"
conn.send("**END**")
conn.close()
break
else:
conn.send("This is the result of command %s\n" % rc)
except Exception:
conn.close()
sys.exit()
client.py
import sys, os, socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('localhost', 50500))
while True:
cmd = raw_input('$ ')
s.send(cmd)
result = s.recv(1024)
print result
if result == "**END**":
print "Ending"
break

Well for one thing you're only connecting on the client once and on the server you're closing the socket after every read.
You should take a look at this example.
http://ilab.cs.byu.edu/python/socket/echoserver.html
You're doing quite a few things incorrectly.