We Keep Coding

Python3 - [Errno 32] Broken Pipe while using sockets

I have the following code for the server:
import socket
import threading
def handle_client(client_socket):
request = client_socket.recv(1024)
print ("[*] Received: %s" % request)
client_socket.send("ACK!".encode("utf-8"))
client_socket.close()
bind_ip = "0.0.0.0"
bind_port = 9998
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind((bind_ip, bind_port))
server.listen(5)
print ("[*] Listening on %s:%d" % (bind_ip, bind_port))
while True:
client, addr = server.accept()
print ("[*] Accepted connection from: %s:%d" % (addr[0], addr[1]))
client_handler = threading.Thread(target = handle_client, args = (client))
client_handler.start()
And the following on the client side:
def client_sender(buffer):
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect((target, port))
if len(buffer) > 0:
client.send(buffer.encode("utf-8"))
data = client.recv(4096).decode("utf-8")
print(" - " + data)
while True:
buffer = input("Input:")
buffer += "\n"
client.send(buffer.encode("utf-8"))
data = client.recv(4096).decode("utf-8")
print (" - " + data)
The function client_sender is called by another function where I can choose if listen incoming connection or send data to someone.
I will call it in this way:
python3 filename.py -ip 0.0.0.0 -port 9998
I can then write some data. The first time I do it I press CTRL + D to send it and the server gets and the client gets the response.But when I try to send some data from inside the "While True" loop the server never gets it and I receive this:
client.send(buffer.encode("utf-8"))
BrokenPipeError: [Errno 32] Broken pipe
How do I solve it? The only solution I found that works is to move these two lines inside the "While True" loop:
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect((target, port))
However it seems inefficient to me to re-connect every time.
EDIT
As suggested by Sam Mason, changing the handle_client in the following way solves the problem:
def handle_client(client_socket):
while True:
request = client_socket.recv(1024).decode("utf-8")
if len(request):
print ("[*] Received: %s" % request)
client_socket.send("ACK!".encode("utf-8"))
else:
client_socket.close()
break

your server is closing the port immediately after a single recv. I'd suggest changing your handle_client code to have some sort of while loop that ends when recv returns an empty string (this indicates the client has shutdown their end of the connection, probably by closeing their connection)

An operation was attempted on something that is not a socket (tried fixing a lot)

Before you say this is a duplicate, I have looked at many articles on this and still can't fix it.
I am making a very basic chat client and server python program.
However after connecting through my client, it says 'Connected' on the server console, but disconnects immediately on the chat one with the error 'OSError: [WinError 10038] An operation was attempted on something that is not a
socket'
CHAT
def chat_client():
if(len(sys.argv) not in (3, 4)):
print("Usage: python chat_client.py <hostname> <port> <optional-username>\n")
sys.exit()
host = sys.argv[1]
port = int(sys.argv[2])
username = ""
if len(sys.argv) == 4:
username = sys.argv[3]
else:
username = "Guest"
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.settimeout(2)
# Connect to remote host
try:
s.connect((host, port))
except:
print("Unable to connect")
sys.exit()
print("Connected to remote host. You can start sending messages")
print("*** Press Control-C to log off ***\n")
sys.stdout.write("[" + username + "] ")
sys.stdout.flush()
while True:
socket_list = [sys.stdin, s]
try:
# Get the list sockets which are readable
ready_to_read, ready_to_write, in_error = select.select(socket_list, [], [])
except KeyboardInterrupt:
system("clear")
sys.stdout.write("\nYou have logged off\n")
sys.stdout.flush()
sys.exit()
SERVER
HOST = ""
SOCKET_LIST = []
RECV_BUFFER = 4096
PORT = 9009
CONVERSATION = ""
def chat_server():
global CONVERSATION
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server_socket.bind((HOST, PORT))
server_socket.listen(10)
# Add server socket object to the list of readable connections
SOCKET_LIST.append(server_socket)
print("Chat server started on port " + str(PORT))
while True:
try:
# Get the list sockets which are ready to be read through select
# 4th arg, time_out = 0 : poll and never block
ready_to_read, ready_to_write, in_error = select.select(SOCKET_LIST, [], [], 0)
for sock in ready_to_read:
# A new connection request recieved
if sock == server_socket:
sockfd, addr = server_socket.accept()
SOCKET_LIST.append(sockfd)
print("Client (%s, %s) connected" % addr)
broadcast(server_socket, sockfd, "[%s, %s] entered our chatting room\n" % addr)
# A message from a client, not a new connection
else:
# Process data recieved from client
try:
# Recieving data from socket
data = sock.recv(RECV_BUFFER)
if data:
# there is something in the socket
# broadcast(server_socket, sock, "\r" + '[' + str(sock.getpeername()) + '] ' + data) # old
broadcast(server_socket, sock, "\r" + data)
else:
# Remove the socket that's broken
if sock in SOCKET_LIST:
SOCKET_LIST.remove(sock)
# at this stage, no data probably means the connection has been broken
broadcast(server_socket, sock, "Client (%s, %s) is offline\n" % addr)
except:
broadcast(server_socket, sock, "Client (%s, %s) is offline\n" % addr)
continue
except KeyboardInterrupt:
server_socket.close()
sys.exit()
server_socket.close()
# broadcast chat messages to all connected clients
def broadcast(server_socket, sock, message):
for socket in SOCKET_LIST:
# send the message only to peer
if socket != server_socket and socket != sock:
try:
socket.send(message)
except:
# Broken socket connection
socket.close()
# Broken socket, remove it
if socket in SOCKET_LIST:
SOCKET_LIST.remove(socket)
if __name__ == "__main__":
sys.exit(chat_server())

From select's documentation:
File objects on Windows are not acceptable, but sockets are. On
Windows, the underlying select() function is provided by the WinSock
library, and does not handle file descriptors that don’t originate
from WinSock.
This rules out using sys.stdin.
Alternatives:
Use Cygwin (No modifications to code needed)
Create a thread that waits on sys.stdin (like here)
Go the full Windows route and use WaitForMultipleObjects
Use some library that abstracts these details away, I like libuv but haven't used it with python
Another thing: Don't use select with a zero timeout in an infinite loop. This busy waiting is really inefficient. Instead omit the timeout to have select block till a descriptor becomes ready.

Problems with a chat program written in Python and PyGTK

I am new to python and I am currently working on a chat room program in Python (still in progress...). I have also made a GUI for my program. Initially, I made two py files, one for the GUI and one for the chatting function. They both worked perfectly when separated. After, I combined the two files. I faced the following two problems:
One of my threads (target = loadMsg) is used to wait for the host's msg and print it out on the screen. The problem is that it delays for one msg every time. For example, I sent a "1" to the host and the host should return a "1" immediately. But, the "1" I received didn't appear on my screen. Then I send a "2" to the host and the host should reply a "2" immediately. Then, my screen shows a "1" but the "2" is still missing until the host reply a "3" to me, after I send a "3" to the host. Where is the problem?
This is a technical problem. I was testing the stability of the chat room and I found that about 10% of my msg disappeared during the transmission and this situation occurs randomly. How can I fix such a problem?
Sorry for my poor English. I hope someone can help me with it.T_T
Here is my code for your reference:
---Client
import pygtk,gtk
import logging
from threading import *
import socket
DEBUG = 1
HOST = ''
PORT = 8018
TIMEOUT = 5
BUF_SIZE = 1024
class Base():
def reload(self):
try:
buf = self.sock.recv(BUF_SIZE)
print buf
self.addMsg(buf)
except:
pass
def reload_butt(self,widget):
try:
self.thread = Thread(target=self.reload)
self.thread.start()
except:
pass
def loadMsg(self):
try:
while True :
buf = self.sock.recv(BUF_SIZE)
print buf
self.addMsg(buf)
except:
self.sock.close()
def sendMsg(self,widget):
if DEBUG : print "Send Msg"
if self.entry.get_text() : self.sock.send(self.entry.get_text())
self.entry.set_text("")
def addMsg(self,string):
if DEBUG : print "Try to add Msg"
if self.entry.get_text() :
iter = self.buffer1.get_iter_at_offset(-1)
self.buffer1.insert(iter,("\n Username: "+string))
self.entry.set_text("")
self.adj = self.scrolled_window.get_vadjustment()
self.adj.set_value( self.adj.upper - self.adj.page_size )
if DEBUG : print "Add msg ok"
def destroy(self,widget):
if DEBUG : print "Destroy function called"
self.sock.close()
gtk.main_quit()
def __init__(self,sock):
if DEBUG : print "Initializing..."
self.sock = sock
self.win=gtk.Window()
self.win.connect("destroy",self.destroy)
self.vbox=gtk.VBox()
self.win.add(self.vbox)
self.view=gtk.TextView()
self.view.set_editable(False)
self.buffer1=self.view.get_buffer()
self.scrolled_window=gtk.ScrolledWindow()
self.scrolled_window.set_policy(gtk.POLICY_AUTOMATIC,gtk.POLICY_AUTOMATIC)
self.scrolled_window.add(self.view)
self.vbox.add(self.scrolled_window)
self.entry=gtk.Entry()
self.entry.connect("activate",self.sendMsg)
self.enter=gtk.Button("Enter")
self.enter.connect("clicked",self.sendMsg)
self.reload=gtk.Button("Reload")
self.reload.connect("clicked",self.reload_butt)
self.hbox=gtk.HBox()
self.hbox.add(self.entry)
self.hbox.pack_start(self.reload,False,False)
self.hbox.pack_start(self.enter,False,False)
self.vbox.pack_start(self.hbox,False,False)
self.win.show_all()
if DEBUG : print "Finish initializing"
def main(self):
try :
gtk.main()
except :
print "Error!!!"
def main() :
try :
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((HOST, PORT))
print ('Connecting to '+ str(HOST) +' ' + str(PORT))
base=Base(sock)
thread1=Thread(target=base.loadMsg)
thread2=Thread(target=base.main)
thread2.start()
thread1.start()
except :
print "Err0r!!!"
sock.close()
main()
---host (an echo host)
import socket
HOST = ''
PORT = 8018
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((HOST, PORT))
s.listen(5)
conn, addr = s.accept()
print 'Connected by', addr
try :
print "Start!"
while True:
data = conn.recv(1024)
print data
reply = data # echo
if not reply : break
if reply== "!q" :
conn.close()
break
conn.send(reply)
conn.close()
except :
print "Error!!!!!"
conn.close()

I would seriously recommend to use the gio library (part of glib). Using that library, you connect functions to the socket operations such as when data is available, or when data can be written to the socket. The library will call these function when necessary, and you don't need a wait loop. Which is more CPU-friendly.
http://jcoppens.com/soft/howto/gtk/chat_socket.php contains an example of communications between a C program and Python, using gio, which might be useful to you.
This way, you can start monitoring the sockets after the GUI has started, and you do not need threads to attend the communications.

Python Socket Listening

All of the below mentioned is on windows machines using python 2.7
Hello,
I am currently attempting to listen on a socket for data send by a remote program. This data is then printed to the screen and user input is requested that is then returned to remote program. In testing I have been able to have the remote program send me a menu of command line programs (cmd, ipconfig, whoami, ftp) and then my program returns with a number as a selection of the menu option.
The remote program receives my response and sends the output of the selected command. ipconfig and whoami work perfectly, but cmd and ftp only returns the output of the terminal once. (I.E. I can enter one command into the FTP program and send that too the remote program before I never hear back)
The part of my code that fails is that
if ready[0]: never becomes ready a second time after the first conversation.
I know the remote program is functioning correctly as I can use netcat to act in lieu of my code and operate the cmd terminal indefinitely.
How do I go about properly implementing a python socket listener that can account for this type of connection?
My "program" in its entirety:
import socket, sys, struct, time, select
host = ''
port = 50000
connectionSevered=0
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except socket.error:
print 'Failed to create socket'
sys.exit()
print '[+] Listening for connections on port '+str(port)+'.'
s.bind((host,port))
s.listen(5)
def recvall(the_socket,timeout=2):
global connectionSevered
data=''; # Data found by recv
total_data=[]; # Finally list of everything
s.setblocking(0) #make socket non blocking
begin=time.time() #beginning time
while 1:
ready = select.select([client], [], [], .2)
if time.time()-begin > timeout:
print 'Timeout reached'
#Leave loop, timer has reached its threshold
break
if ready[0]:
print 'In ready loop!'
try:
data = client.recv(4096) #attempt to fetch data
if data:
begin=time.time() #reset timeout timer
total_data.append(data)
data='';
except socket.error:
print '[+] Lost connection to client. Printing buffer...'
connectionSevered=1 # Let main loop know connection has errored
pass
time.sleep(1)
#join all parts to make final string
return ''.join(total_data)
client, address = s.accept()
print '[+] Client connected!'
while (connectionSevered==0): # While connection hasn't errored
print "connectionSevered="+str(connectionSevered) # DEBUG
recvall(s)
response = raw_input() #take user input
client.sendto(response) #send input
client.close(0)
Please let me know if you need more information, any help would be greatly appreciated, I am very new to this and eager to learn.

Playing around with this for a while finally got it working nice with a telnet session locally using python 2.7.
What it does is it sets up a thread that runs when the client connects listening for client stuff.
When the client sends a return ("\r\n" might have to change that if your interacting with a Linux system?) the message gets printed to the server, while this is happening if there is a raw input at the server side this will get sent to the client:
import socket
import threading
host = ''
port = 50000
connectionSevered=0
class client(threading.Thread):
def __init__(self, conn):
super(client, self).__init__()
self.conn = conn
self.data = ""
def run(self):
while True:
self.data = self.data + self.conn.recv(1024)
if self.data.endswith(u"\r\n"):
print self.data
self.data = ""
def send_msg(self,msg):
self.conn.send(msg)
def close(self):
self.conn.close()
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((host,port))
s.listen(5)
except socket.error:
print 'Failed to create socket'
sys.exit()
print '[+] Listening for connections on port: {0}'.format(port)
conn, address = s.accept()
c = client(conn)
c.start()
print '[+] Client connected: {0}'.format(address[0])
c.send_msg(u"\r\n")
print "connectionSevered:{0}".format(connectionSevered)
while (connectionSevered==0):
try:
response = raw_input()
c.send_msg(response + u"\r\n")
except:
c.close()
The above answer will not work for more than a single connection. I have updated it by adding another thread for taking connections. It it now possible to have more than a single user connect.
import socket
import threading
import sys
host = ''
port = 50000
class client(threading.Thread):
def __init__(self, conn):
super(client, self).__init__()
self.conn = conn
self.data = ""
def run(self):
while True:
self.data = self.data + self.conn.recv(1024)
if self.data.endswith(u"\r\n"):
print self.data
self.data = ""
def send_msg(self,msg):
self.conn.send(msg)
def close(self):
self.conn.close()
class connectionThread(threading.Thread):
def __init__(self, host, port):
super(connectionThread, self).__init__()
try:
self.s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.s.bind((host,port))
self.s.listen(5)
except socket.error:
print 'Failed to create socket'
sys.exit()
self.clients = []
def run(self):
while True:
conn, address = self.s.accept()
c = client(conn)
c.start()
c.send_msg(u"\r\n")
self.clients.append(c)
print '[+] Client connected: {0}'.format(address[0])
def main():
get_conns = connectionThread(host, port)
get_conns.start()
while True:
try:
response = raw_input()
for c in get_conns.clients:
c.send_msg(response + u"\r\n")
except KeyboardInterrupt:
sys.exit()
if __name__ == '__main__':
main()
Clients are not able to see what other clients say, messages from the server will be sent to all clients. I will leave that as an exercise for the reader.

If you're in Python 3 by now and still wondering about sockets, here's a basic way of using them:
server.py
import time
import socket
# creating a socket object
s = socket.socket(socket.AF_INET,
socket.SOCK_STREAM)
# get local Host machine name
host = socket.gethostname() # or just use (host == '')
port = 9999
# bind to pot
s.bind((host, port))
# Que up to 5 requests
s.listen(5)
while True:
# establish connection
clientSocket, addr = s.accept()
print("got a connection from %s" % str(addr))
currentTime = time.ctime(time.time()) + "\r\n"
clientSocket.send(currentTime.encode('ascii'))
clientSocket.close()
client.py
import socket
# creates socket object
s = socket.socket(socket.AF_INET,
socket.SOCK_STREAM)
host = socket.gethostname() # or just use (host = '')
port = 9999
s.connect((host, port))
tm = s.recv(1024) # msg can only be 1024 bytes long
s.close()
print("the time we got from the server is %s" % tm.decode('ascii'))
Run server.py first, then run client.py.
This is just send and receive the currentTime.
What's new in Python 3.4 sockets?
A major difference between python 2.7 sockets and python 3.4 sockets is the sending messages. you have to .encode() (usually using 'ascii' or blank as parameters/arguments)
and then using .decode()
For example use .encode() to send, and use .decode() to receive.
Extra info: client/server socket tutorial

Python client sockets connecting on the same server socket port

I am working for the first time on sockets with python.
I need to connect more than a client socket to the same server socket.
To do this I used the the following code:
import socket
import time
import random
from threading import Thread
import thread
import subprocess, signal, os
class ServerObject:
def __init__(self, port):
self.socket = ''
self.host = ''
self.port = port
self.conn = ''
self.data = ''
print "Server port is: ", self.port
def openSocketConnectionAsServer(self):
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.socket.bind((self.host, self.port))
self.socket.listen(5)
self.conn, addr = self.socket.accept()
print 'Server Connected by', addr
def getData(self):
return self.data
def getHost(self):
return self.host
def getPort(self):
return self.port
def sendMessage(self, message):
try :
self.conn.sendall(message)
except socket.error:
print 'Server Send failed'
sys.exit()
def receiveMessageFromClient(self):
while 1:
reply = self.conn.recv(4096)
if not self.data: break
return reply
def receiveMessageFromServer(self):
reply = self.socket.recv(4096)
return reply
def closeConnectionAsServer(self):
self.socket.shutdown(1)
self.socket.close()
del self.socket
class ClientObject:
def __init__(self):
self.data = str(random.choice('abcdefghil'))
print "Client Data Random: ", self.data
self.host = 'localhost'
self.port = ''
def openSocketConnectionAsClient(self):
self.port = 50000 + random.randint(1,3)
print "Client socket port is: ", self.port
try:
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except socket.error:
print 'Client Failed to create socket'
sys.exit()
print 'Client Socket Created'
try:
remote_ip = socket.gethostbyname( self.host )
except socket.gaierror:
print 'Client Hostname could not be resolved. Exiting'
sys.exit()
self.socket.connect((remote_ip , self.port))
def getData(self):
return self.data
def getHost(self):
return self.host
def getPort(self):
return self.port
def sendMessage(self, message):
try :
self.socket.sendall(message)
except socket.error:
print 'Client Send failed'
os.sys.exit()
def receiveMessageFromServer(self):
reply = self.socket.recv(4096)
return reply
def closeConnection(self):
self.socket.shutdown(1)
self.socket.close()
del self.socket
def clientProcess():
print "Client Start the client process"
client1 = ClientObject()
for i in range(1,10):
try:
print "Client try to open the connection socket: attempt number ",i
client1.openSocketConnectionAsClient()
print "Client connection socket established on port ", client1.getPort()
break
except socket.error:
print "Client connection error on the port ", client1.getPort()
if i == 10:
os.sys.exit()
print "Client send the message"
client1.sendMessage(client1.getData())
print "Client receive the message back"
client1.receiveMessageFromServer()
print "Client Data requested: ", client1.getData(), " Hostname: ", client1.getHost(), " Port: ", client1.getPort()
print "Client Close connection"
client1.closeConnection()
def serverProcess(port=40007):
print "Server oject create"
server = ServerObject(port)
print "Server open connection as server"
server.openSocketConnectionAsServer()
print "Server receive the client message"
receivedMessage = server.receiveMessageFromClient()
message = "Server Data requested are: "+receivedMessage
print "Server send the message back to the client"
server.sendMessage(message)
print "Server close the connection with the client"
server.closeConnectionAsServer()
if __name__ == '__main__':
threadServer = Thread(target=serverProcess, args=[50001])
print "New Server thread created with name: ", threadServer.getName()
threadServer.start()
for i in range(1,3):
threadClient = Thread(target=clientProcess)
print "New Client thread created with name: ", threadClient.getName()
threadClient.start()
threadClient.join()
When I run it, the first client is able to connect to the server, the second one never get connected, even though I close the connection (from server and client side) at the end of the first client thread.
Please, could anybody tell me, what's wrong in my code?
Thanks

I would highly recommend using the excellent socketserver module for doing stuff like this. As for your code, the problem is that your threaded socket server only runs in a single thread. At the very least, you should create a new thread to handle the connection after accepting the new connection in openSocketConnectionAsServer.

What #Ber said was correct but incomplete.
You're problem is here:
self.socket.listen(5)
self.conn, addr = self.socket.accept()
listen will open up the port and prepare to receive connections. accept will wait for the next connection.
While you only need to call listen once, you must call accept multiple times, once per connection. There are several ways you can arrange this. For starters you could call accept again when the current connection closes, but this will only allow one client at a time. Better to have one thread call accept to wait for the next connection, but have it start a worker thread to handle each one.
Or you could use non-blocking I/O, but if you go this way, check out Twisted.