We Keep Coding

Socket data not getting received when the request is made from a raspberry pi

I am trying to send and receive data from a server. When I run the below python program from eclipse or python IDLE I get the response from the server. But, when I run the program from Raspberry Pi I am getting an error. Below is the simple code and error.
import socket,time
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
ip = '127.0.0.1'
port = 1883
address = (ip,port)
client.connect(address)
print("connected")
def close():
global client
client.close()
client=socket.socket(socket.AF_INET,socket.SOCK_STREAM)
client.connect(address)
while 1:
client.send(b'\x01\x04\x00\x00\xxxxxx')
print("sent")
try:
raw_data = client.recv(1024)
except:
close()
raw_data = client.recv(1024)
received_data = " ".join("{:02x}".format(byte) for byte in
(raw_data))
print(received_data)
time.sleep(1)
When I run this code in raspberry pi,I get BrokenPipe error. why it is happening in raspberry pi?
Error:
connected
sent
received
sent
Traceback (most recent call last):
File "exdu.py", line 18, in <module>
client.send(b'\x01\x04\x00\x00\xxxxxxxxx')
socket.error: [Errno 32] Broken pipe

A broken pipe is caused when a pipe (in this case a TCP socket) is closed unexpectedly by the other side and recv gets back an empty response or send tries to write to a closed connection.
There are quite a couple pieces of not so best practice code may be causing a broken pipe error in your code.
I'll try and point out a few things that stand out to me using comments:
import socket,time
# it would be nice to have a function wrapping this but as it is this is not the issue.
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
ip = '127.0.0.1'
port = 1883
address = (ip,port)
client.connect(address)
print("connected")
def close():
global client
# If you read the python socket documentation https://docs.python.org/2/howto/sockets.html
# you will see that they advise sending a shutdown packet to the other side of the connection.
# In my experience this is actually extremely important when properly handling TCP connections and may be causing this issue.
# you must do this from the side that closes the connection, this includes when the server closes a connection.
client.shutdown(SHUT_RDWR) # this is what you want to do here.
client.close()
# connecting to a new socket here makes no sense and may also cause an issue.
client=socket.socket(socket.AF_INET,socket.SOCK_STREAM)
client.connect(address)
raw_data=client.recv(1024)
while 1:
client.send(b'\x01\x04\x00\x00\xxxxxx')
print("sent")
try:
raw_data = client.recv(1024)
except:
# this except statement may cause an exception itself.
# Not only that but you are probably getting your broken pipe error
# here from the recv in close().
close()
raw_data = client.recv(1024)
received_data = " ".join("{:02x}".format(byte) for byte in
(raw_data))
print(received_data)
time.sleep(1)
Apologies for the code comments, but i find they're useful when the segment is copy pasted when you are trying to experiment with a solution.
As an extension i would write the code you have to be much more like the python docs example:
class mysocket:
def __init__(self, sock=None):
if sock is None:
self.sock = socket.socket(
socket.AF_INET, socket.SOCK_STREAM)
else:
self.sock = sock
def connect(self, host, port):
self.sock.connect((host, port))
# important bit of missing code
def disconnect(self):
self.sock.shutdown(SHUT_RDWR)
self.sock.close()
def mysend(self, msg):
totalsent = 0
while totalsent < MSGLEN:
sent = self.sock.send(msg[totalsent:])
if sent == 0:
raise RuntimeError("socket connection broken")
totalsent = totalsent + sent
def myreceive(self):
chunks = []
bytes_recd = 0
while bytes_recd < MSGLEN:
chunk = self.sock.recv(min(MSGLEN - bytes_recd, 2048))
if chunk == '':
raise RuntimeError("socket connection broken")
chunks.append(chunk)
bytes_recd = bytes_recd + len(chunk)
return ''.join(chunks)
You can rewrite the send and recieve functions to do your bidding and then call it like:
socket = mysocket()
socket.connect('127.0.0.1', 1883)
try:
socket.mysend(b'\x01\x04\x00\x00\xxxxxx')
raw_data = socket.myrecieve()
# do more stuff
except:
socket.disconnect()
This is example code but may set you on the right path.

Broken pipe implies something isn't going through exactly as you want it to. Are you sure the server is running correctly on the raspberry pi on localhost?
Otherwise you might want to either run the server seperately(maybe heroku or digitalocean is relevant) or check your pi's firewall(I highly doubt this is the problem for localhost -> localhost connections)

Connection reset by peer when testing web server on different computers

I am testing a Python web server. It works as expected using localhost as the server and client, but when I test on different computers, I am getting
[Errno 54] Connection reset by peer about 20% - 80% of the time, depending on how many client threads I spawn at once. Why?
Code Snippets
Server listens:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind((self.interface, self.port))
sock.listen(5)
Server loops forever, accepts client connection, spawns new thread:
while True:
(clientsock, (ip, port)) = self.sock.accept()
newthread = ClientThread(ip, port, clientsock)
newthread.start()
Spawn a bunch of client threads which connect with server, send message which requests a file, and then closes connection
Server sends message to client when ready
self.socket.sendall(message.encode())
After message is sent, close the write end of connection:
self.socket.shutdown(socket.SHUT_WR)
Client receives message (error occurs here)
def receive(self):
data_string = ''
bytes = self.sock.recv(self.bufsize)
while len(bytes) > 0:
bytes_str = bytes.decode('UTF-8')
data_string += bytes_str
bytes = self.sock.recv(self.bufsize)
return data_string
After client thread has received message, close the connection:
self.socket.close()

Receive function had errors. Changed to this:
def receive(self):
data_string = ''
while True:
bytes = self.sock.recv(self.bufsize)
bytes_str = bytes.decode('UTF-8')
data_string += bytes_str
if not bytes:
break
return data_string
Old receive function would try to call recv a second time when server had already closed socket. New one only calls once.
Also did not know you could increase listening socket backlog > 5 since Python docs say generally 5 is max, when on OS X it is 128. Increasing backlog to 128 helped.

python XMLRPC + socket relay server with "kill client"

I'm trying to make a server with Python to relay traffic from listening ports to remote hosts. I'll call the listening port <--> remote host a proxy, which might be localhost:54321 <--> somehost:2001.
I need the following functionality from another thread running a XMLRPC server:
Add a new proxy
Kill a proxy
Disconnect the client from a proxy listening port.
This is rather trivial to implement with threads. Here's what I came up with:
import logging
import threading
import socket
class ProxyThread(threading.Thread):
""" Thread that serves as a transport proxy """
def __init__(self, server, client):
super(ProxyThread, self).__init__()
self.server = server # (host, port)
self.client = client # listening socket
self.kill_thread = threading.Event()
self.kill_client = threading.Event()
def connect_server(self):
try: # make connection to server
server = socket.create_connection(self.server, timeout=2)
except Exception as e:
return None
return server
def connect_client(self):
try: # accept connection from client
self.client.settimeout(1)
client, saddr = self.client.accept()
except socket.timeout:
return None
return client
def run(self):
client, server = None, None
def close():
if client:
client.close()
if server:
server.close()
return None, None
while not self.kill_thread.isSet():
if not server:
server = self.connect_server()
if not server:
time.sleep(2) # wait before retrying
continue
if not client:
client = self.connect_client()
if not client:
continue
while not (self.kill_client.isSet() or self.kill_thread.isSet()):
# read loop 0.1 timeout to check kill_client event
r, w, d = select.select([client, server], [], [], 0.1)
if client in r:
read = client.recv(1024)
if not read: # disconnected
break
server.sendall(read)
if server in r:
read = server.recv(1024)
if not read: # disconnected
break
client.sendall(read)
# close sockets and reset
client, server = close()
self.kill_client.clear()
# end of thread
close()
self.client.close()
Then from the XMLRPC thread,
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind(("", 0))
sock.listen(1)
proxy = ProxyThread(sock, ('somehost', 2001))
proxy.start()
# later ...
proxy.kill_client.set()
This works but it is pretty inefficient and introduces quite a bit of latency. Further compounding the issue is that this server will be hosting up to hundreds of concurrent proxies, and with threading this becomes a CPU hog and latency is unacceptable.
I'm looking into asynchronous frameworks such as twisted and asyncio, but I can't figure out how I would be able to add and control running proxies from another thread (the XMLRPC server). Can anyone please help guide me in the right direction?

Python Socket - Why connection closed after sending several messages?

I have a program that launches a TCP client as well as a server, and I can send messages and files from client to the server (they are all transferred in this direction). The server is expected to be always listening, and respond each upcoming message. But I found after I sent several messages, the server never responds again, unless I relaunch connect button on GUI.
here we have in the server,
# establish connection
def conn(self):
self.s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.s.bind((self.ip, self.port))
self.s.listen(1)
print 'server ready.'
self.conn, self.addr = self.s.accept()
print 'Connected by', str(self.addr), '\n'
def recv(self):
flag = self.conn.recv(BUFFER_SIZE)
data = self.conn.recv(BUFFER_SIZE)
# this is a message
if flag=='1':
msg = "other >> %s" % data
self.conn.send("success")
print msg
return
# there will be a file
elif flag=='0':
filename = data
with open('new_'+filename, 'wb+') as f:
while True:
data = self.s.recv(BUFFER_SIZE)
if not data: break # transfer finished
f.write(data)
size = os.path.getsize(filename)
self.conn.send(str(size)) # echo size
self.conn.send("success")
return
# do not close connection unless exception raised
def run(self):
self.conn()
while True:
try:
# shoud I connect again each time here?
self.recv()
except:
self.close()
break
and in the client I have,
# expected to establish a connection
def conn(self):
self.s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print 'client ready.'
try:
self.s.connect((self.ip, self.port))
print 'connection established.'
except:
# close socket
self.s.close()
raise
def send(self, msg='', flag=1):
if flag:
self.s.send(str(flag))
self.s.send(msg)
sent = self.s.recv(BUFFER_SIZE)
print sent
else:
self.s.send(str(flag))
self.s.send(msg) # send filename
# send file in buffer-size
with open(msg, 'rb') as f:
while 1:
data = f.read(BUFFER_SIZE)
if not data: break
self.s.send(data) # send block
sent = self.s.recv(BUFFER_SIZE)
print 'sent: %s bytes' % sent
The problem is, should I put client.socket.connect() in each send function or I just leave it established and trust it would not down? And in the server, should I close connection after each message received? And why my connection is mysteriously down after a short time?
Another question is I noticed some code examples transfer files from server to client using conn.send(), instead, I sent files from client to server by socket.send(). Will this cause a problem?

I think there is something wrong with function name which may cause your problem.
Change function name in your server code:
...
def connection(self):
...
...
def run(self):
self.connection()
while True:
...
Tell me if it works.

create multi-thread tcp server python 3

Hello I tried to make a simple server that accept multiple clients simultaneously I'm new to python and I have a difficult to understand it....I try to change my code in multi-thread applications but without positive result...here is the code:
import socket, threading
def message():
while 1:
data = connection.recv(1024)
if not data: break
#connection.sendall(b'-- Message Received --\n')
print(data.decode('utf-8'))
connection.close()
def connection():
address = input("Insert server ip")
port = 44444
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((address, port))
s.listen(1)
print("Server started! Waiting for connections...")
def accept connection():
connection, address = s.accept()
print('Client connected with address:', address)
t=thread.Threading(target=message,args=(connection))
t.run()
I know that there are many errors but I'm new in python sorry :(
The original non-threaded code is:
import socket
address = input("Insert server ip:")
port = 44444
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((address, port))
s.listen(1)
print("Server started! Waiting for connections...")
connection, address = s.accept()
print('Client connected with address:', address)
while 1:
data = connection.recv(1024)
if not data: break
#connection.sendall(b'-- Message Received --\n')
print(data.decode('utf-8'))
connection.close()

Your basic design is close, but you've got a whole lot of little problems making it hard to move forward.
First, you have a function name with a space in it, which isn't allowed. And you have an IndentationError because you didn't indent its contents.
Next, inside that accept_connection function, you're using threading wrong.
thread.Threading doesn't exist; you probably meant threading.Thread.
args has to be a sequence (tuple, list, etc.) of values. You probably expected (connection) to be a tuple of one value, but it's not; tuples are defined by commas, not parentheses, and what you have is just the value connection with superfluous parentheses around it. You wanted (connection,) here.
Also, calling run on a thread object just runs the thread's code in the current thread. You want to call start, which will start a new thread and call the run method on that thread.
Meanwhile, you're never actually calling this function anywhere, so of course it can't do anything. Think about where you want to call it. After creating the listener socket, you want to loop around accept, kicking off a new client thread for each accepted connection, right? So, you want to call it in a loop, either inside connection, or at the top level (in which case connection has to return s).
And finally, your accept_connection function can't access local variables from some other function; if you want it to use a socket named s, you have to pass it as a parameter.
So:
def connection():
address = input("Insert server ip")
port = 44444
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((address, port))
s.listen(1)
print("Server started! Waiting for connections...")
while True:
accept_connection(s)
def accept_connection(s):
connection, address = s.accept()
print('Client connected with address:', address)
t=thread.Threading(target=message, args=(connection,))
t.start()
As a side note, be careful with using sock.recv(1024) and assuming you're going to get the whole message that the other side sent with send(msg). You might get that, or you might get half the message, or the whole message plus half of another message the client sent later. Sockets are just streams of bytes, like files, not streams of separate messages; you need some kind of protocol to separate messages.
The simplest possible protocol is to send each message on its own line. Then you can just do socket.makefile() and for line in f:, just like you would for a real file. Of course this doesn't work if your messages can have newlines, but you can, e.g., backslash-escape them on one side and unescape them on the other.

This is a pretty old post but there's a nice way to do what you're talking about. Here's a link to an example I posted a little while back:
https://bitbucket.org/matthewwachter/tcp_threadedserver/src/master/
And the script:
from datetime import datetime
from json import loads, dumps
from pprint import pprint
import socket
from threading import Thread
class ThreadedServer(Thread):
def __init__(self, host, port, timeout=60, debug=False):
self.host = host
self.port = port
self.timeout = timeout
self.debug = debug
Thread.__init__(self)
# run by the Thread object
def run(self):
if self.debug:
print(datetime.now())
print('SERVER Starting...', '\n')
self.listen()
def listen(self):
# create an instance of socket
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
# bind the socket to its host and port
self.sock.bind((self.host, self.port))
if self.debug:
print(datetime.now())
print('SERVER Socket Bound', self.host, self.port, '\n')
# start listening for a client
self.sock.listen(5)
if self.debug:
print(datetime.now())
print('SERVER Listening...', '\n')
while True:
# get the client object and address
client, address = self.sock.accept()
# set a timeout
client.settimeout(self.timeout)
if self.debug:
print(datetime.now())
print('CLIENT Connected:', client, '\n')
# start a thread to listen to the client
Thread(target = self.listenToClient,args = (client,address)).start()
# send the client a connection message
# res = {
# 'cmd': 'connected',
# }
# response = dumps(res)
# client.send(response.encode('utf-8'))
def listenToClient(self, client, address):
# set a buffer size ( could be 2048 or 4096 / power of 2 )
size = 1024
while True:
try:
# try to receive data from the client
data = client.recv(size).decode('utf-8')
if data:
data = loads(data.rstrip('\0'))
if self.debug:
print(datetime.now())
print('CLIENT Data Received', client)
print('Data:')
pprint(data, width=1)
print('\n')
#send a response back to the client
res = {
'cmd': data['cmd'],
'data': data['data']
}
response = dumps(res)
client.send(response.encode('utf-8'))
else:
raise error('Client disconnected')
except:
if self.debug:
print(datetime.now())
print('CLIENT Disconnected:', client, '\n')
client.close()
return False
if __name__ == "__main__":
ThreadedServer('127.0.0.1', 8008, timeout=86400, debug=True).start()

Here is some example code I have showing a threaded socket connection.
def sock_connection( sock, host ):
"Handle socket"
pass
while 1:
try:
newsock = sock.accept()
thread = Thread( target=sock_connection, args=newsock )
thread.start()
except Exception, e:
print "error on socket connection: " % e)