I am trying to write a simple multithreaded web server in Python.
When a new client socket connects with the server socket, I create a new thread to handle the client socket, like so:
(clientsock, (ip, port)) = self.sock.accept()
newthread = ClientThread(ip, port, clientsock)
newthread.start()
where the ClientThread class has a run method, which in turn calls the following bit of code:
data_string = ''
bytes = self.socket.recv(2046)
bytes_str = bytes.decode('UTF-8')
data_string += bytes_str
where self.socket is the clientsock referenced above.
This code works as expected: 1 thread is created when the server socket accepts a connection from the client socket, and receives the client socket request.
But if I change the above code in order to loop while there is more data, such as:
data_string = ''
bytes = self.socket.recv(2046)
while len(bytes) > 0:
bytes_str = bytes.decode('UTF-8')
data_string += bytes_str
print 'data received: %s' % (bytes_str)
bytes = self.socket.recv(2046)
Then I have a strange issue. It seems that accept is being called multiple times, thereby creating a new, unnecessary thread each time.
I.E - if I start up a client and try to access a file, I get the following output from the server:
We have accepted a connection from (127.0.0.1, 59983)
New thread started for (127.0.0.1, 59983)
We have accepted a connection from (127.0.0.1, 59984)
New thread started for (127.0.0.1, 59984)
Why is accept being called multiple times if only one client has connected to the server?
Edit
Is this expected behavior?
Related
I am trying to establish a client-server communication. The client is written in Ruby whereas the server is written in Python.
client.rb
require 'socket'
hostname = 'localhost'
port = 7778
s = TCPSocket.open(hostname, port)
s.write("2020-06-25T11:11:00+00:00 5 127.0.0.1 printer: event")
while line = s.gets
puts line.chop
end
s.close()
The ruby client sends a log to the Python server and tries to receive it back.
server.py
import socket
#Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
#bind the socket to the port - tuple
server_address = ('localhost', 7778)
print('starting up on %s port %s' %server_address)
sock.bind(server_address)
#Listen for incoming connections
sock.listen(1)
while True:
print('waiting for a connection')
connection, client_address = sock.accept()
while True:
data = connection.recv(1024)
print('received "%s"' % data)
if data:
print('sending data back to the client')
connection.send(data)
else:
print('no more data from', client_address)
break
connection.close()
The log is sent to the python server and when the python server sends it back. When the ruby client receives it, it doesn't receive the full log.
example:
2020-06-25T11:11:00+00:00 5 127.0.0.1 printer: eve
I think this comes from the fact that TCP is a streaming protocol and we never know if we can get the full message each time.
Could you propose me a solution for both the client and the server so I can be sure they always receive the full message between each other? I would really appreciate it if anyone would help.
So the issue is that you're assuming the data received has a new line character - however the data you're sending is not terminated by a new line.
s.write("2020-06-25T11:11:00+00:00 5 127.0.0.1 printer: event") will not write the string with a new line character - you should use puts IO#puts
s.gets will return the data because the socket is closed by the python server after it has sent the data. So even getssays it will read the next line from the socket, in reality its just reading what remained in the buffer after the socket was closed.
line.chop will remove the last character, and you're using it here to strip a newline character (assuming that it has one from gets). However since there is no newline character it will remove the last character instead.
So the fix would be to replace in the ruby client s.write with s.puts.
hi i make model server client which works fine and i also create separate GUI which need to two input server IP and port it only check whether server is up or not. But when i run server and then run my GUI and enter server IP and port it display connected on GUI but on server side it throw this error. The Server Client working fine but integration of GUI with server throw below error on server side.
conn.send('Hi'.encode()) # send only takes string BrokenPipeError: [Errno 32] Broken pip
This is server Code:
from socket import *
# Importing all from thread
import threading
# Defining server address and port
host = 'localhost'
port = 52000
data = " "
# Creating socket object
sock = socket()
# Binding socket to a address. bind() takes tuple of host and port.
sock.bind((host, port))
# Listening at the address
sock.listen(5) # 5 denotes the number of clients can queue
def clientthread(conn):
# infinite loop so that function do not terminate and thread do not end.
while True:
# Sending message to connected client
conn.send('Hi'.encode('utf-8')) # send only takes string
data =conn.recv(1024)
print (data.decode())
while True:
# Accepting incoming connections
conn, addr = sock.accept()
# Creating new thread. Calling clientthread function for this function and passing conn as argument.
thread = threading.Thread(target=clientthread, args=(conn,))
thread.start()
conn.close()
sock.close()
This is part of Gui Code which cause problem:
def isOpen(self, ip, port):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
s.connect((ip, int(port)))
data=s.recv(1024)
if data== b'Hi':
print("connected")
return True
except:
print("not connected")
return False
def check_password(self):
self.isOpen('localhost', 52000)
Your problem is simple.
Your client connects to the server
The server is creating a new thread with an infinite loop
The server sends a simple message
The client receives the message
The client closes the connection by default (!!!), since you returned from its method (no more references)
The server tries to receive a message, then proceeds (Error lies here)
Since the connection has been closed by the client, the server cannot send nor receive the next message inside the loop, since it is infinite. That is the cause of the error! Also there is no error handling in case of closing the connection, nor a protocol for closing on each side.
If you need a function that checks whether the server is online or not, you should create a function, (but I'm sure a simple connect is enough), that works like a ping. Example:
Client function:
def isOpen(self, ip, port):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
s.connect((str(ip), int(port)))
s.send("ping".encode('utf-8'))
return s.recv(1024).decode('utf-8') == "pong" # return whether the response match or not
except:
return False # cant connect
Server function:
def clientthread(conn):
while True:
msg = conn.recv(1024).decode('utf-8') #receiving a message
if msg == "ping":
conn.send("pong".encode('utf-8')) # sending the response
conn.close() # closing the connection on both sides
break # since we only need to check whether the server is online, we break
From your previous questions I can tell you have some problems understanding how TCP socket communication works. Please take a moment and read a few articles about how to communicate through sockets. If you don't need live communications (continous data stream, like a video, game server, etc), only login forms for example, please stick with well-known protocols, like HTTP. Creating your own reliable protocol might be a little complicated if you just got into socket programming.
You could use flask for an HTTP back-end.
I am testing a Python web server. It works as expected using localhost as the server and client, but when I test on different computers, I am getting
[Errno 54] Connection reset by peer about 20% - 80% of the time, depending on how many client threads I spawn at once. Why?
Code Snippets
Server listens:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind((self.interface, self.port))
sock.listen(5)
Server loops forever, accepts client connection, spawns new thread:
while True:
(clientsock, (ip, port)) = self.sock.accept()
newthread = ClientThread(ip, port, clientsock)
newthread.start()
Spawn a bunch of client threads which connect with server, send message which requests a file, and then closes connection
Server sends message to client when ready
self.socket.sendall(message.encode())
After message is sent, close the write end of connection:
self.socket.shutdown(socket.SHUT_WR)
Client receives message (error occurs here)
def receive(self):
data_string = ''
bytes = self.sock.recv(self.bufsize)
while len(bytes) > 0:
bytes_str = bytes.decode('UTF-8')
data_string += bytes_str
bytes = self.sock.recv(self.bufsize)
return data_string
After client thread has received message, close the connection:
self.socket.close()
Receive function had errors. Changed to this:
def receive(self):
data_string = ''
while True:
bytes = self.sock.recv(self.bufsize)
bytes_str = bytes.decode('UTF-8')
data_string += bytes_str
if not bytes:
break
return data_string
Old receive function would try to call recv a second time when server had already closed socket. New one only calls once.
Also did not know you could increase listening socket backlog > 5 since Python docs say generally 5 is max, when on OS X it is 128. Increasing backlog to 128 helped.
I have been self-learning python since few months now , and finally learning Socket programming. As an text book exercise, I am supposed to design a half-duplex chat system . Below is the code. The first request and response are just fine , but everytime I try sending a second message from client, the server seems to be hanging. The program is TCP based.
I am suspecting that since ss.accept() is being called everytime a new message has to be sent, a new thread is being created but since I have made only 1 call to sc.connect() from client , may be my new connection at the server end is hanging there for infinite time.
As a trail : I called ss.accept() outside the while loop, ie making only 1 connection and listening to data over and over on while loop, the conversations works just fine
Can someone please have a look a the code and help me understand where exactly is the issue.
Since, I am learning, I have not moved to twisted yet. I want to learn all the basics first before I move to frameworks.
!bin/usr/env python
import socket, sys
HOST =''
PORT = 1060
ADDR =(HOST,PORT)
def userinput(sock):
usermessage = input('>')
sock.sendall(str(len(usermessage)))
return usermessage
def server():
s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)
s.bind(ADDR)
s.listen(1)
print 'the server is listening at',s.getsockname()
while True:
ss,sockname = s.accept()
#listen to determine the bytes sent by client
msglen = ss.recv(4096)
#accept the complete message
msg = ss.recv(int(msglen))
print 'client:', repr(msg)
servermsg = userinput(ss)
ss.sendall(servermsg)
print " ---------------"
ss.close()
def client():
sc = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
sc.connect(ADDR)
while True:
message = userinput(sc)
sc.sendall(message)
replylen = sc.recv(4096)
reply = sc.recv(int(replylen))
print "reply:",reply
print "----------------"
sc.close()
if sys.argv[1:] == ['server']:
server()
elif sys.argv[1:] == ['client']:
client()
else:
print >> sys.stderr,'usage:tcp_2_7.py server|client[host]'
Your trial - accepting once and then receiving multiple messages - is how you should do this. Calling accept is waiting for a new connection - you don't need to do this every time you want to send or receive a message, just as you don't want to call connect every time you want to send or receive.
Think of it this way:
When you connect to a chat server, do you connect, send a message, then disconnect immediately? No - you have a constant open connection which messages are sent through, and the connection is only closed at the end of a chat session.
From the docs on accept:
socket.accept()
Accept a connection. The socket must be bound to an
address and listening for connections. The return value is a pair
(conn, address) where conn is a new socket object usable to send and
receive data on the connection, and address is the address bound to
the socket on the other end of the connection.
i got this code from http://www.evolt.org/node/60276 and modified it to listen for a single "1" coming from the other side
but whenever i run this program it stops and python IDLE goes to non-responding on "data1,addr = UDPSock.recvfrom(1024)"
def get1():
# Server program, receives 1 if ball found
# ff1 is file w/ received data
import socket
import time
# Set the socket parameters
host = "mysystem"
port = 21567
#buf = 1024
addr = (host,port)
# Create socket (UDP) and bind to address
UDPSock = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
UDPSock.bind(addr)
# Receive messages
while 1:
print "waiting..............."
data1,addr = UDPSock.recvfrom(1024)
print "got 1"
if not data1:
print "Client has exited!"
break
else:
print "\nReceived message '", data1,"'"
UDPSock.close() # Close socket
print "socket closed\n"
#call some other function that uses 1
and client side
def send1():
# Client program, sends 1 if ball found
# mf1 is file with data to be sent
import socket
# Set the socket parameters
host = "mysystem"
port = 21567
buf = 1024
addr = (host,port)
# Create socket (UDP)
UDPSock = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
mf1=1
print mf1
# Send messages
if(UDPSock.sendto(str(mf1),addr)):
print "Sending message '",str(mf1),"'....."
# Close socket
UDPSock.close()
does anyone know what might be the cause of this? (sorry for long post)
As a second guess (I replaced my first guess with this) I suspect that you are running the receiver in IDLE and then IDLE is hanging so you can't run the client. I don't know exactly how IDLE works as I never use it, but the line containing recvfrom will stop the Python thread its running in until data is sent. So you need to start the client in a separate instance of IDLE or from the command line or something.
At any rate, I have tested the program in question on my Python with 127.0.0.1 as the host, and it worked fine, for some values of fine. The recvfrom does hang, but only until some data is sent, then it comes back with the data and prints it out and everything. You do have a bug that happens after that though. :-)