We Keep Coding

I want to implement retransmission. Through the timeout function of the recv method [duplicate]

I need to set timeout on python's socket recv method. How to do it?

The typical approach is to use select() to wait until data is available or until the timeout occurs. Only call recv() when data is actually available. To be safe, we also set the socket to non-blocking mode to guarantee that recv() will never block indefinitely. select() can also be used to wait on more than one socket at a time.
import select
mysocket.setblocking(0)
ready = select.select([mysocket], [], [], timeout_in_seconds)
if ready[0]:
data = mysocket.recv(4096)
If you have a lot of open file descriptors, poll() is a more efficient alternative to select().
Another option is to set a timeout for all operations on the socket using socket.settimeout(), but I see that you've explicitly rejected that solution in another answer.

there's socket.settimeout()

As mentioned both select.select() and socket.settimeout() will work.
Note you might need to call settimeout twice for your needs, e.g.
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind(("",0))
sock.listen(1)
# accept can throw socket.timeout
sock.settimeout(5.0)
conn, addr = sock.accept()
# recv can throw socket.timeout
conn.settimeout(5.0)
conn.recv(1024)

You could set timeout before receiving the response and after having received the response set it back to None:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.settimeout(5.0)
data = sock.recv(1024)
sock.settimeout(None)

The timeout that you are looking for is the connection socket's timeout not the primary socket's, if you implement the server side. In other words, there is another timeout for the connection socket object, which is the output of socket.accept() method. Therefore:
sock.listen(1)
connection, client_address = sock.accept()
connection.settimeout(5) # This is the one that affects recv() method.
connection.gettimeout() # This should result 5
sock.gettimeout() # This outputs None when not set previously, if I remember correctly.
If you implement the client side, it would be simple.
sock.connect(server_address)
sock.settimeout(3)

Got a bit confused from the top answers so I've wrote a small gist with examples for better understanding.
Option #1 - socket.settimeout()
Will raise an exception in case the sock.recv() waits for more than the defined timeout.
import socket
sock = socket.create_connection(('neverssl.com', 80))
timeout_seconds = 2
sock.settimeout(timeout_seconds)
sock.send(b'GET / HTTP/1.1\r\nHost: neverssl.com\r\n\r\n')
data = sock.recv(4096)
data = sock.recv(4096) # <- will raise a socket.timeout exception here
Option #2 - select.select()
Waits until data is sent until the timeout is reached. I've tweaked Daniel's answer so it will raise an exception
import select
import socket
def recv_timeout(sock, bytes_to_read, timeout_seconds):
sock.setblocking(0)
ready = select.select([sock], [], [], timeout_seconds)
if ready[0]:
return sock.recv(bytes_to_read)
raise socket.timeout()
sock = socket.create_connection(('neverssl.com', 80))
timeout_seconds = 2
sock.send(b'GET / HTTP/1.1\r\nHost: neverssl.com\r\n\r\n')
data = recv_timeout(sock, 4096, timeout_seconds)
data = recv_timeout(sock, 4096, timeout_seconds) # <- will raise a socket.timeout exception here

You can use socket.settimeout() which accepts a integer argument representing number of seconds. For example, socket.settimeout(1) will set the timeout to 1 second

try this it uses the underlying C.
timeval = struct.pack('ll', 2, 100)
s.setsockopt(socket.SOL_SOCKET, socket.SO_RCVTIMEO, timeval)

As mentioned in previous replies, you can use something like: .settimeout()
For example:
import socket
s = socket.socket()
s.settimeout(1) # Sets the socket to timeout after 1 second of no activity
host, port = "somehost", 4444
s.connect((host, port))
s.send("Hello World!\r\n")
try:
rec = s.recv(100) # try to receive 100 bytes
except socket.timeout: # fail after 1 second of no activity
print("Didn't receive data! [Timeout]")
finally:
s.close()
I hope this helps!!

#! /usr/bin/python3.6
# -*- coding: utf-8 -*-
import socket
import time
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)
s.settimeout(5)
PORT = 10801
s.bind(('', PORT))
print('Listening for broadcast at ', s.getsockname())
BUFFER_SIZE = 4096
while True:
try:
data, address = s.recvfrom(BUFFER_SIZE)
except socket.timeout:
print("Didn't receive data! [Timeout 5s]")
continue

Shout out to: https://boltons.readthedocs.io/en/latest/socketutils.html
It provides a buffered socket, this provides a lot of very useful functionality such as:
.recv_until() #recv until occurrence of bytes
.recv_closed() #recv until close
.peek() #peek at buffer but don't pop values
.settimeout() #configure timeout (including recv timeout)

Python 2.7 server.socket timeout question

I have some python code that sometimes will block when a connection is opened up but no data is sent. I understand why it is waiting for 64 bits or less of data. It will wait forever. Is there a simple way to time out the connection if no data is received. Any help or suggestions would be greatly appreciated.
serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serversocket.bind(('', port))
serversocket.listen(5) # become a server socket, maximum 5 connections
while 1:
try:
while 1:
# print "Waiting for oonnection..."
connection, address = serversocket.accept()
buf = connection.recv(64)

You can use socket.settimeout(value) where value is number of seconds.
serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serversocket.bind(('', port))
serversocket.listen(5) # become a server socket, maximum 5 connections
while 1:
try:
while 1:
# print "Waiting for oonnection..."
connection, address = serversocket.accept()
connection.settimeout(5) # Set 5 seconds timeout to receive 64 bytes of data
buf = connection.recv(64)
except socket.timeout:
print("Timeout happened -- goting back to accept another connection")

Python socket loop does not break

I created socket for two PC, one is Raspberry Pi and the other one is my laptop. I just connected two then I send string to test the connection. If I send a character "q" from the RPi, my PC should break out of the loop and close the connection but it does not. The part print("Listening") is still running. Why? See code below.
import socket
import time
# IP address of this PC.
TCP_IP = '192.168.137.1'
# Port.
TCP_PORT = 5005
# Size of buffer.
BUFFER_SIZE = 1024
# Create a socket, connect and listen to it.
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((TCP_IP, TCP_PORT))
s.listen(1)
conn, addr = s.accept()
print('Connection address:', addr)
while 1:
print("Listening")
data = conn.recv(BUFFER_SIZE)
data = data.decode()
if data=='q':
break
if data:
print ("Received data:", data)
# Echo back.
conn.send(data.encode())
time.sleep(1)
print("It breaks.")
conn.close()
s.close()

TCP is a stream oriented protocol. So data transmitted is a stream not a sequence of messages. So when you expect data to be q it actually is some_data_sent_before_q_and_finally_q.
The simplest way to repair the code is to use if data.endswith('q') instead of if data=='q'. May work and may not depending on how you actually use the connection. For example, this approach may fail with some_data_sent_before_q pretty long pause more_data_and_q and with some_data_sent_before_q_and_finally_q_plus_something_else_why_not.
Little bit more advanced way to solve the problem is to divide the stream into messages with separators - message_1<separator>message_2<separator>q<separator>. This method will allow you to treat every message separately.

socket connection receive bytes python

I'm trying to send a stream of data via socket in Python. So far I manage to create a dummy_data_gen.py which sends a line containing 4 floats to the server.py. However, I'm still having issues in the stability of the all setup.
server.py:
import sys
import time
import socket
import numpy as np
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Bind the socket to the port
server_address = ('localhost', 5002)
print >>sys.stderr, 'starting up on %s port %s' % server_address
sock.bind(server_address)
# Listen for incoming connections
sock.listen(1)
# Create a list to store the incoming data
data = []
while True:
# Wait for a connection
print >>sys.stderr, 'waiting for a connection'
connection, client_address = sock.accept()
try:
print >>sys.stderr, 'connection from', client_address
while True:
incoming_data = connection.recv(48).split(',')
print incoming_data
event = float(incoming_data[0]), float(incoming_data[1]), float(incoming_data[2]), float(incoming_data[3])
data += [event]
time.sleep(0.01)
finally:
# Clean up the connection
connection.close()
dummy_data_gen.py
import sys
import time
import socket
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Connect the socket to the port where the server is listening
server_address = ('localhost', 5002)
sock.connect(server_address)
file = '../data/myfile.txt'
# Simulating a real-time data stream at 100 Hz
try:
with open(file) as f:
for line in f:
sock.sendall(line)
time.sleep(0.01)
finally:
print >>sys.stderr, 'closing socket'
sock.close()
My problem is that sometimes the communication is working properly, however, I have situations where I receive more data per line than I should. In the following output example the first 7 lines are correct, however the following lines are incorrect and therefore problematic:
['391910745379', '24.134277', '-1.9487305', '-117.373535', '\n']
['391920745379', '24.434082', '-1.3491211', '-117.373535', '\n']
['391930745379', '23.68457', '-0.5996094', '-116.62402', '\n']
['391940745379', '24.434082', '-1.0493164', '-115.57471', '\n']
['391950745379', '24.134277', '-1.0493164', '-116.47412', '\n']
['391960745379', '23.234863', '-1.0493164', '-116.47412', '\n']
['391970745379', '24.583984', '-0.89941406', '-116.92383', '\n']
['391980745379', '23.384766', '-0.2998047', '-116.62402', '\n39']
['1990745379', '23.68457', '-0.5996094', '-115.72461', '\n39200']
['0745379', '23.834473', '-0.44970703', '-115.87451', '\n392010']
['745379', '23.534668', '-1.0493164', '-114.9751', '\n392020745']
['379', '23.384766', '-1.7988281', '-115.72461', '\n39203074537']
['9', '22.935059', '-0.44970703', '-114.9751', '\n392040745379', '']
I tried to play with the connection.recv bytes but I'm still facing this issue.
EDIT1: Following some suggestions I modified the server.py as follows:
del_message = '\n'
del_stream = ','
while True:
_buffer += connection.recv(1)
if del_message in _buffer:
incoming_data = _buffer.split(del_stream)
event = float(incoming_data[0]), \
float(incoming_data[1]), \
float(incoming_data[2]), \
float(incoming_data[3])
This approach seems to solve my issue, however the performance is extremely slow. My files contains approximately 6300 lines that were actually sent in 70 seconds (time interval at which the socket was closed on my dummy data generator). However, I took almost 10 minutes to receive all of the 6300 lines. It seems also that I receive more samples per second on the beginning rather that on the end of the stream.

If you have a message protocol that terminates messages with a newline, you need to write some code to implement that protocol. It won't work by magic.
You need a "receive a message" function, where "message" is defined as "a sequence of bytes delimited by a newline". You've never written any such function, so you're not receiving messages but just the chunks of bytes you're sending.

Connection reset by peer when testing web server on different computers

I am testing a Python web server. It works as expected using localhost as the server and client, but when I test on different computers, I am getting
[Errno 54] Connection reset by peer about 20% - 80% of the time, depending on how many client threads I spawn at once. Why?
Code Snippets
Server listens:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind((self.interface, self.port))
sock.listen(5)
Server loops forever, accepts client connection, spawns new thread:
while True:
(clientsock, (ip, port)) = self.sock.accept()
newthread = ClientThread(ip, port, clientsock)
newthread.start()
Spawn a bunch of client threads which connect with server, send message which requests a file, and then closes connection
Server sends message to client when ready
self.socket.sendall(message.encode())
After message is sent, close the write end of connection:
self.socket.shutdown(socket.SHUT_WR)
Client receives message (error occurs here)
def receive(self):
data_string = ''
bytes = self.sock.recv(self.bufsize)
while len(bytes) > 0:
bytes_str = bytes.decode('UTF-8')
data_string += bytes_str
bytes = self.sock.recv(self.bufsize)
return data_string
After client thread has received message, close the connection:
self.socket.close()

Receive function had errors. Changed to this:
def receive(self):
data_string = ''
while True:
bytes = self.sock.recv(self.bufsize)
bytes_str = bytes.decode('UTF-8')
data_string += bytes_str
if not bytes:
break
return data_string
Old receive function would try to call recv a second time when server had already closed socket. New one only calls once.
Also did not know you could increase listening socket backlog > 5 since Python docs say generally 5 is max, when on OS X it is 128. Increasing backlog to 128 helped.