In MacOS, I used to go with the following solution:
import socket
import time
server = socket.socket()
server.bind(('127.0.0.1', 7777))
time.sleep(5)
server.listen(1)
If another socket tries to connect to this address, it will be hung up until server calls listen.
But this is not the case in Linux, trying to connect before the listen call results in Connection Refused immediately.
So is there a better way to emulate connect timeout when testing?
I don't know the specifics of what you're testing, but you perhaps could go ahead and do an accept, and then sleep - you'll appear to get a connection in the client, but it doesn't do anything useful (for a while). Or play with your firewall settings :)
You can use poll or select. It returns until n seconds , which it is specified by you, or when it is writable, which means connectable for clients.
http://docs.python.org/3/library/multiprocessing.html?highlight=poll#multiprocessing.Connection.poll
Related
I have some code which will connect to a host and do nothing but listen for incoming data until either the client is shut down or the host send a close statement. For this my code works well.
However when the host dies without sending a close statement, my client keeps listening for incoming data forever as expected. To resolve this I made the socket timeout every foo seconds and start the process of checking if the connection is alive or not. From the Python socket howto I found this:
One very nasty problem with select: if somewhere in those input lists of sockets is one which has died a nasty death, the select will fail. You then need to loop through every single damn socket in all those lists and do a select([sock],[],[],0) until you find the bad one. That timeout of 0 means it won’t take long, but it’s ugly.
# Example code written for this question.
from select import select
from socket include socket, AF_INET, SOCK_STREAM
socket = socket(AF_INET, SOCK_STREAM)
socket.connect(('localhost', 12345))
socklist = [socket,]
attempts = 0
def check_socklist(socks):
for sock in socklist:
(r, w, e) = select([sock,], [], [], 0)
...
...
...
while True:
(r, w, e) = select(socklist, [], [], 60)
for sock in r:
if sock is socket:
msg = sock.recv(4096)
if not msg:
attempts +=1
if attempts >= 10:
check_socket(socklist)
break
else:
attempts = 0
print msg
This text creates three questions.
I was taught that to check if a connection is alive or not, one has to write to the socket and see if a response returns. If not, the connection has to be assumed it is dead. In the text it says that to check for bad connections, one single out each socket, pass it to select's first parameter and set the timeout to zero. How will this confirm that the socket is dead or not?
Why not test if the socket is dead or alive by trying to write to the socket instead?
What am I looking for when the connection is alive and when it is dead? Select will timeout at once, so having no data there will prove nothing.
I realize there are libraries like gevent, asyncore and twisted that can help me with this, but I have chosen to do this my self to get a better understanding of what is happening and to get more control over the source my self.
If a connected client crashes or exits, but its host OS and computer are still running, then its OS's TCP stack will send your server a FIN packet to let your computer's TCP stack know that the TCP connection has been closed. Your Python app will see this as select() indicating that the client's socket is ready-for-read, and then when you call recv() on the socket, recv() will return 0. When that happens, you should respond by closing the socket.
If the connected client's computer never gets a chance to send a FIN packet, on the other hand (e.g. because somebody reached over and yanked its Ethernet cord or power cable out of the socket), then your server won't realize that the TCP connection is defunct for quite a while -- possibly forever. The easiest way to avoid having a "zombie socket" is simply to have your server send some dummy data on the socket every so often, e.g. once per minute or something. The client should know to discard the dummy data. The benefit of sending the dummy data is that your server's TCP stack will then notice that it's not getting any ACK packets back for the data packet(s) it sent, and will resend them; and after a few resends your server's TCP stack will give up and decide that the connection is dead, at which point you'll see the same behavior that I described in my first paragraph.
If you write something to a socket and then wait for an answer to check the connection, the server should support this "ping" messages. It is not alway the case. Otherwise the server app may crash itself or disconnect your client if the server doesn't wait this message.
If select failed in the way you described, the socket framework knows which socket is dead. You just need to find it. But if a socket is dead by that nasty death like server's app crash, it doesn't mean mandatory that client's socket framework will detect that. E.g. in the case when a client is waiting some messages from the server and the server crashes, in some cases the client can wait forever. For example Putty, to avoid this scenario, can use application's protocol-level ping (SSH ping option) of the server to check the connection; SSH server can use TCP keepalive to check the connection and to prevent network equipment from dropping connections without activity.
(see p.1).
You are right that select's timeout and having no data proves nothing. As documentation says you have to check every socket when select fails.
My goal is to establish a continuous and robust TCP connection between one server and exactly one client. If one side fails, the other one should wait until it recovers.
I wrote the following code based on this question (that only asks for continuous, but not robust TCP connections and does not handle keepalive issues), this post and my own experience.
I have two questions:
How can I make the keepalive work? If the server dies, the client only recognizes it after trying to send() - which worked also without the KEEPALIVE option as this results in a connection reset. Is there some way that the socket sends an interrupt for a connection that is dead or some keepalive function that I can check on a regular basis?
Is this a robust way of handling a continous TCP connection? Having a stable, continous TCP connection seems to be a standard problem, however, I couldn't find tutorials covering this in detail. There must be some best-practice.
Note, I could handle keep alive messages on my own at the application level. However, as TCP already implements this at transport level, it is better to rely on this service provided by the lower level.
The server:
from socket import *
serverPort = 12000
while True:
# 1. Configure server socket
serverSocket = socket(AF_INET, SOCK_STREAM)
serverSocket.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
serverSocket.bind(('127.0.0.1', serverPort))
serverSocket.listen(1)
print("waiting for client connecting...")
connectionSocket, addr = serverSocket.accept()
connectionSocket.setsockopt(SOL_SOCKET, SO_KEEPALIVE,1)
print(connectionSocket.getsockopt(SOL_SOCKET,SO_KEEPALIVE))
print("...connected.")
serverSocket.close() # Destroy the server socket; we don't need it anymore since we are not accepting any connections beyond this point.
# 2. communication routine
while True:
try:
sentence = connectionSocket.recv(512).decode()
except ConnectionResetError as e:
print("Client connection closed")
break
if(len(sentence)==0): # close if client closed connection
break
else:
print("recv: "+str(sentence))
# 3. proper closure
connectionSocket.shutdown(SHUT_RDWR)
connectionSocket.close()
print("connection closed.")
The client:
from socket import *
import time
while True:
# 1. configure socket dest.
serverName = '127.0.0.1'
serverPort = 12000
clientSocket = socket(AF_INET, SOCK_STREAM)
try:
clientSocket.setsockopt(SOL_SOCKET, SO_KEEPALIVE,1)
clientSocket.connect((serverName, serverPort))
print(clientSocket.getsockopt(SOL_SOCKET,SO_KEEPALIVE))
except ConnectionRefusedError as e:
print("Server refused connection. retrying")
time.sleep(1)
continue
# 2. communication routine
while(1):
sentence = input('input sentence: ')
if(sentence == "close"):
break
try:
clientSocket.send(sentence.encode())
except ConnectionResetError as e:
print("Server connection closed")
break
# 3. proper closure
clientSocket.shutdown(SHUT_RDWR)
clientSocket.close()
I tried to hold this example as minimal as possible. But given the requirement of robustness, it is relativley long.
I also tried some socket options as TCP_KEEPIDLE, TCP_KEEPINTVL and TCP_KEEPCNT.
Thank you!
I will try to answer both questions.
... Is there some way that the socket sends an interrupt for a connection that is dead ...
I know none. TCP_KEEPALIVE only tries to maintain the connection. It is very useful if any equipment on the network flow has a timeout, because it prevents the timeout to abort the connection. But if the connection drops because because of any other reason (that timeout) TCP_KEEPALIVE cannot do anything. The rationale is that there is no need to restore a dropped inactive connection before something has to be exchanged.
Is this a robust way of handling a continous TCP connection?
Not really.
The robust way is to be prepared that the connection fails for any reason at any moment. So you should be prepared to face an error when sending a message (your code is) and if that happens try to re-open the connection and send the message again (your current code does not). Something like:
def connect(...):
# establish and return a connection
...
return clientSocket
clientSocket = connect(...)
while True:
...
while True:
try:
clientSocket.send(message)
break
except OSError:
clientSocket = connect()
...
Unrelated: your graceful shutdown is incorrect. The initiator (the part using shutdown) should not immediately close the socket, but start a read loop and only close when everything has be received and processed.
How can I make the keepalive work? If the server dies, the client only recognizes it after trying to send() - which worked also without the KEEPALIVE option as this results in a connection reset.
Keepalive is more useful on the server or reading side. And it is a tricky beast. The socket won't notify you at all unless you read/write. You can query its state (even though I'm not sure this is possible with the standard Python) but this still doesn't solve the problem of notification. You need to check the state periodically anyway.
Is there some way that the socket sends an interrupt for a connection that is dead or some keepalive function that I can check on a regular basis?
Have you ever heard about the Two Generals' Problem? There is no reliable way to detect whether one side is dead or not. We can however be close enough with pings and timeouts.
Note, I could handle keep alive messages on my own at the application level. However, as TCP already implements this at transport level, it is better to rely on this service provided by the lower level.
No, it is not better. If, for any reason, there's a proxy between the server and the client, then no TCP feature will help you. Because by design these only control a single connection, while with a proxy you have at least two connections. You should not think about your connection in terms of the underlying transport (TCP). Instead create your own protocol with ping command which the server (or client or both) send periodically together with timeouts. This way you can be sure that the peer is alive up to period interval.
Is this a robust way of handling a continous TCP connection? Having a stable, continous TCP connection seems to be a standard problem, however, I couldn't find tutorials covering this in detail. There must be some best-practice.
You won't find tutorials covering this, because that problem has no solution. Most people simulate "I'm still alive" with the combination of pings and timeouts.
So, I have a server socket defined as server_sock the current code looks as follow
# define a variable server_sock
server_sock.bind(("", PORT_ANY))
server_sock.listen(1)
port = server_sock.getsockname()[1]
client_sock, client_info = server_sock.accept()
print("[+] Accepted connection from ", client_info)
server_sock.close()
client_sock.close()
# define a variable sock
sock.connect((host, port)) # This will be the client socket
Now, this code will create a server_sock , listen for incoming connections and after any client is connected it will close those socket and act as client by using another sock.
What I am planning to do is to first let the code run as a server (i.e server_sock should wait for connections) for specific timeout (Let's assume 10 seconds).
After the 10 seconds, the server_sock should get closed by itself and then the next piece of code (i.e client part) should start.
So, it's loosely something like change from Server Mode to Client Mode after a specific Time-Out.
I am having a hard time to solve this issue. Usually server_sock.accept() line would be stuck until it hits a new connection else it won't proceed.
So, how can I implement something which breaks that whole thing after a specific time-out.
Note that I am running this code cross platform on Windows and UNIX. So, I have been looking to some signal specific answers but Windows doesn't support some Signal.
EDIT:
Many people are saying to use settimeout() on the socket. But that doesn't answer the behavior I need.
Because Let's say if I get a connection from a device during the server_sock mode, I would like to continue communication thereafter rather than abruptly closing the socket.
The settimeout() will close socket no matter what actions are being performed. So, that fails to answer my case
You can simply use a timeout function.
# define a socket variable server_sock
server_sock.settimeout(10.0) #setting timeout for 10 sec
# rest of your code
I'm trying to connect a socket to an endpoint until the socket receives data from that endpoint. This is because the endpoint might not exist at that time.
Currently the connect stalls, i'm guessing because it can't resolve the hostname and that takes a while.
Is there any way to set a timeout on a socket connect?
import zmq
import time
endpoint = 'tcp://doesnt_exist:12345'
ctx = zmq.Context.instance()
s = ctx.socket(zmq.SUB)
t = time.time()
try:
s.connect(endpoint)
except Exception:
pass
print time.time() - t
If you provide a host name to connect, ZeroMQ uses synchronous DNS resolution via a call to getaddrinfo, which is why you see the connect call blocking.
If you really need to connect in controllable way, I suggest you do DNS resolve on your own, using one of the asynchronous DNS resolvers already available for Python (check this example based on pyuc/pycares).
Also see my reply to similar question.
The problem is not the connection, but the DNS lookup. The blocking is done at the OS level, on the gethostbyname call.
Since the timeout is controlled by the OS, working around it is hard (but feasible). My suggestion is that you simply hardcode the IP
I have some code which will connect to a host and do nothing but listen for incoming data until either the client is shut down or the host send a close statement. For this my code works well.
However when the host dies without sending a close statement, my client keeps listening for incoming data forever as expected. To resolve this I made the socket timeout every foo seconds and start the process of checking if the connection is alive or not. From the Python socket howto I found this:
One very nasty problem with select: if somewhere in those input lists of sockets is one which has died a nasty death, the select will fail. You then need to loop through every single damn socket in all those lists and do a select([sock],[],[],0) until you find the bad one. That timeout of 0 means it won’t take long, but it’s ugly.
# Example code written for this question.
from select import select
from socket include socket, AF_INET, SOCK_STREAM
socket = socket(AF_INET, SOCK_STREAM)
socket.connect(('localhost', 12345))
socklist = [socket,]
attempts = 0
def check_socklist(socks):
for sock in socklist:
(r, w, e) = select([sock,], [], [], 0)
...
...
...
while True:
(r, w, e) = select(socklist, [], [], 60)
for sock in r:
if sock is socket:
msg = sock.recv(4096)
if not msg:
attempts +=1
if attempts >= 10:
check_socket(socklist)
break
else:
attempts = 0
print msg
This text creates three questions.
I was taught that to check if a connection is alive or not, one has to write to the socket and see if a response returns. If not, the connection has to be assumed it is dead. In the text it says that to check for bad connections, one single out each socket, pass it to select's first parameter and set the timeout to zero. How will this confirm that the socket is dead or not?
Why not test if the socket is dead or alive by trying to write to the socket instead?
What am I looking for when the connection is alive and when it is dead? Select will timeout at once, so having no data there will prove nothing.
I realize there are libraries like gevent, asyncore and twisted that can help me with this, but I have chosen to do this my self to get a better understanding of what is happening and to get more control over the source my self.
If a connected client crashes or exits, but its host OS and computer are still running, then its OS's TCP stack will send your server a FIN packet to let your computer's TCP stack know that the TCP connection has been closed. Your Python app will see this as select() indicating that the client's socket is ready-for-read, and then when you call recv() on the socket, recv() will return 0. When that happens, you should respond by closing the socket.
If the connected client's computer never gets a chance to send a FIN packet, on the other hand (e.g. because somebody reached over and yanked its Ethernet cord or power cable out of the socket), then your server won't realize that the TCP connection is defunct for quite a while -- possibly forever. The easiest way to avoid having a "zombie socket" is simply to have your server send some dummy data on the socket every so often, e.g. once per minute or something. The client should know to discard the dummy data. The benefit of sending the dummy data is that your server's TCP stack will then notice that it's not getting any ACK packets back for the data packet(s) it sent, and will resend them; and after a few resends your server's TCP stack will give up and decide that the connection is dead, at which point you'll see the same behavior that I described in my first paragraph.
If you write something to a socket and then wait for an answer to check the connection, the server should support this "ping" messages. It is not alway the case. Otherwise the server app may crash itself or disconnect your client if the server doesn't wait this message.
If select failed in the way you described, the socket framework knows which socket is dead. You just need to find it. But if a socket is dead by that nasty death like server's app crash, it doesn't mean mandatory that client's socket framework will detect that. E.g. in the case when a client is waiting some messages from the server and the server crashes, in some cases the client can wait forever. For example Putty, to avoid this scenario, can use application's protocol-level ping (SSH ping option) of the server to check the connection; SSH server can use TCP keepalive to check the connection and to prevent network equipment from dropping connections without activity.
(see p.1).
You are right that select's timeout and having no data proves nothing. As documentation says you have to check every socket when select fails.