I want my python application to be able to tell when the socket on the other side has been dropped. Is there a method for this?
Short answer:
use a non-blocking recv(), or a blocking recv() / select() with a very
short timeout.
Long answer:
The way to handle socket connections is to read or write as you need to, and be prepared to handle connection errors.
TCP distinguishes between 3 forms of "dropping" a connection: timeout, reset, close.
Of these, the timeout can not really be detected, TCP might only tell you the time has not expired yet. But even if it told you that, the time might still expire right after.
Also remember that using shutdown() either you or your peer (the other end of the connection) may close only the incoming byte stream, and keep the outgoing byte stream running, or close the outgoing stream and keep the incoming one running.
So strictly speaking, you want to check if the read stream is closed, or if the write stream is closed, or if both are closed.
Even if the connection was "dropped", you should still be able to read any data that is still in the network buffer. Only after the buffer is empty will you receive a disconnect from recv().
Checking if the connection was dropped is like asking "what will I receive after reading all data that is currently buffered ?" To find that out, you just have to read all data that is currently bufferred.
I can see how "reading all buffered data", to get to the end of it, might be a problem for some people, that still think of recv() as a blocking function. With a blocking recv(), "checking" for a read when the buffer is already empty will block, which defeats the purpose of "checking".
In my opinion any function that is documented to potentially block the entire process indefinitely is a design flaw, but I guess it is still there for historical reasons, from when using a socket just like a regular file descriptor was a cool idea.
What you can do is:
set the socket to non-blocking mode, but than you get a system-depended error to indicate the receive buffer is empty, or the send buffer is full
stick to blocking mode but set a very short socket timeout. This will allow you to "ping" or "check" the socket with recv(), pretty much what you want to do
use select() call or asyncore module with a very short timeout. Error reporting is still system-specific.
For the write part of the problem, keeping the read buffers empty pretty much covers it. You will discover a connection "dropped" after a non-blocking read attempt, and you may choose to stop sending anything after a read returns a closed channel.
I guess the only way to be sure your sent data has reached the other end (and is not still in the send buffer) is either:
receive a proper response on the same socket for the exact message that you sent. Basically you are using the higher level protocol to provide confirmation.
perform a successful shutdow() and close() on the socket
The python socket howto says send() will return 0 bytes written if channel is closed. You may use a non-blocking or a timeout socket.send() and if it returns 0 you can no longer send data on that socket. But if it returns non-zero, you have already sent something, good luck with that :)
Also here I have not considered OOB (out-of-band) socket data here as a means to approach your problem, but I think OOB was not what you meant.
It depends on what you mean by "dropped". For TCP sockets, if the other end closes the connection either through
close() or the process terminating, you'll find out by reading an end of file, or getting a read error, usually the errno being set to whatever 'connection reset by peer' is by your operating system. For python, you'll read a zero length string, or a socket.error will be thrown when you try to read or write from the socket.
From the link Jweede posted:
exception socket.timeout:
This exception is raised when a timeout occurs on a socket
which has had timeouts enabled via a prior call to settimeout().
The accompanying value is a string whose value is currently
always “timed out”.
Here are the demo server and client programs for the socket module from the python docs
# Echo server program
import socket
HOST = '' # Symbolic name meaning all available interfaces
PORT = 50007 # Arbitrary non-privileged port
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((HOST, PORT))
s.listen(1)
conn, addr = s.accept()
print 'Connected by', addr
while 1:
data = conn.recv(1024)
if not data: break
conn.send(data)
conn.close()
And the client:
# Echo client program
import socket
HOST = 'daring.cwi.nl' # The remote host
PORT = 50007 # The same port as used by the server
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
s.send('Hello, world')
data = s.recv(1024)
s.close()
print 'Received', repr(data)
On the docs example page I pulled these from, there are more complex examples that employ this idea, but here is the simple answer:
Assuming you're writing the client program, just put all your code that uses the socket when it is at risk of being dropped, inside a try block...
try:
s.connect((HOST, PORT))
s.send("Hello, World!")
...
except socket.timeout:
# whatever you need to do when the connection is dropped
If I'm not mistaken this is usually handled via a timeout.
I translated the code sample in this blog post into Python: How to detect when the client closes the connection?, and it works well for me:
from ctypes import (
CDLL, c_int, POINTER, Structure, c_void_p, c_size_t,
c_short, c_ssize_t, c_char, ARRAY
)
__all__ = 'is_remote_alive',
class pollfd(Structure):
_fields_ = (
('fd', c_int),
('events', c_short),
('revents', c_short),
)
MSG_DONTWAIT = 0x40
MSG_PEEK = 0x02
EPOLLIN = 0x001
EPOLLPRI = 0x002
EPOLLRDNORM = 0x040
libc = CDLL('libc.so.6')
recv = libc.recv
recv.restype = c_ssize_t
recv.argtypes = c_int, c_void_p, c_size_t, c_int
poll = libc.poll
poll.restype = c_int
poll.argtypes = POINTER(pollfd), c_int, c_int
class IsRemoteAlive: # not needed, only for debugging
def __init__(self, alive, msg):
self.alive = alive
self.msg = msg
def __str__(self):
return self.msg
def __repr__(self):
return 'IsRemoteAlive(%r,%r)' % (self.alive, self.msg)
def __bool__(self):
return self.alive
def is_remote_alive(fd):
fileno = getattr(fd, 'fileno', None)
if fileno is not None:
if hasattr(fileno, '__call__'):
fd = fileno()
else:
fd = fileno
p = pollfd(fd=fd, events=EPOLLIN|EPOLLPRI|EPOLLRDNORM, revents=0)
result = poll(p, 1, 0)
if not result:
return IsRemoteAlive(True, 'empty')
buf = ARRAY(c_char, 1)()
result = recv(fd, buf, len(buf), MSG_DONTWAIT|MSG_PEEK)
if result > 0:
return IsRemoteAlive(True, 'readable')
elif result == 0:
return IsRemoteAlive(False, 'closed')
else:
return IsRemoteAlive(False, 'errored')
Trying to improve on #kay response. I made a more pythonic version
(Note that it was not yet tested in a "real-life" environment, and only on Linux)
This detects if the remote side closed the connection, without actually consuming the data:
import socket
import errno
def remote_connection_closed(sock: socket.socket) -> bool:
"""
Returns True if the remote side did close the connection
"""
try:
buf = sock.recv(1, socket.MSG_PEEK | socket.MSG_DONTWAIT)
if buf == b'':
return True
except BlockingIOError as exc:
if exc.errno != errno.EAGAIN:
# Raise on unknown exception
raise
return False
Here is a simple example from an asyncio echo server:
import asyncio
async def handle_echo(reader, writer):
addr = writer.get_extra_info('peername')
sock = writer.get_extra_info('socket')
print(f'New client: {addr!r}')
# Initial of client command
data = await reader.read(100)
message = data.decode()
print(f"Received {message!r} from {addr!r}")
# Simulate a long async process
for _ in range(10):
if remote_connection_closed(sock):
print('Remote side closed early')
return
await asyncio.sleep(1)
# Write the initial message back
print(f"Send: {message!r}")
writer.write(data)
await writer.drain()
writer.close()
async def main():
server = await asyncio.start_server(
handle_echo, '127.0.0.1', 8888)
addrs = ', '.join(str(sock.getsockname()) for sock in server.sockets)
print(f'Serving on {addrs}')
async with server:
await server.serve_forever()
if __name__ == '__main__':
asyncio.run(main())
Related
I'm trying to write a simple daemon that listens for orders on a Unix socket. The following works, but the connection.recv(1024) line blocks, meaning I can't kill the server gracefully:
import socket, os
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as server:
server.bind("/tmp/sock")
server.listen()
connection, __ = server.accept()
with connection:
while True:
data = connection.recv(1024)
print("Hi!") # This line isn't executed 'til data is sent
if data:
print(data.decode())
Ideally, I'd like to place all of this inside a Thread that checks a self.should_stop property every self.LOOP_TIME seconds, and if that value is set to True, then exit. However, as that .recv() line blocks, there's no way for my program to be doing anything other than waiting at any given time.
Surely there's a proper way to do this, but as I'm new to sockets, I have no idea what that is.
Edit
Jeremy Friesner's answer put me on the right track. I realised that I could allow the thread to block and simply set .should_stop then pass an b"" to the socket so that it'd un-block, see that it should stop, and then exit cleanly. Here's the end result:
import os
import socket
from pathlib import Path
from shutil import rmtree
from threading import Thread
class MyThreadThing(Thread):
RUNTIME_DIR = Path(os.getenv("XDG_RUNTIME_DIR", "/tmp")) / "my-project-name"
def __init__(self):
super().__init__(daemon=True)
self.should_stop = False
if self.RUNTIME_DIR.exists():
rmtree(self.RUNTIME_DIR)
self.RUNTIME_DIR.mkdir(0o700)
self.socket_path = self.RUNTIME_DIR / "my-project.sock"
def run(self) -> None:
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as s:
s.bind(self.socket_path.as_posix())
s.listen()
while True:
connection, __ = s.accept()
action = ""
with connection:
while True:
received = connection.recv(1024).decode()
action += received
if not received:
break
# Handle whatever is in `action`
if self.should_stop:
break
self.socket_path.unlink()
def stop(self):
"""
Trigger this when you want to stop the listener.
"""
self.should_stop = True
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as s:
s.connect(self.socket_path.as_posix())
s.send(b"")
Using arbitrary-length timeouts is always a bit unsatisfactory -- either you set the timeout-value to a relatively long time, in which case your program becomes slow to react to the quit-request, because it is pointlessly waiting for timeout period to expire... or you set the timeout-value to a relatively short time, in which case your program is constantly waking up to see if it should quit, wasting CPU power 24/7 to check for an event which might never arrive.
A more elegant way to deal with the problem is to create a pipe, and send a byte on the pipe when you want your event-loop to exit. Your event loop can simultaneously "watch" both the pipe's reading-end file-descriptor and your networking-socket(s) via select(), and when that file-descriptor indicates it is ready-for-read, your event loop can respond by exiting. This approach is entirely event-driven, so it requires no CPU wakeups except when there is actually something to do.
Below is an example version of your program that implements a signal-handler for SIGINT (aka pressing Control-C) to sends the please-quit-now byte on the pipe:
import socket, os
import select
import signal, sys
# Any bytes written to (writePipeFD) will become available for reading on (readPipeFD)
readPipeFD, writePipeFD = os.pipe()
# Set up a signal-handler to handle SIGINT (aka Ctrl+C) events by writing a byte to the pipe
def signal_handler(sig, frame):
print("signal_handler() is executing -- SIGINT detected!")
os.write(writePipeFD, b"\0") # doesn't matter what we write; a single 0-byte will do
signal.signal(signal.SIGINT, signal_handler)
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as serverSock:
serverSock.bind("/tmp/sock")
serverSock.listen()
# Wait for incoming connection (or the please-quit signal, whichever comes first)
connection = None
while True:
readReady,writeReady,exceptReady = select.select([readPipeFD,serverSock], [], [])
if (readPipeFD in readReady):
print("accept-loop: Someone wrote a byte to the pipe; time to go away!");
break
if (connection in readReady):
connection, __ = serverSock.accept()
break
# Read data from incoming connection (or the please-quit signal, whichever comes first)
if connection:
with connection:
while True:
readReady,writeReady,exceptReady = select.select([readPipeFD,connection], [], [])
if (readPipeFD in readReady):
print("Connection-loop: Someone wrote a byte to the pipe; time to go away!");
break
if (connection in readReady):
data = connection.recv(1024)
print("Hi!") # This line isn't executed 'til data is sent
if data:
print(data.decode())
print("Bye!")
Use a timeout identical to your LOOP_TIME like so:
import socket, os
LOOP_TIME = 10
should_stop = False
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as server:
server.bind("/tmp/sock")
server.listen()
connection, __ = server.accept()
connection.settimeout(LOOP_TIME)
with connection:
while not should_stop:
try:
data = connection.recv(1024)
except socket.timeout:
continue
print("Hi!") # This line isn't executed 'til data is sent
if data:
print(data.decode())
You may use select, but if it's only a single simple socket, this way is a bit less complicated.
You can choose to place it in a different thread with a self.should_stop or just at the main - it will now listen to the KeyboardInterrupt.
Overview
I have a simple question with code below. Hopefully I didn't make a mistake in the code.
I'm a network engineer, and I need to test certain linux behavior of our business application keepalives during network outages (I'm going to insert some iptables stuff later to jack with the connection - first I want to make sure I got the client & server right).
As part of a network failure test I'm conducting, I wrote a non-blocking Python TCP client and server that are supposed to blindly send messages to each other in a loop. To understand what's happening I am using loop counters.
The server's loop should be relatively straightforward. I loop through every fd that select says is ready. I never even import sleep anywhere in my server's code. From this perspective, I don't expect the server's code to pause while it loops over the client's socket , but for some reason the server code pauses intermittently (more detail, below).
I initially didn't put a sleep in the client's loop. Without a sleep on the client side, the server and client seem to be as efficient as I want. However, when I put a time.sleep(1) statement after the client does an fd.send() to the server, the TCP server code intermittently pauses while the client is sleeping.
My questions:
Should I be able to write a single-threaded Python TCP server that doesn't pause when the client hits time.sleep() in the client's fd.send() loop? If so, what am I doing wrong? <- ANSWERED
If I wrote this test code correctly and the server shouldn't pause, why is the TCP server intermittently pausing while it polls the client's connection for data?
Reproducing the scenario
I'm running this on two RHEL6 linux machines. To reproduce the issue...
Open two different terminals.
Save the client and server scripts in different files
Change the shebang path to your local python (I'm using Python 2.7.15)
Change the SERVER_HOSTNAME and SERVER_DOMAIN in the client's code to be the hostname and domain of the server you're running this on
Start the server first, then start the client.
After the client connects, you'll see messages as shown in EXHIBIT 1 scrolling quickly in the server's terminal. After a few seconds The scrolling pauses intermittently when the client hits time.sleep(). I don't expect to see those pauses, but maybe I've misunderstood something.
EXHIBIT 1
---
LOOP_COUNT 0
---
LOOP_COUNT 1
---
LOOP_COUNT 2
---
LOOP_COUNT 3
CLIENTMSG: 'client->server 0'
---
LOOP_COUNT 4
---
LOOP_COUNT 5
---
LOOP_COUNT 6
---
LOOP_COUNT 7
---
LOOP_COUNT 8
---
LOOP_COUNT 9
---
LOOP_COUNT 10
---
LOOP_COUNT 11
---
Summary resolution
If I wrote this test code correctly and the server shouldn't pause, why is the TCP server intermittently pausing while it polls the client's connection for data?
Answering my own question. My blocking problem was caused by calling select() with a non-zero timeout.
When I changed select() to use a zero-second timeout, I got expected results.
Final non-blocking code (incorporating suggestions in answers):
tcp_server.py
#!/usr/bin/python -u
from socket import AF_INET, SOCK_STREAM, SO_REUSEADDR, SOL_SOCKET
from socket import MSG_DONTWAIT
#from socket import MSG_OOB <--- for send()
from socket import socket
import socket as socket_module
import select
import errno
import fcntl
import time
import sys
import os
def get_errno_info(e, op='', debugmsg=False):
"""Return verbose information from errno errors, such as errors returned by python socket()"""
VALID_OP = set(['accept', 'connect', 'send', 'recv', 'read', 'write'])
assert op.lower() in VALID_OP, "op must be: {0}".format(
','.join(sorted(VALID_OP)))
## ref: man 3 errno (in linux)... other systems may be man 2 intro
## also see https://docs.python.org/2/library/errno.html
try:
retval_int = int(e.args[0]) # Example: 32
retval_str = os.strerror(e.args[0]) # Example: 'Broken pipe'
retval_code = errno.errorcode.get(retval_int, 'MODULEFAIL') # Ex: EPIPE
except:
## I don't expect to get here unless something broke in python errno...
retval_int = -1
retval_str = '__somethingswrong__'
retval_code = 'BADFAIL'
if debugmsg:
print "DEBUG: Can't {0}() on socket (errno:{1}, code:{2} / {3})".format(
op, retval_int, retval_code, retval_str)
return retval_int, retval_str, retval_code
host = ''
port = 6667 # IRC service
DEBUG = True
serv_sock = socket(AF_INET, SOCK_STREAM)
serv_sock.setsockopt(SOL_SOCKET, SOCK_STREAM, 1)
serv_sock.bind((host, port))
serv_sock.listen(5)
#fcntl.fcntl(serv_sock, fcntl.F_SETFL, os.O_NONBLOCK) # Make the socket non-blocking
serv_sock.setblocking(False)
sock_list = [serv_sock]
from_client_str = '__DEFAULT__'
to_client_idx = 0
loop_count = 0
need_send_select = False
while True:
if need_send_select:
# Only do this after send() EAGAIN or EWOULDBLOCK...
send_sock_list = sock_list
else:
send_sock_list = []
#print "---"
#print "LOOP_COUNT", loop_count
recv_ready_list, send_ready_list, exception_ready = select.select(
sock_list, send_sock_list, [], 0.0) # Last float is the select() timeout...
## Read all sockets which are output-ready... might be client or server...
for sock_fd in recv_ready_list:
# accept() if we're reading on the server socket...
if sock_fd is serv_sock:
try:
clientsock, clientaddr = sock_fd.accept()
except socket_module.error, e:
errstr, errint, errcode = get_errno_info(e, op='accept',
debugmsg=DEBUG)
assert sock_fd.gettimeout()==0.0, "client socket should be in non-blocking mode"
sock_list.append(clientsock)
# read input from the client socket...
else:
try:
from_client_str = sock_fd.recv(1024, MSG_DONTWAIT)
if from_client_str=='':
# Client closed the socket...
print "CLIENT CLOSED SOCKET"
sock_list.remove(sock_fd)
except socket_module.error, e:
errstr, errint, errcode = get_errno_info(e, op='recv',
debugmsg=DEBUG)
if errcode=='EAGAIN' or errcode=='EWOULDBLOCK':
# socket unavailable to read()
continue
elif errcode=='ECONNRESET' or errcode=='EPIPE':
# Client closed the socket...
sock_list.remove(sock_fd)
else:
print "UNHANDLED SOCKET ERROR", errcode, errint, errstr
sys.exit(1)
print "from_client_str: '{0}'".format(from_client_str)
## Adding dynamic_list, per input from EJP, below...
if need_send_select is False:
dynamic_list = sock_list
else:
dynamic_list = send_ready_list
## NOTE: socket code shouldn't walk this list unless a write is pending...
## broadast the same message to all clients...
for sock_fd in dynamic_list:
## Ignore server's listening socket...
if sock_fd is serv_sock:
## Only send() to accept()ed sockets...
continue
try:
to_client_str = "server->client: {0}\n".format(to_client_idx)
send_retval = sock_fd.send(to_client_str, MSG_DONTWAIT)
## send() returns the number of bytes written, on success
## disabling assert check on sent bytes while using MSG_DONTWAIT
#assert send_retval==len(to_client_str)
to_client_idx += 1
need_send_select = False
except socket_module.error, e:
errstr, errint, errcode = get_errno_info(e, op='send',
debugmsg=DEBUG)
if errcode=='EAGAIN' or errcode=='EWOULDBLOCK':
need_send_select = True
continue
elif errcode=='ECONNRESET' or errcode=='EPIPE':
# Client closed the socket...
sock_list.remove(sock_fd)
else:
print "FATAL UNHANDLED SOCKET ERROR", errcode, errint, errstr
sys.exit(1)
loop_count += 1
tcp_client.py
#!/usr/bin/python -u
from socket import AF_INET, SOCK_STREAM
from socket import MSG_DONTWAIT # non-blocking send/recv; see man 2 recv
from socket import gethostname, socket
import socket as socket_module
import select
import fcntl
import errno
import time
import sys
import os
## NOTE: Using this script to simulate a scheduler
SERVER_HOSTNAME = 'myServerHostname'
SERVER_DOMAIN = 'mydomain.local'
PORT = 6667
DEBUG = True
def get_errno_info(e, op='', debugmsg=False):
"""Return verbose information from errno errors, such as errors returned by python socket()"""
VALID_OP = set(['accept', 'connect', 'send', 'recv', 'read', 'write'])
assert op.lower() in VALID_OP, "op must be: {0}".format(
','.join(sorted(VALID_OP)))
## ref: man 3 errno (in linux)... other systems may be man 2 intro
## also see https://docs.python.org/2/library/errno.html
try:
retval_int = int(e.args[0]) # Example: 32
retval_str = os.strerror(e.args[0]) # Example: 'Broken pipe'
retval_code = errno.errorcode.get(retval_int, 'MODULEFAIL') # Ex: EPIPE
except:
## I don't expect to get here unless something broke in python errno...
retval_int = -1
retval_str = '__somethingswrong__'
retval_code = 'BADFAIL'
if debugmsg:
print "DEBUG: Can't {0}() on socket (errno:{1}, code:{2} / {3})".format(
op, retval_int, retval_code, retval_str)
return retval_int, retval_str, retval_code
connect_finished = False
while not connect_finished:
try:
c2s = socket(AF_INET, SOCK_STREAM) # Client to server socket...
# Set socket non-blocking
#fcntl.fcntl(c2s, fcntl.F_SETFL, os.O_NONBLOCK)
c2s.connect(('.'.join((SERVER_HOSTNAME, SERVER_DOMAIN,)), PORT))
c2s.setblocking(False)
assert c2s.gettimeout()==0.0, "c2s socket should be in non-blocking mode"
connect_finished = True
except socket_module.error, e:
errstr, errint, errcode = get_errno_info(e, op='connect',
debugmsg=DEBUG)
if errcode=='EINPROGRESS':
pass
to_srv_idx = 0
need_send_select = False
while True:
socket_list = [c2s]
# Get the list sockets which can: take input, output, etc...
if need_send_select:
# Only do this after send() EAGAIN or EWOULDBLOCK...
send_sock_list = socket_list
else:
send_sock_list = []
recv_ready_list, send_ready_list, exception_ready = select.select(
socket_list, send_sock_list, [])
for sock_fd in recv_ready_list:
assert sock_fd is c2s, "Strange socket failure here"
#incoming message from remote server
try:
from_srv_str = sock_fd.recv(1024, MSG_DONTWAIT)
except socket_module.error, e:
## https://stackoverflow.com/a/16745561/667301
errstr, errint, errcode = get_errno_info(e, op='recv',
debugmsg=DEBUG)
if errcode=='EAGAIN' or errcode=='EWOULDBLOCK':
# Busy, try again later...
print "recv() BLOCKED"
continue
elif errcode=='ECONNRESET' or errcode=='EPIPE':
# Server ended normally...
sys.exit(0)
## NOTE: if we get this far, we successfully received from_srv_str.
## Anything caught above, is some kind of fail...
print "from_srv_str: {0}".format(from_srv_str)
## Adding dynamic_list, per input from EJP, below...
if need_send_select is False:
dynamic_list = socket_list
else:
dynamic_list = send_ready_list
for sock_fd in dynamic_list:
# outgoing message to remote server
if sock_fd is c2s:
try:
to_srv_str = 'client->server {0}'.format(to_srv_idx)
sock_fd.send(to_srv_str, MSG_DONTWAIT)
##
time.sleep(1) ## Client blocks the server here... Why????
##
to_srv_idx += 1
need_send_select = False
except socket_module.error, e:
errstr, errint, errcode = get_errno_info(e, op='send',
debugmsg=DEBUG)
if errcode=='EAGAIN' or errcode=='EWOULDBLOCK':
## Try to send() later...
print "send() BLOCKED"
need_send_select = True
continue
elif errcode=='ECONNRESET' or errcode=='EPIPE':
# Server ended normally...
sys.exit(0)
Original Question Code:
tcp_server.py
#!/usr/bin/python -u
from socket import AF_INET, SOCK_STREAM, SO_REUSEADDR, SOL_SOCKET
#from socket import MSG_OOB <--- for send()
from socket import socket
import socket as socket_module
import select
import fcntl
import os
host = ''
port = 9997
serv_sock = socket(AF_INET, SOCK_STREAM)
serv_sock.setsockopt(SOL_SOCKET, SOCK_STREAM, 1)
serv_sock.bind((host, port))
serv_sock.listen(5)
fcntl.fcntl(serv_sock, fcntl.F_SETFL, os.O_NONBLOCK) # Make the socket non-blocking
sock_list = [serv_sock]
from_client_str = '__DEFAULT__'
to_client_idx = 0
loop_count = 0
while True:
recv_ready_list, send_ready_list, exception_ready = select.select(sock_list, sock_list,
[], 5)
print "---"
print "LOOP_COUNT", loop_count
## Read all sockets which are input-ready... might be client or server...
for sock_fd in recv_ready_list:
# accept() if we're reading on the server socket...
if sock_fd is serv_sock:
clientsock, clientaddr = sock_fd.accept()
sock_list.append(clientsock)
# read input from the client socket...
else:
try:
from_client_str = sock_fd.recv(4096)
if from_client_str=='':
# Client closed the socket...
print "CLIENT CLOSED SOCKET"
sock_list.remove(sock_fd)
except socket_module.error, e:
print "WARNING RECV FAIL"
print "from_client_str: '{0}'".format(from_client_str)
for sock_fd in send_ready_list:
if sock_fd is not serv_sock:
try:
to_client_str = "server->client: {0}\n".format(to_client_idx)
sock_fd.send(to_client_str)
to_client_idx += 1
except socket_module.error, e:
print "TO CLIENT SEND ERROR", e
loop_count += 1
tcp_client.py
#!/usr/bin/python -u
from socket import AF_INET, SOCK_STREAM
from socket import gethostname, socket
import socket as socket_module
import select
import fcntl
import errno
import time
import sys
import os
## NOTE: Using this script to simulate a scheduler
SERVER_HOSTNAME = 'myHostname'
SERVER_DOMAIN = 'mydomain.local'
PORT = 9997
def handle_socket_error_continue(e):
## non-blocking socket info from:
## https://stackoverflow.com/a/16745561/667301
print "HANDLE_SOCKET_ERROR_CONTINUE"
err = e.args[0]
if (err==errno.EAGAIN) or (err==errno.EWOULDBLOCK):
print 'CLIENT DEBUG: No data input from server'
return True
else:
print 'FROM SERVER RECV ERROR: {0}'.format(e)
sys.exit(1)
c2s = socket(AF_INET, SOCK_STREAM) # Client to server socket...
c2s.connect(('.'.join((SERVER_HOSTNAME, SERVER_DOMAIN,)), PORT))
# Set socket non-blocking...
fcntl.fcntl(c2s, fcntl.F_SETFL, os.O_NONBLOCK)
to_srv_idx = 0
while True:
socket_list = [c2s]
# Get the list sockets which can: take input, output, etc...
recv_ready_list, send_ready_list, exception_ready = select.select(
socket_list, socket_list, [])
for sock_fd in recv_ready_list:
assert sock_fd is c2s, "Strange socket failure here"
#incoming message from remote server
try:
from_srv_str = sock_fd.recv(4096)
except socket_module.error, e:
## https://stackoverflow.com/a/16745561/667301
err_continue = handle_socket_error_continue(e)
if err_continue is True:
continue
else:
if len(from_srv_str)==0:
print "SERVER CLOSED NORMALLY"
sys.exit(0)
## NOTE: if we get this far, we successfully received from_srv_str.
## Anything caught above, is some kind of fail...
print "from_srv_str: {0}".format(from_srv_str)
for sock_fd in send_ready_list:
#incoming message from remote server
if sock_fd is c2s:
#to_srv_str = raw_input('Send to server: ')
try:
to_srv_str = 'client->server {0}'.format(to_srv_idx)
sock_fd.send(to_srv_str)
##
time.sleep(1) ## Client blocks the server here... Why????
##
to_srv_idx += 1
except socket_module.error, e:
print "TO SERVER SEND ERROR", e
TCP sockets are almost always ready for writing, unless their socket send buffer is full.
It is therefore incorrect to always select on writability for a socket. You should only do so after you've encountered a send failure due to EAGAIN/EWOULDBLOCK. Otherwise your server will spin mindlessly processing writeable sockets, which will usually be all of them.
However, when I put a time.sleep(1) statement after the client does an
fd.send() to the server, the TCP server code intermittently pauses
while the client is sleeping.
AFAICT from running the provided code (nice self-contained example, btw), the server is behaving as intended.
In particular, the semantics of the select() call are that select() shouldn't return until there is something for the thread to do. Having the thread block inside select() is a good thing when there is nothing that the thread can do right now anyway, since it prevents the thread from spinning the CPU for no reason.
So in this case, your server program has told select() that it wants select() to return only when at least one of the following conditions is true:
serv_sock is ready-for-read (which is to say, a new client wants to connect to the server now)
serv_sock is ready-for-write (I don't believe this ever actually happens on a listening-socket, so this criterion can probably be ignored)
clientsock is ready-for-read (that is, the client has sent some bytes to the server and they are waiting in clientsock's buffer for the server thread to recv() them)
clientsock is ready-for-write (that is, clientsock has some room in its outgoing-data-buffer that the server could send() data into if it wants to send some data back to the client)
Five seconds have passed since the call to select() started blocking.
I see (via print-debugging) that when your server program blocks, it is blocking inside select(), which indicates that none of the 5 conditions above are being met during the blocking-period.
Why is that? Well, let's go down the list.
Not met because no other clients are trying to connect
Not met because this never happens
Not met because the server has read all of the data that the connected client has sent (and since the connected client is itself sleeping, it's not sending any more data)
Not met because the server has filled up the outgoing-data buffer of its clientsock (because the client program is sleeping, it's only reading the data coming from the server intermittently, and the TCP layer guarantees lossless/in-order transmission, so once clientsock's outgoing-data-buffer is full, clientsock won't select-as-ready-for-write unless/until the client reads at least some data from its end of the conenction)
Not met because 5 seconds haven't elapsed yet since select() started blocking.
So is this behavior actually a problem for the server? In fact it is not, because the server will still be responsive to any other clients that connect to the server. In particular, select() will still return right away whenever serv_sock or any other client's socket select()s as ready-for-read (or ready-for-write) and so the server can handle the other clients just fine while waiting for your hacked/slow client to wake up.
The hacked/slow client might be a problem for the user, but there's nothing the server can really do about that (short of forcibly disconnecting the client's TCP connection, or maybe printing out a log message requesting that someone debug the connected client program, I suppose :)).
I agree with EJP, btw -- selecting on ready-for-write should only be done on sockets that you actually want to write some data to. If you don't actually have any desire to write to the socket ASAP, then it's pointless and counterproductive to instruct select() to return as soon as that socket is ready-for-write: the problem with doing so is that you're likely to spin the CPU a lot whenever any socket's outgoing-data-buffer is less-than-full (which in most applications, is most of the time!). The user-visible symptom of the problem would be that your server program is using up 100% of a CPU core even when it ought to be idle or mostly-idle.
Referencing this example (and the docs): https://pymotw.com/2/socket/tcp.html I am trying to achieve bidirectional communication with blocking sockets between a client and a server using TCP.
I can get one-way communication to work from client->server or server->client, but the socket remains blocked or "hangs" when trying to receive messages on both the server and client. I am using a simple algorithm(recvall), which uses recv, to consolidate the packets into the full message.
I understand the sockets remain blocked by design until all the data is sent or read(right?), but isn't that what sendall and recvall take care of? How come disabling recv on either the client or server "unblocks" it and causes it to work? And ultimately what am I doing wrong that is causing the socket to stay blocked?
Here is my code, the only fundamental difference really being the messages that are sent:
recvall(socket)(shared between client and server):
def recvall(socket):
data = ''
while True:
packet = socket.recv(16)
if not packet: break
data += packet
return data
server.py (run first):
import socket
host = 'localhost'
port = 8080
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((host, port))
s.listen(5)
while True:
(client, address) = s.accept()
print 'client connected'
try:
print recvall(client)
client.sendall('hello client')
finally:
client.close()
client.py:
import socket
s = socket.create_connection((args.ip, args.port))
try:
s.sendall('hello server')
print recvall(s)
finally:
s.close()
From my understanding (epiphany here), the main problem is that recv inside recvall is only concerned with retrieving the stream (in the same way send is only concerned with sending the stream), it has no concept of a "message" and therefore cannot know when to finish reading. It read all the bytes and did not return any additional bytes, but that is NOT a signal that the message is finished sending, there could be more bytes waiting to be sent and it would not be safe to assume otherwise.
This requires us to have an explicit indicator for when to stop reading. recv and send are only concerned with managing the stream and therefore have no concept of a message (our "unit"). This article has some great solutions to this problem. Since I am sending fixed-length messages, I opted to check that the length is as expected before finishing recv. Here is the updated version of recvall, note MSG_LENGTH must be defined and enforced in order for recvall to not block the socket.
def recvall(socket):
data = ''
while len(data) < MSG_LENGTH:
packet = socket.recv(BUFFER_SIZE)
if not packet: break
data += packet
return data
Bidirectional communication now works, the only catch being the client and server must know the length of the message they will receive, again this is not an issue in my case. This is all new to me so someone please correct me on terminology and concepts.
I am doing a client-server project for my college project,
we have to allocate the login to the client.
Client system will request its status for every 2 seconds(to check whether the client is locked or unlocked). and server will accept the client request and reply the client status to the system.
But the problem is server thread is not responding to the client request.
CLIENT THREAD:
def checkPort():
while True:
try:
s = socket.socket()
s.connect((host, port))
s.send('pc1') # send PC name to the server
status = s.recv(1024) # receive the status from the server
if status == "unlock":
disableIntrrupts() # enable all the functions of system
else:
enableInterrupts() # enable all the functions of system
time.sleep(5)
s.close()
except Exception:
pass
SERVER THREAD:
def check_port():
while True:
try:
print "hello loop is repeating"
conn, addr = s.accept()
data = conn.recv(1024)
if exit_on_click == 1:
break
if (any(sublist[0] == data for sublist in available_sys)):
print "locked"
conn.send("lock")
elif (any(sublist[0] == data for sublist in occupied_sys)):
conn.send("unlock")
print "unlocked"
else:
print "added to gui for first time"
available_sys.append([data,addr[0],nameText,usnText,branchText])
availSysList.insert('end',data)
except Exception:
pass
But my problem is server thread is not executing more than 2 time,
So its unable to accept client request more than one time.
can't we handle multiple client sockets using single server socket?
How to handle multiple client request from server ?
Thanks for any help !!
Its because your server, will block waiting for a new connection on this line
conn, addr = s.accept()
This is because calls like .accept and .read are blocking calls that hold the process
You need to consider an alternative design, where in you either.
Have one process per connection (this idea is stupid)
One thread per connection (this idea is less stupid than the first but still mostly foolish)
Have a non blocking design that allows multiple clients and read/write without blocking execution.
To achieve the first, look at multiprocessing, the second is threading the third is slightly more complicated to get your head around but will yield the best results, the go to library for event driven code in Python is twisted but there are others like
gevent
tulip
tornado
And so so many more that I haven't listed here.
here's an full example of implementing a threaded server. it's fully functional and comes with the benefit of using SSL as well. further, i use threaded event objects to signal another class object after storing my received data in a database.
please note, _sni and _cams_db are additional modules purely of my own. if you want to see the _sni module (provides SNI support for pyOpenSSL), let me know.
what follows this, is a snippet from camsbot.py, there's a whole lot more that far exceeds the scope of this question. what i've built is a centralized message relay system. it listens to tcp/2345 and accepts SSL connections. each connection passes messages into the system. short lived connections will connect, pass message, and disconnect. long lived connections will pass numerous messages after connecting. messages are stored in a database and a threading.Event() object (attached to the DB class) is set to tell the bot to poll the database for new messages and relay them.
the below example shows
how to set up a threaded tcp server
how to pass information from the listener to the accept handler such as config data and etc
in addition, this example also shows
how to employ an SSL socket
how to do some basic certificate validations
how to cleanly wrap and unwrap SSL from a tcp socket
how to use poll() on the socket instead of select()
db.pending is a threading.Event() object in _cams_db.py
in the main process we start another thread that waits on the pending object with db.pending.wait(). this makes that thread wait until another thread does db.pending.set(). once it is set, our waiting thread immediately wakes up and continues to work. when our waiting thread is done, it calls db.pending.clear() and goes back to the beginning of the loop and starts waiting again with db.pending.wait()
while True:
db.pending.wait()
# after waking up, do code. for example, we wait for incoming messages to
# be stored in the database. the threaded server will call db.pending.set()
# which will wake us up. we'll poll the DB for new messages, relay them, clear
# our event flag and go back to waiting.
# ...
db.pending.clear()
snippet from camsbot.py:
import sys, os, sys, time, datetime, threading, select, logging, logging.handlers
import configparser, traceback, re, socket, hashlib
# local .py
sys.path.append('/var/vse/python')
import _util, _webby, _sni, _cams_db, _cams_threaded_server, _cams_bot
# ...
def start_courier(config):
# default values
host = '::'
port = 2345
configp = config['configp']
host = configp.get('main', 'relay msp hostport')
# require ipv6 addresses be specified in [xx:xx:xx] notation, therefore
# it is safe to look for :nnnn at the end
if ':' in host and not host.endswith(']'):
port = host.split(':')[-1]
try:
port = int(port, 10)
except:
port = 2345
host = host.split(':')[:-1][0]
server = _cams_threaded_server.ThreadedTCPServer((host, port), _cams_threaded_server.ThreadedTCPRequestHandler, config)
t = threading.Thread(target=server.serve_forever, name='courier')
t.start()
_cams_threaded_server.py:
import socket, socketserver, select, datetime, time, threading
import sys, struct
from OpenSSL.SSL import SSLv23_METHOD, SSLv3_METHOD, TLSv1_METHOD, OP_NO_SSLv2
from OpenSSL.SSL import VERIFY_NONE, VERIFY_PEER, VERIFY_FAIL_IF_NO_PEER_CERT, Context, Connection
from OpenSSL.SSL import FILETYPE_PEM
from OpenSSL.SSL import WantWriteError, WantReadError, WantX509LookupError, ZeroReturnError, SysCallError
from OpenSSL.crypto import load_certificate
from OpenSSL import SSL
# see note at beginning of answer
import _sni, _cams_db
class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
def __init__(self, server_address, HandlerClass, config):
socketserver.BaseServer.__init__(self, server_address, HandlerClass)
self.address_family = socket.AF_INET6
self.connected = []
self.logger = config['logger']
self.config = config
self.socket = socket.socket(self.address_family, self.socket_type)
self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sc = Context(TLSv1_METHOD)
sc.set_verify(VERIFY_PEER|VERIFY_FAIL_IF_NO_PEER_CERT, _sni.verify_cb)
sc.set_tlsext_servername_callback(_sni.pick_certificate)
self.sc = sc
self.server_bind()
self.server_activate()
class ThreadedTCPRequestHandler(socketserver.BaseRequestHandler):
def handle(self):
config = self.server.config
logger = self.server.logger
connected = self.server.connected
sc = self.server.sc
try:
self.peer_hostname = socket.gethostbyaddr(socket.gethostbyname(self.request.getpeername()[0]))[0]
except:
self.peer_hostname = '!'+self.request.getpeername()[0]
logger.info('peer: {}'.format(self.peer_hostname))
ssl_s = Connection(sc, self.request)
ssl_s.set_accept_state()
try:
ssl_s.do_handshake()
except:
t,v,tb = sys.exc_info()
logger.warn('handshake failed {}'.format(v))
ssl_s.setblocking(True)
self.ssl_s = ssl_s
try:
peercert = ssl_s.get_peer_certificate()
except:
peercert = False
t,v,tb = sys.exc_info()
logger.warn('SSL get peer cert failed: {}'.format(v))
if not peercert:
logger.warn('No peer certificate')
else:
acl = config['configp']['main'].get('client cn acl', '').split(' ')
cert_subject = peercert.get_subject().CN
logger.info('Looking for {} in acl: {}'.format(cert_subject,acl))
if cert_subject in acl:
logger.info('{} is permitted'.format(cert_subject))
else:
logger.warn('''client CN not approved''')
# it's ok to block here, every socket has its own thread
ssl_s.setblocking(True)
self.db = config['db']
msgcount = 0
p = select.poll()
# don't want writable, just readable
p.register(self.request, select.POLLIN|select.POLLPRI|select.POLLERR|select.POLLHUP|select.POLLNVAL)
peername = ssl_s.getpeername()
x = peername[0]
if x.startswith('::ffff:'):
x = x[7:]
peer_ip = x
try:
host = socket.gethostbyaddr(x)[0]
except:
host = peer_ip
logger.info('{}/{}:{} connected'.format(host, peer_ip, peername[1]))
connected.append( [host, peername[1]] )
if peercert:
threading.current_thread().setName('{}/port={}/CN={}'.format(host, peername[1], peercert.get_subject().CN))
else:
threading.current_thread().setName('{}/port={}'.format(host, peername[1]))
sockclosed = False
while not sockclosed:
keepreading = True
#logger.debug('starting 30 second timeout for poll')
pe = p.poll(30.0)
if not pe:
# empty list means poll timeout
# for SSL sockets it means WTF. we get an EAGAIN like return even if the socket is blocking
continue
logger.debug('poll indicates: {}'.format(pe))
#define SSL_NOTHING 1
#define SSL_WRITING 2
#define SSL_READING 3
#define SSL_X509_LOOKUP 4
while keepreading and not sockclosed:
data,sockclosed,keepreading = self._read_ssl_data(2, head=True)
if sockclosed or not keepreading:
time.sleep(5)
continue
plen = struct.unpack('H', data)[0]
data,sockclosed,keepreading = self._read_ssl_data(plen)
if sockclosed or not keepreading:
time.sleep(5)
continue
# send thank you, ignore any errors since we appear to have gotten
# the message
try:
self.ssl_s.sendall(b'ty')
except:
pass
# extract the timestamp
message_ts = data[0:8]
msgtype = chr(data[8])
message = data[9:].decode()
message_ts = struct.unpack('d', message_ts)[0]
message_ts = datetime.datetime.utcfromtimestamp(message_ts).replace(tzinfo=datetime.timezone.utc)
self.db.enqueue(config['group'], peer_ip, msgtype, message, message_ts)
self.db.pending.set()
# we're recommended to use the return socket object for any future operations rather than the original
try:
s = ssl_s.unwrap()
s.close()
except:
pass
connected.remove( [host, peername[1]] )
t_name = threading.current_thread().getName()
logger.debug('disconnect: {}'.format(t_name))
def _read_ssl_data(self, wantsize=16384, head=False):
_w = ['WANT_NOTHING','WANT_READ','WANT_WRITE','WANT_X509_LOOKUP']
logger = self.server.logger
data = b''
sockclosed = False
keepreading = True
while len(data) < wantsize and keepreading and not sockclosed:
rlen = wantsize - len(data)
try:
w,wr = self.ssl_s.want(),self.ssl_s.want_read()
#logger.debug(' want({}) want_read({})'.format(_w[w],wr))
x = self.ssl_s.recv(rlen)
#logger.debug(' recv(): {}'.format(x))
if not ( x or len(x) ):
raise ZeroReturnError
data += x
if not (len(x) == len(data) == wantsize):
logger.info(' read={}, len(data)={}, plen={}'.format(len(x),len(data),wantsize))
except WantReadError:
# poll(), when ready, read more
keepreading = False
logger.info(' got WantReadError')
continue
except WantWriteError:
# poll(), when ready, write more
keepreading = False
logger.info(' got WantWriteError')
continue
except ZeroReturnError:
# socket got closed, a '0' bytes read also means the same thing
keepreading = False
sockclosed = True
logger.info(' ZRE, socket closed normally')
continue
except SysCallError:
keepreading = False
sockclosed = True
t,v,tb = sys.exc_info()
if v.args[0] == -1: # normal EOF
logger.info(' EOF found, keepreading=False')
else:
logger.info('{} terminated session abruptly while reading plen'.format(self.peer_hostname))
logger.info('t: {}'.format(t))
logger.info('v: {}'.format(v))
continue
except:
t,v,tb = sys.exc_info()
logger.warning(' fucked? {}'.format(v))
raise
if not head and not len(data) == wantsize:
logger.warn(' short read {} of {}'.format(len(data), wantsize))
return data,sockclosed,keepreading
let's start with a bare bones threaded tcp server.
class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
def __init__(self, server_address, HandlerClass):
socketserver.BaseServer.__init__(self, server_address, HandlerClass)
self.address_family = socket.AF_INET
self.socket = socket.socket(self.address_family, self.socket_type)
self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.server_bind()
self.server_activate()
class ThreadedTCPRequestHandler(socketserver.BaseRequestHandler):
def handle(self):
# self.request is your accepted socket, do all your .read() and .wirte() on it
s = self.request
request = s.read(1024)
# decide locked or unlocked. this example arbitrarily writes back 'locked'
s.write('locked')
# we're done, close the socket and exit with a default return of None
s.close()
ok, start your threaded server with this in your main() function:
server = threading.ThreadedTCPServer(('127.0.0.1', 1234), ThreadedTCPRequestHandler)
t = threading.Thread(target=server.serve_forever, name='optional_name')
t.start()
now you can let the threading module handle the semantics of concurrency and not worry about it.
You might want to take a look at 0MQ and concurrent.futures. 0MQ has a Tornado event loop in the library and it reduces the complexity of socket programming. concurrent.futures is a high level interface over threading or multiprocessing.
You can see different concurrent server approaches at
https://bitbucket.org/arco_group/upper/src
These will help you to choose the better way for you.
Cheers
I have a main thread that waits for connection. It spawns client threads that will echo the response from the client (telnet in this case). But say that I want to close down all sockets and all threads after some time, like after 1 connection.
How would I do it? If I do clientSocket.close() from the main thread, it won't stop doing the recv. It will only stop if I first send something through telnet, then it will fail doing further sends and recvs.
My code looks like this:
# Echo server program
import socket
from threading import Thread
import time
class ClientThread(Thread):
def __init__(self, clientSocket):
Thread.__init__(self)
self.clientSocket = clientSocket
def run(self):
while 1:
try:
# It will hang here, even if I do close on the socket
data = self.clientSocket.recv(1024)
print "Got data: ", data
self.clientSocket.send(data)
except:
break
self.clientSocket.close()
HOST = ''
PORT = 6000
serverSocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serverSocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
serverSocket.bind((HOST, PORT))
serverSocket.listen(1)
clientSocket, addr = serverSocket.accept()
print 'Got a new connection from: ', addr
clientThread = ClientThread(clientSocket)
clientThread.start()
time.sleep(1)
# This won't make the recv in the clientThread to stop immediately,
# nor will it generate an exception
clientSocket.close()
I know this is an old thread and that Samuel probably fixed his issue a long time ago. However, I had the same problem and came across this post while google'ing. Found a solution and think it is worthwhile to add.
You can use the shutdown method on the socket class. It can prevent further sends, receives or both.
socket.shutdown(socket.SHUT_WR)
The above prevents future sends, as an example.
See Python docs for more info.
I don't know if it's possible to do what you're asking, but it shouldn't be necessary. Just don't read from the socket if there is nothing to read; use select.select to check the socket for data.
change:
data = self.clientSocket.recv(1024)
print "Got data: ", data
self.clientSocket.send(data)
to something more like this:
r, _, _ = select.select([self.clientSocket], [], [])
if r:
data = self.clientSocket.recv(1024)
print "Got data: ", data
self.clientSocket.send(data)
EDIT: If you want to guard against the possibility that the socket has been closed, catch socket.error.
do_read = False
try:
r, _, _ = select.select([self.clientSocket], [], [])
do_read = bool(r)
except socket.error:
pass
if do_read:
data = self.clientSocket.recv(1024)
print "Got data: ", data
self.clientSocket.send(data)
I found a solution using timeouts. That will interrupt the recv (actually before the timeout has expired which is nice):
# Echo server program
import socket
from threading import Thread
import time
class ClientThread(Thread):
def __init__(self, clientSocke):
Thread.__init__(self)
self.clientSocket = clientSocket
def run(self):
while 1:
try:
data = self.clientSocket.recv(1024)
print "Got data: ", data
self.clientSocket.send(data)
except socket.timeout:
# If it was a timeout, we want to continue with recv
continue
except:
break
self.clientSocket.close()
HOST = ''
PORT = 6000
serverSocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serverSocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
serverSocket.bind((HOST, PORT))
serverSocket.listen(1)
clientSocket, addr = serverSocket.accept()
clientSocket.settimeout(1)
print 'Got a new connection from: ', addr
clientThread = ClientThread(clientSocket)
clientThread.start()
# Close it down immediatly
clientSocket.close()
I must apologize for the comments below. The earlier comment by #Matt Anderson works. I had made a mistake when trying it out which led to my post below.
Using timeout is not a very good solution. It may seem that waking up for an instant and then going back to sleep is no big deal, but I have seen it greatly affect the performance of an application. You have an operation that for the most part wants to block until data is available and thus sleep forever. However, if you want to abort for some reason, like shutting down your application, then the trick is how to get out. For sockets, you can use select and listen on two sockets. Your primary one, and a special shutdown one. Creating the shutdown one though is a bit of a pain. You have to create it. You have to get the listening socket to accept it. You have to keep track of both ends of this pipe. I have the same issue with the Synchronized Queue class. There however, you can at least insert a dummy object into the queue to wake up the get(). This requires that the dummy object not look like your normal data though. I sometimes wish Python had something like the Windows API WaitForMultipleObjects.