We Keep Coding

Shutdown for socketserver based Python 3 server hangs

I am working on a "simple" server using a threaded SocketServer in Python 3.
I am going through a lot of trouble implementing shutdown for this. The code below I found on the internet and shutdown works initially but stops working after sending a few commands from the client via telnet. Some investigation tells me it hangs in threading._shutdown... threading._wait_for_tstate_lock but so far this does not ring a bell.
My research tells me that there are ~42 different solutions, frameworks, etc. on how to do this in different python versions. So far I could not find a working approach for python3. E.g. I love telnetsrv
(https://pypi.python.org/pypi/telnetsrv/0.4) for python 2.7 (it uses greenlets from gevent) but this one does not work for python 3. So if there is a more pythonic, std lib approach or something that works reliably I would love to hear about it!
My bet currently is with socketserver but I could not figure out yet how to deal with the hanging server. I removed all the log statements and most functionality so I can post this minimal server which exposes the issue:
# -*- coding: utf-8 -*-
import socketserver
import threading
SERVER = None
def shutdown_cmd(request):
global SERVER
request.send(bytes('server shutdown requested\n', 'utf-8'))
request.close()
SERVER.shutdown()
print('after shutdown!!')
#SERVER.server_close()
class service(socketserver.BaseRequestHandler):
def handle(self):
while True:
try:
msg = str(self.request.recv(1024).strip(), 'utf-8')
if msg == 'shutdown':
shutdown_cmd(msg, self.request)
else:
self.request.send(bytes("You said '{}'\n".format(msg), "utf-8"))
except Exception as e:
pass
class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
pass
def run():
global SERVER
SERVER = ThreadedTCPServer(('', 1520), service)
server_thread = threading.Thread(target=SERVER.serve_forever)
server_thread.daemon = True
server_thread.start()
input("Press enter to shutdown")
SERVER.shutdown()
if __name__ == '__main__':
run()
It would be great being able to stop the server from the handler, too (see shutdown_cmd)

shutdown() works as expected, the server has stopped accepting new connections, but python still waiting for alive threads to terminate.
By default, socketserver.ThreadingMixIn will create new threads to handle incoming connection and by default, those are non-daemon threads, so python will wait for all alive non-daemon threads to terminate.
Of course, you could make the server spawn daemon threads, then python will not waiting:
The ThreadingMixIn class defines an attribute daemon_threads, which indicates whether or not the server should wait for thread termination. You should set the flag explicitly if you would like threads to behave autonomously; the default is False, meaning that Python will not exit until all threads created by ThreadingMixIn have exited.
class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
daemon_threads = True
But that is not the ideal solution, you should check why threads never terminate, usually, the server should stop processing connection when no new data available or client shutdown connection:
import socketserver
import threading
shutdown_evt = threading.Event()
class service(socketserver.BaseRequestHandler):
def handle(self):
self.request.setblocking(False)
while True:
try:
msg = self.request.recv(1024)
if msg == b'shutdown':
shutdown_evt.set()
break
elif msg:
self.request.send(b'you said: ' + msg)
if shutdown_evt.wait(0.1):
break
except Exception as e:
break
class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
pass
def run():
SERVER = ThreadedTCPServer(('127.0.0.1', 10000), service)
server_thread = threading.Thread(target=SERVER.serve_forever)
server_thread.daemon = True
server_thread.start()
input("Press enter to shutdown")
shutdown_evt.set()
SERVER.shutdown()
if __name__ == '__main__':
run()

I tried two solutions to implement a tcp server which runs on Python 3 on both Linux and Windows (I tried Windows 7):
using socketserver (my question) - shutdown is not working
using asyncio (posted an answer for that) - does not work on Windows
Both solutions have been based upon search results on the web. In the end I had to give up on the idea of finding a proven solution because I could not find one. Consequently I implemented my own solution (based on gevent). I post it here because I hope it will be helpful for others to avoid stuggeling the way I did.
# -*- coding: utf-8 -*-
from gevent.server import StreamServer
from gevent.pool import Pool
class EchoServer(StreamServer):
def __init__(self, listener, handle=None, spawn='default'):
StreamServer.__init__(self, listener, handle=handle, spawn=spawn)
def handle(self, socket, address):
print('New connection from %s:%s' % address[:2])
socket.sendall(b'Welcome to the echo server! Type quit to exit.\r\n')
# using a makefile because we want to use readline()
rfileobj = socket.makefile(mode='rb')
while True:
line = rfileobj.readline()
if not line:
print("client disconnected")
break
if line.strip().lower() == b'quit':
print("client quit")
break
if line.strip().lower() == b'shutdown':
print("client initiated server shutdown")
self.stop()
break
socket.sendall(line)
print("echoed %r" % line.decode().strip())
rfileobj.close()
srv = EchoServer(('', 1520), spawn=Pool(20))
srv.serve_forever()

after more research I found a sample that works using asyncio:
# -*- coding: utf-8 -*-
import asyncio
# after further research I found this relevant europython talk:
# https://www.youtube.com/watch?v=pi49aiLBas8
# * protocols and transport are useful if you do not have tons of socket based code
# * event loop pushes data in
# * transport used to push data back to the client
# found decent sample in book by wrox "professional python"
class ServerProtocol(asyncio.Protocol):
def connection_made(self, transport):
self.transport = transport
self.write('Welcome')
def connection_lost(self, exc):
self.transport = None
def data_received(self, data):
if not data or data == '':
return
message = data.decode('ascii')
command = message.strip().split(' ')[0].lower()
args = message.strip().split(' ')[1:]
#sanity check
if not hasattr(self, 'command_%s' % command):
self.write('Invalid command: %s' % command)
return
# run command
try:
return getattr(self, 'command_%s' % command)(*args)
except Exception as ex:
self.write('Error: %s' % str(ex))
def write(self, msg):
self.transport.write((msg + '\n').encode('ascii', 'ignore'))
def command_shutdown(self):
self.write('Okay. shutting down')
raise KeyboardInterrupt
def command_bye(self):
self.write('bye then!')
self.transport.close()
self.transport = None
if __name__ == '__main__':
loop = asyncio.get_event_loop()
coro = loop.create_server(ServerProtocol, '127.0.0.1', 8023)
asyncio.async(coro)
try:
loop.run_forever()
except KeyboardInterrupt:
pass
I understand that this is the most useful way to do this kind of network programming. If necessary the performance could be improved using the same code with uvloop (https://magic.io/blog/uvloop-blazing-fast-python-networking/).

Another way to shut down the server is by creating a process/thread for the serve_forever call.
After server_forever is started, simply wait for a custom flag to trigger and use server_close on the server, and terminate the process.
streaming_server = StreamingServer(('', 8000), StreamingHandler)
FLAG_KEEP_ALIVE.value = True
process_serve_forever = Process(target=streaming_server.serve_forever)
process_serve_forever.start()
while FLAG_KEEP_ALIVE.value:
pass
streaming_server.server_close()
process_serve_forever.terminate()

global name 'WebServer' is not defined

Please find below code as it is throwing the error - global name 'webserver' is not defined.
import SimpleHTTPServer
import SocketServer
import os
from threading import Thread
import threading
class WebServer(Thread):
def __init__(self, stream_path):
"""
Class to create a Web server and add given folder to the web server which is passed as an argument.
In this case it creates the web server to the incoming streams pushed by VLC to temp folder
"""
Thread.__init__(self)
self.stream_path = stream_path
def run(self):
global WebServer
"""
This method is built in Thread object method invoked by start()
and the code which is under run() will be executed.
"""
os.chdir(self.stream_path)
PORT = 8000
Handler = SimpleHTTPServer.SimpleHTTPRequestHandler
httpd = SocketServer.TCPServer(("", PORT), Handler)
print "serving at port", PORT
print "function thread", threading.currentThread()
httpd.serve_forever()
print "test1"
def create_web_server(self,stream_path):
global WebServer
"""
This method is to create the web server to a given path
"""
obj1 = WebServer(self,stream_path)
obj1.start()
print "server created"
def stop_web_server(self):
cmd='tskill python /A'
os.system(cmd)
if __name__ == '__main__':
create_web_server("","C:\\\\QED")

You don't need the two global WebServer lines in your code. The global keyword is used to grant write access to a global variable, and you don't have a global variable named WebServer.
Your code should look like the following to resolve your error, though in its state, it will still throw errors because your code has multiple problems, including calling the create_web_server method by itself, as it's a method that needs to be run on a WebServer object.
I've included a working prototype example of your code at the bottom of my post.
import SimpleHTTPServer
import SocketServer
import os
from threading import Thread
import threading
class WebServer(Thread):
def __init__(self, stream_path):
"""
Class to create a Web server and add given folder to the web server which is passed as an argument.
In this case it creates the web server to the incoming streams pushed by VLC to temp folder
"""
Thread.__init__(self)
self.stream_path = stream_path
def run(self):
"""
This method is built in Thread object method invoked by start()
and the code which is under run() will be executed.
"""
os.chdir(self.stream_path)
PORT = 8000
Handler = SimpleHTTPServer.SimpleHTTPRequestHandler
httpd = SocketServer.TCPServer(("", PORT), Handler)
print "serving at port", PORT
print "function thread", threading.currentThread()
httpd.serve_forever()
print "test1"
def create_web_server(self,stream_path):
"""
This method is to create the web server to a given path
"""
obj1 = WebServer(self,stream_path)
obj1.start()
print "server created"
def stop_web_server(self):
cmd='tskill python /A'
os.system(cmd)
if __name__ == '__main__':
create_web_server("","C:\\\\QED")
Here is a simplified version of your code that should do the same thing (serve a directory with a basic SimpleHTTPRequestHandler).
It runs the thread in daemon mode so that it can be interrupted with Ctrl-C. I removed several methods as they didn't seem to serve a purpose in your original code, and to show you a basic framework of what your WebServer class would probably look like.
import threading
import os
import time
import SimpleHTTPServer
import SocketServer
class WebServer(threading.Thread):
# Let's specify additional constructor arguments for host/port
# so you don't have to hardcode those values.
# We can specify default values for them as well.
def __init__(self, stream_path, host='localhost', port=8000):
super(WebServer, self).__init__()
self.stream_path = stream_path
self.host = host
self.port = port
# This is the method that will be called when you call
# .start() on a WebServer object.
def run(self):
os.chdir(self.stream_path)
handler = SimpleHTTPServer.SimpleHTTPRequestHandler
httpd = SocketServer.TCPServer((self.host, self.port), handler)
httpd.serve_forever()
if __name__ == '__main__':
# Create a WebServer thread object, set it to daemonize
# and start it
ws = WebServer(r'C:\QED')
ws.daemon = True
ws.start()
# Since the main thread doesn't do anything after daemonizing
# the WebServer, it would exit immediately.
# This forever loop just ensures that the main thread runs
while True: time.sleep(1)
And to take it a step further, you don't need to create a Thread subclass. It's allowed because it feels more 'natural' to developers who are familiar with Java, where subclassing Thread or implementing Runnable is standard practice.
You can just do something like:
import threading
import os
import time
import SimpleHTTPServer
import SocketServer
def run_http_server(serve_dir, host='localhost', port=8000):
os.chdir(serve_dir)
handler = SimpleHTTPServer.SimpleHTTPRequestHandler
httpd = SocketServer.TCPServer((host, port), handler)
httpd.serve_forever()
if __name__ == '__main__':
thread = threading.Thread(target=run_http_server, args=(r'C:\QED',))
thread.daemon = True
thread.start()
while True: time.sleep(1)

Bottle web framework - How to stop?

When starting a bottle webserver without a thread or a subprocess, there's no problem. To exit the bottle app -> CTRL + c.
In a thread, how can I programmatically stop the bottle web server ?
I didn't find a stop() method or something like that in the documentation. Is there a reason ?

For the default (WSGIRef) server, this is what I do (actually it is a cleaner approach of Vikram Pudi's suggestion):
from bottle import Bottle, ServerAdapter
class MyWSGIRefServer(ServerAdapter):
server = None
def run(self, handler):
from wsgiref.simple_server import make_server, WSGIRequestHandler
if self.quiet:
class QuietHandler(WSGIRequestHandler):
def log_request(*args, **kw): pass
self.options['handler_class'] = QuietHandler
self.server = make_server(self.host, self.port, handler, **self.options)
self.server.serve_forever()
def stop(self):
# self.server.server_close() <--- alternative but causes bad fd exception
self.server.shutdown()
app = Bottle()
#app.route('/')
def index():
return 'Hello world'
#app.route('/stop') # not working from here, it has to come from another thread
def stopit():
server.stop()
server = MyWSGIRefServer(port=80)
try:
app.run(server=server)
except:
print('Bye')
When I want to stop the bottle application, from another thread, I do the following:
server.stop()

I had trouble closing a bottle server from within a request as bottle seems to run requests in subprocesses.
I eventually found the solution was to do:
sys.stderr.close()
inside the request (that got passed up to the bottle server and axed it).

An updated version of mike's answer.
from bottlepy.bottle import WSGIRefServer, run
from threading import Thread
import time
class MyServer(WSGIRefServer):
def run(self, app): # pragma: no cover
from wsgiref.simple_server import WSGIRequestHandler, WSGIServer
from wsgiref.simple_server import make_server
import socket
class FixedHandler(WSGIRequestHandler):
def address_string(self): # Prevent reverse DNS lookups please.
return self.client_address[0]
def log_request(*args, **kw):
if not self.quiet:
return WSGIRequestHandler.log_request(*args, **kw)
handler_cls = self.options.get('handler_class', FixedHandler)
server_cls = self.options.get('server_class', WSGIServer)
if ':' in self.host: # Fix wsgiref for IPv6 addresses.
if getattr(server_cls, 'address_family') == socket.AF_INET:
class server_cls(server_cls):
address_family = socket.AF_INET6
srv = make_server(self.host, self.port, app, server_cls, handler_cls)
self.srv = srv ### THIS IS THE ONLY CHANGE TO THE ORIGINAL CLASS METHOD!
srv.serve_forever()
def shutdown(self): ### ADD SHUTDOWN METHOD.
self.srv.shutdown()
# self.server.server_close()
def begin():
run(server=server)
server = MyServer(host="localhost", port=8088)
Thread(target=begin).start()
time.sleep(2) # Shut down server after 2 seconds
server.shutdown()
The class WSGIRefServer is entirely copied with only 1 line added to the run() method is added. Also add a simple shutdown() method. Unfortunately, this is necessary because of the way bottle creates the run() method.

You can make your thread a daemon by setting the daemon property to True before calling start.
mythread = threading.Thread()
mythread.daemon = True
mythread.start()
A deamon thread will stop whenever the main thread that it is running in is killed or dies. The only problem is that you won't be able to make the thread run any code on exit and if the thread is in the process of doing something, it will be stopped immediately without being able to finish the method it is running.
There's no way in Python to actually explicitly stop a thread. If you want to have more control over being able to stop your server you should look into Python Processes from the multiprocesses module.

Since bottle doesn't provide a mechanism, it requires a hack. This is perhaps the cleanest one if you are using the default WSGI server:
In bottle's code the WSGI server is started with:
srv.serve_forever()
If you have started bottle in its own thread, you can stop it using:
srv.shutdown()
To access the srv variable in your code, you need to edit the bottle source code and make it global. After changing the bottle code, it would look like:
srv = None #make srv global
class WSGIRefServer(ServerAdapter):
def run(self, handler): # pragma: no cover
global srv #make srv global
...

Here's one option: provide custom server (same as default), that records itself:
import bottle
class WSGI(bottle.WSGIRefServer):
instances = []
def run(self, *args, **kw):
self.instances.append(self)
super(WSGI, self).run(*args, **kw)
# some other thread:
bottle.run(host=ip_address, port=12345, server=WSGI)
# control thread:
logging.warn("servers are %s", WSGI.instances)

This is exactly the same method than sepero's and mike's answer, but now much simpler with Bottle version 0.13+:
from bottle import W, run, route
from threading import Thread
import time
#route('/')
def index():
return 'Hello world'
def shutdown():
time.sleep(5)
server.srv.shutdown()
server = WSGIRefServer(port=80)
Thread(target=shutdown).start()
run(server=server)
Also related: https://github.com/bottlepy/bottle/issues/1229 and https://github.com/bottlepy/bottle/issues/1230.
Another example with a route http://localhost/stop to do the shutdown:
from bottle import WSGIRefServer, run, route
from threading import Thread
#route('/')
def index():
return 'Hello world'
#route('/stop')
def stopit():
Thread(target=shutdown).start()
def shutdown():
server.srv.shutdown()
server = WSGIRefServer(port=80)
run(server=server)
PS: it requires at least Bottle 0.13dev.

Console log of Bottle server tells us that the official way of shutting down the server is "Hit Ctrl-C":
Bottle v0.12.19 server starting up (using WSGIRefServer())...
Listening on http://localhost:8080/
Hit Ctrl-C to quit.
Why not simply follow it programmatically?
Hitting "Ctrl-C" is nothing but sending SIGINT to the process, and we can achieve it with just built-in modules:
Get current PID with os.getpid().
Kill the process with os.kill(). Remember passing SIGINT so it will be exactly same as "Hit Ctrl-C".
Wrap the 'kill' in another thread and start it after a few seconds so the client won't get error.
Here is the server code:
from bottle import route, run
import os
import signal
from threading import Thread
import time
#route('/hello')
def return_hello():
return 'Hello'
#route('/stop')
def handle_stop_request():
# Handle "stop server" request from client: start a new thread to stop the server
Thread(target=shutdown_server).start()
return ''
def shutdown_server():
time.sleep(2)
pid = os.getpid() # Get process ID of the current Python script
os.kill(pid, signal.SIGINT)
# Kill the current script process with SIGINT, which does same as "Ctrl-C"
run(host='localhost', port=8080)
Here is the client code:
import requests
def request_service(service_key):
url = f'http://127.0.0.1:8080/{service_key}'
response = requests.get(url)
content = response.content.decode('utf-8')
print(content)
request_service('hello')
request_service('stop')
Note that in function "handle_stop_request" we didn't stop the server immediately but rather started a thread then returned empty string. With this mechanism, when a client requests "http://127.0.0.1:8080/stop", it can get response (the empty string) normally. After that, the server will shutdown. If we otherwise shutdown the server in function "handle_stop_request", the server will close the connection before returning to the client, and hence the client will get "ConnectionError".
Server side output:
Bottle v0.12.19 server starting up (using WSGIRefServer())...
Listening on http://localhost:8080/
Hit Ctrl-C to quit.
127.0.0.1 - - [23/Nov/2021 11:18:08] "GET /hello HTTP/1.1" 200 5
127.0.0.1 - - [23/Nov/2021 11:18:08] "GET /stop HTTP/1.1" 200 0
Client side output:
Hello
The code was tested under Python 3.7 and Bottle 0.12.

I suppose that the bottle webserver runs forever until it terminates. There are no methonds like stop().
But you can make something like this:
from bottle import route, run
import threading, time, os, signal, sys, operator
class MyThread(threading.Thread):
def __init__(self, target, *args):
threading.Thread.__init__(self, target=target, args=args)
self.start()
class Watcher:
def __init__(self):
self.child = os.fork()
if self.child == 0:
return
else:
self.watch()
def watch(self):
try:
os.wait()
except KeyboardInterrupt:
print 'KeyBoardInterrupt'
self.kill()
sys.exit()
def kill(self):
try:
os.kill(self.child, signal.SIGKILL)
except OSError: pass
def background_process():
while 1:
print('background thread running')
time.sleep(1)
#route('/hello/:name')
def index(name='World'):
return '<b>Hello %s!</b>' % name
def main():
Watcher()
MyThread(background_process)
run(host='localhost', port=8080)
if __name__ == "__main__":
main()
Then you can use Watcher.kill() when you need to kill your server.
Here is the code of run() function of the bottle:
try:
app = app or default_app()
if isinstance(app, basestring):
app = load_app(app)
if not callable(app):
raise ValueError("Application is not callable: %r" % app)
for plugin in plugins or []:
app.install(plugin)
if server in server_names:
server = server_names.get(server)
if isinstance(server, basestring):
server = load(server)
if isinstance(server, type):
server = server(host=host, port=port, **kargs)
if not isinstance(server, ServerAdapter):
raise ValueError("Unknown or unsupported server: %r" % server)
server.quiet = server.quiet or quiet
if not server.quiet:
stderr("Bottle server starting up (using %s)...\n" % repr(server))
stderr("Listening on http://%s:%d/\n" % (server.host, server.port))
stderr("Hit Ctrl-C to quit.\n\n")
if reloader:
lockfile = os.environ.get('BOTTLE_LOCKFILE')
bgcheck = FileCheckerThread(lockfile, interval)
with bgcheck:
server.run(app)
if bgcheck.status == 'reload':
sys.exit(3)
else:
server.run(app)
except KeyboardInterrupt:
pass
except (SyntaxError, ImportError):
if not reloader: raise
if not getattr(server, 'quiet', False): print_exc()
sys.exit(3)
finally:
if not getattr(server, 'quiet', False): stderr('Shutdown...\n')
As you can see there are no other way to get off the run loop, except some exceptions.
The server.run function depends on the server you use, but there are no universal quit-method anyway.

This equally kludgy hack has the advantage that is doesn't have you copy-paste any code from bottle.py:
# The global server instance.
server = None
def setup_monkey_patch_for_server_shutdown():
"""Setup globals to steal access to the server reference.
This is required to initiate shutdown, unfortunately.
(Bottle could easily remedy that.)"""
# Save the original function.
from wsgiref.simple_server import make_server
# Create a decorator that will save the server upon start.
def stealing_make_server(*args, **kw):
global server
server = make_server(*args, **kw)
return server
# Patch up wsgiref itself with the decorated function.
import wsgiref.simple_server
wsgiref.simple_server.make_server = stealing_make_server
setup_monkey_patch_for_server_shutdown()
def shutdown():
"""Request for the server to shutdown."""
server.shutdown()

I've found this solution to be the easiest, but it does require that the "psutil" package is installed, to get the current process. It also requires the "signals" module, but that's part of the standard library.
#route('/shutdown')
def shutdown():
current_process = psutil.Process()
current_process.send_signal(signal.CTRL_C_EVENT)
return 'Shutting down the web server'
Hope that's of use to someone!

This question was top in my google search, so i will post my answer:
When the server is started with the Bottle() class, it has a method close() to stop the server. From the source code:
""" Close the application and all installed plugins. """
For example:
class Server:
def __init__(self, host, port):
self._host = host
self._port = port
self._app = Bottle()
def stop(self):
# close ws server
self._app.close()
def foo(self):
# More methods, routes...
Calling stop method will stop de server.

Python atexit for cleaning up server running in different process -- port still in use

I have some integration testing code that spawns a HTTP server in a different process for calling against. This server could potentially get polluted by activity so I'd like the ability to start and stop new instances of it on demand.
This unfortunately isn't working... I am running into a situation where the port my server was running on is still locked after my process exits(meaning if I run the test two times quickly, it fails the second time because the port is locked).
I've tried using atexit.register to bind the shutdown method and it's not working either.
Here's the code for the server:
from BaseHTTPServer import BaseHTTPRequestHandler
import SocketServer
import atexit
class RestHTTPRequestHandler(BaseHTTPRequestHandler):
def do_GET(self):
if self.path == '/sanitycheck':
self.send_response(200)
self.send_header('Content-type', 'application/json')
self.end_headers()
self.wfile.write("{ 'text': 'You are sane.' }")
else:
self.wfile.write(self.path)
def kill_server(httpd):
open("/tmp/log", "w").write("KILLING")
httpd.shutdown()
def start_simple_server(port):
httpd = SocketServer.TCPServer(("", port), RestHTTPRequestHandler)
atexit.register(kill_server, httpd)
httpd.serve_forever()
return httpd
Nothing ever gets written to /tmp/log which makes me think that the atexit isn't getting called.
Here's how I instantiate the server:
p = Process(target=start_simple_server, args=(port,))
p.start()
And then when I'm done to terminate it, I just call:
p.terminate()
Which does kill the process and should(to my understanding) trigger the atexit call -- but it's not :(
Any thoughts?

Python atexit doesn't run when you terminate a process.
>>>import atexit
>>> def hook():
... print "hook ran"
...
>>> atexit.register(hook)
<function hook at 0x100414aa0>
>>>
# in another terminal: kill <python process id>
>>> Terminated

I wound up taking a slightly different approach inspired by some code from David Beazley... server code:
from BaseHTTPServer import BaseHTTPRequestHandler
import SocketServer
import multiprocessing
import cgi
import urlparse
class StoppableHTTPServerProcess(multiprocessing.Process):
def __init__(self, address, handler):
multiprocessing.Process.__init__(self)
self.exit = multiprocessing.Event()
self.server = StoppableHTTPServer(address, handler)
def run(self):
while not self.exit.is_set():
self.server.handle_request()
def shutdown(self):
self.exit.set()
class RestHTTPRequestHandler(BaseHTTPRequestHandler):
def do_GET(self):
self.wfile.write(self.path)
class StoppableHTTPServer(SocketServer.TCPServer):
allow_reuse_address = True
timeout = 0.5
def start_simple_server(port):
process = StoppableHTTPServerProcess(("", port), RestHTTPRequestHandler)
return process
Calling code:
p = start_simple_server(port)
p.start()
And when I'm done...
p.shutdown()

Python socket accept blocks - prevents app from quitting

I've written a very simple python class which waits for connections on a socket. The intention is to stick this class into an existing app and asyncronously send data to connecting clients.
The problem is that when waiting on an socket.accept(), I cannot end my application by pressing ctrl-c. Neither can I detect when my class goes out of scope and notify it to end.
Ideally the application below should quit after the time.sleep(4) expires. As you can see below, I tried using select, but this also prevents the app from responding to ctrl-c. If I could detect that the variable 'a' has gone out of scope in the main method, I could set the quitting flag (and reduce the timeout on select to make it responsive).
Any ideas?
thanks
import sys
import socket
import threading
import time
import select
class Server( threading.Thread ):
def __init__( self, i_port ):
threading.Thread.__init__( self )
self.quitting = False
self.serversocket = socket.socket( socket.AF_INET, socket.SOCK_STREAM )
self.serversocket.bind( (socket.gethostname(), i_port ) )
self.serversocket.listen(5)
self.start()
def run( self ):
# Wait for connection
while not self.quitting:
rr,rw,err = select.select( [self.serversocket],[],[], 20 )
if rr:
(clientsocket, address) = self.serversocket.accept()
clientsocket.close()
def main():
a = Server( 6543 )
time.sleep(4)
if __name__=='__main__':
main()

Add self.setDaemon(True) to the __init__ before self.start().
(In Python 2.6 and later, self.daemon = True is preferred).
The key idea is explained here:
The entire Python program exits when
no alive non-daemon threads are left.
So, you need to make "daemons" of those threads who should not keep the whole process alive just by being alive themselves. The main thread is always non-daemon, by the way.

I don't recommend the setDaemon feature for normal shutdown. It's sloppy; instead of having a clean shutdown path for threads, it simply kills the thread with no chance for cleanup. It's good to set it, so your program doesn't get stuck if the main thread exits unexpectedly, but it's not a good normal shutdown path except for quick hacks.
import sys, os, socket, threading, time, select
class Server(threading.Thread):
def __init__(self, i_port):
threading.Thread.__init__(self)
self.setDaemon(True)
self.quitting = False
self.serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.serversocket.bind((socket.gethostname(), i_port))
self.serversocket.listen(5)
self.start()
def shutdown(self):
if self.quitting:
return
self.quitting = True
self.join()
def run(self):
# Wait for connection
while not self.quitting:
rr,rw,err = select.select([self.serversocket],[],[], 1)
print rr
if rr:
(clientsocket, address) = self.serversocket.accept()
clientsocket.close()
print "shutting down"
self.serversocket.close()
def main():
a = Server(6543)
try:
time.sleep(4)
finally:
a.shutdown()
if __name__=='__main__':
main()
Note that this will delay for up to a second after calling shutdown(), which is poor behavior. This is normally easy to fix: create a wakeup pipe() that you can write to, and include it in the select; but although this is very basic, I couldn't find any way to do this in Python. (os.pipe() returns file descriptors, not file objects that we can write to.) I havn't dig deeper, since it's tangental to the question.