Python socket accept blocks - prevents app from quitting - python

I've written a very simple python class which waits for connections on a socket. The intention is to stick this class into an existing app and asyncronously send data to connecting clients.
The problem is that when waiting on an socket.accept(), I cannot end my application by pressing ctrl-c. Neither can I detect when my class goes out of scope and notify it to end.
Ideally the application below should quit after the time.sleep(4) expires. As you can see below, I tried using select, but this also prevents the app from responding to ctrl-c. If I could detect that the variable 'a' has gone out of scope in the main method, I could set the quitting flag (and reduce the timeout on select to make it responsive).
Any ideas?
thanks
import sys
import socket
import threading
import time
import select
class Server( threading.Thread ):
def __init__( self, i_port ):
threading.Thread.__init__( self )
self.quitting = False
self.serversocket = socket.socket( socket.AF_INET, socket.SOCK_STREAM )
self.serversocket.bind( (socket.gethostname(), i_port ) )
self.serversocket.listen(5)
self.start()
def run( self ):
# Wait for connection
while not self.quitting:
rr,rw,err = select.select( [self.serversocket],[],[], 20 )
if rr:
(clientsocket, address) = self.serversocket.accept()
clientsocket.close()
def main():
a = Server( 6543 )
time.sleep(4)
if __name__=='__main__':
main()

Add self.setDaemon(True) to the __init__ before self.start().
(In Python 2.6 and later, self.daemon = True is preferred).
The key idea is explained here:
The entire Python program exits when
no alive non-daemon threads are left.
So, you need to make "daemons" of those threads who should not keep the whole process alive just by being alive themselves. The main thread is always non-daemon, by the way.

I don't recommend the setDaemon feature for normal shutdown. It's sloppy; instead of having a clean shutdown path for threads, it simply kills the thread with no chance for cleanup. It's good to set it, so your program doesn't get stuck if the main thread exits unexpectedly, but it's not a good normal shutdown path except for quick hacks.
import sys, os, socket, threading, time, select
class Server(threading.Thread):
def __init__(self, i_port):
threading.Thread.__init__(self)
self.setDaemon(True)
self.quitting = False
self.serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.serversocket.bind((socket.gethostname(), i_port))
self.serversocket.listen(5)
self.start()
def shutdown(self):
if self.quitting:
return
self.quitting = True
self.join()
def run(self):
# Wait for connection
while not self.quitting:
rr,rw,err = select.select([self.serversocket],[],[], 1)
print rr
if rr:
(clientsocket, address) = self.serversocket.accept()
clientsocket.close()
print "shutting down"
self.serversocket.close()
def main():
a = Server(6543)
try:
time.sleep(4)
finally:
a.shutdown()
if __name__=='__main__':
main()
Note that this will delay for up to a second after calling shutdown(), which is poor behavior. This is normally easy to fix: create a wakeup pipe() that you can write to, and include it in the select; but although this is very basic, I couldn't find any way to do this in Python. (os.pipe() returns file descriptors, not file objects that we can write to.) I havn't dig deeper, since it's tangental to the question.

Related

Python socket with worker threads

I'm playing with Python to understand basics of client-server and threading programming.
This is my main.py application:
from threading import Thread
from Server import Server
class Worker(object):
def __init__(self, clients):
#constructor
self.clients = clients
def start(self):
while True:
print("it work as expected!")
conn, addr = self.clients.get()
print( conn.recv(1024) )
#create a instance of the server
server = Server()
#put on a new thread
def serverThreadCallback(server):
server.start()
def createWorkerCallback():
worker = Worker(server.getClients())
worker.start()
#put the server on a new thread
serverThread = Thread( target=serverThreadCallback, args=(server, ) )
serverThread.daemon = True
serverThread.start()
'''
workerThread = Thread(target=createWorkerCallback)
workerThread.daemon = True
workerThread.start()
'''
and this is my server.py
import socket
import queue
class Server(object):
'''
classdocs
'''
def __init__(self):
'''
Constructor
'''
self.clients = queue.Queue()
def create(self):
print("Creating server...")
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR , 1)
self.socket = server
def listen(self):
print("Starting server...")
self.socket.bind( ("0.0.0.0", 9002) )
self.socket.listen(10)
print("Server started. Listening on localhost port 9002")
while(True):
#conn, addr = self.server.accept()
self.clients.put(self.socket.accept())
def start(self):
self.create()
self.listen()
def getClients(self):
return self.clients
When I try to start the server in that thread, the listening method
doesn't call. Something goes stuck in create() method from the server.
What I've did wrong?
First, there are a couple of syntax errors on the coded you posted:
main.py:
1) Indentation starting from your class declaration is wrong
2) You should call from server import Server (note the module name is server - in lower case)
server.py
3) You should call import Queue (and not import queue)
4) Therefore the call to self.clients = queue.Queue() should become self.clients = Queue.Queue() (the module name is Queue with upper case Q)
Other than that, you have to add a serverThread.join() after serverThread.start(), otherwise the main thread finishes and the program terminates before your server thread has had the chance to start.
Finally, you might want to change the call to accept() inside the thread, such that it times out from time to time to handle Ctrl+C interrupts properly (otherwise you can only finish your program by killing it)
serverThread.daemon = True
Daemon threads are abruptly stopped when Python program exits.
Maybe the listen() method hasn't been called yet.

Gevent.monkey.patch_all breaks code that relies on socket.shutdown()

I'm currently working to add support for gevent-socketio to an existing django project. I'm finding that gevent.monkey.patch_all() call is breaking the cancellation mechanism of a thread which is responsible for receiving data from a socket, we'll call the class SocketReadThread for now.
SocketReadThread is pretty simple, it calls recv() on a blocking socket. When it receives data is processes it and calls recv() again. The thread stops when an exception occurs or when recv() returns 0 bytes as occurs when socket.shutdown(SHUT_RDWR) is called in SocketReadThread.stop_reading()
The problem occurs when the gevent.monkey.patch_all() replaces the default socket implementation. Instead of shutting down nicely I get the following exception:
error: [Errno 9] File descriptor was closed in another greenlet
I'm assuming this is occurring because gevent makes my socket non-blocking in order to work its magic. This means that when I call socket.shutdown(socket.SHUT_RDWR) the greenlet that was doing the work for the monkey patched socket.recv call tried to read from the closed file descriptor.
I coded an example to isolate this issue:
from gevent import monkey
monkey.patch_all()
import socket
import sys
import threading
import time
class SocketReadThread(threading.Thread):
def __init__(self, socket):
super(SocketReadThread, self).__init__()
self._socket = socket
def run(self):
connected = True
while connected:
try:
print "calling socket.recv"
data = self._socket.recv(1024)
if (len(data) < 1):
print "received nothing, assuming socket shutdown"
connected = False
else :
print "Recieved something: {}".format(data)
except socket.timeout as e:
print "Socket timeout: {}".format(e)
connected = false
except :
ex = sys.exc_info()[1]
print "Unexpected exception occurrred: {}".format(str(ex))
raise ex
def stop_reading(self):
self._socket.shutdown(socket.SHUT_RDWR)
self._socket.close()
if __name__ == '__main__':
sock = socket.socket()
sock.connect(('127.0.0.1', 4242))
st = SocketReadThread(sock)
st.start()
time.sleep(3)
st.stop_reading()
st.join()
If you open a terminal an run nc -lp 4242 & (to give this program something to connect to) and then run this program you will see the exception mentioned above. If you remove the call to monkey.patch_all() you will see that it works just fine.
My question is: How can support cancellation of the SocketReadThread in a way that works with or without gevent monkey patching and doesn't require the use of an arbitrary timeout that would make cancellation slow (i.e. calling recv() with a timeout and checking a conditional)?
I found that there were two different workarounds for this. The first was to simply catch and suppress the exception. This appears to work fine since it is common practice for one thread to close a socket in order to cause another thread to exit from a blocking read. I don't know or understand why greenlets would complain about this other than a debugging aid. It is really just an annoyance.
The second option was to use the self-pipe trick (a quick search yields many explanations) as a mechanism to wake up a blocked thread. Essentially we create a second file descriptor (a socket is like a type of file descriptor to the OS) for signaling cancellation. We then use select as our blocking to wait for either incoming data on the socket or a cancellation request to come in on the cancellation file descriptor. See the example code below.
from gevent import monkey
monkey.patch_all()
import os
import select
import socket
import sys
import threading
import time
class SocketReadThread(threading.Thread):
def __init__(self, socket):
super(SocketReadThread, self).__init__()
self._socket = socket
self._socket.setblocking(0)
r, w = os.pipe()
self._cancelpipe_r = os.fdopen(r, 'r')
self._cancelpipe_w = os.fdopen(w, 'w')
def run(self):
connected = True
read_fds = [self._socket, self._cancelpipe_r]
while connected:
print "Calling select"
read_list, write_list, x_list = select.select(read_fds, [], [])
print "Select returned"
if self._cancelpipe_r in read_list :
print "exiting"
self._cleanup()
connected = False
elif self._socket in read_list:
print "calling socket.recv"
data = self._socket.recv(1024)
if (len(data) < 1):
print "received nothing, assuming socket shutdown"
connected = False
self._cleanup()
else :
print "Recieved something: {}".format(data)
def stop_reading(self):
print "writing to pipe"
self._cancelpipe_w.write("\n")
self._cancelpipe_w.flush()
print "joining"
self.join()
print "joined"
def _cleanup(self):
self._cancelpipe_r.close()
self._cancelpipe_w.close()
self._socket.shutdown(socket.SHUT_RDWR)
self._socket.close()
if __name__ == '__main__':
sock = socket.socket()
sock.connect(('127.0.0.1', 4242))
st = SocketReadThread(sock)
st.start()
time.sleep(3)
st.stop_reading()
Again, before running the above program run netcat -lp 4242 & to give it a listening socket to connect to.

How to exit a multithreaded program?

I was just messing around with threading in python, wrote this basic IM thingy [code at bottom]
I noticed that when I kill the program with C-c it doesn't exit, it just hangs forever.
I'm just guessing it's waiting for each thread to finish what they are doing, but since it's an endless loop that will never happen.
So I guess I need to kill each thread manually, or end the loop when the killsignal comes in.
How would I do that?
#!/usr/bin/env python
import threading
import socket
class Listen(threading.Thread):
def run(self):
conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
conn.bind(('', 2727))
conn.listen(1)
while True:
channel, details = conn.accept()
print str(details)+": "+channel.recv(250)
channel.send("got it")
channel.close()
class Shout(threading.Thread):
def run(self):
while True:
try:
address = raw_input("who u talking to? ")
conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
conn.connect((address, 2727))
break
except:
print "can't connect to "+ str(address)
while True:
conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
conn.connect((address, 2727))
conn.send(raw_input())
conn.close()
listen = Listen().start()
shout = Shout().start()
I see several causes of the misbehavior in your code.
Ctrl+C causes a "KeyboardInterrupt" exception in the main thread. So you should handle it there.
Your socket is in blocking mode. This causes several socket functions to block the calling thread until the function returns. During this state the thread cannot react to any termination event.
As you already said: your endless loop in the thread's run() function is ... really endless. So the thread execution is never ending (at least not without an unexpected exception). You should use some kind of synchronization object, like an threading.Event object, to be able to tell a thread externally that it should terminate itself.
I would discourage the use of raw_input() out of the main thread. Imagine what happens when you have more than one Shout thread.
Why are you always closing and reconnecting the socket when a message has been transmitted in your Shout class? Network connections should be re-established only in special cases because of the setup costs.
Without a frame protocol for the communication you can never expect to have received all data that was sent by the other host when the recv() function returns.
The start() function of the thread object does not return a value or object. So saving the returned value (=None) doesn't make much sense.
You can never expect the send() function to transmit all passed data. Therefore you must check the result of the function and appropriately handle the situation when not all bytes were really transmitted.
To learn threading there are surely better problems to solve than network communication, since that topic is in itself really complex.
Beside all these things, here is my try for a solution. Still there is much that can be improved. You should consider the answer from Mark Tolonen too, since the SocketServer class is surely provided to ease several things in handling this kind of stuff. But you should keep on studying the basics too.
#!/usr/bin/env python
import threading
import socket
import time
import errno
class StoppableThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.stop_event = threading.Event()
def stop(self):
if self.isAlive() == True:
# set event to signal thread to terminate
self.stop_event.set()
# block calling thread until thread really has terminated
self.join()
class Accept(StoppableThread):
def __init__(self, port):
StoppableThread.__init__(self)
self.port = port
self.threads = []
def run(self):
# handle connection acception
conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
conn.bind(('', self.port ))
conn.listen(5)
# set socket timeout to ~10ms
conn.settimeout(0.01)
while self.stop_event.is_set() == False:
try:
csock, caddr = conn.accept()
# spawn a new thread to handle the client connection
listen_thread = Listen(csock, caddr)
self.threads.append(listen_thread)
listen_thread.start()
except socket.timeout:
# socket operation timeout
# clear all terminated threads from thread list
for thread in self.threads:
if thread.isAlive() == False:
self.threads.remove(thread)
self.stop_threads()
def stop_threads(self):
# stop all running threads
for listen_thread in self.threads:
if listen_thread.isAlive() == True:
listen_thread.stop()
self.threads = []
class Listen(StoppableThread):
def __init__(self, csock, caddr):
StoppableThread.__init__(self)
self.csock = csock
self.caddr = caddr
self.csock.setblocking(False)
def run(self):
while self.stop_event.is_set() == False:
try:
recv_data = self.csock.recv(250)
if len(recv_data) > 0:
print str(self.caddr)+": " + recv_data
self.csock.send("got it")
else:
# connection was closed by foreign host
self.stop_event.set()
except socket.error as (sock_errno, sock_errstr):
if (sock_errno == errno.EWOULDBLOCK):
# socket would block - sleep sometime
time.sleep(0.1)
else:
# unexpected / unhandled error - terminate thread
self.stop_event.set()
channel.close()
class Shout(StoppableThread):
def __init__(self, sport):
StoppableThread.__init__(self)
self.sport = sport
def run(self):
while self.stop_event.is_set() == False:
try:
address = raw_input("who u talking to? ")
conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
conn.connect((address, self.sport))
break
except socket.error:
# handle connection problems
print "can't connect to "+ str(address)
except:
# exit thread in case of an unexpected error
self.stop_event.set()
while self.stop_event.is_set() == False:
try:
# chat loop: send messages to remote host
print "what to send? :",
msg = raw_input()
# beware: send() function may block indefinitly here and it might not send all bytes as expected !!
conn.send(msg)
except:
# exit thread in case of an unexpected error
self.stop_event.set()
# close socket before thread terminates
conn.close()
def main():
do_exit = False
server_port = 2727
# start server socket thread
accept = Accept(server_port)
accept.start()
# start transmitting client socket thread
shout = Shout(server_port)
shout.start()
while do_exit == False:
try:
# sleep some time
time.sleep(0.1)
except KeyboardInterrupt:
# Ctrl+C was hit - exit program
do_exit = True
# stop all running threads
shout.stop()
accept.stop()
# exit main program after all threads were terminated gracefully
if __name__ == "__main__":
main()
Look at the Python library source for SocketServer.py, in particular the implementation of server_forever() to see how a server implements a quit. It uses select() to poll the server socket for new connections and tests a quit flag. Here's a hack on your source to use SocketServer, and I added a quit flag to Shout(). It will run the Shout and Listen threads for 5 seconds and then stop them.
import socket
import SocketServer
import threading
import time
class Handler(SocketServer.StreamRequestHandler):
def handle(self):
print str(self.client_address) + ": " + self.request.recv(250)
self.request.send("got it\n")
class Listen(threading.Thread):
def run(self):
self.server = SocketServer.TCPServer(('',2727),Handler)
self.server.serve_forever()
def stop(self):
self.server.shutdown()
class Shout(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.quit = False
def run(self):
while not self.quit:
conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
conn.connect(('localhost', 2727))
conn.send('sending\n')
print conn.recv(100)
conn.close()
def stop(self):
self.quit = True
listen = Listen()
listen.start()
shout = Shout()
shout.start()
time.sleep(5)
shout.stop()
listen.stop()

Running SimpleXMLRPCServer in separate thread and shutting down

I have a class that I wish to test via SimpleXMLRPCServer in python. The way I have my unit test set up is that I create a new thread, and start SimpleXMLRPCServer in that. Then I run all the test, and finally shut down.
This is my ServerThread:
class ServerThread(Thread):
running = True
def run(self):
self.server = #Creates and starts SimpleXMLRPCServer
while (self.running):
self.server.handle_request()
def stop(self):
self.running = False
self.server.server_close()
The problem is, that calling ServerThread.stop(), followed by Thread.stop() and Thread.join() will not cause the thread to stop properly if it's already waiting for a request in handle_request. And since there doesn't seem to be any interrupt or timeout mechanisms here that I can use, I am at a loss for how I can cleanly shut down the server thread.
I had the same problem and after hours of research i solved it by switching from using my own handle_request() loop to serve_forever() to start the server.
serve_forever() starts an internal loop like yours. This loop can be stopped by calling shutdown(). After stopping the loop it is possible to stop the server with server_close().
I don't know why this works and the handle_request() loop don't, but it does ;P
Here is my code:
from threading import Thread
from xmlrpc.server import SimpleXMLRPCServer
from pyWebService.server.service.WebServiceRequestHandler import WebServiceRquestHandler
class WebServiceServer(Thread):
def __init__(self, ip, port):
super(WebServiceServer, self).__init__()
self.running = True
self.server = SimpleXMLRPCServer((ip, port),requestHandler=WebServiceRquestHandler)
self.server.register_introspection_functions()
def register_function(self, function):
self.server.register_function(function)
def run(self):
self.server.serve_forever()
def stop_server(self):
self.server.shutdown()
self.server.server_close()
print("starting server")
webService = WebServiceServer("localhost", 8010)
webService.start()
print("stopping server")
webService.stop_server()
webService.join()
print("server stopped")
Two suggestions.
Suggestion One is to use a separate process instead of a separate thread.
Create a stand-alone XMLRPC server program.
Start it with subprocess.Popen().
Kill it when the test is done. In standard OS's (not Windows) the kill works nicely. In Windows, however, there's no trivial kill function, but there are recipes for this.
The other suggestion is to have a function in your XMLRPC server which causes server self-destruction. You define a function that calls sys.exit() or os.abort() or raises a similar exception that will stop the process.
This is my way. send SIGTERM to self. (Works for me)
Server code
import os
import signal
import xmlrpc.server
server = xmlrpc.server.SimpleXMLRPCServer(("0.0.0.0", 8000))
server.register_function(lambda: os.kill(os.getpid(), signal.SIGTERM), 'quit')
server.serve_forever()
Client code
import xmlrpc.client
c = xmlrpc.client.ServerProxy("http://localhost:8000")
try:
c.quit()
except ConnectionRefusedError:
pass

How to stop BaseHTTPServer.serve_forever() in a BaseHTTPRequestHandler subclass?

I am running my HTTPServer in a separate thread (using the threading module which has no way to stop threads...) and want to stop serving requests when the main thread also shuts down.
The Python documentation states that BaseHTTPServer.HTTPServer is a subclass of SocketServer.TCPServer, which supports a shutdown method, but it is missing in HTTPServer.
The whole BaseHTTPServer module has very little documentation :(
Another way to do it, based on http://docs.python.org/2/library/basehttpserver.html#more-examples, is: instead of serve_forever(), keep serving as long as a condition is met, with the server checking the condition before and after each request. For example:
import CGIHTTPServer
import BaseHTTPServer
KEEP_RUNNING = True
def keep_running():
return KEEP_RUNNING
class Handler(CGIHTTPServer.CGIHTTPRequestHandler):
cgi_directories = ["/cgi-bin"]
httpd = BaseHTTPServer.HTTPServer(("", 8000), Handler)
while keep_running():
httpd.handle_request()
I should start by saying that "I probably wouldn't do this myself, but I have in the past". The serve_forever (from SocketServer.py) method looks like this:
def serve_forever(self):
"""Handle one request at a time until doomsday."""
while 1:
self.handle_request()
You could replace (in subclass) while 1 with while self.should_be_running, and modify that value from a different thread. Something like:
def stop_serving_forever(self):
"""Stop handling requests"""
self.should_be_running = 0
# Make a fake request to the server, to really force it to stop.
# Otherwise it will just stop on the next request.
# (Exercise for the reader.)
self.make_a_fake_request_to_myself()
Edit: I dug up the actual code I used at the time:
class StoppableRPCServer(SimpleXMLRPCServer.SimpleXMLRPCServer):
stopped = False
allow_reuse_address = True
def __init__(self, *args, **kw):
SimpleXMLRPCServer.SimpleXMLRPCServer.__init__(self, *args, **kw)
self.register_function(lambda: 'OK', 'ping')
def serve_forever(self):
while not self.stopped:
self.handle_request()
def force_stop(self):
self.server_close()
self.stopped = True
self.create_dummy_request()
def create_dummy_request(self):
server = xmlrpclib.Server('http://%s:%s' % self.server_address)
server.ping()
The event-loops ends on SIGTERM, Ctrl+C or when shutdown() is called.
server_close() must be called after server_forever() to close the listening socket.
import http.server
class StoppableHTTPServer(http.server.HTTPServer):
def run(self):
try:
self.serve_forever()
except KeyboardInterrupt:
pass
finally:
# Clean-up server (close socket, etc.)
self.server_close()
Simple server stoppable with user action (SIGTERM, Ctrl+C, ...):
server = StoppableHTTPServer(("127.0.0.1", 8080),
http.server.BaseHTTPRequestHandler)
server.run()
Server running in a thread:
import threading
server = StoppableHTTPServer(("127.0.0.1", 8080),
http.server.BaseHTTPRequestHandler)
# Start processing requests
thread = threading.Thread(None, server.run)
thread.start()
# ... do things ...
# Shutdown server
server.shutdown()
thread.join()
In my python 2.6 installation, I can call it on the underlying TCPServer - it still there inside your HTTPServer:
TCPServer.shutdown
>>> import BaseHTTPServer
>>> h=BaseHTTPServer.HTTPServer(('',5555), BaseHTTPServer.BaseHTTPRequestHandler)
>>> h.shutdown
<bound method HTTPServer.shutdown of <BaseHTTPServer.HTTPServer instance at 0x0100D800>>
>>>
I think you can use [serverName].socket.close()
In python 2.7, calling shutdown() works but only if you are serving via serve_forever, because it uses async select and a polling loop. Running your own loop with handle_request() ironically excludes this functionality because it implies a dumb blocking call.
From SocketServer.py's BaseServer:
def serve_forever(self, poll_interval=0.5):
"""Handle one request at a time until shutdown.
Polls for shutdown every poll_interval seconds. Ignores
self.timeout. If you need to do periodic tasks, do them in
another thread.
"""
self.__is_shut_down.clear()
try:
while not self.__shutdown_request:
# XXX: Consider using another file descriptor or
# connecting to the socket to wake this up instead of
# polling. Polling reduces our responsiveness to a
# shutdown request and wastes cpu at all other times.
r, w, e = select.select([self], [], [], poll_interval)
if self in r:
self._handle_request_noblock()
finally:
self.__shutdown_request = False
self.__is_shut_down.set()
Heres part of my code for doing a blocking shutdown from another thread, using an event to wait for completion:
class MockWebServerFixture(object):
def start_webserver(self):
"""
start the web server on a new thread
"""
self._webserver_died = threading.Event()
self._webserver_thread = threading.Thread(
target=self._run_webserver_thread)
self._webserver_thread.start()
def _run_webserver_thread(self):
self.webserver.serve_forever()
self._webserver_died.set()
def _kill_webserver(self):
if not self._webserver_thread:
return
self.webserver.shutdown()
# wait for thread to die for a bit, then give up raising an exception.
if not self._webserver_died.wait(5):
raise ValueError("couldn't kill webserver")
This is a simplified version of Helgi's answer for python 3.7:
import threading
import time
from http.server import ThreadingHTTPServer, SimpleHTTPRequestHandler
class MyServer(threading.Thread):
def run(self):
self.server = ThreadingHTTPServer(('localhost', 8000), SimpleHTTPRequestHandler)
self.server.serve_forever()
def stop(self):
self.server.shutdown()
if __name__ == '__main__':
s = MyServer()
s.start()
print('thread alive:', s.is_alive()) # True
time.sleep(2)
s.stop()
print('thread alive:', s.is_alive()) # False
This method I use successfully (Python 3) to stop the server from the web application itself (a web page):
import http.server
import os
import re
class PatientHTTPRequestHandler(http.server.SimpleHTTPRequestHandler):
stop_server = False
base_directory = "/static/"
# A file to use as an "server stopped user information" page.
stop_command = "/control/stop.html"
def send_head(self):
self.path = os.path.normpath(self.path)
if self.path == PatientHTTPRequestHandler.stop_command and self.address_string() == "127.0.0.1":
# I wanted that only the local machine could stop the server.
PatientHTTPRequestHandler.stop_server = True
# Allow the stop page to be displayed.
return http.server.SimpleHTTPRequestHandler.send_head(self)
if self.path.startswith(PatientHTTPRequestHandler.base_directory):
return http.server.SimpleHTTPRequestHandler.send_head(self)
else:
return self.send_error(404, "Not allowed", "The path you requested is forbidden.")
if __name__ == "__main__":
httpd = http.server.HTTPServer(("127.0.0.1", 8080), PatientHTTPRequestHandler)
# A timeout is needed for server to check periodically for KeyboardInterrupt
httpd.timeout = 1
while not PatientHTTPRequestHandler.stop_server:
httpd.handle_request()
This way, pages served via base address http://localhost:8080/static/ (example http://localhost:8080/static/styles/common.css) will be served by the default handler, an access to http://localhost:8080/control/stop.html from the server's computer will display stop.html then stop the server, any other option will be forbidden.
I tried all above possible solution and ended up with having a "sometime" issue - somehow it did not really do it - so I ended up making a dirty solution that worked all the time for me:
If all above fails, then brute force kill your thread using something like this:
import subprocess
cmdkill = "kill $(ps aux|grep '<name of your thread> true'|grep -v 'grep'|awk '{print $2}') 2> /dev/null"
subprocess.Popen(cmdkill, stdout=subprocess.PIPE, shell=True)
import http.server
import socketserver
import socket as sck
import os
import threading
class myserver:
def __init__(self, PORT, LOCATION):
self.thrd = threading.Thread(None, self.run)
self.Directory = LOCATION
self.Port = PORT
hostname = sck.gethostname()
ip_address = sck.gethostbyname(hostname)
self.url = 'http://' + ip_address + ':' + str(self.Port)
Handler = http.server.SimpleHTTPRequestHandler
self.httpd = socketserver.TCPServer(("", PORT), Handler)
print('Object created, use the start() method to launch the server')
def run(self):
print('listening on: ' + self.url )
os.chdir(self.Directory)
print('myserver object started')
print('Use the objects stop() method to stop the server')
self.httpd.serve_forever()
print('Quit handling')
print('Sever stopped')
print('Port ' + str(self.Port) + ' should be available again.')
def stop(self):
print('Stopping server')
self.httpd.shutdown()
self.httpd.server_close()
print('Need just one more request before shutting down'
def start(self):
self.thrd.start()
def help():
helpmsg = '''Create a new server-object by initialising
NewServer = webserver3.myserver(Port_number, Directory_String)
Then start it using NewServer.start() function
Stop it using NewServer.stop()'''
print(helpmsg)
Not a experience python programmer, just wanting to share my comprehensive solution. Mostly based on snippets here and there. I usually import this script in my console and it allows me to set up multiple servers for different locations using their specific ports, sharing my content with other devices on the network.
Here's a context-flavored version for Python 3.7+ which I prefer because it cleans up automatically and you can specify the directory to serve:
from contextlib import contextmanager
from functools import partial
from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer
from threading import Thread
#contextmanager
def http_server(host: str, port: int, directory: str):
server = ThreadingHTTPServer(
(host, port), partial(SimpleHTTPRequestHandler, directory=directory)
)
server_thread = Thread(target=server.serve_forever, name="http_server")
server_thread.start()
try:
yield
finally:
server.shutdown()
server_thread.join()
def usage_example():
import time
with http_server("127.0.0.1", 8087, "."):
# now you can use the web server
time.sleep(100)

Categories