How do you kill a websocket server programmatically? I'll be deploying this server to production along side other things. And I like to build a single python script that sends a kill signal to everything. I cannot figure out how to kill this thing without a user keyboard interrupt or a kill -9.
sys.exit() didn't work.
psutil and terminate() didn't work either
import os
import psutil
current_system_pid = os.getpid()
ThisSystem = psutil.Process(current_system_pid)
ThisSystem.terminate()
I'm out of ideas. For now I'm killing it on the command line with kill -9.
When I kill it varous ways, it tend to see this message below, but the scrip is still running
2020-12-12 12:24:54-0500 [autobahn.twisted.websocket.WebSocketServerFactory] (TCP Port 8080 Closed)
2020-12-12 12:24:54-0500 [-] Stopping factory <autobahn.twisted.websocket.WebSocketServerFactory object at 0x110680f28>
autobahn install:
pip install autobahn[twisted]
Code:
from autobahn.twisted.websocket import WebSocketServerProtocol, WebSocketServerFactory
import sys
from twisted.python import log
from twisted.internet import reactor
class MyServerProtocol(WebSocketServerProtocol):
def onConnect(self, request):
print("Client connecting: {0}".format(request.peer))
def onOpen(self):
print("WebSocket connection open.")
def onMessage(self, payload, isBinary):
print("Text message received: {0}".format(payload.decode('utf8')))
# echo back message verbatim
# self.sendMessage(payload, isBinary)
def onClose(self, wasClean, code, reason):
print("WebSocket connection closed: {0}".format(reason))
def StopWebsocketServer():
PrintAndLog_FuncNameHeader("Begin")
reactor.stop()
PrintAndLog_FuncNameHeader("End")
if __name__ == '__main__':
# TODO remove the logging that came in the example
log.startLogging(sys.stdout)
factory = WebSocketServerFactory("ws://127.0.0.1:8080")
factory.protocol = MyServerProtocol
# note to self: if using putChild, the child must be bytes...
reactor.listenTCP(Port_ws, factory)
reactor.run()
Solution using #Jean-Paul Calderone's answer:
import os
import signal
os.kill(os.getpid(), signal.SIGKILL)
I have an external python script that sends a kill signal to each of my python scripts. The kill signal is simply the existence of a file that every script knows to look for. Once that kill signal appears, each script knows it has x seconds before it will be killed. This way they have a few seconds to gracefully finish something.
twisted.internet.reactor.stop() is how you cause the reactor to shut down. This is usually what results in a Twisted-based program exiting (though of course it doesn't necessarily have to, if the program does more things after the reactor shuts down - but this is uncommon).
However, it sounds like you don't want to know what Python code to run inside the process to end it. You want to know what some other process can do to your Twisted-based process to make it exit. You gave two solutions - KeyboardInterrupt and SIGKILL. You didn't mention why either of these two solutions is inappropriate. They seem fine to me.
If you're uncomfortable with SIGKILL (which you shouldn't be, after all, your program might meet an untimely demise for many reasons and you should be prepared to deal with this) then what you might have overlooked about KeyboardInterrupt is that it is merely the exception that is raised inside a Python program by the default SIGINT handler.
If you send SIGINT to a Twisted-based process then, under normal usage, this will stop the reactor and allow an orderly shutdown.
I have a thread waiting on input, but in the event that no input is provided, I need to exit the program. How can i exit the program? in this example the exit should be triggered by keyboard ctrl+c however I would also like to do this without interaction ie via a timeout or other event.
import threading
import signal
import sys
import time
shutdown = False
def shutdownHook(sigNum, currentStackFrame):
global shutdown
print('shutdown')
shutdown = True
def readInput():
print('readInput')
print(sys.stdin.readline())
print('done reading input')
if __name__ == '__main__':
signal.signal(signal.SIGINT, shutdownHook)
signal.signal(signal.SIGTERM, shutdownHook)
inputThread = threading.Thread(name='input', target=readInput)
inputThread.start()
print('started input')
while not shutdown:
time.sleep(1)
print('waiting ' + str(shutdown))
print('current thread' + str(threading.current_thread()))
print('end of program ' + str(shutdown))
sys.exit(0)
You may use signal.alarm() to send a SIGALRM to your program after a certain amount of time (define here in second):
if __name__ == '__main__':
# Set the signal handler and a 5-second alarm
signal.signal(signal.SIGALRM, shutdownHook)
signal.alarm(5)
Here is the complete working example from the documentation:
Here is a minimal example program. It uses the alarm() function to
limit the time spent waiting to open a file; this is useful if the
file is for a serial device that may not be turned on, which would
normally cause the os.open() to hang indefinitely. The solution is to
set a 5-second alarm before opening the file; if the operation takes
too long, the alarm signal will be sent, and the handler raises an
exception.
import signal, os
def handler(signum, frame):
print('Signal handler called with signal', signum)
raise OSError("Couldn't open device!")
# Set the signal handler and a 5-second alarm
signal.signal(signal.SIGALRM, handler)
signal.alarm(5)
# This open() may hang indefinitely
fd = os.open('/dev/ttyS0', os.O_RDWR)
signal.alarm(0) # Disable the alarm
As for why your program is not quitting is because quoted the doc
Python signal handlers are always executed in the main Python thread,
even if the signal was received in another thread. This means that
signals can’t be used as a means of inter-thread communication. You
can use the synchronization primitives from the threading module
instead. Besides, only the main thread is allowed to set a new signal handler.
That means your thread cannot receive no signals the way you design the program. In fact if you try to set a signal in your thread you will receive a ValueError:
ValueError: signal only works in main thread
That's why your program keeps turning after receiving a SIGTERM. Because the thread did not received the signal.
See here: Kill python thread using os for alternative solution.
Make the thread as Deamon thread, this way it will also shutdown when main thread is exited.
inputThread = threading.Thread(name='input', target=readInput)
inputThread.setDaemon(True) # add this line
inputThread.start()
Also you can add a time lapse for no activity within specified period.
time_limit_for_shutdown_in_secs = 10
secs = 0
while not shutdown:
if secs > time_limit_for_shutdown_in_secs: break
time.sleep(1)
print('waiting ' + str(shutdown))
secs += 1
print('current thread' + str(threading.current_thread()))
print('end of program ' + str(shutdown))
sys.exit(0)
I have a unit test script that needs tests all the rest api's. At the same time I also have a xmpp server that generates messages.
I need to run an instance of the xmpp inside my unit test to receive those messages. But the problem is that the xmpp is a blocking process.
---> self.process(block=True)
This results in the unit test stalling.
Is there any way I can run this xmpp on a background thread and continue to receive msgs and run the unit test on the main thread. If yes, can I have a code snippet which I could implement.
Thanks in advance.
One solution is start the server in the background in your setUp() routine -- ie os.system('myserver &'), then kill it when the test is over, in tearDown()
If you want direct control over the server, use fork() and follow roughly the same pattern as #1.
Example:
import os, sys, subprocess, time, unittest
def server():
try:
for _ in xrange(5, 0, -1):
print 'ding'
time.sleep(1)
except KeyboardInterrupt:
pass
class TestClient(unittest.TestCase):
def setUp(self):
self.server_pid = None
pid = os.fork()
if not pid: # child
return server()
# parent
self.server_pid = pid
def test1(self):
print 'test server, PID',self.server_pid
time.sleep(2)
def tearDown(self):
if not self.server_pid:
return
import signal
os.kill(self.server_pid, signal.SIGINT)
Run with:
python -m unittest ptest
Output:
test server, PID 16490 ding ding .test server, PID None
---------------------------------------------------------------------- Ran 1 test in 2.003s
OK
I have a class that I wish to test via SimpleXMLRPCServer in python. The way I have my unit test set up is that I create a new thread, and start SimpleXMLRPCServer in that. Then I run all the test, and finally shut down.
This is my ServerThread:
class ServerThread(Thread):
running = True
def run(self):
self.server = #Creates and starts SimpleXMLRPCServer
while (self.running):
self.server.handle_request()
def stop(self):
self.running = False
self.server.server_close()
The problem is, that calling ServerThread.stop(), followed by Thread.stop() and Thread.join() will not cause the thread to stop properly if it's already waiting for a request in handle_request. And since there doesn't seem to be any interrupt or timeout mechanisms here that I can use, I am at a loss for how I can cleanly shut down the server thread.
I had the same problem and after hours of research i solved it by switching from using my own handle_request() loop to serve_forever() to start the server.
serve_forever() starts an internal loop like yours. This loop can be stopped by calling shutdown(). After stopping the loop it is possible to stop the server with server_close().
I don't know why this works and the handle_request() loop don't, but it does ;P
Here is my code:
from threading import Thread
from xmlrpc.server import SimpleXMLRPCServer
from pyWebService.server.service.WebServiceRequestHandler import WebServiceRquestHandler
class WebServiceServer(Thread):
def __init__(self, ip, port):
super(WebServiceServer, self).__init__()
self.running = True
self.server = SimpleXMLRPCServer((ip, port),requestHandler=WebServiceRquestHandler)
self.server.register_introspection_functions()
def register_function(self, function):
self.server.register_function(function)
def run(self):
self.server.serve_forever()
def stop_server(self):
self.server.shutdown()
self.server.server_close()
print("starting server")
webService = WebServiceServer("localhost", 8010)
webService.start()
print("stopping server")
webService.stop_server()
webService.join()
print("server stopped")
Two suggestions.
Suggestion One is to use a separate process instead of a separate thread.
Create a stand-alone XMLRPC server program.
Start it with subprocess.Popen().
Kill it when the test is done. In standard OS's (not Windows) the kill works nicely. In Windows, however, there's no trivial kill function, but there are recipes for this.
The other suggestion is to have a function in your XMLRPC server which causes server self-destruction. You define a function that calls sys.exit() or os.abort() or raises a similar exception that will stop the process.
This is my way. send SIGTERM to self. (Works for me)
Server code
import os
import signal
import xmlrpc.server
server = xmlrpc.server.SimpleXMLRPCServer(("0.0.0.0", 8000))
server.register_function(lambda: os.kill(os.getpid(), signal.SIGTERM), 'quit')
server.serve_forever()
Client code
import xmlrpc.client
c = xmlrpc.client.ServerProxy("http://localhost:8000")
try:
c.quit()
except ConnectionRefusedError:
pass
I am running my HTTPServer in a separate thread (using the threading module which has no way to stop threads...) and want to stop serving requests when the main thread also shuts down.
The Python documentation states that BaseHTTPServer.HTTPServer is a subclass of SocketServer.TCPServer, which supports a shutdown method, but it is missing in HTTPServer.
The whole BaseHTTPServer module has very little documentation :(
Another way to do it, based on http://docs.python.org/2/library/basehttpserver.html#more-examples, is: instead of serve_forever(), keep serving as long as a condition is met, with the server checking the condition before and after each request. For example:
import CGIHTTPServer
import BaseHTTPServer
KEEP_RUNNING = True
def keep_running():
return KEEP_RUNNING
class Handler(CGIHTTPServer.CGIHTTPRequestHandler):
cgi_directories = ["/cgi-bin"]
httpd = BaseHTTPServer.HTTPServer(("", 8000), Handler)
while keep_running():
httpd.handle_request()
I should start by saying that "I probably wouldn't do this myself, but I have in the past". The serve_forever (from SocketServer.py) method looks like this:
def serve_forever(self):
"""Handle one request at a time until doomsday."""
while 1:
self.handle_request()
You could replace (in subclass) while 1 with while self.should_be_running, and modify that value from a different thread. Something like:
def stop_serving_forever(self):
"""Stop handling requests"""
self.should_be_running = 0
# Make a fake request to the server, to really force it to stop.
# Otherwise it will just stop on the next request.
# (Exercise for the reader.)
self.make_a_fake_request_to_myself()
Edit: I dug up the actual code I used at the time:
class StoppableRPCServer(SimpleXMLRPCServer.SimpleXMLRPCServer):
stopped = False
allow_reuse_address = True
def __init__(self, *args, **kw):
SimpleXMLRPCServer.SimpleXMLRPCServer.__init__(self, *args, **kw)
self.register_function(lambda: 'OK', 'ping')
def serve_forever(self):
while not self.stopped:
self.handle_request()
def force_stop(self):
self.server_close()
self.stopped = True
self.create_dummy_request()
def create_dummy_request(self):
server = xmlrpclib.Server('http://%s:%s' % self.server_address)
server.ping()
The event-loops ends on SIGTERM, Ctrl+C or when shutdown() is called.
server_close() must be called after server_forever() to close the listening socket.
import http.server
class StoppableHTTPServer(http.server.HTTPServer):
def run(self):
try:
self.serve_forever()
except KeyboardInterrupt:
pass
finally:
# Clean-up server (close socket, etc.)
self.server_close()
Simple server stoppable with user action (SIGTERM, Ctrl+C, ...):
server = StoppableHTTPServer(("127.0.0.1", 8080),
http.server.BaseHTTPRequestHandler)
server.run()
Server running in a thread:
import threading
server = StoppableHTTPServer(("127.0.0.1", 8080),
http.server.BaseHTTPRequestHandler)
# Start processing requests
thread = threading.Thread(None, server.run)
thread.start()
# ... do things ...
# Shutdown server
server.shutdown()
thread.join()
In my python 2.6 installation, I can call it on the underlying TCPServer - it still there inside your HTTPServer:
TCPServer.shutdown
>>> import BaseHTTPServer
>>> h=BaseHTTPServer.HTTPServer(('',5555), BaseHTTPServer.BaseHTTPRequestHandler)
>>> h.shutdown
<bound method HTTPServer.shutdown of <BaseHTTPServer.HTTPServer instance at 0x0100D800>>
>>>
I think you can use [serverName].socket.close()
In python 2.7, calling shutdown() works but only if you are serving via serve_forever, because it uses async select and a polling loop. Running your own loop with handle_request() ironically excludes this functionality because it implies a dumb blocking call.
From SocketServer.py's BaseServer:
def serve_forever(self, poll_interval=0.5):
"""Handle one request at a time until shutdown.
Polls for shutdown every poll_interval seconds. Ignores
self.timeout. If you need to do periodic tasks, do them in
another thread.
"""
self.__is_shut_down.clear()
try:
while not self.__shutdown_request:
# XXX: Consider using another file descriptor or
# connecting to the socket to wake this up instead of
# polling. Polling reduces our responsiveness to a
# shutdown request and wastes cpu at all other times.
r, w, e = select.select([self], [], [], poll_interval)
if self in r:
self._handle_request_noblock()
finally:
self.__shutdown_request = False
self.__is_shut_down.set()
Heres part of my code for doing a blocking shutdown from another thread, using an event to wait for completion:
class MockWebServerFixture(object):
def start_webserver(self):
"""
start the web server on a new thread
"""
self._webserver_died = threading.Event()
self._webserver_thread = threading.Thread(
target=self._run_webserver_thread)
self._webserver_thread.start()
def _run_webserver_thread(self):
self.webserver.serve_forever()
self._webserver_died.set()
def _kill_webserver(self):
if not self._webserver_thread:
return
self.webserver.shutdown()
# wait for thread to die for a bit, then give up raising an exception.
if not self._webserver_died.wait(5):
raise ValueError("couldn't kill webserver")
This is a simplified version of Helgi's answer for python 3.7:
import threading
import time
from http.server import ThreadingHTTPServer, SimpleHTTPRequestHandler
class MyServer(threading.Thread):
def run(self):
self.server = ThreadingHTTPServer(('localhost', 8000), SimpleHTTPRequestHandler)
self.server.serve_forever()
def stop(self):
self.server.shutdown()
if __name__ == '__main__':
s = MyServer()
s.start()
print('thread alive:', s.is_alive()) # True
time.sleep(2)
s.stop()
print('thread alive:', s.is_alive()) # False
This method I use successfully (Python 3) to stop the server from the web application itself (a web page):
import http.server
import os
import re
class PatientHTTPRequestHandler(http.server.SimpleHTTPRequestHandler):
stop_server = False
base_directory = "/static/"
# A file to use as an "server stopped user information" page.
stop_command = "/control/stop.html"
def send_head(self):
self.path = os.path.normpath(self.path)
if self.path == PatientHTTPRequestHandler.stop_command and self.address_string() == "127.0.0.1":
# I wanted that only the local machine could stop the server.
PatientHTTPRequestHandler.stop_server = True
# Allow the stop page to be displayed.
return http.server.SimpleHTTPRequestHandler.send_head(self)
if self.path.startswith(PatientHTTPRequestHandler.base_directory):
return http.server.SimpleHTTPRequestHandler.send_head(self)
else:
return self.send_error(404, "Not allowed", "The path you requested is forbidden.")
if __name__ == "__main__":
httpd = http.server.HTTPServer(("127.0.0.1", 8080), PatientHTTPRequestHandler)
# A timeout is needed for server to check periodically for KeyboardInterrupt
httpd.timeout = 1
while not PatientHTTPRequestHandler.stop_server:
httpd.handle_request()
This way, pages served via base address http://localhost:8080/static/ (example http://localhost:8080/static/styles/common.css) will be served by the default handler, an access to http://localhost:8080/control/stop.html from the server's computer will display stop.html then stop the server, any other option will be forbidden.
I tried all above possible solution and ended up with having a "sometime" issue - somehow it did not really do it - so I ended up making a dirty solution that worked all the time for me:
If all above fails, then brute force kill your thread using something like this:
import subprocess
cmdkill = "kill $(ps aux|grep '<name of your thread> true'|grep -v 'grep'|awk '{print $2}') 2> /dev/null"
subprocess.Popen(cmdkill, stdout=subprocess.PIPE, shell=True)
import http.server
import socketserver
import socket as sck
import os
import threading
class myserver:
def __init__(self, PORT, LOCATION):
self.thrd = threading.Thread(None, self.run)
self.Directory = LOCATION
self.Port = PORT
hostname = sck.gethostname()
ip_address = sck.gethostbyname(hostname)
self.url = 'http://' + ip_address + ':' + str(self.Port)
Handler = http.server.SimpleHTTPRequestHandler
self.httpd = socketserver.TCPServer(("", PORT), Handler)
print('Object created, use the start() method to launch the server')
def run(self):
print('listening on: ' + self.url )
os.chdir(self.Directory)
print('myserver object started')
print('Use the objects stop() method to stop the server')
self.httpd.serve_forever()
print('Quit handling')
print('Sever stopped')
print('Port ' + str(self.Port) + ' should be available again.')
def stop(self):
print('Stopping server')
self.httpd.shutdown()
self.httpd.server_close()
print('Need just one more request before shutting down'
def start(self):
self.thrd.start()
def help():
helpmsg = '''Create a new server-object by initialising
NewServer = webserver3.myserver(Port_number, Directory_String)
Then start it using NewServer.start() function
Stop it using NewServer.stop()'''
print(helpmsg)
Not a experience python programmer, just wanting to share my comprehensive solution. Mostly based on snippets here and there. I usually import this script in my console and it allows me to set up multiple servers for different locations using their specific ports, sharing my content with other devices on the network.
Here's a context-flavored version for Python 3.7+ which I prefer because it cleans up automatically and you can specify the directory to serve:
from contextlib import contextmanager
from functools import partial
from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer
from threading import Thread
#contextmanager
def http_server(host: str, port: int, directory: str):
server = ThreadingHTTPServer(
(host, port), partial(SimpleHTTPRequestHandler, directory=directory)
)
server_thread = Thread(target=server.serve_forever, name="http_server")
server_thread.start()
try:
yield
finally:
server.shutdown()
server_thread.join()
def usage_example():
import time
with http_server("127.0.0.1", 8087, "."):
# now you can use the web server
time.sleep(100)