We Keep Coding

How to kill a twisted websocket server programmatically

How do you kill a websocket server programmatically? I'll be deploying this server to production along side other things. And I like to build a single python script that sends a kill signal to everything. I cannot figure out how to kill this thing without a user keyboard interrupt or a kill -9.
sys.exit() didn't work.
psutil and terminate() didn't work either
import os
import psutil
current_system_pid = os.getpid()
ThisSystem = psutil.Process(current_system_pid)
ThisSystem.terminate()
I'm out of ideas. For now I'm killing it on the command line with kill -9.
When I kill it varous ways, it tend to see this message below, but the scrip is still running
2020-12-12 12:24:54-0500 [autobahn.twisted.websocket.WebSocketServerFactory] (TCP Port 8080 Closed)
2020-12-12 12:24:54-0500 [-] Stopping factory <autobahn.twisted.websocket.WebSocketServerFactory object at 0x110680f28>
autobahn install:
pip install autobahn[twisted]
Code:
from autobahn.twisted.websocket import WebSocketServerProtocol, WebSocketServerFactory
import sys
from twisted.python import log
from twisted.internet import reactor
class MyServerProtocol(WebSocketServerProtocol):
def onConnect(self, request):
print("Client connecting: {0}".format(request.peer))
def onOpen(self):
print("WebSocket connection open.")
def onMessage(self, payload, isBinary):
print("Text message received: {0}".format(payload.decode('utf8')))
# echo back message verbatim
# self.sendMessage(payload, isBinary)
def onClose(self, wasClean, code, reason):
print("WebSocket connection closed: {0}".format(reason))
def StopWebsocketServer():
PrintAndLog_FuncNameHeader("Begin")
reactor.stop()
PrintAndLog_FuncNameHeader("End")
if __name__ == '__main__':
# TODO remove the logging that came in the example
log.startLogging(sys.stdout)
factory = WebSocketServerFactory("ws://127.0.0.1:8080")
factory.protocol = MyServerProtocol
# note to self: if using putChild, the child must be bytes...
reactor.listenTCP(Port_ws, factory)
reactor.run()
Solution using #Jean-Paul Calderone's answer:
import os
import signal
os.kill(os.getpid(), signal.SIGKILL)
I have an external python script that sends a kill signal to each of my python scripts. The kill signal is simply the existence of a file that every script knows to look for. Once that kill signal appears, each script knows it has x seconds before it will be killed. This way they have a few seconds to gracefully finish something.

twisted.internet.reactor.stop() is how you cause the reactor to shut down. This is usually what results in a Twisted-based program exiting (though of course it doesn't necessarily have to, if the program does more things after the reactor shuts down - but this is uncommon).
However, it sounds like you don't want to know what Python code to run inside the process to end it. You want to know what some other process can do to your Twisted-based process to make it exit. You gave two solutions - KeyboardInterrupt and SIGKILL. You didn't mention why either of these two solutions is inappropriate. They seem fine to me.
If you're uncomfortable with SIGKILL (which you shouldn't be, after all, your program might meet an untimely demise for many reasons and you should be prepared to deal with this) then what you might have overlooked about KeyboardInterrupt is that it is merely the exception that is raised inside a Python program by the default SIGINT handler.
If you send SIGINT to a Twisted-based process then, under normal usage, this will stop the reactor and allow an orderly shutdown.

exit program stuck on sys.stdin.readline

I have a thread waiting on input, but in the event that no input is provided, I need to exit the program. How can i exit the program? in this example the exit should be triggered by keyboard ctrl+c however I would also like to do this without interaction ie via a timeout or other event.
import threading
import signal
import sys
import time
shutdown = False
def shutdownHook(sigNum, currentStackFrame):
global shutdown
print('shutdown')
shutdown = True
def readInput():
print('readInput')
print(sys.stdin.readline())
print('done reading input')
if __name__ == '__main__':
signal.signal(signal.SIGINT, shutdownHook)
signal.signal(signal.SIGTERM, shutdownHook)
inputThread = threading.Thread(name='input', target=readInput)
inputThread.start()
print('started input')
while not shutdown:
time.sleep(1)
print('waiting ' + str(shutdown))
print('current thread' + str(threading.current_thread()))
print('end of program ' + str(shutdown))
sys.exit(0)

You may use signal.alarm() to send a SIGALRM to your program after a certain amount of time (define here in second):
if __name__ == '__main__':
# Set the signal handler and a 5-second alarm
signal.signal(signal.SIGALRM, shutdownHook)
signal.alarm(5)
Here is the complete working example from the documentation:
Here is a minimal example program. It uses the alarm() function to
limit the time spent waiting to open a file; this is useful if the
file is for a serial device that may not be turned on, which would
normally cause the os.open() to hang indefinitely. The solution is to
set a 5-second alarm before opening the file; if the operation takes
too long, the alarm signal will be sent, and the handler raises an
exception.
import signal, os
def handler(signum, frame):
print('Signal handler called with signal', signum)
raise OSError("Couldn't open device!")
# Set the signal handler and a 5-second alarm
signal.signal(signal.SIGALRM, handler)
signal.alarm(5)
# This open() may hang indefinitely
fd = os.open('/dev/ttyS0', os.O_RDWR)
signal.alarm(0) # Disable the alarm
As for why your program is not quitting is because quoted the doc
Python signal handlers are always executed in the main Python thread,
even if the signal was received in another thread. This means that
signals can’t be used as a means of inter-thread communication. You
can use the synchronization primitives from the threading module
instead. Besides, only the main thread is allowed to set a new signal handler.
That means your thread cannot receive no signals the way you design the program. In fact if you try to set a signal in your thread you will receive a ValueError:
ValueError: signal only works in main thread
That's why your program keeps turning after receiving a SIGTERM. Because the thread did not received the signal.
See here: Kill python thread using os for alternative solution.

Make the thread as Deamon thread, this way it will also shutdown when main thread is exited.
inputThread = threading.Thread(name='input', target=readInput)
inputThread.setDaemon(True) # add this line
inputThread.start()
Also you can add a time lapse for no activity within specified period.
time_limit_for_shutdown_in_secs = 10
secs = 0
while not shutdown:
if secs > time_limit_for_shutdown_in_secs: break
time.sleep(1)
print('waiting ' + str(shutdown))
secs += 1
print('current thread' + str(threading.current_thread()))
print('end of program ' + str(shutdown))
sys.exit(0)

How to really test signal handling in Python?

My code is simple:
def start():
signal(SIGINT, lambda signal, frame: raise SystemExit())
startTCPServer()
So I register my application with signal handling of SIGINT, then I start a start a TCP listener.
here are my questions:
How can I using python code to send a SIGINT signal?
How can I test whether if the application receives a signal of SIGINT, it will raise a SystemExit exception?
If I run start() in my test, it will block and how can I send a signal to it?

First of, testing the signal itself is a functional or integration test, not a unit test. See What's the difference between unit, functional, acceptance, and integration tests?
You can run your Python script as a subprocess with subprocess.Popen(), then use the Popen.send_signal() method to send signals to that process, then test that the process has exited with Popen.poll().

How can I using python code to send a SIGINT signal?
You can use os.kill, which slightly misleadingly, can used to send any signal to any process by its ID. The process ID of the application/test can be found by os.getpid(), so you would have...
pid = os.getpid()
# ... other code discussed later in the answer ...
os.kill(pid, SIGINT)
How can I test whether if the application receives a signal of SIGINT, it will raise a SystemExit exception?
The usual way in a test you can check that some code raises SystemExit, is with unittest.TestCase::assertRaises...
import start
class TestStart(unittest.TestCase):
def test_signal_handling(self):
# ... other code discussed later in the answer ...
with self.assertRaises(SystemExit):
start.start()
If I run start() in my test, it will block and how can I send a signal to it?
This is the trick: you can start another thread which then sends a signal back to the main thread which is blocking.
Putting it all together, assuming your production start function is in start.py:
from signal import (
SIGINT,
signal,
)
import socketserver
def startTCPServer():
# Taken from https://docs.python.org/3.4/library/socketserver.html#socketserver-tcpserver-example
class MyTCPHandler(socketserver.BaseRequestHandler):
def handle(self):
self.data = self.request.recv(1024).strip()
self.request.sendall(self.data.upper())
HOST, PORT = "localhost", 9999
server = socketserver.TCPServer((HOST, PORT), MyTCPHandler)
server.serve_forever()
def start():
def raiseSystemExit(_, __):
raise SystemExit
signal(SIGINT, raiseSystemExit)
startTCPServer()
Then your test code could be like the following, say in test.py
import os
from signal import (
SIGINT,
)
import threading
import time
import unittest
import start
class TestStart(unittest.TestCase):
def test_signal_handling(self):
pid = os.getpid()
def trigger_signal():
# You could do something more robust, e.g. wait until port is listening
time.sleep(1)
os.kill(pid, SIGINT)
thread = threading.Thread(target=trigger_signal)
thread.daemon = True
thread.start()
with self.assertRaises(SystemExit):
start.start()
if __name__ == '__main__':
unittest.main()
and run using
python test.py
The above is the same technique as in the answer at https://stackoverflow.com/a/49500820/1319998

Signal Handling in Windows

In Windows I am trying to create a python process that waits for SIGINT signal.And when it receives SIGINT I want it to just print a message and wait for another occurrence of SIGINT.So I used signal handler.
Here is my signal_receiver.py code.
import signal, os, time
def handler(signum, frame):
print 'Yes , Received', signum
signal.signal(signal.SIGINT, handler)
print 'My process Id' , os.getpid()
while True:
print 'Waiting for signal'
time.sleep(10)
When this process running ,I just send SIGINT to this procees from some other python process using,
os.kill(pid,SIGINT).
But when the signal_receiver.py receives SIGINT it just quits the execution .But expected behavior is to print the message inside the handler function and continue execution.
Can some one please help me to solve this issue.Is it a limitation in windows ,because the same works fine in linux.
Thanks in advance.

When you press CTRL+C, the process receives a SIGINT and you are catching it correctly, because otherwise it would throw a KeyboardInterrupt error.
On Windows, when time.sleep(10) is interrupted, although you catch SIGINT, it still throws an InterruptedError. Just add a try/except statement inside time.sleep to catch this exception, for example:
import signal
import os
import time
def handler(signum, frame):
if signum == signal.SIGINT:
print('Signal received')
if __name__ == '__main__':
print('My PID: ', os.getpid())
signal.signal(signal.SIGINT, handler)
while True:
print('Waiting for signal')
try:
time.sleep(5)
except InterruptedError:
pass
Note: tested on Python3.x, it should also work on 2.x.

How do I make a Twisted application handle SIGTERM?

I have the following sample code. The code for handling SIGINT works perfectly but it does not seem to be doing the same when I send a SIGTERM. What could be wrong?
def signal_handler(signal, frame):
print 'Terminating...'
reactor.removeAll()
reactor.stop()
def run():
signal.signal(signal.SIGINT, signal_handler)
signal.signal(signal.SIGTERM, signal_handler)
site = server.Site(stuff)
reactor.listenTCP(8080, site)
reactor.run()
Why isn't SIGTERM calling signal_handler?
Updated with the answer
I added reactor.addSystemEventTrigger('before', 'shutdown', shutdown) before running the reactor and then called reactor.sigTerm() from the shutdown method.

There can only be one handler for a particular signal. reactor.run() installs its own handler for SIGTERM that replaces yours.
Fortunately, the reactor's SIGTERM handler essentially does the same thing as yours (but more correctly).