I have an app with two threads. One is a pygame thread which runs a simple game, the other thread is a listening server which accepts messages which are used to control the game.
Here is the stripped down pseudo code:
class ServerThread(threading.Thread):
def run(self):
class SingleTCPHandler(SocketServer.BaseRequestHandler):
try:
while(1):
...
#Receive messages from socket. Add them to pygame event queue
...
except KeyboardInterrupt:
sys.exit(0)
...
...
class PygameThread(threading.Thread):
def run(self):
...
#pygame stuff
...
#The following pygame code closed the app when closing the pygame window while running as a single thread
for event in pygame.event.get():
if event.type==QUIT:
exit()
...
try:
server_thread = ServerThread()
server_thread.start()
pygame_thread = PygameThread()
pygame_thread.start()
except KeyboardInterrupt:
sys.exit(0)
It seems that none of the exceptions are being caught. I've tried running just the server without the pygame thread and the:
try:
while(1):
...
#Receive messages from socket. Add them to pygame event queue
...
except KeyboardInterrupt:
sys.exit(0)
doesn't respond to Ctrl + c
The pygame window standard close button (the little x op right) doesn't work anymore.
And my try of a workaround:
try:
server_thread = ServerThread()
server_thread.start()
pygame_thread = PygameThread()
pygame_thread.start()
except KeyboardInterrupt:
sys.exit(0)
also isn't working.
I'm looking for ideas to close the app without having to kill the shell from which the app has been started.
Updated
Based on the suggestion i did the following:
Changed the former while True in both treads to while not self.stop_requested:.
And also:
try:
pygame_thread = PygameThread()
pygame_thread.start()
server_thread = ServerThread()
server_thread.start()
except KeyboardInterrupt:
pygame_thread.stop_requested = True
server_thread.stop_requested = True
It still isn't working. I also noticed that in the console which runs this code when I try to terminate with Ctrl+c, it only gets printed out.
alan#alan ~/.../py $ python main.py
^C^C^C^C^C^C^C
Updated
I did a little shortcut and changed the server thread to daemon, so it closes once the pygame window (which is tha pygame thread) is closed.
In the except-block of your main program, you should somehow notify your Threads to stop on their own. You can look at my answer in this thread to get an idea of what I mean.
Basically, substitute the while(1):-loop by a while not self.stop_requested:-loop. You can then set this field of your class from inside your main thread, where the KeyboardInterrupt is actually caught. Then you should also join() each thread from your main thread, and then you safely know everthing stopped.
BTW: I would not use while(1) at all. while True is more intuitive, as the 1 is evaluated as a bool each iteration of the loop. Why not write a bool where it is expected? The parentheses are redundant as well. This kind of notation goes back to good-old C, which doesn't have a boolean-type.
sys.exit is somewhat confusing name, since it doesn't actually terminate or "exit" anything. It only throws an exception, and if you do that in a thread, the exception remains local to that thread. To throw SystemExit in the main context you're going to need thread.interrupt_main.
Related
I have a program that has quite a few functions, each running on a separate thread.
When the user presses Ctrl+C, only 1 thread crashes with an exception, but because of this, the whole program may not work correctly.
Of course, I can write this construction in each function:
try:
do_something()
except KeyboardInterrupt as e:
pass
but, as I said, there are many functions, perhaps there is an option not to prescribe this construction in each function?
Or is it possible to disable Ctrl+C interrupt in cmd settings?
For example, in the registry. The program creates its own registry key in HKEY_CURRENT_USER\Console\MyProgrammKey
UPD 1
signal.signal(signal.SIGINT, signal.SIG_IGN)
It helped in almost all cases except one: a thread that has an infinite loop with the input() function anyway interrupts.
UPD 2
Here is a sample code
import signal, time
from threading import Thread
def one():
while True:
inp = input("INPUT: ")
def two():
while True:
print("I just printing...")
time.sleep(1)
if __name__ == '__main__':
signal.signal(signal.SIGINT, signal.SIG_IGN)
Thread(target=one).start()
Thread(target=two).start()
UPD 3
Screenshot of exception.
Ctrl+C will send SIGINT signal to program, so you could define a global signal handler to ignore that SIGINT, something like next:
test.py:
import signal, os, time
def handler(signum, frame):
pass
signal.signal(signal.SIGINT, handler)
time.sleep(10)
print("done")
During the program run, if you input Ctrl+c, the program will ignore it, and continue to run, finally print done:
$ python3 test.py
^Cdone
I am having a main program which is defined like this:
main.py
def main():
try:
registry.start_server()
except:
print("Shutting down the program")
pass
if __name__ == '__main__':
main()
registry.start_server() is the method in another module which looks like this:
def start_server():
t_server = threading.Thread(target=server.start)
t_server.start()
try:
t_server.join()
except KeyboardInterrupt:
print("Error")
raise ValueError
finally:
fp.close()
server.start is the method in another module which does some listening work in a while(True) manner. I am not sure how to stop the whole program when clicking Stop in PyCharm which is Ctrl + C (Signal). I tried with Event but without success. I get to the main.py by raising an exception when the signal gets caught but that does not terminate the whole program. It shows Waiting for program to detach. The only way is to use SIGKILL. I don't understand where does the program keeps hanging? I have also tried calling sys.exit(0) when the signal gets caught and creating the thread as Deamon but that didnt help either.
EDIT
While True method in another module
def start(self, event):
try:
while True:
if event.is_set():
if self.pubsub.channels:
print("It enters here")
message = self.pubsub.get_message(True)
if message:
.
.
.
else:
return
To solve the problem, all you need to do is:
let the child-thread exit, and
let main thread join the child-thread.
I have an infinite loop in which there are operations that are mandatory to be completely executed before exiting the loop. Namely, I am using the socket library for connecting to an external device and I need to wait the read instructions to be finished before interrupting the loop.
I have tried using a signal handler (like in this question) for raising a flag when a Keyboard interrupt is detected.
Current code:
import videosensor
import signal
def signal_handler(signal, frame):
"""Raises a flag when a keyboard interrupt is raised."""
global interrupted
interrupted = True
if __name__ == '__main__':
camera = videosensor.VideoSensor(filename)
interrupted = False
signal.signal(signal.SIGINT, signal_handler)
while not interrupted:
location = camera.get_register()
#...
#More irrelevant stuff is executed.
#...
time.sleep(0.01)
#This code has to be executed after exiting while loop
camera_shutdown(camera)
In the previous code, videosensor.VideoSensor is a class containing socket operations for getting data from an external device. The get_register() method used in the main routine is the following:
def get_register(self):
"""Read the content of the specified register.
"""
#Do some stuff
value = socket.recv(2048)
return value
The problem:
I wanted the while loop to be continually executed until the user pressed a key or used the Keyboard Interrupt, but after the current iteration was finished. Instead, using the previous solution does not work as desired, as it interrupts the ongoing instruction, and if it is reading the socket, an error is raised:
/home/.../client.pyc
in read_register(self, regkey)
164 reg = self._REGISTERS[regkey]
165 self.send('r,{}\n'.format(reg))
--> 166 value = socket.recv(2048)
167 #Convert the string input into a valid value e.g. list or int
168 formatted_result = ast.literal_eval(value)
error: [Errno 4] Interrupted system
EDIT: It seems, from an answer below, that there is no way of using the Keyboard Interrupt and avoid the socket read function to be aborted. Despite there are solutions for catching the error, they don't avoid the read cancellation.
I am interested, though, in finding a way of getting a user input e.g. specific key press, that raises the flag, which will be checked at the end of the loop, without interrupting the main routine execution until this check.
EDIT2: The used OS is the Linux distribution Ubuntu 14.04
After quick SO search I found this solution for your issue
Basically, there's nothing you can do: when you send a SIGINT to your process, the socket will return a SIGINT as well. The best you can do, then, is to actively ignore the issue, by catching the socket EINTR error and going on with your loop:
import errno
try:
# do something
value = conn.recv(2048)
except socket.error as (code, msg):
if code != errno.EINTR:
raise
An alternative solution to avoid issues with C-c breaking reads, is to use parallel execution, to read your socket in a routine, and handle user input on the other:
import asyncio
async def camera_task(has_ended, filename):
camera = videosensor.VideoSensor(filename)
try:
while not has_ended.is_set():
location = camera.get_register()
#...
#More irrelevant stuff is executed.
#...
await asyncio.sleep(0.01)
finally:
#This code has to be executed after exiting while loop
camera_shutdown(camera)
async def input_task(shall_end):
while True:
i = input("Press 'q' to stop the script…")
if i == 'q':
shall_end.set()
def main():
filename = …
#
end_event = asyncio.Event()
asyncio.Task(camera_task(end_event, filename))
asyncio.Task(input_task(end_event))
asyncio.get_event_loop().run_forever()
or with threading
import threading, time
def camera_task(has_ended, filename):
camera = videosensor.VideoSensor(filename)
try:
while not has_ended.is_set():
location = camera.get_register()
#...
#More irrelevant stuff is executed.
#...
time.sleep(0.01)
finally:
#This code has to be executed after exiting while loop
camera_shutdown(camera)
def input_task(shall_end):
while True:
i = input("Press 'q' to stop the script…")
if i == 'q':
shall_end.set()
def main():
filename = …
#
end_event = threading.Event()
threads = [
threading.Thread(target=camera_task, args=(end_event, filename)),
threading.Thread(target=input_task, args=(end_event,))
]
# start threads
for thread in threads:
thread.start()
# wait for them to end
for thread in threads:
thread.join()
or with multiprocessing:
import multiprocessing, time
def camera_task(has_ended, filename):
camera = videosensor.VideoSensor(filename)
try:
while not has_ended.is_set():
location = camera.get_register()
#...
#More irrelevant stuff is executed.
#...
time.sleep(0.01)
finally:
#This code has to be executed after exiting while loop
camera_shutdown(camera)
def input_task(shall_end):
while True:
i = input("Press 'q' to stop the script…")
if i == 'q':
shall_end.set()
def main():
filename = …
#
end_event = multiprocessing.Event()
processes = [
multiprocessing.Process(target=camera_task, args=(end_event, filename)),
multiprocessing.Process(target=input_task, args=(end_event,))
]
# start processes
for process in processes:
process.start()
# wait for them to end
for process in processes:
process.join()
disclaimer: those codes are untested, and there might be some typos or little errors, but I believe the overall logic should be 👌
You created your custom signal handler but did not overide the default keyboard interrupt behaviour. Add signal.signal(signal.SIGINT, signal_handler) to your code to accomplish this:
import videosensor
import signal
# Custom signal handler
def signal_handler(signal, frame):
"""Raises a flag when a keyboard interrupt is raised."""
global interrupted
interrupted = True
# Necessary to override default keyboard interrupt
signal.signal(signal.SIGINT, signal_handler)
if __name__ == '__main__':
# Main programme
If I understand correctly, you do not want socket.recv() to be interrupted, but you do want to use signals to let the user indicate that the I/O loop should be terminated once the current I/O operation has completed.
With the assumption that you are using Python 2 on a Unix system, you can solve your problem by calling signal.siginterrupt(signal.SIGINT, False) before entering the loop. This will cause system calls to be restarted when a signal occurs rather than interrupting it and raising an exception.
In your case this means that the socket.recv() operation will be restarted after your signal handler is called and therefore get_register() will not return until a message is received on the socket. If that is what you want your code will be:
interrupted = False
old_handler = signal.signal(signal.SIGINT, signal_handler) # install signal handler
signal.siginterrupt(signal.SIGINT, False) # do not interrupt system calls
while not interrupted:
location = camera.get_register()
if location == '':
# remote connection closed
break
#...
#More irrelevant stuff is executed.
#...
time.sleep(0.01)
That's one way to do it, but it does require that your code is running on a Unix platform.
Another way, which might work on other platforms, is to handle the exception, ignore further SIGINT signals (in case the user hits interrupt again), and then perform a final socket.recv() before returning from the get_register() function:
import errno
def get_register(s):
"""Read the content of the specified register.
"""
#Do some stuff
try:
old_handler = None
return s.recv(2048)
except socket.error as exc:
if exc.errno == errno.EINTR:
old_handler = signal.signal(signal.SIGINT, signal.SIG_IGN) # ignore this signal
return s.recv(2048) # system call was interrupted, restart it
else:
raise
finally:
if old_handler is not None:
signal.signal(signal.SIGINT, old_handler) # restore handler
Signal handling can get tricky and there might be race conditions in the above that I am not aware of. Try to use siginterrupt() if possible.
The following program hangs the terminal such that it ignores Ctrl+C. This is rather annoying since I have to restart the terminal every time one of the threads hang.
Is there any way to catch the KeyboardInterrupt while waiting on an event?
import threading
def main():
finished_event = threading.Event()
startThread(finished_event)
finished_event.wait()#I want to stop the program here
print('done!')
def startThread(evt):
"""Start a thread that will trigger evt when it is done"""
#evt.set()
if __name__ == '__main__':
main()
If you want to avoid polling, you can use the pause() function of the signal module instead of finished_event.wait(). signal.pause() is a blocking function and gets unblocked when a signal is received by the process. In this case, when ^C is pressed, SIGINT signal unblocks the function. Note that the function does not work on Windows according to the documentation. I've tried it on Linux and it worked for me.
I came across this solution in this SO thread.
Update: On the current Python 3 finished_event.wait() works on my Ubuntu machine (starting with Python 3.2). You don't need to specify the timeout parameter, to interrupt it using Ctrl+C. You need to pass the timeout parameter on CPython 2.
Here's a complete code example:
#!/usr/bin/env python3
import threading
def f(event):
while True:
pass
# never reached, otherwise event.set() would be here
event = threading.Event()
threading.Thread(target=f, args=[event], daemon=True).start()
try:
print('Press Ctrl+C to exit')
event.wait()
except KeyboardInterrupt:
print('got Ctrl+C')
There could be bugs related to Ctrl+C. Test whether it works in your environment.
Old polling answer:
You could try to allow the interpreter to run the main thread:
while not finished_event.wait(.1): # timeout in seconds
pass
If you just want to wait until the child thread is done:
while thread.is_alive():
thread.join(.1)
You could also patch the Event.wait() function in the following manner:
def InterruptableEvent():
e = threading.Event()
def patched_wait():
while not e.is_set():
e._wait(3)
e._wait = e.wait
e.wait = patched_wait
return e
>>> event = InterruptableEvent()
>>> try:
... event.wait()
... except KeyboardInterrupt:
... print "Received KeyboardInterrupt"
...
^CReceived KeyboardInterrupt
This works because wait() with a timeout argument will raise a KeyboardInterrupt.
Based on #Pete's answer, but with subclassing and using the actual Event.wait method, just with smaller timeouts to allow handling of KeyboardInterrupts and such in between:
class InterruptableEvent(threading.Event):
def wait(self, timeout=None):
wait = super().wait # get once, use often
if timeout is None:
while not wait(0.01): pass
else:
wait(timeout)
I cannot get a way to terminate a thread that is hung in a socket.recvfrom() call. For example, ctrl+c that should trigger KeyboardInterrupt exception can't be caught. Here is a script I've used for testing:
from socket import *
from threading import Thread
from sys import exit
class TestThread(Thread):
def __init__(self,host="localhost",port=9999):
self.sock = socket(AF_INET,SOCK_DGRAM)
self.sock.bind((host,port))
super(TestThread,self).__init__()
def run(self):
while True:
try:
recv_data,addr = self.sock.recvfrom(1024)
except (KeyboardInterrupt, SystemExit):
sys.exit()
if __name__ == "__main__":
server_thread = TestThread()
server_thread.start()
while True: pass
The main thread (the one that executes infinite loop) exits. However the thread that I explicitly create, keeps hanging in recvfrom().
Please, help me resolve this.
Keyboard interrupts are always caught on the main thread -- never on "child" threads. To avoid server_thread keeping the process alive when the main thread exits, do
server_thread.daemon = True
before you call server_thread.start().
BTW, your while True: pass in the main thread is needlessly burning CPU cycles. You should at least change it to something like while True: time.sleep(1.0). But that doesn't change the semantics of your code -- just gets it down from 99% CPU or so, to (I'd guess) < 5%;-).
You should open a pipe from the main thread to the network thread and 'select' on both the socket and the pipe. When you want to terminate the network thread, just send a byte through the pipe from the main thread and act accordingly in the network thread.
Just my 2 cents.