I'm having trouble with some seemingly simple code which basically starts a thread to read a serial device, and then in the main thread writes some data to the device. The intended shutdown mechanism is a keyboard interrupt, but that doesn't seem to be caught how I expect.
readData = True
dev = serial.Serial('/dev/ttyX', 115200)
readThread = threading.Thread(target=read_loop, args=())
readThread.start()
send_loop()
def read_loop():
while readData:
try:
print dev.read(2)
except Exception, e:
print 'Continue'
dev.close()
def send_loop():
global readData
for i in xrange(5):
try:
dev.write('a')
time.sleep(1)
except Exception,e:
break
readData = False
readThread.join()
A keyboard interrupt, or any other external signal, always goes to the main thread only -- not to sub-threads. If you want everything to stop when the main thread terminates, make sub-threads daemons, so they won't keep the whole process alive by themselves!
Related
I have an infinite loop in which there are operations that are mandatory to be completely executed before exiting the loop. Namely, I am using the socket library for connecting to an external device and I need to wait the read instructions to be finished before interrupting the loop.
I have tried using a signal handler (like in this question) for raising a flag when a Keyboard interrupt is detected.
Current code:
import videosensor
import signal
def signal_handler(signal, frame):
"""Raises a flag when a keyboard interrupt is raised."""
global interrupted
interrupted = True
if __name__ == '__main__':
camera = videosensor.VideoSensor(filename)
interrupted = False
signal.signal(signal.SIGINT, signal_handler)
while not interrupted:
location = camera.get_register()
#...
#More irrelevant stuff is executed.
#...
time.sleep(0.01)
#This code has to be executed after exiting while loop
camera_shutdown(camera)
In the previous code, videosensor.VideoSensor is a class containing socket operations for getting data from an external device. The get_register() method used in the main routine is the following:
def get_register(self):
"""Read the content of the specified register.
"""
#Do some stuff
value = socket.recv(2048)
return value
The problem:
I wanted the while loop to be continually executed until the user pressed a key or used the Keyboard Interrupt, but after the current iteration was finished. Instead, using the previous solution does not work as desired, as it interrupts the ongoing instruction, and if it is reading the socket, an error is raised:
/home/.../client.pyc
in read_register(self, regkey)
164 reg = self._REGISTERS[regkey]
165 self.send('r,{}\n'.format(reg))
--> 166 value = socket.recv(2048)
167 #Convert the string input into a valid value e.g. list or int
168 formatted_result = ast.literal_eval(value)
error: [Errno 4] Interrupted system
EDIT: It seems, from an answer below, that there is no way of using the Keyboard Interrupt and avoid the socket read function to be aborted. Despite there are solutions for catching the error, they don't avoid the read cancellation.
I am interested, though, in finding a way of getting a user input e.g. specific key press, that raises the flag, which will be checked at the end of the loop, without interrupting the main routine execution until this check.
EDIT2: The used OS is the Linux distribution Ubuntu 14.04
After quick SO search I found this solution for your issue
Basically, there's nothing you can do: when you send a SIGINT to your process, the socket will return a SIGINT as well. The best you can do, then, is to actively ignore the issue, by catching the socket EINTR error and going on with your loop:
import errno
try:
# do something
value = conn.recv(2048)
except socket.error as (code, msg):
if code != errno.EINTR:
raise
An alternative solution to avoid issues with C-c breaking reads, is to use parallel execution, to read your socket in a routine, and handle user input on the other:
import asyncio
async def camera_task(has_ended, filename):
camera = videosensor.VideoSensor(filename)
try:
while not has_ended.is_set():
location = camera.get_register()
#...
#More irrelevant stuff is executed.
#...
await asyncio.sleep(0.01)
finally:
#This code has to be executed after exiting while loop
camera_shutdown(camera)
async def input_task(shall_end):
while True:
i = input("Press 'q' to stop the script…")
if i == 'q':
shall_end.set()
def main():
filename = …
#
end_event = asyncio.Event()
asyncio.Task(camera_task(end_event, filename))
asyncio.Task(input_task(end_event))
asyncio.get_event_loop().run_forever()
or with threading
import threading, time
def camera_task(has_ended, filename):
camera = videosensor.VideoSensor(filename)
try:
while not has_ended.is_set():
location = camera.get_register()
#...
#More irrelevant stuff is executed.
#...
time.sleep(0.01)
finally:
#This code has to be executed after exiting while loop
camera_shutdown(camera)
def input_task(shall_end):
while True:
i = input("Press 'q' to stop the script…")
if i == 'q':
shall_end.set()
def main():
filename = …
#
end_event = threading.Event()
threads = [
threading.Thread(target=camera_task, args=(end_event, filename)),
threading.Thread(target=input_task, args=(end_event,))
]
# start threads
for thread in threads:
thread.start()
# wait for them to end
for thread in threads:
thread.join()
or with multiprocessing:
import multiprocessing, time
def camera_task(has_ended, filename):
camera = videosensor.VideoSensor(filename)
try:
while not has_ended.is_set():
location = camera.get_register()
#...
#More irrelevant stuff is executed.
#...
time.sleep(0.01)
finally:
#This code has to be executed after exiting while loop
camera_shutdown(camera)
def input_task(shall_end):
while True:
i = input("Press 'q' to stop the script…")
if i == 'q':
shall_end.set()
def main():
filename = …
#
end_event = multiprocessing.Event()
processes = [
multiprocessing.Process(target=camera_task, args=(end_event, filename)),
multiprocessing.Process(target=input_task, args=(end_event,))
]
# start processes
for process in processes:
process.start()
# wait for them to end
for process in processes:
process.join()
disclaimer: those codes are untested, and there might be some typos or little errors, but I believe the overall logic should be 👌
You created your custom signal handler but did not overide the default keyboard interrupt behaviour. Add signal.signal(signal.SIGINT, signal_handler) to your code to accomplish this:
import videosensor
import signal
# Custom signal handler
def signal_handler(signal, frame):
"""Raises a flag when a keyboard interrupt is raised."""
global interrupted
interrupted = True
# Necessary to override default keyboard interrupt
signal.signal(signal.SIGINT, signal_handler)
if __name__ == '__main__':
# Main programme
If I understand correctly, you do not want socket.recv() to be interrupted, but you do want to use signals to let the user indicate that the I/O loop should be terminated once the current I/O operation has completed.
With the assumption that you are using Python 2 on a Unix system, you can solve your problem by calling signal.siginterrupt(signal.SIGINT, False) before entering the loop. This will cause system calls to be restarted when a signal occurs rather than interrupting it and raising an exception.
In your case this means that the socket.recv() operation will be restarted after your signal handler is called and therefore get_register() will not return until a message is received on the socket. If that is what you want your code will be:
interrupted = False
old_handler = signal.signal(signal.SIGINT, signal_handler) # install signal handler
signal.siginterrupt(signal.SIGINT, False) # do not interrupt system calls
while not interrupted:
location = camera.get_register()
if location == '':
# remote connection closed
break
#...
#More irrelevant stuff is executed.
#...
time.sleep(0.01)
That's one way to do it, but it does require that your code is running on a Unix platform.
Another way, which might work on other platforms, is to handle the exception, ignore further SIGINT signals (in case the user hits interrupt again), and then perform a final socket.recv() before returning from the get_register() function:
import errno
def get_register(s):
"""Read the content of the specified register.
"""
#Do some stuff
try:
old_handler = None
return s.recv(2048)
except socket.error as exc:
if exc.errno == errno.EINTR:
old_handler = signal.signal(signal.SIGINT, signal.SIG_IGN) # ignore this signal
return s.recv(2048) # system call was interrupted, restart it
else:
raise
finally:
if old_handler is not None:
signal.signal(signal.SIGINT, old_handler) # restore handler
Signal handling can get tricky and there might be race conditions in the above that I am not aware of. Try to use siginterrupt() if possible.
I'm trying to see how multi thread are working in order to use them in an automation project. I can run the thread but I cannot find a way to exit completely the two threads: the thread restart after each keyboard interupt. Is there a way to exit both thread with a keyboard interupt ?
import thread
from time import sleep
*parameters when starting
temp_c = 32
T_hot = 30
T_cold = 27
interval_temp = 2
def ctrl_fan(temp_c, T_hot,interval_temp):
while True:
if temp_c >= T_hot:
print 'refreshing'
else:
print ' fan stopped'
sleep(interval_temp)
print 'shutting everything off'
def ctrl_light(temp_c, T_cold,interval_temp):
while True:
if temp_c <= T_cold:
print 'warming'
else:
print 'light stopped'
sleep(interval_temp)
print 'shutting everything off'
try:
thread.start_new_thread(ctrl_fan, (temp_c, T_hot,interval_temp, ) )
sleep(1)
thread.start_new_thread(ctrl_light, (temp_c, T_cold,interval_temp, ) )
except (KeyboardInterrupt, SystemExit):
thread.exit()
print "Error: unable to start thread"
Sure,
Firstly I'd recommend using the slightly higher level threading module instead of the thread module.
To start a thread with threading use the following
import threading
t = threading.Thread(target=ctrl_fan, args=(temp_c, T_hot, interval_temp))
t.start()
There's a few things you'll need to do to get the program to exit with a Ctrl-C interupt.
Firstly you will want to set the threads to be daemon, so that they allow the program to exit when the main thread exits (t.daemon = True)
You will also want the main thread to wait on the completion of the threads, you can use t.join() to do this. However this wont raise out a KeyboardInterrupt exception until the thread finishes, there is a work around for this though
while t.is_alive():
t.join(1)
Providing a timeout value gets around this.
I'd be tempted to pull this together into a subclass, to get the behaviour you want
import threading
class CustomThread(threading.Thread):
def __init__(self, *args, **kwargs):
threading.Thread.__init__(self, *args, **kwargs)
self.daemon = True
def join(self, timeout=None):
if timeout is None:
while self.is_alive():
threading.Thread.join(self, 10)
else:
return threading.Thread.join(self, timeout)
t1 = CustomThread(target=ctrl_fan, args=(temp_c, T_hot, interval_temp))
t1.start()
t2 = CustomThread(target=ctrl_light, args=(temp_c, T_cold, interval_temp))
t2.start()
t1.join()
t2.join()
The explanation is, again, in the documentation (https://docs.python.org/2/library/thread.html) :
Threads interact strangely with interrupts: the KeyboardInterrupt exception will be received by an arbitrary thread. (When the signal module is available, interrupts always go to the main thread.)
You'd certainly find answers in https://stackoverflow.com/, like :
Propagate system call interruptions in threads
I want to write a program for my network course and i have a socket that listen to receive data if it listen and receive no data i should terminate the program, i use threading.Timer to act like timer and have a line like t = threading.Timer(5, end_data) in my function that listen for receive data but i cant terminate program in end_data that is:
def end_data():
sys.exit()
can any one help me?
i also test below code bud did not terminate running program in terminal :(
def end_data():
try:
sys.exit()
except:
print"exception"
i expect that when stop terminal print Tinas-MacBook-Pro:~ tina$
i'm listening to socket in function named receive not main and when elapse 5 second with no data receiving it will run end_data and seems never return to receive function that part of this function is like below
def receive():
s2 = socket(AF_INET, SOCK_DGRAM)
s2.bind(addr3_2)
global end_call
global base_window
write=open('pictur.jpg','wb')
s = socket(AF_INET, SOCK_DGRAM)
s.bind(addr3)
while(end_call==0):
t = threading.Timer(5, end_data)
t.start()
recv_data, addr = s.recvfrom(3500)#size ro hala badan check kon
t.cancel()
first i decide to set global var end_call after 5 second but it didn't work because it never come back to receive function
some thing that is very interesting for me is if define data_end like:
def end_data():
os._exit
print "Hi"
Hi will print in output :O
Maybe try a setup like this
run_program = True
def end_data() :
global run_program
run_program = False
t = threading.Timer(5, end_data)
t.start()
while run_program:
#do stuff
time.sleep(1)
t.join() #maybe not needed
Make sure you called t.start() :)
To answer your second question, with try, except.
This is the expected behavior per the pydoc:
"Since exit() ultimately “only” raises an exception, it will only exit
the process when called from the main thread, and the exception is not intercepted."
Example:
def end_data():
try:
sys.exit("error!")
except SystemExit, e:
print "Exception caught: ", e.args
print "begin"
end_data()
print "end"
Output:
$ ./test.py
begin
Exception caught: ('error!',)
end
What about using threading Events ?
import threading
event = threading.Event()
def signal_stop():
try:
# do what you have to do
finally:
event.set()
t = threading.Timer(5, signal_stop)
print 'start thread and wait for it'
t.start()
event.wait()
print 'done'
I have an app with two threads. One is a pygame thread which runs a simple game, the other thread is a listening server which accepts messages which are used to control the game.
Here is the stripped down pseudo code:
class ServerThread(threading.Thread):
def run(self):
class SingleTCPHandler(SocketServer.BaseRequestHandler):
try:
while(1):
...
#Receive messages from socket. Add them to pygame event queue
...
except KeyboardInterrupt:
sys.exit(0)
...
...
class PygameThread(threading.Thread):
def run(self):
...
#pygame stuff
...
#The following pygame code closed the app when closing the pygame window while running as a single thread
for event in pygame.event.get():
if event.type==QUIT:
exit()
...
try:
server_thread = ServerThread()
server_thread.start()
pygame_thread = PygameThread()
pygame_thread.start()
except KeyboardInterrupt:
sys.exit(0)
It seems that none of the exceptions are being caught. I've tried running just the server without the pygame thread and the:
try:
while(1):
...
#Receive messages from socket. Add them to pygame event queue
...
except KeyboardInterrupt:
sys.exit(0)
doesn't respond to Ctrl + c
The pygame window standard close button (the little x op right) doesn't work anymore.
And my try of a workaround:
try:
server_thread = ServerThread()
server_thread.start()
pygame_thread = PygameThread()
pygame_thread.start()
except KeyboardInterrupt:
sys.exit(0)
also isn't working.
I'm looking for ideas to close the app without having to kill the shell from which the app has been started.
Updated
Based on the suggestion i did the following:
Changed the former while True in both treads to while not self.stop_requested:.
And also:
try:
pygame_thread = PygameThread()
pygame_thread.start()
server_thread = ServerThread()
server_thread.start()
except KeyboardInterrupt:
pygame_thread.stop_requested = True
server_thread.stop_requested = True
It still isn't working. I also noticed that in the console which runs this code when I try to terminate with Ctrl+c, it only gets printed out.
alan#alan ~/.../py $ python main.py
^C^C^C^C^C^C^C
Updated
I did a little shortcut and changed the server thread to daemon, so it closes once the pygame window (which is tha pygame thread) is closed.
In the except-block of your main program, you should somehow notify your Threads to stop on their own. You can look at my answer in this thread to get an idea of what I mean.
Basically, substitute the while(1):-loop by a while not self.stop_requested:-loop. You can then set this field of your class from inside your main thread, where the KeyboardInterrupt is actually caught. Then you should also join() each thread from your main thread, and then you safely know everthing stopped.
BTW: I would not use while(1) at all. while True is more intuitive, as the 1 is evaluated as a bool each iteration of the loop. Why not write a bool where it is expected? The parentheses are redundant as well. This kind of notation goes back to good-old C, which doesn't have a boolean-type.
sys.exit is somewhat confusing name, since it doesn't actually terminate or "exit" anything. It only throws an exception, and if you do that in a thread, the exception remains local to that thread. To throw SystemExit in the main context you're going to need thread.interrupt_main.
I cannot get a way to terminate a thread that is hung in a socket.recvfrom() call. For example, ctrl+c that should trigger KeyboardInterrupt exception can't be caught. Here is a script I've used for testing:
from socket import *
from threading import Thread
from sys import exit
class TestThread(Thread):
def __init__(self,host="localhost",port=9999):
self.sock = socket(AF_INET,SOCK_DGRAM)
self.sock.bind((host,port))
super(TestThread,self).__init__()
def run(self):
while True:
try:
recv_data,addr = self.sock.recvfrom(1024)
except (KeyboardInterrupt, SystemExit):
sys.exit()
if __name__ == "__main__":
server_thread = TestThread()
server_thread.start()
while True: pass
The main thread (the one that executes infinite loop) exits. However the thread that I explicitly create, keeps hanging in recvfrom().
Please, help me resolve this.
Keyboard interrupts are always caught on the main thread -- never on "child" threads. To avoid server_thread keeping the process alive when the main thread exits, do
server_thread.daemon = True
before you call server_thread.start().
BTW, your while True: pass in the main thread is needlessly burning CPU cycles. You should at least change it to something like while True: time.sleep(1.0). But that doesn't change the semantics of your code -- just gets it down from 99% CPU or so, to (I'd guess) < 5%;-).
You should open a pipe from the main thread to the network thread and 'select' on both the socket and the pipe. When you want to terminate the network thread, just send a byte through the pipe from the main thread and act accordingly in the network thread.
Just my 2 cents.