I've run into a very strange problem, that is making me wonder if I understand exception handling at all.
I have a code (that I'll post at the end) that looks more or less like this:
try:
doSomething()
finally:
print 'bye'
The code in the finally clause is not being executed when I exit my program via ctrl+c.
To make matters worse, now consider the following:
try:
doSomething()
except: # this could be replaced by except Exception, it doesn't matter
print 'something'
finally:
print 'bye'
Now the code in the except clause is not executed.. but the code in the finally clause is!
I realize this has to be the fault of the code executed by doSomething(). But my question is, how is it even possible? I thought we could be 100% confident that finally clauses always got executed.
Here goes the real code. It's running on a raspberry pi 3. It's an adaptation of the code found here.
import RPi.GPIO as GPIO, time
GPIO.setmode(GPIO.BCM)
# Define function to measure charge time
def RCtime (PiPin):
# Discharge capacitor
GPIO.setup(PiPin, GPIO.OUT)
GPIO.output(PiPin, GPIO.LOW)
time.sleep(.1)
time1 = time.time()
GPIO.setup(PiPin, GPIO.IN)
if (GPIO.input(PiPin) == GPIO.LOW):
GPIO.wait_for_edge(PiPin, GPIO.RISING, timeout=1000)
time_elap = time.time()-time1
return time_elap*1e3
# Main program loop
try:
while True:
print RCtime(4) # Measure timing using GPIO4
except Exception:
print '---------got ya-----------------'
finally:
print '---Finaly---'
GPIO.cleanup() # this ensures a clean exit
To be more specific, the behaviour depicted appears when the program is waiting at the GPIO.wait_for_edge(PiPin, GPIO.RISING, timeout=1000) line.
If your code does not handle the KeyboardInterrupt exception, the system has to do it: which means it kills the code. Instead, if you have an exception handler, your code takes over, leaves the loop and then executes the finally block.
Well, this seems to be a bug related to the way RPi.GPIO handles signals (such as KeyboardInterrupt). This module invokes some functions written in c. The function is sort of catching the KeyboardInterrupt and throwing an exception, but it doesn't do it properly. Instead of getting only one exception, the two exceptions get "stacked". Because of that, the first thing that run after an exception in my code is going to be terminated by the second exception. If I don't include an except block, the finally block gets executed after the first exception, and is terminated by the second exception. If I include an except block, it tries to run after the first exception, it fails because of the second exception, and then it goes to the finally block.
I only found one forum where people deal with a similar problem.
After my previous experiment I was curious how this works with
input() [https://svn.python.org/projects/python/trunk/Parser/myreadline.c]. I replaced the sem.acquire() with raw_input() and ran the same
tests. Now the inner exception is really taken so it works like the OP
expected. The exception, however is KeyboardInterrupt, not the special
exception from the IPC module.
So I looked in the source code how they did it:
The code is in Parser/myreadline.c.
This code for input in function calls PyErr_CheckSignals() and
PyOS_InterruptOccurred() for a proper handling of the interrupt. So it
seems the OP should do something similar. Onl;y to deliver the custom
error you will have to do some other stuff. I don't know what but maybe
calling PyErr_SetString is sufficient as it might overwrite the
KeyboardInterrupt stuff.
Related
I'm desperate.
My code reads nframe in videos, sometimes the code just stop for no reason, and no error.
So I decided to somehow raise an error.
The thing is, the code does raise an error, but it ignores it for some reason, and just works as normal.
*Ive provided a code block on which exactly the same method works.
handler:
def handler(signum,frame):
print("error") ## This is printed
raise Exception('time out') ## I guess this is getting raised
Code part i want to wrap:
for i in range(0,int(frame_count), nframe): # basicly loads every nframe from the video
try:
frame = video.set(1,i)
signal.signal(signal.SIGALRM), handler)
signal.alarm(1) # At this point, the 'handler' did raise the error, but it did not kill this 'try' block.
_n,frame = video.read() # This line sometimes gets for infinit amount of time, and i want to wrap it
except Exception as e:
print('test') # Code does not get here, yet the 'handler' does raise an exception
raise e
# Here i need to return False, or rise an error, but the code just does not get here.
An example where exactly the same method will work:
import signal
import time
def handler(signum, frame):
raise Exception('time out')
def function():
try:
signal.signal(signal.SIGALRM,handler)
signal.alarm(5) # 5 seconds till raise
time.sleep(10) # does not get here, an Exception is raised after 5 seconds
except Exception as e:
raise e # This will indeed work
My guess is that the read() call is blocked somewhere inside C code. The signal handler runs, puts an exception into the Python interpreter somewhere, but the exception isn't handled until the Python interpreter regains control. This is a limitation documented in the signal module:
A long-running calculation implemented purely in C (such as regular expression matching on a large body of text) may run uninterrupted for an arbitrary amount of time, regardless of any signals received. The Python signal handlers will be called when the calculation finishes.
One possible workaround is to read frames on a separate process using the multiprocessing module, and return them to the main process using a multiprocessing.Queue (from which you can get with a timeout). However, there will be extra overhead in sending the frames between processes.
Another approach might be to try and avoid the root of the problem. OpenCV has different video backends (V4L, GStreamer, ffmpeg, ...); one of them might work where another doesn't. Using the second argument to the VideoCapture constructor, you can indicate a preference for which backend to use:
cv.VideoCapture(..., cv.CAP_FFMPEG)
See the documentation for the full list of backends. Depending on your platform and OpenCV build, not all of them will be available.
I have a python script that runs on an infinite loops. In the loop multiprocess and async functions happen so normally I catch KeyboardInterrupt to properly kill all the processes.
Using similar code somehow on one of the loops I am unable to catch the KeyboardInterrupt the loop just keeps going.
logic goes like that:
try:
while True:
do stuff
except (KeyboardInterrupt, SystemExit):
exit cleanly
Normally I would suspect a blanket try ... except somewhere in the children functions but I went over the whole code base and while there is a lot of error catching everything is specific.
Is there a way to trace errors and somehow figure out where the KeyboardInterrupt is caught ?
Thank you
****** Edit after some debugging...
So I disabled the code part by part until I cornered the bug:
Somewhere in the code I was calling a method that was missing self and was not marked as #staticmethod.
Changing that fixed my issue.
This works for me.
try:
while True:
print(1)
except KeyboardInterrupt:
raise
EDIT:
Actually read your question, I won't be able to tell you why it's not raising an error without seeing the rest or more of the code.
It is conventional to use pass statement in python like the following piece of code.
try:
os.makedirs(dir)
except OSError:
pass
So, 'pass' bascially does not do anything here. In this case, why would we still put a few codes like this in the program? I am confused. Many thanks for your time and attention.
It's for the parser. If you wrote this:
try:
# Code
except Error:
And then put nothing in the except spot, the parser would signal an error because it would incorrectly identify the next indentation level. Imagine this code:
def f(x):
try:
# Something
except Error:
def g(x):
# More code
The parser was expecting a statement with a greater indentation than the except statement but got a new top-level definition. pass is simply filler to satisfy the parser.
This is in case you want the code to continue right after the lines in the try block. If you won't catch it - it either skips execution until it is caught elsewhere - or fails the program altogether.
Suppose you're creating a program that attempts to print to a printer, but also prints to the standard output - you may not want it to file if the printer is not available:
try:
print_to_printer('hello world')
except NoPrinterError:
pass # no printer - that's fine
print("hello world")
If you would not use a try-catch an error would stop execution until the exception is caught (or would fail the program) and nothing would be printed to standard output.
The pass is used to tell the program what to do when it catches an error. In this particular case you're pretty much ignoring it. So you're running your script and if you experience an error keep going without worrying as to why and how.
That particular case is when you are definite on what is expected. There are other cases where you can break and end the program, or even assign the error to a variable so you can debug your program by using except Error as e.
try:
os.makedirs(dir)
except OSError:
break
or:
try:
os.makedirs(dir)
except OSError as e:
print(str(e))
try:
# Do something
except:
# again some code
# few more code
There are two uses of pass. First, and most important use :- if exception arises for the code under try, the execution will jump to except block. And if you have nothing inside the except block, it will throw IndentationError at the first place. So, to avoid this error, even if you have nothing to do when exception arises, you need to put pass inside except block.
The second use, if you have some more code pieces after the try-except block (e.g. again some code and few more code), and you don't put pass inside except, then that code piece will not be executed (actually the whole code will not be executed since compiler will throw IndentationError). So, in order to gracefully handle the scenario and tell the interpreter to execute the lines after except block, we need to put pass inside the except block, even though we don't want to do anything in case of exception.
So, here pass as indicated from name, handles the except block and then transfers the execution to the next lines below the except block.
I am using thread to invoke a function which runs as :
def fillIQ(ipno):
global inp_width
global no_exit
try:
while 1 :
if no_exit==1:
sys.exit() # <exit line>
tags=[]
for i in range(ipno):
yn=random.randint(0,1)
if yn==1:
voqno=random.randint(0,ipno-1)
if inpQ[i][voqno]<10:
inpQ[i][voqno]+=1
tag="iq"
tags.append(tag)
d.update()
time.sleep(2)
d.delete("iq")
drawIQ(ipno)
except BaseException ,e:
print "fillIQ > "+e
I am changing the value of no_exit in the main function. But once I change it the thread is not getting exit. Because next time I create a thread for another instance with different inputs( its a GUI program. for one input i execute the thread and later, change input and execute it again) the odler thread appears to run.
To exit a thread you must either return from it or raise SystemExit(). Returning actually raises a SystemExit exception in the context of a thread. But you are catching that exception and not re-raising it.
>>> isinstance(SystemExit(1), BaseException)
True
There's your problem. Either catch SystemExit exceptions separately and re-raise it explicitly, or be more selective with your general catching, or re-raise everything you catch. For brevity, I give an example of the former
except SystemExit:
raise
except BaseException:
print "fillIQ > ", e
Also, note that it's , e and not +e since e is not a string. There were a couple of other issues I encountered just trying to get your thread to run, but I'm sure you'll figure them out quickly enough, mostly missing imports and other values that are not listed in your example.
I'm trying to raise an exception within an except: block but the interpreter tries to be helpful and prints stack traces 'by force'. Is it possible to avoid this?
A little bit of background information:
I'm toying with urwid, a TUI library for python. The user interface is started by calling urwid.MainLoop.run() and ended by raising urwid.ExitMainLoop(). So far this works fine but what happens when another exception is raised? E.g. when I'm catching KeyboardInterrupt (the urwid MainLoop does not), I do some cleanup and want to end the user interface - by raising the appropriate exception. But this results in a screen full of stack traces.
Some little research showed python3 remembers chained exceptions and one can explicitly raise an exception with a 'cause': raise B() from A(). I learned a few ways to change or append data regarding the raised exceptions but I found no way to 'disable' this feature. I'd like to avoid the printing of stack traces and lines like The above exception was the direct cause of... and just raise the interface-ending exception within an except: block like I would outside of one.
Is this possible or am I doing something fundamentally wrong?
Edit:
Here's an example resembling my current architecture, resulting in the same problem:
#!/usr/bin/env python3
import time
class Exit_Main_Loop(Exception):
pass
# UI main loop
def main_loop():
try:
while True:
time.sleep(0.1)
except Exit_Main_Loop as e:
print('Exit_Main_Loop')
# do some UI-related clean up
# my main script
try:
main_loop()
except KeyboardInterrupt as e:
print('KeyboardInterrupt')
# do some clean up
raise Exit_Main_Loop() # signal the UI to terminate
Unfortunately I can't change main_loop to except KeyboardInterrupt as well. Is there a pattern to solve this?
I still don't quite understand your explanation, but from the code:
try:
main_loop()
except KeyboardInterrupt as e:
print('KeyboardInterrupt')
# do some clean up
raise Exit_Main_Loop() # signal the UI to terminate
There is no way that main_loop could ever see the Exit_Main_Loop() exception. By the time you get to the KeyboardInterrupt handle, main_loop is guaranteed to have already finished (in this case, because of an unhandled KeyboardInterrupt), so its exception handler is no longer active.
So, what happens is that you raise a new exception that nobody catches. And when an exception gets to the top of your code without being handled, Python handles it automatically by printing a traceback and quitting.
If you want to convert one type of exception into another so main_loop can handle it, you have to do that somewhere inside the try block.
You say:
Unfortunately I can't change main_loop to except KeyboardInterrupt as well.
If that's true, there's no real answer to your problem… but I'm not sure there's a problem in the first place, other than the one you created. Just remove the Exit_Main_Loop() from your code, and isn't it already doing what you wanted? If you're just trying to prevent Python from printing a traceback and exiting, this will take care of it for you.
If there really is a problem—e.g., the main_loop code has some cleanup code that you need to get executed no matter what, and it's not getting executed because it doesn't handle KeyboardInterrupt—there are two ways you could work around this.
First, as the signal docs explain:
The signal.signal() function allows to define custom handlers to be executed when a signal is received. A small number of default handlers are installed: … SIGINT is translated into a KeyboardInterrupt exception.
So, all you have to do is replace the default handler with a different one:
def handle_sigint(signum, frame):
raise ExitMainLoop()
signal.signal(signal.SIGINT, handle_sigint)
Just do this before you start main_loop, and you should be fine. Keep in mind that there are some limitations with threaded programs, and with Windows, but if none of those limitations apply, you're golden; a ctrl-C will trigger an ExitMainLoop exception instead of a KeyboardInterrupt, so the main loop will handle it. (You may want to also add an except ExitMainLoop: block in your wrapper code, in case there's an exception outside of main_loop. However, you could easily write a contextmanager that sets and restores the signal around the call to main_loop, so there isn't any outside code that could possibly raise it.)
Alternatively, even if you can't edit the main_loop source code, you can always monkeypatch it at runtime. Without knowing what the code looks like, it's impossible to explain exactly how to do this, but there's almost always a way to do it.