I am trying to create a watchdog class, that will throw an exception after specified time:
from threading import Timer
from time import sleep
class watchdog():
def _timeout(self):
#raise self
raise TypeError
def __init__(self):
self.t = Timer(1, self._timeout)
def start(self):
self.t.start()
try:
w = watchdog()
w.start()
sleep(2)
except TypeError, e:
print "Exception caught"
else:
print "Of course I didn't catch the exception"
This exception is not caught, as the exception is thrown from completely different context, hence we will see the last message.
My question is, how can I modify the code, so the exception will be caught?
This is not possible, as you suggested, and there is no api for abruptly stopping a thread, either, which rules out other potential solutions.
I believe your best solution is to let the watchdog set a flag, and let the test read it at certain points. Similarly, your test can simply check the duration from time to time.
Note that if the "flag" would set in a way that will cause the main thread to raise an exception (for example, deleting attributes from objects), it'll be just as effective.
The other possibility is to use multiprocessing instead of multythreading, if it is possible for your application.
Related
I have some code that is multi-threaded and any time I introduce a bug in any non-main thread, that thread fails silently, and that makes it difficult to debug the program. What is the best way to handle exceptions in threads that aren't the main thread? Ideally I just want uncaught exceptions to kill the process like they would in the main thread.
For example in this code the result will never be changed because I try to reference a function on an object that is set to None:
from threading import Thread
def thread_func(args):
global result
bad_obj = None
bad_obj.call_function() # Should cause error
result = 2
result = 1
thread = Thread(target=thread_func, args=None)
thread.start()
while True:
print(result)
I really would prefer not to surround the entire thread function in a try catch, is there a way to make uncaught exceptions non silent at least?
Thread class catches exceptions in the child thread and only reports them when Thread.join is called.
One work-around is to override Thread.run method in a derived class and use that instead of Thread:
from threading import Thread
import sys
import os
class MyThread(Thread):
def run(self):
try:
super(MyThread, self).run()
except Exception as e:
print("error:", e, file=sys.stderr)
os._exit(1)
Note that it uses os._exit because sys.exit works by throwing SystemExit exception and that doesn't terminate the process until Thread.join is called.
Problem
I have a library which I must use, written in Python 2.7. There are several bugs in it, and one of them occasionally causes the calling thread to crash (rarely, however). I would like to generate the stack trace so I can determine which thread is dying when the library crashes. I get a trace dumped to STDERR of what went wrong in the library, i.e.:
A problem occurred in a Python script. Here is the sequence of
function calls leading up to the error, in the order they occurred.
I've also tried GDB (which works wonders for my C/C++ projects), using a guide I found on StackOverflow to get "Python plus GDB" working (so I can attach to a running Python application). However, I don't see anything helpful that relates to the (now dead) thread.
Question
Is it possible, in Python 2.7, to force a thread (when it crashes) to report a full stack trace to STDOUT, STDERR, or a log file, when this sort of issue (i.e. a library call crashing the calling thread) occurs?
Thank you.
If you have access to the thread definition -- you can write a wrapper thread
import logger
log = logger.getLogger(__name__)
class WrapperThread(threading.Thread):
def __init__(self, innerThread):
self.innerThread = innerThread
def start(self):
try:
# run the thread in the current context with run.
self.innerThread.run()
except Exception as e:
log.error("%s has crashed.", self, exc_info=True) #Exec info makes loggin print the stack trace.
Depending on the library you are using you may be able to apply a decorator to the thread definition. Though I don't recommend code like this ever being included in released code
import threading
import logging
logging.basicConfig()
def loggingThread(clazz, logger):
class _Thread(clazz):
def __init__(self, *args, **kwargs):
clazz.__init__(self, *args, **kwargs)
def run(self):
try:
clazz.run(self)
except Exception as e:
logger.error("%s has Crashed!", self, exc_info=True)
return _Thread
threading.Thread = loggingThread(threading.Thread, logging)
import random
def ohNo(range1, range2):
for x in xrange(1, range1):
if x % random.randint(1, range2) == 0:
raise ValueError("Oh no. %d is an illeagal value!" % (x,))
test = threading.Thread(target=ohNo, args=(500,100))
test.start()
Here is my code for timer
import signal, time
def timeoutHandler():
raise Exception()
try:
signal.signal(signal.SIGALRM, timeoutHandler)
signal.setitimer(signal.ITIMER_REAL, 5)
count = 0
while True:
print count
count += 1
time.sleep(1)
except Exception as e:
print e
It works great, what bothers me is how is this implemented by signal module(i do not posses great knowledge about signals)
I thought the implementation would be like
signal module would be monitoring the SIGALRM in a separate thread
As soon as it receives one it has opportunity to execute the handler
What I dont understand it how does it resume its control back to main thread(The exception is caught)
Sure thread is not what is in picture here and I think some OS concept I am missing over here.
The "separate thread" is the OS itself. signal.settimer effectively asks the OS to send a SIGALRM to this process in 5 seconds.
I'm using the code solution mentioned here.
I'm new to decorators, and don't understand why this solution doesn't work if I want to write something like the following:
#timeout(10)
def main_func():
nested_func()
while True:
continue
#timeout(5)
def nested_func():
print "finished doing nothing"
=> Result of this will be no timeout at all. We will be stuck on endless loop.
However if I remove #timeout annotation from nested_func I get a timeout error.
For some reason we can't use decorator on function and on a nested function in the same time, any idea why and how can I correct it to work, assume that containing function timeout always must be bigger than the nested timeout.
This is a limitation of the signal module's timing functions, which the decorator you linked uses. Here's the relevant piece of the documentation (with emphasis added by me):
signal.alarm(time)
If time is non-zero, this function requests that a SIGALRM signal be sent to the process in time seconds. Any previously scheduled alarm is canceled (only one alarm can be scheduled at any time). The returned value is then the number of seconds before any previously set alarm was to have been delivered. If time is zero, no alarm is scheduled, and any scheduled alarm is canceled. If the return value is zero, no alarm is currently scheduled. (See the Unix man page alarm(2).) Availability: Unix.
So, what you're seeing is that when your nested_func is called, it's timer cancels the outer function's timer.
You can update the decorator to pay attention to the return value of the alarm call (which will be the time before the previous alarm (if any) was due). It's a bit complicated to get the details right, since the inner timer needs to track how long its function ran for, so it can modify the time remaining on the previous timer. Here's an untested version of the decorator that I think gets it mostly right (but I'm not entirely sure it works correctly for all exception cases):
import time
import signal
class TimeoutError(Exception):
def __init__(self, value = "Timed Out"):
self.value = value
def __str__(self):
return repr(self.value)
def timeout(seconds_before_timeout):
def decorate(f):
def handler(signum, frame):
raise TimeoutError()
def new_f(*args, **kwargs):
old = signal.signal(signal.SIGALRM, handler)
old_time_left = signal.alarm(seconds_before_timeout)
if 0 < old_time_left < second_before_timeout: # never lengthen existing timer
signal.alarm(old_time_left)
start_time = time.time()
try:
result = f(*args, **kwargs)
finally:
if old_time_left > 0: # deduct f's run time from the saved timer
old_time_left -= time.time() - start_time
signal.signal(signal.SIGALRM, old)
signal.alarm(old_time_left)
return result
new_f.func_name = f.func_name
return new_f
return decorate
as Blckknght pointed out, You can't use signals for nested decorators - but you can use multiprocessing to achieve that.
You might use this decorator, it supports nested decorators : https://github.com/bitranox/wrapt_timeout_decorator
and as ABADGER1999 points out in his blog https://anonbadger.wordpress.com/2018/12/15/python-signal-handlers-and-exceptions/
using signals and the TimeoutException is probably not the best idea - because it can be caught in the decorated function.
Of course you can use your own Exception, derived from the Base Exception Class, but the code might still not work as expected -
see the next example - you may try it out in jupyter: https://mybinder.org/v2/gh/bitranox/wrapt_timeout_decorator/master?filepath=jupyter_test_wrapt_timeout_decorator.ipynb
import time
from wrapt_timeout_decorator import *
# caveats when using signals - the TimeoutError raised by the signal may be caught
# inside the decorated function.
# So You might use Your own Exception, derived from the base Exception Class.
# In Python-3.7.1 stdlib there are over 300 pieces of code that will catch your timeout
# if you were to base an exception on Exception. If you base your exception on BaseException,
# there are still 231 places that can potentially catch your exception.
# You should use use_signals=False if You want to make sure that the timeout is handled correctly !
# therefore the default value for use_signals = False on this decorator !
#timeout(5, use_signals=True)
def mytest(message):
try:
print(message)
for i in range(1,10):
time.sleep(1)
print('{} seconds have passed - lets assume we read a big file here'.format(i))
# TimeoutError is a Subclass of OSError - therefore it is caught here !
except OSError:
for i in range(1,10):
time.sleep(1)
print('Whats going on here ? - Ooops the Timeout Exception is catched by the OSError ! {}'.format(i))
except Exception:
# even worse !
pass
except:
# the worst - and exists more then 300x in actual Python 3.7 stdlib Code !
# so You never really can rely that You catch the TimeoutError when using Signals !
pass
if __name__ == '__main__':
try:
mytest('starting')
print('no Timeout Occured')
except TimeoutError():
# this will never be printed because the decorated function catches implicitly the TimeoutError !
print('Timeout Occured')
There's a better version of timeout decorator that's currently on Python's PyPI library. It supports both UNIX and non-UNIX based operating system. The part where SIGNALS are mentioned - that specifically for UNIX.
Assuming you aren't using UNIX. Below is a code snippet from the decorator that shows you a list of parameters that you can use as required.
def timeout(seconds=None, use_signals=True, timeout_exception=TimeoutError, exception_message=None)
For implementation on NON-UNIX base operating system. This is what I would do:
import time
import timeout_decorator
#timeout_decorator.timeout(10, use_signals=False)
def main_func():
nested_func()
while True:
continue
#timeout_decorator.timeout(5, use_signals=False)
def nested_func():
print "finished doing nothing"
If you notice, I'm doing use_signals=False. That's all, you should be good to go.
I need to pass exceptions across a thread boundary.
I'm using python embedded in a non thread safe app which has one thread safe call, post_event(callable), which calls callable from its main thread.
I am running a pygtk gui in a seperate thread, so when a button is clicked I post an event with post_event, and wait for it to finish before continuing. But I need the caller to know if the callee threw an exception, and raise it if so. I'm not too worried about the traceback, just the exception itself.
My code is roughly:
class Callback():
def __init__(self,func,*args):
self.func=func
self.args=args
self.event=threading.Event()
self.result=None
self.exception=None
def __call__(self):
gtk.gdk.threads_enter()
try:
self.result=self.func(*self.args)
except:
#what do I do here? How do I store the exception?
pass
finally:
gtk.gdk.threads_leave()
self.event.set()
def post(self):
post_event(self)
gtk.gdk.threads_leave()
self.event.wait()
gtk.gdk.threads_enter()
if self.exception:
raise self.exception
return self.result
Any help appreciated, thanks.
#what do I do here? How do I store the exception?
Use sys.exc_info()[:2], see this wiki
Best way to communicate among threads is Queue. Have the main thread instantiate a Queue.Queue instance and pass it to subthreads; when a subthread has something to communicate back to the master it uses .put on that queue (e.g. a tuple with thread id, exception type, exception value -- or, other useful info, not necessarily exception-related, just make sure the first item of a tuple identifies the kind of info;-). The master can .get those info-units back when it wants, with various choices for synchronization and so on.