Python version 2.7.13
I was debugging an issue in my program. I use pycharm debug to find that there is a key error, dict try to get a key not exist in it, but there is no traceback reported. In my program, we use sys.excepthook = <my_callback_func> to catch the uncaught trace, it usually works, but in my issue, it didn't work, the code didn't enter <my_callback_func>.
Maybe there is some other try catch code which catch this exception but I can't find it, because the program is quite large and complex.
So here is my question, is there a way to find which try catch code capture this exception, I use pycharm debug tools, but when I step debug it, the code never go to the catch code. or maybe there is some other reason that the sys.excepthook not working.
Related
Under wxPython when an unhandled exception arises in the main loop the traceback is printed and the exception is ignored.
Not so when debugging under PyCharm. Then the unhandled exception leads to termination of the process.
Also, this is new behavior as previous versions of PyCharm did not behave this way.
Take the following test code:
import wx
import wx.lib.sized_controls as sc
def Raise(evt=None):
raise Exception('error')
app = wx.App()
frame = sc.SizedFrame(None,-1,'hello')
p = frame.GetContentsPane()
b = wx.Button(p,-1,'press')
frame.Bind(wx.EVT_BUTTON,Raise,b)
frame.Show()
app.MainLoop()
Pressing the button when running this normally results in an exception printed and ignored.
When running this under PyCharm debug mode the process terminates.
Is there a way to change this behavior?
Windows10, Python3.6, wxPython 4.0.4, PyCharm 2019.3
Like most Python debuggers, PyCharm installs its own exception handler as sys.excepthook, so whenever there is different behavior between normal runs and debugger runs that is the place to start looking. For example, if you look at the list of Breakpoints for your project you'll also see options for breaking on exceptions. That stuff is implemented in the new excepthoot function. If all else fails, you can look at the pydev code that PyCharm uses for launching and communicating with the debuggee, and see what it's doing in it's excepthook-related code, which may give some clues about how to work around this issue.
I'm not using it as much anymore so I may be misremembering, but I seem to recall that there was an option somewhere in PyCharm for stopping on or ignoring unhandled exceptions, or perhaps for ignoring exceptions raised from specific locations, but I'm not seeing either of them now.
OTOH, catching exceptions before the end of an event handler or other callback function is always a good idea.
I made a typo on a variable in my script. Unfortunately, I did not see it before I began debugging.
After 20 minutes of runtime, the script reached the piece of code with the misnamed variable, and it raised an ValueError Exception, which was break into.
Using the console, I can reassign the value to the variable, this time with its proper name. However, after an exception is raised, it seems there's no way around to continue execution. Is this true?
Is there any method of forcing/bypassing a particular uncaught exception breakpoint so I don't have to restart my script and wait 20 minutes all over again just because of a typo?
It sounds like you want to somehow suppress the exception mid-raise to allow continued execution by pretending the exception never occurred. If that's correct then the answer is no as Python doesn't provide a way to do this.
I am using logging throughout my code for easier control over logging level. One thing that's been confusing me is
when do you prefer logging.error() to raise Exception()?
To me, logging.error() doesn't really make sense, since the program should stop when encountered with an error, like what raise Exception() does.
In what scenarios, do we put up an error message with logging.error() and let the program keep running?
Logging is merely to leave a trace of events for later inspection but does absolutely nothing to influence the program execution; raising an exception allows you to signal an error condition to higher up callers and let them handle it. Typically you use both together, it's not an either-or.
That purely depends on your process flow but it basically boils down to whether you want to deal with an encountered error (and how), or do you want to propagate it to the user of your code - or both.
logging.error() is typically used to log when an error occurs - that doesn't mean that your code should halt and raise an exception as there are recoverable errors. For example, your code might have been attempting to load a remote resource - you can log an error and try again at later time without raising an exception.
btw. you can use logging to log an exception (full, with a stack trace) by utilizing logging.exception().
so I have this script in selenium, that it sometimes crashes for various reasons. Sometimes it fails to click a button, gets confused, gets messed up and displays an error.
How can I command the script, so whenever it crashes, to re-run the script from the beginning again? I've heard about try and except functions but I'm not sure how to use them.
Any help is appreciated! :)
[Using Python 2.7 with Selenium Webdriver]
generic answer to retry on any exception:
while True:
try:
# run your selenium code which sometimes breaks
pass
except Exception as e:
print("something went wrong: "+repr(e))
you may try to refine the exception type to avoid retrying say, because of a python error like ValueError or IOError. Check the exception type and change Exception by the qualified type.
I'm looking for a way to debug a python exception "retrospectively". Essentially if my program raises an exception that isn't handled, I want it to save off the program state so I can go back later and debug the problem.
I've taken a look at the pdb docs, and it seems that you can do this, but only if you can interact with the program at the point of the exception. This won't work for me as the program will be run in the background (without a controlling terminal).
My first (doomed!) approach was to put a try/except block at the highest level of my program, and in the except block extract the traceback object from the current exception and write it to disk using pickle. I planned to then write a separate program that would unpickle the object and use pdb.post_mortem to debug the crashed program. But traceback objects aren't pickleable, but I wouldn't expect that to work anyway, as it wouldn't save off the entire program state.
As far as I know, there isn't any way to do what you're asking. That said, it sounds like you might be looking for a remote debugger. There are a couple of options:
rconsole - This isn't really a debugger, but it allows you to get an interactive prompt inside another process. This can be useful for debugging purposes. I haven't tried this, but it looks relatively straightforward.
rpdb2's embedded debugger - This lets you start a debugger and then connect to it from another shell.
What you can do is use twisted.python and write the traceback to a file, it gives you an exact traceback including the exception
At the moment the exception is caught, before the stack is unwound, the state is available for inspection with the inspect module: http://docs.python.org/2/library/inspect.html
Generically you would use inspect.getinnerframes on your traceback object. The local variables in each stack frame are available as .f_locals so you can see what they are.
The hard part is serializing all of them correctly: depending on what types you have in the local scope you may or may not be able to pickle them, dump them to JSON, or whatever.
You could create an entirely separate execution environment in the top level:
myEnv = {}
myEnv.update(globals)
Then execute your code within that execution environment. If an exception occurs you have the traceback (stack) and all the globals, so you can pretty well reconstruct the program state.
I hope this helps (it helped me):
import logging, traceback
_logger = logging.getLogger(__name__)
try:
something_bad()
except Exception as error:
_logger.exception("Oh no!") # Logs original traceback
store_exception_somewhere(error)
Also, in Python 3 there are a few new options like raise new_exc from original_exc
or raise OtherException(...).with_traceback(tb).