I am running an IronPython script inside a c# application, i am catching exceptions within the script and i wish to find out the script line at which the exception is thrown. This has to be done while the script is running ie. i do not wish the script to terminate in order to print the exception.
Is this even possible?
If inspect is working as expected under IronPython (not really sure) this could do the trick:
import inspect
filename, linenum, funcname = inspect.getframeinfo(inspect.currentframe())[:3]
print linenum
Edit: alternate solution:
import sys
frame = sys._getframe()
print frame.f_lineno
Haven't tried this on ironpython but:
import traceback
try:
# something that raises exception
except YourException, _:
traceback.print_exc()
This should show you the stack trace of the place where the exception was raised. You can also do other stuff than just print, like print to string, or get the stack frames.
The traceback module documentation will tell you more.
Related
I have a python file (app.py) which makes a call to a function as follows:
answer = fn1()
The fn1() is actually written in C++ and I've built a wrapper so that I can use it in Python.
The fn1() can either return a valid result, or it may sometimes fail and terminate. Now the issue is that at the times when fn1() fails and aborts, the calling file (i.e. app.py) also terminates and does not go forward to the error handling part.
I would like the calling file to move to my error handling part (i.e. 'except' and 'finally') if fn1() aborts and dumps core. Is there any way to achieve this?
From the OP:
The C++ file that I have built wrapper around aborts in case of exception and dumps core. Python error code is not executed
This was not evident in your question. To catch this sort of error, you can use the signal.signal function in the python standard library (relevant SO answer).
import signal
def sig_handler(signum, frame):
print("segfault")
signal.signal(signal.SIGSEGV, sig_handler)
answer = fn1()
You basically wrote the answer in your question. Use a try except finally block. Refer also to the Python3 documentation on error handling
try:
answer = fn1()
except Exception: # You can use an exception more specific to your failing code.
# do stuff
finally:
# do stuff
What you need to do is to catch the exception in your C++ function and then convert it to a python exception and return that to the python code.
name = input('name? ')
if len(name) == 0:
print('error.\n')
raise SystemExit
I receive an error when using python 3.3.2 (which is the version is school sadly) but it works fine on other versions e.g. 2.7.10 and 3.5
This is the error
Looking at the screenshot I can see Python prompt at the bottom:
This means the script is run in an interactive session (IDLE on Windows I guess). I haven't found any documentation, but other users have discovered that raising SystemExit in an interactive session does print the traceback.
So you should check and ensure that you are not launching the script in an interactive session.
Old answer:
Looks like it's a bug (or a particularity) in Python 3.3.2. According to this blog post:
If nothing catches the exception, then the python interpreter catches
it at the end, does not print a stack trace, and then calls exit.
I tried to raise SystemExit('asd') and the program just printed asd, so looks like it's true.
Either upgrade Python or try os._exit(1).
Not sure if this is what you want, but if you wanna exit use this:
import sys
name = raw_input('name? ')
if len(name) == 0:
print('error.\n')
sys.exit()
This exits the interpreter by raising SystemExit.
Why don't you use sys.exit()?
sys.exit([arg])
Exit from Python. This is implemented by raising the SystemExit exception, so cleanup actions specified by finally clauses of try statements are honored, and it is possible to intercept the exit attempt at an outer level.
You probably have an handler set for excepthook:
https://docs.python.org/3/library/sys.html#sys.excepthook
You should be able to reset it by doing
sys.excepthook = sys.__excepthook__
Nevermind, the hook works correctly for BaseException, but weirdly enough not for SystemExit (which is a subclass of BaseException).
You're probably executing your program with
python3 -i whatever.py
This gives me the same behavior you witnessed:
dario#feynman ~> python3 -i /tmp/a.py
Traceback (most recent call last):
File "/tmp/a.py", line 11, in <module>
raise SystemExit()
SystemExit
>>>
Note the >>> at the end.
Just remove the -i flag, from whatever is executing your program
Alternatively, it's bad practice, but you can also use os._exit(1)
TL;DR: The Standard Library fails to close a file when an exception is raised. I'm looking for the best way to handle this situation. Feel free to read from the paragraph beginning with "Upon closer inspection of CPython's source code". Also scroll down to the end of the question to grab a self-contained script that reproduces this issue on Windows.
I'm writing a Python package in which I use STL's ConfigParser (2.x) or configparser (3.x) to parse user config file (I will refer to both as ConfigParser since the problem mainly lies in the 2.x implementation). From now on my relevant lines of code on GitHub will be linked when appropriate throughout. ConfigParser.ConfigParser.read(filenames) (used in my code here) raises a ConfigParser.Error exception when the config file is malformed. I have some code in my test suite targeting this situation, using unittest.TestCase.assertRaises(ConfigParser.Error). The malformed config file is properly generated with tempfile.mkstemp (the returned fd is closed with os.close before anything) and I attempt to remove the temp file with os.remove.
The os.remove is where trouble begins. My tests fail on Windows (while working on both OS X and Ubuntu) with Python 2.7 (see this AppVeyor build):
Traceback (most recent call last):
File "C:\projects\storyboard\tests\test_util.py", line 235, in test_option_reader
os.remove(malformed_conf_file)
WindowsError: [Error 32] The process cannot access the file because it is being used by another process: 'c:\\users\\appveyor\\appdata\\local\\temp\\1\\storyboard-test-3clysg.conf'
Note that as I said above, malformed_conf_file is generated with tempfile.mkstemp and immediately closed with os.close, so the only time it is opened is when I call ConfigParser.ConfigParser.read([malformed_conf_file]) here inside the unittest.TestCase.assertRaises(ConfigParser.Error) context. So the culprit seems to be the STL rather than my own code.
Upon closer inspection of CPython's source code, I found that ConfigParser.ConfigPaser.read indeed doesn't close the file properly when an exception is raised. The read method from 2.7 (here on CPython's Mercurial) has the following lines:
for filename in filenames:
try:
fp = open(filename)
except IOError:
continue
self._read(fp, filename)
fp.close()
read_ok.append(filename)
The exception (if any) is raised by self._read(fp, filename), but as you can see, if self._read raises, then fp won't be closed, since fp.close() is only called after self._read returns.
Meanwhile, the read method from 3.4 (here) does not suffer from the same problem since this time they properly embedded file handling in a context:
for filename in filenames:
try:
with open(filename, encoding=encoding) as fp:
self._read(fp, filename)
except OSError:
continue
read_ok.append(filename)
So I think it's pretty clear that the problem is a defect in 2.7's STL. And what's the best way to handle this situation? Specifically:
Is there anything I can do from my side to close that file?
Is it worth reporting to bugs.python.org?
For now I guess I'll just add a try .. except OSError .. to that os.remove (any suggestions?).
Update: A self-contained script that can be used to reproduce this issue on Windows:
#!/usr/bin/env python2.7
import ConfigParser
import os
import tempfile
def main():
fd, path = tempfile.mkstemp()
os.close(fd)
with open(path, 'w') as f:
f.write("malformed\n")
config = ConfigParser.ConfigParser()
try:
config.read(path)
except ConfigParser.Error:
pass
os.remove(path)
if __name__ == '__main__':
main()
When I run it with Python 2.7 interpreter:
Traceback (most recent call last):
File ".\example.py", line 19, in <module>
main()
File ".\example.py", line 16, in main
os.remove(path)
WindowsError: [Error 32] The process cannot access the file because it is being used by another process: 'c:\\users\\redacted\\appdata\\local\\temp\\tmp07yq2v'
This is an interesting problem. As Lukas Graf noted in a comment, the problem seems to be that the exception traceback object holds a reference to the call frame where the exception was raised. This call frame includes the local variables that existed at that time, one of which is a reference to the open file. So that file object still has a reference to it and is not properly closed.
For your self-contained example, simply removing the try/except ConfigParser.Error "works": the exception about the malformed config file is uncaught and stops the program. However, in your actual application, assertRaises is catching the exception in order to check that it is the one you're testing for. I'm not 100% sure why the traceback persists even after the with assertRaises block, but apparently it does.
For your example, another more promising fix is to change the pass in your except clause to sys.exc_clear():
try:
config.read(path)
except ConfigParser.Error:
sys.exc_clear()
This will get rid of the pesky traceback object and allow the file to be closed.
However, it's not clear exactly how to do that in your real application, because the offending except clause there is inside unittest. I think the easiest thing might be to not use assertRaises directly. Instead, write a helper function that does your test, checks for the exception you want, cleans up with the sys.exc_clear() trick, and then raises another custom exception. Then wrap a call to that helper method in assertRaises. This way you gain control of the problematic exception raised by ConfigParser and can properly clean it up (which unittest isn't doing).
Here's a sketch of what I mean:
# in your test method
assertRaises(CleanedUpConfigError, helperMethod, conf_file, malformed_conf_file)
# helper method that you add to your test case class
def helperMethod(self, conf_file, malformed_conf_file):
gotRightError = False
try:
or6 = OptionReader(
config_files=[conf_file, malformed_conf_file],
section='sec',
)
except ConfigParser.Error:
gotRightError = True
sys.exc_clear()
if gotRightError:
raise CleanedUpConfigError("Config error was raised and cleaned up!")
I haven't actually tested this, of course, because I don't have the whole unittest thing set up with your code. You might have to tweak it a bit. (Come to think of it, you might not even need exc_clear() if you do this, because since the exception handler is now in a separate function, the traceback should be correctly cleared when helperMethod exits.) However, I think this idea may get you somewhere. Basically you need to make sure the except clause that catches this particular ConfigParser.Error is written by you, so that it can be cleaned up before you try to remove your test file.
Addendum: it seems that if the context manager handles an exception, the traceback is actually stored until the function containing the with block ends, as this example shows:
class CM(object):
def __init__(self):
pass
def __enter__(self):
return self
def __exit__(self, exc_type, exc_value, tb):
return True
def foo():
with CM():
raise ValueError
print(sys.exc_info())
Even though the with block has ended by the time the print occurs, so the exception handling should be finished, sys.exc_info still returns exception info as if there were an active exception. This is what is happening in your code too: the with assertRaises block causes the traceback to persist all the way to the end of that function, interfering with your os.remove. This seems like buggy behavior, and I notice that it no longer works this way in Python 3 (the print prints (None, None None)), so I imagine this was a wart that was fixed with Python 3.
Based on this, I suspect it may be sufficient to just insert a sys.exc_clear() right before your os.remove (after the with assertRaises block).
When I execute logger.info(traceback.print_exc())
the trace gets on console rather than in the log file
I have logger.propagate = False also still the same issue
print_exc prints the stack trace to stderr.
Just use the exc_info=1 argument and it will automaticaly include the exception.
logging.exception("Exception") #or
logging.error("exception ",exc_info=1) #or
logging.info("Exception has occured" ,exc_info=1)
I am using python 2.7 and sadly exc_info=1 never worked for me so I had to use this:
import traceback
...
log.debug(traceback.format_exc())
I'm using Twister to build a server. I am also maintaining a server error log. The issue is that if I let an exception run all the way up the stack, it'll crash the current connection and disconnect the user, so obviously I attach a bare except to grab everything else.
Once I've caught something, is there a way to get the traceback as a string so that I can store it somewhere/print it myself without raising it and letting Python print it for me once the program crashes?
The traceback module contains some helper functions for printing and inspecting the traceback (for exameble, traceback.print_tb ) - but the important thing is that the traceback information itself is stored in a "interpreter global" variable - sys.exc_traceback, on the module sys.
Quoting from:
http://docs.python.org/reference/compound_stmts.html#try
Before an except clause’s suite is executed, details about the
exception are assigned to three variables in the sys module:
sys.exc_type receives the object identifying the exception;
sys.exc_value receives the exception’s parameter; sys.exc_traceback
receives a traceback object...
You can pass the sys.exc_traceback object as a parameter to traceback.print_tb to have the traceback printed to stdout within the except clause.
Using the logging module, you could log the traceback to a file:
import logging
logging.basicConfig(level = logging.DEBUG, filename = logfile)
logger = logging.getLogger(__name__)
try:
1/0
except ZeroDivisionError as err:
logger.exception(err)
Try this:
import traceback, sys
try:
# Do something that might raise an exception
open("/does not exist",'rb')
except:
traceback.print_exc( file=sys.stderr )
# Or
traceback.print_exc( file=your_open_log_file )
That should do the trick and print full stack traces too.
Yep, there's a module for that. The traceback module contains functions to print or format exception information, or to return the raw stack frames so you can do whatever you want with them.
For a reasonably sophisticated application, though, I would actually recommend using the logging system instead of plain old traceback functions. In particular the method logging.Logger.exception will write information about the exception to whatever logging destination you (or your software's user) has configured. The default formatter will just print out a traceback, as Python would on the console, but you can customize the display of exceptions in your log by creating a Formatter and overriding the format_exception method. Your overridden method is the place to call traceback functions if you need them to format the exception output.