I am writing an interpreter for my simple programing language in Python. When there is an error during interpreting, a Python exception is thrown. The problem is that the stack trace is really ugly because it contains many recursive calls as I am navigating program's structure and interpreting it. And from the stack trace you cannot really see where in my program's structure the code was when it failed.
I know that I could capture all exceptions myself and rethrow a new exception adding some information about where an exception happened, but because it is recursive, I would have to do such rethrowing again and again.
I wonder if there isn't an easier way. For example, that next to path, line numbers, and function name in interpreter's code, I could print out also information where in the interpreted program's code that function is at that moment in the stack.
You can write a sys.excepthook that prints whatever traceback you want. I regularly use such a hook that uses the traceback module to print a normal traceback but with added lines that give the values of interesting local variables (via f_locals introspection).
Related
I'm writing a Python function which takes data from an online source and copies it into a local data dump. If there's a file present already on the intended path for said data dump, I'd like my program to stop abruptly, giving a short message to the user - I'm just building a tiny CLI-type program - explaining why what he or she was about to attempt could destroy valuable data.
Would it be appropriate to raise a FileExists error in the above circumstances? If so, I imagine my code would look something like this:
def make_data_dump():
if os.path.exists("path/to/dump"):
raise FileExistsError("Must not overwrite dump at path/to/dump.")
data = get_data_from_url()
write_to_path(data, "path/to/dump")
Apologies if this is a silly question, but I couldn't find any guidance on when to raise a FileExistsError manually, only on what to do if one's program raises such an exception unexpectedly - hence my asking if raising said exception manually is ever good practice.
The Python documentation explicitly states that this is allowed:
User code can raise built-in exceptions. This can be used to test an exception handler or to report an error condition “just like” the situation in which the interpreter raises the same exception; but beware that there is nothing to prevent user code from raising an inappropriate error.
However, your code example is wrong for a different reason. The problem with this code is that it's using the LBYL (Look Before You Leap) pattern, which might read to race conditions. (In the time between checking if the file exists and writing the file, another process could have created the file, which now would be overwritten). A better pattern for these type of scenarios is the EAFP (Easier to Ask for Forgiveness than Permission) pattern.
See What is the EAFP principle in Python? for more information and examples.
Having said that, I think that for most Python code, manually raising a FileExistsError isn't that common, since you would use the standard Python libraries that already throw this error when necessary. However, one valid reason I can think of is when you would write a wrapper for a low-level function (implemented in another language like C) and would want to translate the error to Python.
A hypothetical code example to demonstrate this:
def make_data_dump():
data = get_data_from_url()
# Assuming write_to_path() is a function in a C library, which returns an error code.
error = write_to_path(data, "path/to/dump")
if error == EEXIST:
raise FileExistsError("Must not overwrite dump at path/to/dump.")
Note that some other built-in exceptions are much more common to raise manually, for example a StopIteration, when implementing an iterator, or a ValueError, for example when your method gets an argument with the wrong value (example).
Some exceptions you will rarely use yourself, like a SyntaxError for example.
As per NobbyNobbs's comment above: if the programmer raises standard exception in his code, it's difficult to work out, during error handling, if a given exception was raised on the application or the system level. Therefore, it's a practice best avoided.
Problem
I have a logging and infra system which I can't (and don't want to modify), in Python, that relies on the Tracebacks of the exceptions.
I have C++ code wrapped with Cython. This C++ code can potentially raise exceptions (std::runtime_errors). Cython gives the ability to translate exception (e.g. with except +). However, and that makes sense of course, the translation only translates the exception type and message.
In my case, I would want to translate the stack trace in the C++ exception (which I can recover to a string), into a Python's Traceback. Obviously, the translation is not fully meaningful, but it allows me to "cheat" and continue (or replace) the stack trace in Python with something that represents the stack trace in C++.
Assuming I can parse the C++ exception's stack trace into meaningful values, I wonder what's the best way to achieve that. I have one idea listed below, and I will be happy to get your opinions on it or share alternatives.
I'll add that this whole thing feels very flaky, but I'm both curious about it and want to avoid doing stuff like putting the C++ stack trace in the Python's exception message.
My Idea
So after looking a bit, I found that CPython has a method that is called _PyTraceback_Add. Specifically it's signature is void _PyTraceback_Add(const char *funcname, const char *filename, int lineno) and looks like it adds the values (funcname, filename and lineno) into the current trace. Link to implementation: https://github.com/python/cpython/blob/66f77caca39ba39ebe1e4a95dba6d19b20d51951/Python/traceback.c#L257
My thoughts were to use that right when I raise the Python exception in Cython. Then, I could use the exception string from C++ and insert the values into the existing trace.
However, this method seems to be private in CPython and I'm not sure it will be sound to rely on it in production code.
Are there any good alternatives to this solution?
Thanks!
Well the function seems to be using only public API's, so can just do the same steps instead.
With that said, it looks unlikely to change anytime soon.
Also, it seems somebody's already doing exactly what you're doing: see issue #24743.
I am running a middle sized django project, whenever a error occurs a traceback is printed, but many functions where Django modules.
But the actual culprit in my own project code, it has only one line, and useful context were hidden because of the traceback depth limit. So I am thinking of filter the call stack by module path instead of call depth.
In case you haven't seen this problem, I have an very similar example in Java. What I need is to make sure the business logic code shows upfront in an exception.
While I am aware of traceback.print_exc(), but you need to wrap every code in a try ... except.
Is it possible to change python's default traceback behavior so django's call stack is fewer and my own code is larger portion?
Whenever an unhandled exception arises, this is the function that Python calls to do the printing
sys.excepthook(type, value, traceback)
type: the exception class
value: the exception instance that wasn't handled
traceback: a trackeback object; the same as what is stored in sys.last_traceback
Read more
Suppose, I have a 3rd party library that I am not allowed to modify. Suppose, it is called Fabric, but that is important only to explain the symptoms.
The script processes a list of existing files to get them using fabric.operations.get, which in its turn calls fabric.sftp.SFTP.get. Using fabric.sftp.SFTP.get produced some Warning: get() encountered an exception while downloading ... Underlying exception: Permission denied. I noticed the implementation was too old, and swapped the implementation of that function for one that uses sudo to work around the Permission denied:
import fabric.sftp
def sftpget(....same args as in current implementation....):
...here I pasted fabric.sftp.SFTP.get from the Internet
# swapping the implementation
fabric.sftp.SFTP.get=sftpget
This worked in 99.999% of cases. But getting three files still results in the same error. I tried to see if that is caused by some other codepath, but the only place where that string is printed is in fabric.operations.get in except: clause (grepped /usr/lib/python2.6/site-packages/ for get() encountered an exception while downloading). I tried to swap that function for a implementation that will print the stack trace of the exception, but I still only get the Permission denied message, and no stack trace.
It looks like the function does not get swapped in this case.
What could be the reasons for some invocations to use the original fabric.operations.get (since I don't see the stack traces printed) (and possibly the unpatched fabric.sftp.SFTP.get, since it seems sudo fix is not being used - I did check manually that those operations can be done on those files)?
during import, before you replace the get function some other piece of code might save a reference to the get function, for example:
class a():
def __init__(self,getter):
self.getter=getter
b=a(sftp.SFTP.get)
class a would then still hold a reference to the old piece of code despite it being replaced by you in the namespace.
I am trying to do some parsing on some text. I have the binary version of the parser to in my program'd body,I use call to run this parser and pass my sentences one by one. However, for some reasons sometimes the parser is not able to parse the sentence and generates an error.It might be a bit difficult to put it into words, it just prints some error messages but does not crash and ends normally. My understanding is that there is some sort of exception handling done in the parser itself that it doesn't crash. However, I want to keep track of these problematic sentences. In other words, if the parser couldn't parse the sentence I want to write that sentence in a file. I used the normal exception handling as I do with all of my programs, but it seems it cannot catch exception as the exception has been taken care of inside the parser program. Does anyone know how I should catch these kind of external exception?
thanks
Check the return code of call. Is it any different when you get the exception compared to normal/correct execution? If you want to get an exception you could use check_call.
Another solution could be the usage of check_output to call the parser-program and examine the output.
Documentation for all functions: Python subprocess module