Catching SyntaxError from ast.literal_eval - python

I have the following code to evaluate some configuration values stored in a file:
from ast import literal_eval
for key, value in dict_read_from_file.items():
try:
cfg[key] = literal_eval(value)
except ValueError:
cfg[key] = value
However, with certain inputs literal_eval raises a SyntaxError, rather than a ValueError:
>>> literal_eval('asdf')
Traceback (most recent call last):
[...]
ValueError: malformed node or string on line 1: <ast.Name object at 0x000001A065B69AE0>
>>> literal_eval('123 4')
Traceback (most recent call last):
[...]
File "ast.py", line 50, in parse
return compile(source, filename, mode, flags,
File "<unknown>", line 1
123 4
^
SyntaxError: invalid syntax
But Python doesn't seem to let me catch the SyntaxError:
>>> try:
... literal_eval('123 4')
... except ValueError | SyntaxError:
... print("Fixing")
...
Traceback (most recent call last):
[...]
SyntaxError: invalid syntax
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 3, in <module>
TypeError: catching classes that do not inherit from BaseException is not allowed
How do I catch and properly handle this SyntaxError?

Related

Test exception chaining and traceback output with doctest

How to test a 'multi traceback' using doctest?
It seems that using several ELLIPSIS and <BLANKLINE> won't do the trick:
def myfunc():
"""
>>> myfunc()
Traceback (most recent call last):
...
ValueError: this is
<BLANKLINE>
The above exception was the direct cause of the following exception:
<BLANKLINE>
Traceback (most recent call last):
...
TypeError: it
"""
try:
raise ValueError('this is')
except ValueError as err:
raise TypeError('it') from err
import doctest
doctest.testmod(optionflags=doctest.REPORT_NDIFF|doctest.ELLIPSIS)
Result:
"test.py" 23L, 490C written
**********************************************************************
File "test.py", line 4, in __main__.myfunc
Failed example:
myfunc()
Differences (ndiff with -expected +actual):
Traceback (most recent call last):
- ...
+ File "test.py", line 17, in myfunc
+ raise ValueError('this is')
ValueError: this is
<BLANKLINE>
The above exception was the direct cause of the following exception:
<BLANKLINE>
Traceback (most recent call last):
- ...
+ File "/usr/lib/python3.7/doctest.py", line 1329, in __run
+ compileflags, 1), test.globs)
+ File "<doctest __main__.myfunc[0]>", line 1, in <module>
+ myfunc()
+ File "test.py", line 19, in myfunc
+ raise TypeError('it') from err
TypeError: it
**********************************************************************
1 items had failures:
1 of 1 in __main__.myfunc
***Test Failed*** 1 failures.
But if I squash all, it will pass:
>>> myfunc()
Traceback (most recent call last):
...
TypeError: it
I'm afraid that is not possible to check "multi tracebacks" in that way.
The problem is that doctest ignores everything except the exception class and its message.
In your example, it will be just:
TypeError: it
If you are interested in how this works, check the doctest.py and search for
exc_msg = traceback.format_exception_only(*exception[:2])[-1]
That "exc_msg" will only contain the raised exception's details:
TypeError: it
Alternatives
If it is possible, you could change your test to not raise any exception but print the wanted message.
Another possibility could be use another "doctest engine" like byexample. It works in the same way that doctest does but it's more flexible (quick overview here).
If you have a lot of tests, you may want to try its compatibility mode with doctest to avoid rewriting everything.
For your example, this should be:
"""
>>> from your_module import myfunc
"""
def myfunc():
"""
>>> myfunc()
Traceback (most recent call last):
...
ValueError: this is
<BLANKLINE>
The above exception was the direct cause of the following exception:
<BLANKLINE>
Traceback (most recent call last):
...
TypeError: it
"""
try:
raise ValueError('this is')
except ValueError as err:
raise TypeError('it') from err
And to run it you do from the shell:
byexample -l python -o '+py-doctest -py-pretty-print +ELLIPSIS' your_module.py
Disclaimer: I'm the author of byexample. I'm a really fan of doctest but I know that has its limitations and checking exceptions is one of them (specially if you work in a dual Python 2.x / 3.x project).
For that reason I created byexample: It is really useful to me and I really hope that it would be useful to others.
Ask me any question that you have, here or in github

What are the pros and cons of raising an error class vs. an instance of the class

The following appears to have the same effect:
>>> raise NotImplementedError
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NotImplementedError
>>> raise NotImplementedError()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NotImplementedError
Is there a difference, and if so, what are the pros and cons of each?
There is essentially no difference if the exception class requires no arguments for initialization:
If an exception class is passed, it will be implicitly instantiated by
calling its constructor with no arguments.
[Emphasis mine]
Otherwise, you'll get another exception complaining about the initialization of the instance:
class MyException(Exception):
def __init__(self, arg):
pass
raise MyException
Traceback (most recent call last):
File "python", line 6, in <module>
TypeError: __init__() takes exactly 2 arguments (1 given)
Apparently, you can do more by passing custom arguments to custom exception classes, or passing a custom message to a builtin exception class:
>>> raise ValueError('number must be 42')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: number must be 42
The above is much more informative (quite desirable from a users' perspective) than a barren ValueError.
There is no difference:
>>> try:
... raise NotImplementedError
... except Exception as e:
... pass
...
>>> e
NotImplementedError()
>>> type(e)
<type 'exceptions.NotImplementedError'>
>>> type(NotImplementedError)
<type 'type'>
>>> type(NotImplementedError())
<type 'exceptions.NotImplementedError'>
>>>
See section 8.4 of the docs: https://docs.python.org/3/tutorial/errors.html#raising-exceptions

Formatting exceptions as Python does

If I raise an Exception in Python, here's what I get:
raise Exception("Hello world")
Traceback (most recent call last):
File "<ipython-input-24-dd3f3f45afbe>", line 1, in <module>
raise Exception("Hello world")
Exception: Hello world
Note the last line that says Exception: Hello world. Given an Exception (foo = Exception("Hello world")), how can I produce text like this? None of the following work:
str(foo)
Out[27]: 'Hello world'
repr(foo)
Out[28]: "Exception('Hello world',)"
"{}".format(foo)
Out[29]: 'Hello world'
"{}: {}".format(type(foo), foo)
Out[30]: "<type 'exceptions.Exception'>: Hello world"
If your exception object is exc, then:
The part before the colon is type(exc).__name__.
The part after the colon is str(exc).
So you can just do this:
print('{}: {}'.format(type(exc).__name__, exc))
Making tdelaney's answer formal and demonstrating the difference...
Strings
#test.py
import traceback
try :
raise TypeError("Wrong Type baby!")
except Exception as e:
print( "EXCEPTION FORMAT PRINT:\n{}".format( e ) )
print( "EXCEPTION TRACE PRINT:\n{}".format( "".join(traceback.format_exception(type(e), e, e.__traceback__))
Resulting console output
EXCEPTION FORMAT PRINT:
Wrong Type baby!
EXCEPTION TRACE PRINT:
Traceback (most recent call last):
File "test.py", line 4, in <module>
raise TypeError("Wrong Type baby!")
TypeError: Wrong Type baby!
Logging
If you're in the context of logging there's also the exception method and exc_info kwarg that will do the formatting for you. We should note that the debug and info level log messages are ignored out on account of the root logger holding warning as its default log level.
# logTest.py
import logging
try :
raise ValueError("my bad value")
except Exception as e :
logging.exception( e )
logging.debug("\n{}\nDEBUG LEVEL EXAMPLE".format('-'*30), exc_info=e)
logging.info("\n{}\nINFO LEVEL EXAMPLE".format('-'*30), exc_info=e)
logging.warning("\n{}\nWARNING LEVEL EXAMPLE".format('-'*30), exc_info=e)
logging.error("\n{}\nERROR LEVEL EXAMPLE".format('-'*30), exc_info=e)
with the resulting console output...
ERROR:root:my bad value
Traceback (most recent call last):
File "/Users/me/logTest.py", line 5, in <module>
raise ValueError("my bad value")
ValueError: my bad value
WARNING:root:
------------------------------
WARNING LEVEL EXAMPLE
Traceback (most recent call last):
File "/Users/me/logTest.py", line 5, in <module>
raise ValueError("my bad value")
ValueError: my bad value
ERROR:root:
------------------------------
ERROR LEVEL EXAMPLE
Traceback (most recent call last):
File "/Users/me/logTest.py", line 5, in <module>
raise ValueError("my bad value")
ValueError: my bad value

Python print last traceback only?

Consider the following code and traceback:
>>> try:
... raise KeyboardInterrupt
... except KeyboardInterrupt:
... raise Exception
...
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
KeyboardInterrupt
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 4, in <module>
Exception
>>>
I'd like to print only the most recent traceback (the one in which Exception was raised).
How can this be achieved?
From the above example, I'd like to print the following, as if raise Exception had been called outside the except clause.
Traceback (most recent call last):
File "<stdin>", line 4, in <module>
Exception
The perfect question for me.
You can suppress the exception context, that is the first part of the traceback, by explicitly raising the exception from None:
>>> try:
raise KeyboardInterrupt
except:
raise Exception from None
Traceback (most recent call last):
File "<pyshell#4>", line 4, in <module>
raise Exception from None
Exception
This was formalized in PEP 409 and further improved in PEP 415. The original bug request for this was filed by myself btw.
Note that suppressing the context will not actually remove the context from the new exception. So you can still access the original exception:
try:
try:
raise Exception('inner')
except:
raise Exception('outer') from None
except Exception as e:
print(e.__context__) # inner

Custom python traceback or debug output

I have a traceback print and want to customize the last part of it.
What: The error occurred in another process and traceback lies there (as is the case in multiprocessing).
Problem: I want to have the full traceback and error report.
Similar to this code:
>>> def f():
g()
>>> def g():
raise Exception, Exception(), None ## my traceback here
>>> f()
Traceback (most recent call last):
File "<pyshell#14>", line 1, in <module>
f()
File "<pyshell#8>", line 2, in f
g()
File "<pyshell#11>", line 2, in g
raise Exception, Exception(), None ## my traceback starts here
my traceback appears here
my traceback appears here
Exception
Impossible "Solutions": subclass and mock-object
>>> from types import *
>>> class CostomTB(TracebackType):
pass
Traceback (most recent call last):
File "<pyshell#125>", line 1, in <module>
class CostomTB(TracebackType):
TypeError: Error when calling the metaclass bases
type 'traceback' is not an acceptable base type
>>> class CostomTB(object):
pass
>>> try: zzzzzzzzz
except NameError:
import sys
ty, err, tb = sys.exc_info()
raise ty, err, CostomTB()
Traceback (most recent call last):
File "<pyshell#133>", line 5, in <module>
raise ty, err, CostomTB()
TypeError: raise: arg 3 must be a traceback or None
I am using python 2.7.
I guess you want the full traceback stack. See this which is having very good examples python logging module.
If some confusion comes See the logging documentation.
You mentioned a separate process: if your problem is to capture the traceback in process A and show it in process B, as if the exception was actually raised in the latter, then I'm afraid there is no clean way to do it.
I would suggest to serialize the traceback in process A, send it to process B and from there raise a new exception that includes the former in its description. The result is a somewhat longer output, but it carries information about both processes stacks.
In the following example there aren't really two separate processes, but I hope it makes my point clearer:
import traceback, StringI
def functionInProcessA():
raise Exception('Something happened in A')
class RemoteException(Exception):
def __init__(self, tb):
Exception.__init__(self, "Remote traceback:\n\n%s" % tb)
def controlProcessB():
try:
functionInProcessA()
except:
fd = StringIO.StringIO()
traceback.print_exc(file=fd)
tb = fd.getvalue()
raise RemoteException(tb)
if __name__ == '__main__':
controlProcessB()
Output:
Traceback (most recent call last):
File "a.py", line 20, in <module>
controlProcessB()
File "a.py", line 17, in controlProcessB
raise RemoteException(tb)
__main__.RemoteException: Remote traceback:
Traceback (most recent call last):
File "a.py", line 12, in controlProcessB
functionInProcessA()
File "a.py", line 4, in functionInProcessA
raise Exception('Something happened in A')
Exception: Something happened in A

Categories