We Keep Coding

Why does my message for exception not change with my if statement

I currently have the following code which throws an error on exception just how i want:
try:
something ....
except Exception as e:
print(
'You have encountered the following in the main function \n ERROR: {}'.format(e))
However in some cases if I get a specific exception such as:
invalid literal for int() with base 10: ''
I want to change the message of e in the exception to what i want.. how would i go about this?
If e == "invalid literal for int() with base 10: ''":
e = 'my new message'
print(e)
but it doesnt seem to be working

Try catching the type of error instead of parsing the text of the error.
More info can be found at Handling Exceptions section of Python help but to be fully thorough (because I feel dumb for initially answering a Python question in C#) you can sort out what exception type you're looking for with something like this:
>>> # Create the error
>>> int('3.6')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: invalid literal for int() with base 10: '3.6'
Where ValueError is the error type you need to catch.
More realistically, you can incorporate figuring it your uncaught error types into your program and (hopefully) identify them during testing:
>>> try:
... # something ....
... int('3.6') # for the example, we'll generate error on purpose
... # Assume we've already figured out what to do with these 3 errors
... except (RuntimeError, TypeError, NameError):
... print("We know what to do with these errors")
... # Our generic except to catch unhandled errors.
... except:
... print("Unhandled error: {0}".format(err))
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
ValueError: invalid literal for int() with base 10: '3.6'
Once you identify a new error type, add a specific handler for it:
>>> try:
... # something ....
... int('3.6')
... except (RuntimeError, TypeError, NameError):
... print("We know what to do with these errors")
... # The newly added handler for ValueError type
... except ValueError:
... print("And now we know what to do with a ValueError")
... print("My new message")
... except:
... print("Unhandled error: {0}".format(err))
And now we know what to do with a ValueError
My new message
Original (completely useless) answer kept here for posterity (and so the comments make sense)...
Try catching the type of error instead of parsing the text of the error.
e.g.
catch (FileNotFoundException e)
{
// FileNotFoundExceptions are handled here.
}
catch (IOException e)
{
// Extract some information from this exception, and then
// throw it to the parent method.
if (e.Source != null)
Console.WriteLine("IOException source: {0}", e.Source);
throw;
}
which is copied directly from here:
Microsoft try-catch (C# Reference)

print help text in try-exception in python

in a try-exception block in python, like shown below, I want my help message to be printed, instead of python's own error message. Is this possible?
def genpos(a):
''' Generate POSCAR :
Some error message'''
try:
tposcar = aio.read(os.path.join(root,"nPOSCAR"))
cell = (tposcar.get_cell())
cell[0][0] = a
cell[1][1] = math.sqrt(3)*float(a)
tposcar.set_cell(cell, scale_atoms=True)
aio.write("POSCAR", tposcar, direct=True)
except:
help(genpos)
sys.exit()
So, say, when this code is called without an argument, I want to get "Generate POSCAR : Some error message" instead of, python's
Traceback (most recent call last):
File "submit.py", line 41, in <module>
main()
File "submit.py", line 36, in __init__
ase_mod.genpos()
TypeError: genpos() missing 1 required positional argument: 'a'

You can define a new exception:
class CustomError(Exception): pass
raise CustomError('Generate POSCAR : Some error message')
Although, the error you're receiving has nothing to do with the try-except statement. Instead, your gen_pos() function is missing an argument.

Error while testing the raise of self-defined exceptions (using assertRaises())

I am creating tests for a python project. The normal tests work just fine, however I want to test if in a certain condition my function raises a self-defined exception. Therefor I want to use assertRaises(Exception, Function). Any ideas?
The function that raises the exception is:
def connect(comp1, comp2):
if comp1 == comp2:
raise e.InvalidConnectionError(comp1, comp2)
...
The exception is:
class InvalidConnectionError(Exception):
def __init__(self, connection1, connection2):
self._connection1 = connection1
self._connection2 = connection2
def __str__(self):
string = '...'
return string
The test method is the following:
class TestConnections(u.TestCase):
def test_connect_error(self):
comp = c.PowerConsumer('Bus', True, 1000)
self.assertRaises(e.InvalidConnectionError, c.connect(comp, comp))
However I get the following error:
Error
Traceback (most recent call last):
File "C:\Users\t5ycxK\PycharmProjects\ElectricPowerDesign\test_component.py", line 190, in test_connect_error
self.assertRaises(e.InvalidConnectionError, c.connect(comp, comp))
File "C:\Users\t5ycxK\PycharmProjects\ElectricPowerDesign\component.py", line 428, in connect
raise e.InvalidConnectionError(comp1, comp2)
InvalidConnectionError: <unprintable InvalidConnectionError object>

assertRaises expects to actually perform the call. Yet, you already perform it by yourself, thereby throwing the error before assertRaises actually executes.
self.assertRaises(e.InvalidConnectionError, c.connect(comp, comp))
# run this ^ with first static argument ^ and second argument ^ from `c.connect(comp, comp)`
Use either of those instead:
self.assertRaises(e.InvalidConnectionError, c.connect, comp, comp)
with self.assertRaises(e.InvalidConnectionError):
c.connect(comp, comp)

properly logging unicode & utf-8 exceptions in python 2

I'm trying to log various exceptions from libraries in python 2.7. I find that sometimes the exceptions contain a unicode string and sometimes a utf8 bytestring. I thought that logging.exception(e) was the right approach to log them, but the following doesn't seem to work:
# encoding: utf-8
import logging
try:
raise Exception('jörn')
except Exception as e:
logging.exception(e)
try:
raise Exception(u'jörn')
except Exception as e:
logging.exception(e)
saving this into a file and running it results in this:
$ python test.py
ERROR:root:jörn
Traceback (most recent call last):
File "test.py", line 4, in <module>
raise Exception('jörn')
Exception: jörn
Traceback (most recent call last):
File "/usr/local/Cellar/python/2.7.10/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/__init__.py", line 859, in emit
msg = self.format(record)
File "/usr/local/Cellar/python/2.7.10/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/__init__.py", line 732, in format
return fmt.format(record)
File "/usr/local/Cellar/python/2.7.10/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/__init__.py", line 474, in format
s = self._fmt % record.__dict__
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 1: ordinal not in range(128)
Logged from file test.py, line 12
So as you see the utf8 exception worked fine, but the unicode exception broke logging, swallowing the real exception and hiding it behind a UnicodeEncodeError.
Is there some standard logging facility for exceptions that won't break my code? What am i missing?

Actually, i think i finally found the mistake and a proper way myself: I seem to have used logging.exception('msg') wrong the whole time. You're not meant to pass in the exception, but a message:
# encoding: utf-8
import logging
try:
raise Exception('jörn')
except Exception as e:
logging.exception('exception occurred')
try:
raise Exception(u'jörn')
except Exception as e:
logging.exception('exception occurred')
running the above correctly logs the exception:
$ python test.py
ERROR:root:exception occurred
Traceback (most recent call last):
File "test.py", line 4, in <module>
raise Exception('jörn')
Exception: jörn
ERROR:root:exception occurred
Traceback (most recent call last):
File "test.py", line 10, in <module>
raise Exception(u'jörn')
Exception: j\xf6rn
The reason why logging.exception(e) seems to fail is that it passes the exception e up into logging.Formatter.format() where the it arrives as record.message variable which still is an Exception object.
Then in Line 474 the following happens:
s = self._fmt % record.__dict__
which is equivalent to the following:
s = '%(levelname)s:%(name)s:%(message)s' % {
'levelname': 'ERROR',
'name': 'ROOT',
'message': Exception(u'jörn')
}
It turns out this is why if message is one of ['jörn', u'jörn', Exception('jörn')] it works and not if it is Exception(u'jörn'):
>>> 'foo %s' % 'jörn'
'foo j\xc3\xb6rn'
>>> 'foo %s' % u'jörn'
u'foo j\xf6rn'
>>> 'foo %s' % Exception('jörn')
'foo j\xc3\xb6rn'
>>> 'foo %s' % Exception(u'jörn')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 1: ordinal not in range(128)
As you can see there is an automatic upcasting that happens for unicode strings which is why the following works:
>>> logging.error('jörn')
ERROR:root:jörn
>>> logging.error(u'jörn')
ERROR:root:jörn
This transformation into unicode fails when attempting it with an Exception object that doesn't properly handle the encoding of its message (which sadly seems to be the case in a lot of libraries).
The logging.exception(msg) call seems to properly use the repr() for formatting the exception for logging and prefixes it with your msg. So if you don't make the mistake and pass the exception to logging.exception it will correctly log it.
So long story short:
Don't use logging.exception(e) but logging.exception('exception occurred'). It will automagically and correctly append the formatted exception to your log. If you really want to use the exception's message without assuming some encoding, the safest you can do is logging.exception(repr(e)).

It's not the logging that fails to deal with unicode, it's the Exception.__str__ method which does not support unicode strings as exception arguments. When you invoke logging.exception(e) it will do something like logging.exception(str(e)) which in turn does something like str(self.args) on the exception instance. That's where the error comes from, your self.args is a unicode string which cannot be encoded in ascii.
You have two options, either do logging.exception(unicode(e)) or implement your own exception class that provides a __str__ method that can deal with unicode objects in self.args.
The reason why your first test passes is that the editor encodes the string into UTF-8 and Python sees a string instance with encoded unicode characters.

There's a lot of ways of doing, but most simple is to "patch" logger in order for it to pre-convert any non unicode string so logging won't fail, regardless of source, which is quite useful since you can't control sources like tracebacks, when using something like logger.error(exc_info=True)
Patching sounds bad, but fortunately there is a mechanism called a contextfilter that does the job without modifying the code.
TL;DR: I have built the following code as a package Tested with all Python versions from 2.7 to 3.10. Use it with pip install ofunctions.logger_utils
Usage:
from ofunctions.logger_utils import logger_get_logger
logger = logger_get_logger(log_file='somepath')
logger.info('Ûnicode café")
TL;DR end
Consider the following code which sole purpose is to call function safe_string_convert() on a given logged message:
class FixPython2Logging(logging.Filter):
def __init__(self):
if sys.version_info[0] < 3:
# pylint: disable=E1003 (bad-super-call)
super(logging.Filter, self).__init__()
else:
super().__init__()
def filter(self, record):
# type: (str) -> bool
# Fix python2 unicodedecodeerrors when non unicode strings are sent to logger
if sys.version_info[0] < 3:
record.msg = safe_string_convert(record.msg)
return True
Of course we'd have to design a fool proof (or unicode decode error proof) function that makes sure we'll have a string that can be logged:
def safe_string_convert(string):
"""
Allows to encode strings for hacky UTF-8 logging in python 2.7
"""
try:
return string.decode("utf8")
except Exception: # noqa
try:
return string.decode("unicode-escape")
except Exception: # noqa
try:
return string.decode("latin1")
except Exception: # noqa
if sys.version_info[0] < 3:
# pylint: disable=E0602 (undefined-variable)
if isinstance(string, unicode): # noqa
return string
try:
return (
b"Cannot convert logged string. Passing it as binary blob: "
+ bytes(string)
)
except Exception: # noqa
return string
Now we can make sure FixPython2Logging filter will get executed by any logger call. Let's just add it as filter to our standard logger class:
log_filter = FixPython2Logging()
logger = logging.getLogger()
# Remove earlier handlers if exist
while _logger.handlers:
_logger.handlers.pop()
# Add context filter
logger.addFilter(log_filter)
# Try it
logger.info("Ûnicode café")
Here we go ;)
Of course the contextfilter works for your initial logger.exception() question ;)

implementing a deferred exception in Python

I would like to implement a deferred exception in Python that is OK to store somewhere but as soon as it is used in any way, it raises the exception that was deferred. Something like this:
# this doesn't work but it's a start
class DeferredException(object):
def __init__(self, exc):
self.exc = exc
def __getattr__(self, key):
raise self.exc
# example:
mydict = {'foo': 3}
try:
myval = obtain_some_number()
except Exception as e:
myval = DeferredException(e)
mydict['myval'] = myval
def plus_two(x):
print x+2
# later on...
plus_two(mydict['foo']) # prints 5
we_dont_use_this_val = mydict['myval'] # Always ok to store this value if not used
plus_two(mydict['myval']) # If obtain_some_number() failed earlier,
# re-raises the exception, otherwise prints the value + 2.
The use case is that I want to write code to analyze some values from incoming data; if this code fails but the results are never used, I want it to fail quietly; if it fails but the results are used later, then I'd like the failure to propagate.
Any suggestions on how to do this? If I use my DeferredException class I get this result:
>>> ke = KeyError('something')
>>> de = DeferredException(ke)
>>> de.bang # yay, this works
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 6, in __getattr__
KeyError: 'something'
>>> de+2 # boo, this doesn't
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +: 'DeferredException' and 'int'

Read section 3.4.12 of the docs, "Special method lookup for new-style classes." It explains exactly the problem you have encountered. The normal attribute lookup is bypassed by the interpreter for certain operators, such as addition (as you found out the hard way). Thus the statement de+2 in your code never calls your getattr function.
The only solution, according to that section, is to insure that "the special method must be set on the class object itself in order to be consistently invoked by the interpreter."
Perhaps you'd be better off storing all your deferred exceptions in a global list, wrapping your entire program in a try:finally: statement, and printing out the whole list in the finally block.