We Keep Coding

Pytest - Altering the outcome based on the exception error raised

I was wondering how I could alter a pytest test outcome (from a fail to a skip) in the case that my error message includes a specific string.
Occasionally we get test failures using appium where the response from the appium server is a 500 error with the failure message: "An unknown server-side error occurred while processing the command." Its an issue that we need to solve, but for the meantime we want to basically say, if the test failed because of an error message similar to that, skip the test instead of failing it.
Ive considered and tried something like this:
def pytest_runtest_setup(item):
excinfo = None
try:
item.obj()
except Exception as e:
excinfo = sys.exc_info()
if excinfo and "An unknown server-side error occurred while processing the command." in str(excinfo[1]):
pytest.skip("Skipping test due to error message")
And this obviously won't work.
But I was hoping for a similar approach.
The successful answer:
In order for me to get this working #Teejay pointed out below I needed to use the runtest_call hook and assess the message there. Currently working well in my test suite!
#pytest.hookimpl(hookwrapper=True)
def pytest_runtest_call(item):
output = yield
if output.excinfo:
for potential_error_message in expected_failure_messages.keys():
if output._excinfo[1].stacktrace and potential_error_message in output._excinfo[1].stacktrace:
pytest.skip(reason=expected_failure_messages[potential_error_message])

I recommend utilizing a hook wrapper to inspect the exception raised and act accordingly
#pytest.hookimpl(hookwrapper=True)
def pytest_runtest_setup(item):
output = yield
if output.excinfo:
# Additional logic to target specific error
pytest.skip()

Your sketched idea is close to workable. I'd skip using sys.exc_info() and just inspect the str() value of the exception, and I'd restrict the classes of exception caught to the smallest set that covers the failures you're trying to ignore.
Something like:
try:
item.obj()
except OSError as e: # ideally a narrower subclass or tuple of classes
if 'An unknown server-side error occurred while processing the command.' in str(e):
pytest.skip(f'Skipping test due to error message {e}')
else:
raise e
The only additional logical change I've made is moving the if/skip into the exception handler and re-raising if it doesn't match the message you're expecting.
Restricting the classes of errors matched is a best practice to avoid catching situations you didn't intend to catch. It's probably harmless to over-catch here where you're inspecting the message, just a good habit to cultivate. But it might also let you identify a specific field of the exception class to check rather than just the string representation - eg OSError has the strerror attribute containing the error message from the OS, so if you've limited your except block to catching just those you know you'll have that attribute available.
I chose to include the exception in the skip message, you might decide differently if they're uninformative.

How to catch when *any* error occurs in a whole script? [duplicate]

How do you best handle multiple levels of methods in a call hierarchy that raise exceptions, so that if it is a fatal error the program will exit (after displaying an error dialog)?
I'm basically coming from Java. There I would simply declare any methods as throws Exception, re-throw it and catch it somewhere at the top level.
However, Python is different. My Python code basically looks like the below.
EDIT: added much simpler code...
Main entry function (plugin.py):
def main(catalog):
print "Executing main(catalog)... "
# instantiate generator
gen = JpaAnnotatedClassGenerator(options)
# run generator
try:
gen.generate_bar() # doesn't bubble up
except ValueError as error:
Utilities.show_error("Error", error.message, "OK", "", "")
return
... usually do the real work here if no error
JpaAnnotatedClassGenerator class (engine.py):
class JpaAnnotatedClassGenerator:
def generate_bar(self):
self.generate_value_error()
def generate_value_error(self):
raise ValueError("generate_value_error() raised an error!")
I'd like to return to the caller with an exception that is to be thrown back to that ones call until it reaches the outermost try-except to display an error dialog with the exception's message.
QUESTION:
How is this best done in Python? Do I really have to repeat try-except for every method being called?
BTW: I am using Python 2.6.x and I cannot upgrade due to being bound to MySQL Workbench that provides the interpreter (Python 3 is on their upgrade list).

If you don't catch an exception, it bubbles up the call stack until someone does. If no one catches it, the runtime will get it and die with the exception error message and a full traceback. IOW, you don't have to explicitely catch and reraise your exception everywhere - which would actually defeat the whole point of having exceptions. Actually, despite being primarily used for errors / unexpected conditions, exceptions are first and foremost a control flow tool allowing to break out of the normal execution flow and pass control (and some informations) to any arbitrary place up in the call stack.
From this POV your code seems mostlt correct (caveat: I didn't bother reading the whole thing, just had a quick look), except (no pun indented) for a couple points:
First, you should define your own specific exception class(es) instead of using the builtin ValueError (you can inherit from it if it makes sense to you) so you're sure you only catch the exact exceptions you expect (quite a few layers "under" your own code could raise a ValueError that you didn't expect).
Then, you may (or not, depending on how your code is used) also want to add a catch-all top-level handler in your main() function so you can properly log (using the logger module) all errors and eventually free resources, do some cleanup etc before your process dies.
As a side note, you may also want to learn and use proper string formatting, and - if perfs are an issue at least -, avoid duplicate constant calls like this:
elif AnnotationUtil.is_embeddable_table(table) and AnnotationUtil.is_secondary_table(table):
# ...
elif AnnotationUtil.is_embeddable_table(table):
# ...
elif AnnotationUtil.is_secondary_table(table):
# ...
Given Python's very dynamic nature, neither the compiler nor runtime can safely optimize those repeated calls (the method could have been dynamically redefined between calls), so you have to do it yourself.
EDIT:
When trying to catch the error in the main() function, exceptions DON'T bubble up, but when I use this pattern one level deeper, bubbling-up seems to work.
You can easily check that it works correctly with a simple MCVE:
def deeply_nested():
raise ValueError("foo")
def nested():
return deeply_nested()
def firstline():
return nested()
def main():
try:
firstline()
except ValueError as e:
print("got {}".format(e))
else:
print("you will not see me")
if __name__ == "__main__":
main()
It appears the software that supplies the Python env is somehow treating the main plugin file in a wrong way. Looks I will have to check the MySQL Workbench guys
Uhu... Even embedded, the mechanism expection should still work as expected - at least for the part of the call stack that depends on your main function (can't tell what happens upper in the call stack). But given how MySQL treats errors (what about having your data silently truncated ?), I wouldn't be specially suprised if they hacked the runtime to silently pass any error in plugins code xD

It is fine for errors to bubble up
Python's exceptions are unchecked, meaning you have no obligation to declare or handle them. Even if you know that something may raise, only catch the error if you intend to do something with it. It is fine to have exception-transparent layers, which gracefully abort as an exception bubbles through them:
def logged_get(map: dict, key: str):
result = map[key] # this may raise, but there is no state to corrupt
# the following is not meaningful if an exception occurred
# it is fine for it to be skipped by the exception bubbling up
print(map, '[%s]' % key, '=>', result)
return result
In this case, logged_get will simply forward any KeyError (and others) that are raised by the lookup.
If an outer caller knows how to handle the error, it can do so.
So, just call self.create_collection_embeddable_class_stub the way you do.
It is fine for errors to kill the application
Even if nothing handles an error, the interpreter does. You get a stack trace, showing what went wrong and where. Fatal errors of the kind "only happens if there is a bug" can "safely" bubble up to show what went wrong.
In fact, exiting the interpreter and assertions use this mechanism as well.
>>> assert 2 < 1, "This should never happen"
Traceback (most recent call last):
File "<string>", line 1, in <module>
AssertionError: This should never happen
For many services, you can use this even in deployment - for example, systemd would log that for a Linux system service. Only try to suppress errors for the outside if security is a concern, or if users cannot handle the error.
It is fine to use precise errors
Since exceptions are unchecked, you can use arbitrary many without overstraining your API. This allows to use custom errors that signal different levels of problems:
class DBProblem(Exception):
"""Something is wrong about our DB..."""
class DBEntryInconsistent(DBProblem):
"""A single entry is broken"""
class DBInconsistent(DBProblem):
"""The entire DB is foobar!"""
It is generally a good idea not to re-use builtin errors, unless your use-case actually matches their meaning. This allows to handle errors precisely if needed:
try:
gen.generate_classes(catalog)
except DBEntryInconsistent:
logger.error("aborting due to corrupted entry")
sys.exit(1)
except DBInconsistent as err:
logger.error("aborting due to corrupted DB")
Utility.inform_db_support(err)
sys.exit(1)
# do not handle ValueError, KeyError, MemoryError, ...
# they will show up as a stack trace

inform the user if a certain exception is raised while running unittest

I have been writing unittests. Although there must be a way, I find it exceptionally difficult to get any print statement in a unittest class method to print anything, but can see what is going on when unittest shows me the diffs.
Have previously tested for exceptions with assertRaises as described here, but this is about detecting not asserting.
The tests assume a certain hardware device is plugged into the USB port when they run. The program being tested raises the following if someone tries to run it without the device connected:
RuntimeError('Device not reachable. Please connect it to USB')
I just tested sys.exit(0) inside a unittest method and that didn't exit either. Still, exiting is not that important, is probably better if all the tests fail, but it would be polite if the first test detects that Exception then prints a message to the user such as "The device needs to be connected before running unittests.". Does anyone know how to achieve this?
It is the standard class as described in the docs inheriting from unittest.TestCase with a setUp and tearDown method and the tests in between.

There are a few solutions you could try:
log and raise your RuntimeError in the setUpClass method
Use the skipUnless decorator
Intercept every error raised to test its type and log if it is a RuntimeError (see below)
To intercept every exception raised:
SYS_EXCEPT_HOOK = sys.excepthook
def _excepthook(typ, value, trace):
if typ is RuntimeError:
print("{}\n{}\n{}".format(typ.__name__, value, ''.join(traceback.format_tb(trace))))
SYS_EXCEPT_HOOK(typ, value, trace)
sys.excepthook = _excepthook

Handling Exceptions with else clause

The Python Tutorial states:
The try ... except statement has an optional else clause, which,
when present, must follow all except clauses. It is useful for code
that must be executed if the try clause does not raise an exception.
For example:
for arg in sys.argv[1:]:
try:
f = open(arg, 'r')
except IOError:
print 'cannot open', arg
else:
print arg, 'has', len(f.readlines()), 'lines'
f.close()
The use of the else clause is better than adding additional code
to the try clause because it avoids accidentally catching an exception
that wasn’t raised by the code being protected by the try ... except
statement.
Question 1>　Ａｆｔｅｒ reading the above document, I still don't get the idea why we cannot simply move the code from else clause into try clause.
Question 2> How does try clause can accidentally catch an exception since all those catches are done in the except clause, right?

You could put the else code in the try suite, but then you'd catch any exceptions that might be raised there. If you didn't intend that to happen, it would be "accidental," hence the wording of the document you linked to.
Best practice is to put as little code as possible in a try block, so that when an error occurs, you know what operation caused it and can handle it appropriately. If you have five lines of code in a try block and only expect one of them to ever raise an exception, your exception-handling code will be ill-prepared when an exception occurs in a line you didn't expect it to. Better in that case to let the exception be raised than handle it the wrong way.

If you move the code from the else into the try then that becomes part of the "critical path" which can raise an exception. If f.readlines() raises some sort of exception (perhaps an I/O error while reading the file because of a bad sector on the disk) then that error will be conflated with the one error that you currently catch. (Technically the "cannot open" error message would be wrong at that point ... because opening a file can succeed when reading it later fails; in fact opening it must succeed before you can even get an I/O error while processing it).
Normally you'd use a pattern more like:
foo = None
try:
# some code to access/load/initialize foo from an external source
except ...:
# handle various types of file open/read, database access, etc errors
else:
foo = something
... so that your subsequently run code and simply check if foo is None and use it or
work around it's unavailability in whatever way you see fit.

Answer for both questions is similar,
If you move code to the try clause, then you can not be sure from where the exception is coming from. Thus if you have another line of code that produces an unexpected IOError you could end searching for a problem where there is not.
So to better disect your code you want to simplify as much as possible the lines in the try making the catch as especific as possible.

1) Of course you could just move code from the else clause into the try clause. You could move it outside of the try block entirely, but this allows extra flexibility, and further modulation of the code. Also, perhaps specificity with the errors being caught. You could list a whole load of different exceptions that might be likely to happen, each with different statements. The stuff in the else clause would still only happen if no exception was raised, after the execution of the final line in the try block. e.g. printing a successful return message.
Also, try clauses add some extra CPU overhead with regards to garbage collection management and error tracing, so anything outside of the try clause is not protected in the same manner and may run more efficiently.
2) Error catching is quite specific. e.g. The except clause in your above example will only be run if an IOError is raised when running the f = open(arg,'r') line. If you want to catch any form of exception, use except Exception:.

Python Ignore Exception and Go Back to Where I Was

I know using below code to ignore a certain exception, but how to let the code go back to where it got exception and keep executing? Say if the exception 'Exception' raises in do_something1, how to make the code ignore it and keep finishing do_something1 and process do_something2? My code just go to finally block after process pass in except block. Please advise, thanks.
try:
do_something1
do_something2
do_something3
do_something4
except Exception:
pass
finally:
clean_up
EDIT:
Thanks for the reply. Now I know what's the correct way to do it. But here's another question, can I just ignore a specific exception (say if I know the error number). Is below code possible?
try:
do_something1
except Exception.strerror == 10001:
pass
try:
do_something2
except Exception.strerror == 10002:
pass
finally:
clean_up
do_something3
do_something4

There's no direct way for the code to go back inside the try-except block. If, however, you're looking at trying to execute these different independant actions and keep executing when one fails (without copy/pasting the try/except block), you're going to have to write something like this:
actions = (
do_something1, do_something2, #...
)
for action in actions:
try:
action()
except Exception, error:
pass

update. The way to ignore specific exceptions is to catch the type of exception that you want, test it to see if you want to ignore it and re-raise it if you dont.
try:
do_something1
except TheExceptionTypeThatICanHandleError, e:
if e.strerror != 10001:
raise
finally:
clean_up
Note also, that each try statement needs its own finally clause if you want it to have one. It wont 'attach itself' to the previous try statement. A raise statement with nothing else is the correct way to re-raise the last exception. Don't let anybody tell you otherwise.
What you want are continuations which python doesn't natively provide. Beyond that, the answer to your question depends on exactly what you want to do. If you want do_something1 to continue regardless of exceptions, then it would have to catch the exceptions and ignore them itself.
if you just want do_something2 to happen regardless of if do_something1 completes, you need a separate try statement for each one.
try:
do_something1()
except:
pass
try:
do_something2()
except:
pass
etc. If you can provide a more detailed example of what it is that you want to do, then there is a good chance that myself or someone smarter than myself can either help you or (more likely) talk you out of it and suggest a more reasonable alternative.

This is pretty much missing the point of exceptions.
If the first statement has thrown an exception, the system is in an indeterminate state and you have to treat the following statement as unsafe to run.
If you know which statements might fail, and how they might fail, then you can use exception handling to specifically clean up the problems which might occur with a particular block of statements before moving on to the next section.
So, the only real answer is to handle exceptions around each set of statements that you want to treat as atomic

you could have all of the do_something's in a list, and iterate through them like this, so it's no so wordy. You can use lambda functions instead if you require arguments for the working functions
work = [lambda: dosomething1(args), dosomething2, lambda: dosomething3(*kw, **kwargs)]
for each in work:
try:
each()
except:
pass
cleanup()

Exceptions are usually raised when a performing task can not be completed in a manner intended by the code due to certain reasons. This is usually raised as exceptions. Exceptions should be handled and not ignored. The whole idea of exception is that the program can not continue in the normal execution flow without abnormal results.
What if you write a code to open a file and read it? What if this file does not exist?
It is much better to raise exception. You can not read a file where none exists. What you can do is handle the exception, let the user know that no such file exists. What advantage would be obtained for continuing to read the file when a file could not be opened at all.
In fact the above answers provided by Aaron works on the principle of handling your exceptions.

I posted this recently as an answer to another question. Here you have a function that returns a function that ignores ("traps") specified exceptions when calling any function. Then you invoke the desired function indirectly through the "trap."
def maketrap(*exceptions):
def trap(func, *args, **kwargs):
try:
return func(*args, **kwargs)
except exceptions:
return None
return trap
# create a trap that ignores all exceptions
trapall = maketrap(Exception)
# create a trap that ignores two exceptions
trapkeyattrerr = maketrap(KeyError, AttributeError)
# Now call some functions, ignoring specific exceptions
trapall(dosomething1, arg1, arg2)
trapkeyattrerr(dosomething2, arg1, arg2, arg3)
In general I'm with those who say that ignoring exceptions is a bad idea, but if you do it, you should be as specific as possible as to which exceptions you think your code can tolerate.

Python 3.4 added contextlib.suppress(), a context manager that takes a list of exceptions and suppresses them within the context:
with contextlib.suppress(IOError):
print('inside')
print(pathlib.Path('myfile').read_text()) # Boom
print('inside end')
print('outside')
Note that, just as with regular try/except, an exception within the context causes the rest of the context to be skipped. So, if an exception happens in the line commented with Boom, the output will be:
inside
outside