We Keep Coding

How to avoid duplicates when python multiple except executes the same method?

I have the code block like below:
try:
method()
except ErrorType1:
todo()
return
except ErrorType2 as e:
todo()
raise e
Basically for the two error types, I need to execute todo() first, then either return or raise e. Is it possible to just write todo() once? I was thinking using finally but don't think that actually works.

You could catch both exceptions in one except clause, execute todo and then decide to do based on the exception type:
try:
method()
except (ErrorType1, ErrorType2) as e:
todo()
if isinstance(e, ErrorType1):
return
raise
Note - as pointed out by #ShadowRanger in the comments to the question - you should just use raise to re-raise the existing exception, using raise e will raise a second copy of it, resulting in the traceback including the line with raise e on it as well as the line where the original error occurred.

If you have an common set of instructions (either encapsulated as a function or series of functions) that must be executed as part of an exception handling, consider using a context manager to encapsulate the common bits. The following two results in identical outcome, albeit with different construction (one using try..finally, the other using try..except).
from contextlib import contextmanager
#contextmanager
def context1(method):
print("starting context1")
completed = False
try:
yield method()
completed = True
finally:
if completed:
commit()
else:
rollback()
print("finished")
#contextmanager
def context2(method):
print("starting context2")
try:
yield method()
except Exception:
rollback()
raise
else:
commit()
print("finished")
The latter one will not be able to deal with KeyboardInterrupt or other exceptions that subclass off BaseException, so for certain use case this is not exactly ideal, though it is included to follow suite of the question. The first one is more of a response to the fact that you never appeared to have tried using finally, but rather simply thinking it does not actually works, and thus provided to show it can be used to achieve your goal (where only todo() in the question is executed if failure, through the use of a boolean variable).
In both cases, note how the common control flow is fully encapsulated inside the context manager, and usage is fairly straightforward like so such that all the unique extra cases can be done with another try..except block around the with context block.
try:
with context1(f) as result:
pass # or use the result to do something
except Exception: ...
# all the special unique cases be handled here.
To complete demo, more code is below; the commit and rollback functions I defined the following:
def commit():
print("commit")
def rollback():
print("rollback")
Now to test it, I defined the following helpers:
from functools import partial
class ErrorType1(Exception):
pass
class ErrorType2(Exception):
pass
def raise_e(e):
raise e
subject = [
object,
partial(raise_e, ErrorType1),
partial(raise_e, ErrorType2),
]
With the tests defined as such (replace context1 with context2 for the other demonstration):
for f in subject:
try:
with context1(f) as result:
print('done - got result %r' % result)
except ErrorType2:
print("ErrorType2 will be raised")
# raise # uncomment to actually re-raise the exception
except Exception as e:
print("Exception trapped: %r raised by %r" % (e, f))
Note the output of both the above should look about like so (aside from context1 vs context2):
starting context1
done - got result <object object at 0x7f20ccd3e180>
commit
finished
starting context1
rollback
Exception trapped: ErrorType1() raised by functools.partial(<function raise_e at 0x7f20ccb30af0>, <class '__main__.ErrorType1'>)
starting context1
rollback
ErrorType2 will be raised

Variable scope difference between Python versions [duplicate]

This question already has an answer here:
Why do I get a `NameError` (or `UnboundLocalError`) from using a named exception after the `except` block?
(1 answer)
Closed 3 years ago.
Consider the following piece of code:
import shutil
import time
t = time.time()
exception = None
while time.time() < (t + 10.0):
try:
shutil.rmtree('/path-to-non-existent-directory')
break
except OSError as exception:
pass
time.sleep(0.1)
else:
if exception:
raise exception
In Python 2.7 this code is perfectly valid but in Python 3.7 I get the following warning:
Local variable exception might be referenced before assignment
In the else clause.
Does anyone have an idea what is wrong with this snippet when run in Python 3.7?

In Python 3, to resolve a circular reference issue caused by the introduction of the __traceback__ attribute, an except target is automatically deleted at the end of an except block. It behaves as if you had written
except OSError as exception:
pass
del exception
This is documented in PEP 3110.
If you want to preserve the exception object, you should save it to a second variable:
except OSError as exception:
saved_exception = exception
exception will still be deleted, but you can use saved_exception to inspect the exception object after the except block is over.

Python is a block scoped language, you can't reference a variable outside of the block in which it was defined and you can't use a new value outside of the block in which that variable was updated.
In other words, you can't reference the exception variable's error data outside of the except block, if you try to do so, the value of the exception variable will be None (which you set at the top-level).
Try moving the contents of your else block to the except block and get rid of the exception = None and if exception, like this:
timer = Timer(10.0)
while timer.alive:
try:
shutil.rmtree(cls.workspace)
break
except OSError as exception:
raise exception
time.sleep(0.1)
If you don't want fatal errors, you could just use the print() function instead of the raise keyword:
timer = Timer(10.0)
while timer.alive:
try:
shutil.rmtree(cls.workspace)
break
except OSError as exception:
print(exception)
time.sleep(0.1)
Here is another example (which won't work):
def hello():
message = "Hello World"
print(message)
Because it'll raise the the following error:
NameError: name 'message' is not defined
NOTE: I'd advice against calling your exception exception, because there's an error class called Exception and doing so could lead to confusion later on.
Good luck.

Python: How to test the path which handles the exception?

Given this code:
def main(b):
try:
a = 2 / b
return "Hello World"
except Exception:
return "Exception"
How to test the path which handles the exception using patching?
Would it possible to make a = 2 / b as a valid target to patch?
Similar questions:
How to use Python Mock to raise an exception - but with Errno set to a given value
Mocking - How do I raise exception on the caller?
Mocking a function to raise an Exception to test an except block

Unable to pass assertRaises test in Python

So, I have the most trivial in the world example. This is my class to be tested:
# My_Class.py
class My_Class(object):
#staticmethod
def doit(name, params):
try:
raise Exception("This is my error message")
except Exception:
print("Exception: I raised Exception")
And this is the tester itself:
# test.py
import unittest
from My_Class import My_Class
class Test_MyClass(unittest.TestCase):
def setUp(self):
self.my_class = My_Class()
def test_my_class(self):
name = "Abrakadabra"
params = {}
self.assertRaises(Exception, self.my_class.doit, name, params)
And this is what I see in the console, when I'm running my test.py:
$ nosetests test.py
F
======================================================================
FAIL: test_my_class (test.Test_MyClass)
----------------------------------------------------------------------
Traceback (most recent call last):
File ....
nose.proxy.AssertionError: Exception not raised by doit
-------------------- >> begin captured stdout << ---------------------
Exception: I raised Exception
--------------------- >> end captured stdout << ----------------------
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (failures=1)
It is reaaly iteresting, because it is controversial. On the one hand the test says that "Exception not raised by doit", but one line below it clearly prints out a message from the Exception block. So, what I'm doing wrong here??? Thanks!

To directly answer your question, the reason why you are getting that message is because with this assertion:
self.assertRaises(Exception, self.my_class.doit, name, params)
You are testing to make sure an exception was raised. But your try/except suppresses this. If you actually remove your try/except your test will in fact pass, because now your method will raise.
Since you do not want to do this, what you should be doing instead is testing the behaviour of your method when an exception is raised. Ultimately, you want to make sure that your print method is called in your except. I have put together an example below to help understand this.
Keeping in mind what #user2357112 mentioned, which is very important to keep in mind when unittesting, here is an example to help expand on that to provide a practical use for what you are trying to do:
Let us just put together some method:
def some_method():
pass
We will now put this in to your staticmethod you defined as such:
# My_Class.py
class My_Class(object):
#staticmethod
def doit(name, params):
try:
some_method()
except Exception:
print("Exception: I raised Exception")
So now, when it comes to your unittesting, you want to test the behaviour of your method doit. With that in mind, what you will do in this case, is test that some_method will raise an exception and you will validate how your doit method behaves to that exception being raised.
At this point, I suggest taking a look at the documentation behind unittest and mock to get more familiar with what you can do with your testing, but here is an example using mock patching to test the behaviour of your code if an exception is being raised:
#patch('builtins.print')
#patch('__main__.some_method')
def test_my_class(self, m_some_method, m_print):
name = "Abrakadabra"
params = {}
# have the side_effect raise the exception when some_method is called in doit
m_some_method.side_effect = Exception()
self.my_class.doit(name, params)
# check to make sure you caught the exception by checking print was called
self.assertEqual(m_print.call_count, 1)
When you put it all together, the following is functional code that I ran on my end that you can play around with to understand what is happening:
def some_method():
pass
# My_Class.py
class My_Class(object):
#staticmethod
def doit(name, params):
try:
some_method()
except Exception:
print("Exception: I raised Exception")
# test.py
import unittest
from mock import patch
class Test_MyClass(unittest.TestCase):
def setUp(self):
self.my_class = My_Class()
#patch('builtins.print')
#patch('__main__.some_method')
def test_my_class(self, m_some_method, m_print):
name = "Abrakadabra"
params = {}
m_some_method.side_effect = Exception()
self.my_class.doit(name, params)
self.assertEqual(m_print.call_count, 1)
if __name__ == '__main__':
unittest.main()

assertRaises is an assertion about the function's visible behavior, not its internals. It asserts that the stated exception passes out of the function. Any exceptions that are handled inside the function are not assertRaises's concern.

assertRaises failed since there was actually no exception raised. Well, it was raised but handled with except inside the doit() method. The problem is here:
try:
raise Exception("This is my error message")
except Exception:
print("Exception: I raised Exception")
You are raising an exception and then catching it without re-raising. From a caller (assertRaises is the caller in your case) perspective, no errors were thrown during the function call. Re-raising an exception allows a caller to handle an exception as well. Put a raise after the print:
try:
raise Exception("This is my error message")
except Exception:
print("Exception: I raised Exception")
raise # re-raising
Also see Handling Exceptions.

Python custom exception and sub-exceptions

What is a good way of raising sub-exceptions (is that's the term)?
Scenario:
I want to raise a custom ConnectivityException when http or ftp exception occurs. Is there any way to raise ConnectivityException such that exceptions are categorized properly (i.e. I should be able to tell if ConnectivityException is raised because of http ot ftp)?

A standard technique would be to subclass ConnectivityException to create exception classes specific to each kind of error condition:
class ConnectivityException(Exception): pass
class HTTPConnectivityException(ConnectivityException): pass
class FTPConnectivityException(ConnectivityException): pass
Then instead of raise ConnectivityException you can use raise HTTPConnectivityException or raise FTPConnectivityException, depending on which specific type of error you want to indicate.
Multiple exception blocks can be used to dispatch error handling according to the exception type:
try:
some_network_operation()
except HTTPConnectivityException as ex:
# This will execute if the error is an HTTPConnectivityException.
except FTPConnectivityException as ex:
# Likewise for FTPConnectivityException.
except ConnectivityException as ex:
# The generic case; this block will execute if the ConnectivityException isn't
# an instance of one of the earlier specified subclasses.
Note that the exception-handling blocks are tried in lexical order; the first block specifying a class to which the exception object belongs will be used. In this case, that means that you need to put the ConnectivityException block last, or else it will catch HTTPConnectivityException and FTPConnectivityException as well.

you can add an attribute named 'source' to ConnectivityException, and set it to 'http' or 'ftp' according to specific situation, when catch ConnectivityException, check the source attribute and decide what to do
here i recommend another way which uses inherit class
class ConnectivityException(Exception):
pass # you can define some attributes and methods, here I just escape
class HTTPConnectivityException(ConnectivityException):
pass
class FTPConnectivityException(ConnectivityException):
pass
def func():
if some_condition:
raise HTTPConnectivityException()
if some_other_condition:
raise FTPConnectivityException()
def another_func():
try:
func()
except HTTPConnectivityException as e:
pass # you can do something here
except FTPConnectivityException as e:
pass