We Keep Coding

Python: Best practice to report an error during closing a resource

What is the best practice in Python for reporting an error which occurs when closing a resource?
In particular, if I implement __enter__ and __exit__, I can use
with my_library.connect(endpoint) as connection:
connection.store_data("...")
But what should I do if closing my connection and persisting the changes fails, e.g. due to a network outage? I know that I could technically raise an error from within __exit__ but is this best practice/idiomatic in Python? Or should I, e.g., provide a separate persistChanges method, let the __exit__ swallow all errors, and then write in the documentation "if you don't call persistChanges you might lose your changes in error cases"?
My specific use case is: I am providing a Python API to other developers, and I wonder how to handle this case of "error on closing the resource" such that my API follows Python best practices/meets the expectations of Python devs using my library.

I would recommend making a custom error/warning class for your library. this can be very, very simple. There already exists a set of built-in exceptions that you can extend from. Based on your description above, I would recommend extending the RuntimeError like this:
class MyLibraryConnectionError(RuntimeError):
pass
or, if you want to only throw a warning, using the ResourceWarning like this:
class MyLibraryConnectionWarning(ResourceWarning):
pass
There is also the RuntimeWarning that could be extended for similar effect.
If you feel that ResourceWarning, RuntimeWarning, and RuntimeError don't accurately describe the exception, you can also just have them inherit directly from Exception or Warning, depending on whether you want them to only be flagged in Developer mode(Warning), or if you want the full exception functionality.
You can throw these like any other exception:
throw MyLibraryConnectionError("The underlying resource failed to close")
throw MyLibraryConnectionWarning("The underlying resource failed to close")
or even catch your dependency's thrown exception:
def __exit__(...):
try:
# potentially dangerous code with connections
underlyingConnection.close()
except TheErrorYouSeeThrown as e: # you can probably make this Exception instead of TheErrorYouSeeThrown. The more specific the better, so you don't accidentally catch random errors you didn't mean to.
throw MyLibraryConnectionError(e.message) # or whatever this needs to be, depending on the thrown exception type
Then your users could implement like so:
try:
with my_library.connect(endpoint) as connection:
connection.store_data("...")
except my_library.MyLibraryConnectionError as e:
# handle appropriately, or ignore
except Exception as e:
# Handle other errors that happen, that your library doesn't cause.

Is there a use case for an alternative "exception cause" in Python?

Background
In Python it is possible to suppress the context of an Exception if you raise your Exception from None. PEP 409 describes the rationale behind it. Sometimes you want to show only one meaningful (custom) Exception. With PEP 415 the implementation changes with the following argument:
The main problem with this scheme [from PEP 409] is it complicates the role of __cause__. __cause__ should indicate the cause of the exception not whether __context__ should be printed or not. This use of __cause__ is also not easily extended in the future. For example, we may someday want to allow the programmer to select which of __context__ and __cause__ will be printed.
Question
PEP 419 talks about future use cases. Are there any valid use cases right now (Python 3.3+) for using an alternative exception cause? For example, consider the following code:
class CustomError(BaseException):
pass
class StupidError(BaseException):
def __init__(self, message='This is just a stupid error.'):
super(StupidError, self).__init__(message)
self.message = message
try:
value = int('a')
except Exception:
raise CustomError('Custom error message') from StupidError
Output:
StupidError: This is just a stupid error.
The above exception was the direct cause of the following exception:
Traceback (most recent call last): (...)
CustomError: Custom error message
Are there any real use cases in which you want to hide the ValueError but show the StupidError? I mean you could give some relevant information in the StupidError which are not present in a mere ValueError? Maybe I am just overthinking this whole thing.

Sure. Say you're writing a web app and need to hide sensitive/complicated data from the user.
Say a user tries a request, but the web app backend has trouble reading the necessary data from a MySQL database (for whatever reason). Instead of letting the MySQL module raise the error, which could either expose sensitive information about how the app works internally or just simply confuse the user, I would want to catch it and then throw my custom exception (let's call it serverError). That custom exception would show the user a HTTP 500 page as well as report the error internally so it can be analyzed by a developer to figure out what went wrong and how to prevent it.
This means I only have to write my general error handling code once, and then when I catch an error, I can raise serverError, which takes care of the error reporting for me.

How can I inspect a ValidationError exception while testing with django TestCase?

I'm working with Django testing using a django.test.TestCase and I would like to know the pythonic way to inspect a validation error. I've come up with the below, which works, but it feels a little long winded and prone to error over a large number of tests
self.valid_account.contact_number = "1234567" # Too short
with self.assertRaises(ValidationError):
try:
self.valid_account.full_clean()
self.fail("Did not raise validation error")
except ValidationError as e:
self.assertTrue('contact_number' in e.message_dict)
raise e
Is there a better way to inspect and test an exception?

There's no point in using assertRaises if you're manually handling the exception. If you need to inspect the exception too, the solution is documented:
The context manager will store the caught exception object in its exception attribute. This can be useful if the intention is to perform additional checks on the exception raised
So in your case this would look like:
self.valid_account.contact_number = "1234567" # Too short
with self.assertRaises(ValidationError) as cm:
self.valid_account.full_clean()
self.assertTrue('contact_number' in cm.exception.message_dict)

what is error parameter in flask errorhandler

I have a question about flask error handler.
When I want to handle 404 error I use this code:
#app.errorhandler(404)
def page_not_found(e):
return render_template("404.html")
Why I should pass the (e) to the function?
Thanks! :)

e is the exception raised, triggering the handler to be called.
You can register the same error handling function for multiple error codes, and you can use that argument passed in to determine exactly for what error it was called or use that code in a generic template:
#application.errorhandler(404)
#application.errorhandler(401)
#application.errorhandler(500)
def http_error_handler(error):
return render_template("error.html", error=error)
From the Error Handlers documentation:
An error handler is a function, just like a view function, but it is called when an error happens and is passed that error.
Bold emphasis mine.
Note that it is an exception instance; for HTTP error codes, that'll be an instance of a subclass of the HTTPException class (Werkzeug defines several such subclasses). Such instances have a .code attribute if you really want to match against the HTTP code:
if error.code == 404:
# not found error

I'm guessing it is holding the exception so if you want to return it to know what went wrong. Though I am not familiar with this, or it if it is a parent it may need an argument supplied to it to check the error and handle it properly.

Does "Exception" capture all other "Concrete Exceptions"

I'm having issues with tweepy while running the Streaming API, but my question isn't directly related only to tweepy.
I have been getting multiple exceptions and I thought I could "catch/pass" for the time being, as a temporary solution, until I find out where the problem is.
As of now, tweepy has been throwing 5 different errors (IncompleteRead, ProtocolError, UnicodeDecodeError, AttributeError, TypeError), and they're all resulting from the filter to the API not from me obtaining the data.
The line they all have in common from the Traceback is:
twitterStream.filter(locations=[-125.22, 31.61, -104.86, 49.0, -104.86, 26.11, -66.94, 49.03])
IncompleteRead and ProtocolError are related to different packages that tweepy uses. But (UnicodeDecodeError, AttributeError, TypeError) are Concrete Exceptions.
My question:
Am I right to assume that the Exception base class can capture all those (the last 3)? Or is that incorrect?

The documentation for Exception states
All built-in, non-system-exiting exceptions are derived from this class
so the Exception base class should be able to capture those, as they are non-system-exiting (an example of a system-exiting exception is SystemExit). You can test this quickly if you'd like:
try:
raise AttributeError
except Exception:
print("We caught an exception!")
That said, blindly capturing the Exception base class is generally considered a Bad Idea because you will likely end up capturing exceptions that you won't want to.