I'm trying to diagnose a read failure with motor during a Mongo Atlas cluster failover. The Retryable-Reads specification defines that aggregate calls are retryable, but Cursor.getMore is not. I have code that looks like this:
cursor = db.foo.aggregate([...])
if not await cursor.fetch_next:
raise SomeException
doc = cursor.next_object()
This code appears to not retry during a cluster failover, because it's internally calling getMore I assume. I'm not entirely clear whether that's the case or not. Not to mention that fetch_next is deprecated anyway.
Would changing it to this make it a retryable read?
async for doc in cursor:
break
else:
raise SomeException
Or does this result in the same internal processing, and the problem is elsewhere?
The goal is to try to read the single result document from an aggregation pipeline (there's either one or none) in a retryable manner and raise an exception if there's none.
You need to rescue whatever exception the driver raises on a network error outside of this code and repeat the entire iteration in that case.
The driver automatically retries initial queries but not getMores, because those don't have a way of specifying the position in the result set. Thus the driver's retryable reads logic is insufficient for a robust application, you still need to handle the possibility of read errors in your application on the level of complete iterations.
If you are retrieving a single document, it should generally be included in the initial query response and no getMores would be needed, thus in practice this question wouldn't apply.
Related
Python tutorial have a section called Errors and Exceptions which use the structure as;
try:
[statement]
except [Built-in ExceptionType]:
do something when an exception has been captured.
finally:
do something whether exception occurred or not.
This can also handle by raise an Error directly, too.
try:
raise [Built-in ExceptionType]
except [Built-in ExceptionType above] as e:
print(f'foo error: {e}')
There are several built-in ExceptionType which the good practice is developer should catch every specific exceptions type that developer should look for. (refer from Why is "except: pass" a bad programming practice?)
However, after I reading logging section. I'm thinking about I want to log the error into the log file and let user to aware the error (maybe just print the text and user can inform IT support) rather than throwing an exception on screen. Therefore, instead of try / except combo above then I can use the following error message and logged it.
if len(symbol) != 1:
error_message = f'Created box with symbol={symbol}, width={width} and height={height}, Symbol needs to be a string of length 1'
logger.error(error_message)
print(f'\nError! symbol={symbol}, width={width} and height={height} -- Symbol needs to be a string of length 1')
return
I doubt that what the better current practice is between:
a) raise Error on screen and
b) logging the Error for further investigation?
c) Other method (please suggest me)
I hope that I'm not try to comparing two different things.For me, I want to choose b) and store the error in log. this is easier for investigation and does not give unnecessary information to the users. but I do not sure I'm on the right track or not. Thank you very much.
Logging and raising exceptions are two different things for two different purposes. Logs let you inspect what your program did after the fact. Raising exceptions has important effects on the program flow right now. Sometimes you want one, sometimes you want the other, sometimes you want both.
The question is always whether an error is expected or unexpected, whether you have some plan what to do in case it occurs, and whether it is useful to notify anyone about the occurrence of the error or not.
Expected errors that you have a "backup plan" for should be caught and handled, that is regular program control flow. Unexpected errors probably should halt the program, or at least the particular function in which they occurred. If a higher up caller feels like handling the exception, let them. And whether to log an error or not (in addition to handling or not handling it) depends on whether anyone can glean any useful insight from that log entry or whether it would just be noise.
As deceze already mentions, logging and exceptions are two totally distinct things. As a general rule:
you raise an exception when you encounter an unexpected / invalid condition that you cannot handle at this point of the code. What will became of this exception (whether someone up in the call stack will handle it or not) is none of your concern at this point
you only catch an exception when you can handle it or when you want to log it (eventually with additionnal context informations) THEN reraise it
at your application's top-level, you eventually add a catchall exception handler that can log the exception and decide on the best way to deal with the situation depending on the type of application (command-line script, GUI app, webserver etc...).
Logging is mostly a developper/admin tool used for post-mortem inspection, program behaviour analysis etc, it's neither an error handling tool nor an end-user UI feature. Your example:
if len(symbol) != 1:
error_message = f'Created box with symbol={symbol}, width={width} and height={height}, Symbol needs to be a string of length 1'
logger.error(error_message)
print(f'\nError! symbol={symbol}, width={width} and height={height} -- Symbol needs to be a string of length 1')
return
is a perfect antipattern. If your function expects a one character string and the caller passed anything else then the only sane solution is to raise an exception and let the caller deal with it. How it does is, once again, none of your concerns.
You should log all errors no matter if they are thrown or handled for log analysis and refactoring amongst other purposes.
Whether to throw or to handle the error usually depends on the intended purpose of the code and the severity of the error.
Although "throwing" should be only used in debugging and handled
'gracefully' by dedicated exception code in the production version of the
application. No user likes crashes.
If the error impacts the business logic, end result of the code, or can cause damages to the user of the software, you want to terminate the execution. You do it either by throwing the error or handling it through a dedicated error handling procedure that shutdowns the program.
If the error is not severe and does not impact the normal functionality of the program, you can choose whether you want to handle it or throw it based on the best practices in your team and requirements of the project.
From the Docs:
Returns a new instance of the SMTPHandler class. The instance is initialized with the from and to addresses and subject line of the email. The toaddrs should be a list of strings.
logging.handlers.SMTPHandler can be used to send logged error message.
import logging
import logging.handlers
smtp_handler = logging.handlers.SMTPHandler(mailhost=("example.com", 25),
fromaddr="from#example.com",
toaddrs="to#example.com",
subject="Exception notification")
logger = logging.getLogger()
logger.addHandler(smtp_handler)
you can break the logic here if the code execution needs to be stopped.
Also look around this answer i have referred earlier collate-output-in-python-logging-memoryhandler-with-smtphandler
Consider you have this kind of code for flask api:
#app.route('/doing/foo')
def foo():
try:
...
except Exception as ex:
doing some stuff
Now doing some stuff accidentally threw another exception. How do you prevent the server from showing another generic internal server error without any clue what happened?
In other words, how do I catch exceptions that may have originated from the except clause?
Aha! That is a 100 years question.
First option is to move to Python 3.
Python 3 has exception chaining and will print both of the exceptions - the one being handled and the one that was thrown due to the mishandling.
Second option is to implement exception chaining by yourself:
Manually inject the new exception into the one being handled using another try-except clause inside the current except:.
Keep in mind it WILL cause circular reference. Since it's a one-time exception, I wouldn't bother dealing with it. Further usage of this method will require cleaning the traceback using traceback.clear_frames().
In a request handler, log the error and return an HTTP 500 error and some HTML noting that an "Internal error has occurred" and containing a link back to some safe point for users to restart whatever they were doing.
You can attempt to catch more specific exceptions (from most specific and common towards Exception (which is the most general of the general exceptions). You can also attempt to introspect on the exception/traceback object (ex in your example).
Mostly you just want to ensure that you don't display internal application state to the user through their browser (possible security implications) and that you don't simply drop the user's request on the floor.
In queries like aggregations and cardinality search there might be a timeout.
I noticed that when executing queries from python client the response sometimes contains:
{
"took":1200184,
"timed_out":true,
"_shards":{
"total":84,
"successful":84,
"failed":0
}
And returns less results than the expected.
My main problem is that when timeout occurs, response still contains a number of results.
I could check if timeout is true before parsing response results but there is probably a better way to do that :)... like raise an exception or somehow catch timeout and retry
You can increase the timeout for elasticsearch using:-
es.search(index="my_index",
doc_type="document",
body=get_req_body(),
request_timeout=30)
By default the value assigned is 10. If ,on the other hand you want to catch exception you can use a scheduler and check the time elapsed and catch the exception if it exceeds the time limit.
Elasticsearch-py client has a named argument you can pass that will let you set timeout value for the search request.
But I'd suggest using scrolling to obtain results in such scenarios, it is similar to a cursor for database query. Here's a really good example of how to use scrolling. With a limited scroll size, the request is less likely to timeout and you will be able to fetch all the results instead of receiving partial results.
Example search call with timeout parameter
es.search(index="index", doc_type="doc_type", body=body, timeout=50)
I've written an http downloader using webclient.Agent.
The connectionLost function of a body consumer object is called with a reason parameter.
def connectionLost(self, reason):
if isinstance(reason.value, ResponseDone):
self.df.callback(None)
else:
self.df.errback(reason.value)
I would really want to know what are the possible exception classes in 'reason'
because I need to catch them all and deal with them later in the calling
function which uses inlineCallbacks.
So far I identified:
ConnectError, BindError, ConnectionClosed, ResponseFailed
Is this documented somewhere?
Most of the documentation contains just vague
"...errback with a description of the error..." statements.
You don't need to catch them all specifically. Exception handling respects inheritance: if you try to catch a base class and a subclass is raised, you'll catch that too. Exception is the base class for just about all exceptions in Python, so if you catch that, you'll catch just about everything.
try:
yield agent.request(...)
except Exception as e:
print 'Oh no, some failure'
else:
print 'Success'
There is no complete list of all the exceptions that Agent.request may fail with, because it is impossible to know this list in advance. Failures may be due to connection setup - but is it a plain TCP connection or an SSL connection for an HTTPS URL? Perhaps it's a connection over a SOCKS proxy due to an unfortunate user's network configuration. These may all fail in different ways. The same applies to the actual HTTP request itself - who knows how the server will behave? Perhaps it will respond with an unexpected error code, or perhaps it will respond with something Agent isn't even capable of parsing and trigger an exception from the low-level HTTP client protocol implementation. The parser is hand crafted, so there's probably a lot of different exceptions that could come from that kind of problem. And then there's the variation on content and transfer encoding, which may invoke lots of different functionality (eg, a gzip decoder) which adds still more possible exceptions.
So, instead of trying to come up with the full list of all of these exceptions at every place you use Agent.request, only specifically handle the exception types you know about and have special code for and then handle everything else with an except Exception.
I'm using MySQLdb in Python.
I have an update that may succeed or fail:
UPDATE table
SET reserved_by = PID
state = "unavailable"
WHERE state = "available"
AND id = REQUESTED_ROW_ID
LIMIT 1;
As you may be able to infer, multiple processes are using the database, and I need processes to be able to securely grab rows for themselves, without race conditions causing problems.
My theory (perhaps incorrect) is that only one process will be able to succeed with this query (.rowcount=1) -- the others will fail (.rowcount=0) or get a different row (.rowcount=1).
The problem is, it appears that everything that happens through MySQLdb happens in a virtual world -- .rowcount reads =1, but you can't really know whether anything really happened, until you perform a .commit().
My questions:
In MySQL, is a single UPDATE atomic within itself? That is, if the same UPDATE above (with different PID values, but the same REQUESTED_ROW_ID) were sent to the same MySQL server at "once," am I guaranteed that one will succeed and the other will fail?
Is there a way to know, after calling "conn.commit()", whether there was a meaningful change or not?
** Can I get a reliable .rowcount for the actual commit operation?
Does the .commit operation send the actual query (SET's and WHERE conditions intact,) or does it just perform the SETs on affected rows, independent the WHERE clauses that inspired them?
Is my problem solved neatly by .autocommit?
Turn autocommit on.
The commit operation just "confirms" updates already done. The alternative is rollback, which "undoes" any updates already made.