I have searched for other posts, as I felt this is a rather common problem, but all other Python exception questions I have found didn't reflect my problem.
I will try to be as specific here as I can, so I will give a direct example. And pleeeeease do not post any workarounds for this specific problem. I am not specifically interested how you can send an email much nicer with xyz.
I want to know how you generally deal with dependent, error prone statements.
My question is, how to handle exceptions nicely, ones that depend on one another, meaning:
Only if the first step was successful, try the next, and so on. One more criterion is: All exceptions have to be caught, this code has to be robust.
For your consideration, an example:
try:
server = smtplib.SMTP(host) #can throw an exception
except smtplib.socket.gaierror:
#actually it can throw a lot more, this is just an example
pass
else: #only if no exception was thrown we may continue
try:
server.login(username, password)
except SMTPAuthenticationError:
pass # do some stuff here
finally:
#we can only run this when the first try...except was successful
#else this throws an exception itself!
server.quit()
else:
try:
# this is already the 3rd nested try...except
# for such a simple procedure! horrible
server.sendmail(addr, [to], msg.as_string())
return True
except Exception:
return False
finally:
server.quit()
return False
This looks extremely unpythonic to me, and the error handling code is triple the real business code, but on the other hand how can I handle several statements that are dependent on one another, meaning statement1 is prerequisite for statement2 and so on?
I am also interested in proper resource cleanup, even Python can manage that for itself.
Thanks, Tom
Instead of using the try/except's else block, you could simply return when it errors:
def send_message(addr, to, msg):
## Connect to host
try:
server = smtplib.SMTP(host) #can throw an exception
except smtplib.socket.gaierror:
return False
## Login
try:
server.login(username, password)
except SMTPAuthenticationError:
server.quit()
return False
## Send message
try:
server.sendmail(addr, [to], msg.as_string())
return True
except Exception: # try to avoid catching Exception unless you have too
return False
finally:
server.quit()
That's perfectly readable and Pythonic..
Another way of doing this is, rather than worry about the specific implementation, decide how you want your code to look, for example..
sender = MyMailer("username", "password") # the except SocketError/AuthError could go here
try:
sender.message("addr..", ["to.."], "message...")
except SocketError:
print "Couldn't connect to server"
except AuthError:
print "Invalid username and/or password!"
else:
print "Message sent!"
Then write the code for the message() method, catching any errors you expect, and raising your own custom one, and handle that where it's relevant. Your class may look something like..
class ConnectionError(Exception): pass
class AuthError(Exception): pass
class SendError(Exception): pass
class MyMailer:
def __init__(self, host, username, password):
self.host = host
self.username = username
self.password = password
def connect(self):
try:
self.server = smtp.SMTP(self.host)
except smtplib.socket.gaierror:
raise ConnectionError("Error connecting to %s" % (self.host))
def auth(self):
try:
self.server.login(self.username, self.password)
except SMTPAuthenticationError:
raise AuthError("Invalid username (%s) and/or password" % (self.username))
def message(self, addr, to, msg):
try:
server.sendmail(addr, [to], msg.as_string())
except smtplib.something.senderror, errormsg:
raise SendError("Couldn't send message: %s" % (errormsg))
except smtp.socket.timeout:
raise ConnectionError("Socket error while sending message")
In general, you want to use as few try blocks as possible, distinguishing failure conditions by the kinds of exceptions they throw. For instance, here's my refactoring of the code you posted:
try:
server = smtplib.SMTP(host)
server.login(username, password) # Only runs if the previous line didn't throw
server.sendmail(addr, [to], msg.as_string())
return True
except smtplib.socket.gaierror:
pass # Couldn't contact the host
except SMTPAuthenticationError:
pass # Login failed
except SomeSendMailError:
pass # Couldn't send mail
finally:
if server:
server.quit()
return False
Here, we use the fact that smtplib.SMTP(), server.login(), and server.sendmail() all throw different exceptions to flatten the tree of try-catch blocks. In the finally block we test server explicitly to avoid invoking quit() on the nil object.
We could also use three sequential try-catch blocks, returning False in the exception conditions, if there are overlapping exception cases that need to be handled separately:
try:
server = smtplib.SMTP(host)
except smtplib.socket.gaierror:
return False # Couldn't contact the host
try:
server.login(username, password)
except SMTPAuthenticationError:
server.quit()
return False # Login failed
try:
server.sendmail(addr, [to], msg.as_string())
except SomeSendMailError:
server.quit()
return False # Couldn't send mail
return True
This isn't quite as nice, as you have to kill the server in more than one place, but now we can handle specific exception types different ways in different places without maintaining any extra state.
If it was me I would probably do something like the following:
try:
server = smtplib.SMTP(host)
try:
server.login(username, password)
server.sendmail(addr, [to], str(msg))
finally:
server.quit()
except:
debug("sendmail", traceback.format_exc().splitlines()[-1])
return True
All errors are caught and debugged, the return value == True on success, and the server connection is properly cleaned up if the initial connection is made.
Just using one try-block is the way to go. This is exactly what they
are designed for: only execute the next statement if the previous
statement did not throw an exception. As for the resource clean-ups,
maybe you can check the resource if it needs to be cleaned up
(e.g. myfile.is_open(), ...) This does add some extra conditions, but
they will only be executed in the exceptional case. To handle the case
that the same Exception can be raised for different reasons, you
should be able to retrieve the reason from the Exception.
I suggest code like this:
server = None
try:
server = smtplib.SMTP(host) #can throw an exception
server.login(username, password)
server.sendmail(addr, [to], msg.as_string())
server.quit()
return True
except smtplib.socket.gaierror:
pass # do some stuff here
except SMTPAuthenticationError:
pass # do some stuff here
except Exception, msg:
# Exception can have several reasons
if msg=='xxx':
pass # do some stuff here
elif:
pass # do some other stuff here
if server:
server.quit()
return False
It is no uncommon, that error handling code exceeds business code. Correct error handling can be complex.
But to increase maintainability it helps to separate the business code from the error handling code.
I would try something like this:
class Mailer():
def send_message(self):
exception = None
for method in [self.connect,
self.authenticate,
self.send,
self.quit]:
try:
if not method(): break
except Exception, ex:
exception = ex
break
if method == quit and exception == None:
return True
if exception:
self.handle_exception(method, exception)
else:
self.handle_failure(method)
def connect(self):
return True
def authenticate(self):
return True
def send(self):
return True
def quit(self):
return True
def handle_exception(self, method, exception):
print "{name} ({msg}) in {method}.".format(
name=exception.__class__.__name__,
msg=exception,
method=method.__name__)
def handle_failure(self, method):
print "Failure in {0}.".format(method.__name__)
All of the methods (including send_message, really) follow the same protocol: they return True if they succeeded, and unless they actually handle an exception, they don't trap it. This protocol also makes it possible to handle the case where a method needs to indicate that it failed without raising an exception. (If the only way your methods fail is by raising an exception, that simplifies the protocol. If you're having to deal with a lot of non-exception failure states outside of the method that failed, you probably have a design problem that you haven't worked out yet.)
The downside of this approach is that all of the methods have to use the same arguments. I've opted for none, with the expectation that the methods I've stubbed out will end up manipulating class members.
The upside of this approach is considerable, though. First, you can add dozens of methods to the process without send_message getting any more complex.
You can also go crazy and do something like this:
def handle_exception(self, method, exception):
custom_handler_name = "handle_{0}_in_{1}".format(\
exception.__class__.__name__,
method.__name__)
try:
custom_handler = self.__dict__[custom_handler_name]
except KeyError:
print "{name} ({msg}) in {method}.".format(
name=exception.__class__.__name__,
msg=exception,
method=method.__name__)
return
custom_handler()
def handle_AuthenticationError_in_authenticate(self):
print "Your login credentials are questionable."
...though at that point, I might say to myself, "self, you're working the Command pattern pretty hard without creating a Command class. Maybe now is the time."
Why not one big try: block? This way, if any exception is caught, you'll go all the way to the except. And as long as all the exceptions for the different steps are different, you can always tell which part it was that fired the exception.
I like David's answer but if you are stuck on the server exceptions you can also check for server if is None or states. I flattened out the method a bit bit it is still a but unpythonic looking but more readable in the logic at the bottom.
server = None
def server_obtained(host):
try:
server = smtplib.SMTP(host) #can throw an exception
return True
except smtplib.socket.gaierror:
#actually it can throw a lot more, this is just an example
return False
def server_login(username, password):
loggedin = False
try:
server.login(username, password)
loggedin = True
except SMTPAuthenticationError:
pass # do some stuff here
finally:
#we can only run this when the first try...except was successful
#else this throws an exception itself!
if(server is not None):
server.quit()
return loggedin
def send_mail(addr, to, msg):
sent = False
try:
server.sendmail(addr, to, msg)
sent = True
except Exception:
return False
finally:
server.quit()
return sent
def do_msg_send():
if(server_obtained(host)):
if(server_login(username, password)):
if(send_mail(addr, [to], msg.as_string())):
return True
return False
Related
I'm working on a basic socket client program in python and I'm not totally sure how to handle exceptions. This is what I did up to now:
TCP_IP = '..............'
TCP_PORT = 4950
MESSAGE = "o3"
BUFFER_SIZE = 2048
data = ""
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.settimeout(5.0)
try:
s.connect((TCP_IP, TCP_PORT))
except socket.error:
#write error code to file
s.close()
try:
s.sendall(MESSAGE)
except socket.error:
#write to file or whatever
s.close()
try:
data = s.recv(BUFFER_SIZE)
except socket.error:
#write to file or whatever
s.close()
finally:
s.close()
The code is working as I want, but I'm not sure if I should nest try/catch blocks or not? Should I put socket.socket into try/catch block too?
Second question, what will s.settimeout() do in my case? As far as I understood the documentation, it will throw an exception after 5 seconds, but for what? Just connect or will it do the same for sendall and recv?
Since you're doing exactly the same actions in all the exception blocks and catching the same socket.error exception, you could put s.connect, s.sendall and s.recv in the same try: block. Like so:
try:
s.connect((TCP_IP, TCP_PORT))
s.sendall(MESSAGE)
data = s.recv(BUFFER_SIZE)
except socket.error:
#write error code to file
finally:
s.close()
Note that since s.close is also in the finally section in your example, it will always get called, even after an exception has occurred. So you'd end up with another exception occurring when you try to close an already closed socket. By not putting it in the except block and only in finally, you can avoid that situation.
If you do intend to handle each error in a different way, then you can leave them separate as you already have. But make sure to break/return at the end of the exception block so that you don't try the next. It's done that way in the socket examples, by using a continue in the loop.
Nesting them would help if you wanted to do something different in the exception block. But if not you'd be repeating the except block every time. And if you wanted to do something different, when you exit the nested-trys, you wouldn't be certain of which level it has completed or raised an exception - would need to use flag values etc. to merely track that. So for your example of the same error handling code, at the very least, do something like this in your except block:
except socket.error as e:
socket_error_handler(e, s)
def socket_error_handler(exception, socket):
#write error code to file
etc.
Should I put socket.socket into try/catch block too?
They do that in the examples, linked above.
Apart from logging, you shouldn't really be doing the same exception handling at each stage. Probably need to handle those separately.
Part 2:
s.settimeout(5.0) sets the timeout for each socket operation, not just the first connect. Also implies that it's in blocking mode.
I have a function that returns the DB connection handler from MongoDB. I have various other functions that makes a call to the DB, I figure let's throw the connection handler into a function so I don't have to define it in every function.
Does this look right? I guess my question is, if it can't make a connection to the DB server, it will print both messages Could not connect to server and No hosts found How can I go about only printing "Could not connect to the server."
def mongodb_conn():
try:
conn = pymongo.MongoClient()
except pymongo.errors.ConnectionFailure, e:
print "Could not connect to server: %s" % e
return conn
def get_hosts()
try:
conn = mongodb_conn()
mongodb = conn.dbname.collection
b = []
hosts_obj = mongodb.find({'_id': 'PR'})
for x in hosts_obj:
print x
except:
print "No hosts found"
get_hosts()
Move your conn = mongodb_conn() call out of the try .. except handler, and test if None was returned:
def get_hosts()
conn = mongodb_conn()
if conn is None:
# no connection, exit early
return
try:
mongodb = conn.dbname.collection
b = []
hosts_obj = mongodb.find({'_id': 'PR'})
for x in hosts_obj:
print x
except:
print "No hosts found"
You should, at all cost, avoid using a blanket except however; you are catching everything now, including memory errors and keyboard interrupts, see Why is "except: pass" a bad programming practice?
Use specific exceptions only; you can use one except statement to catch multiple exception types:
except (AttributeError, pymongo.errors.OperationFailure):
or you can use multiple except statements handle different exceptions in different ways.
Limit the exception handler to just those parts of the code where the exception can be thrown. The for x in hosts_obj: loop for example is probably not going to throw an AttributeError exception, so it should probably not be part of the try block.
Note that you'll need to adjust your mongodb_conn() function to not try and use the conn local if it has never been set; you'll get an UnboundLocal error if you do:
def mongodb_conn():
try:
return pymongo.MongoClient()
except pymongo.errors.ConnectionFailure, e:
print "Could not connect to server: %s" % e
Now the function returns the connection if successful, None if the connection failed.
You can also check if the server is available
like this:
from pymongo.errors import ConnectionFailure
client = MongoClient()
try:
# The ismaster command is cheap and does not require auth.
client.admin.command('ismaster')
except ConnectionFailure:
print("Server not available")
Here's the code giving me issues:
def connect(self):
self.chatlog['state'] = NORMAL
self.chatlog.insert(END, ("===ATTEMPTING TO CONNECT TO SERVER\n"))
self.chatlog['state'] = DISABLED
self.chatlog.yview(END)
try:
s.connect((host,port))
self.chatlog['state'] = NORMAL
self.chatlog.insert(END, ("===CONNECTED TO SERVER\n"))
self.chatlog['state'] = DISABLED
self.chatlog.yview(END)
self.conn=True
print("Connected")
except ConnectionRefusedError:
self.chatlog['state'] = NORMAL
self.chatlog.insert(END, ("===SERVER COULD NOT BE FOUND\n" + "===PLEASE MAKE SURE THE SERVER IS ON, AND YOU'RE CONNECTED TO THE NETWORK\n"))
self.chatlog['state'] = DISABLED
self.chatlog.yview(END)
else:
self.chatlog['state'] = NORMAL
self.chatlog.insert(END, ("===THERE'S AN ERROR WITH THE PROGRAM\n" + "===PLEASE TURN IT OFF AND ON AGAIN\n"))
self.chatlog['state'] = DISABLED
self.chatlog.yview(END)
When I connect to a server, it both connects (the "try" section completes fine, the print statement occurs) however the "else" section also occurs. I can't seem to find why. Does anyone have any idea what's causing this?
EDIT: I should say that when I remove the "else:" section, it works fine as well.
I'm guessing you are trying to catch all exceptions here, but that's not the correct syntax.
The correct way to do it would be:
try:
# do something dangerous
except ConnectionRefusedError:
# handle known error
except:
# handle unknown error
else in this case means that the code will be executed if no errors occur.
The else part of a try: except: else: finally: block is similar to the else in an if: else:, effectively:
if someerror:
...
else:
...
The else block runs when no error is raised from the try block. Using this block allows you to minimise the code in the try block; in your case:
try:
s.connect((host,port))
except ConnectionRefusedError:
self.chatlog['state'] = NORMAL
self.chatlog.insert(END, ("===SERVER COULD NOT BE FOUND\n" + "===PLEASE MAKE SURE THE SERVER IS ON, AND YOU'RE CONNECTED TO THE NETWORK\n"))
self.chatlog['state'] = DISABLED
self.chatlog.yview(END)
else:
self.chatlog['state'] = NORMAL
self.chatlog.insert(END, ("===CONNECTED TO SERVER\n"))
self.chatlog['state'] = DISABLED
self.chatlog.yview(END)
self.conn=True
print("Connected")
I want to create socket errors (By doing things, obviously) but I've no idea how I should test if my script handles errors properly (If it dectes them.)
Currently, my code is this:
except socket.error as err:
print "Connection lost, waiting..."
time.sleep(5)
In theory, it should handle all the socket errors, print and then sleep (It's a part of a while loop.).
Any idea of how can I test it to see how it handles errors?
Use the raise statement:
try:
raise socket.error
except socket.error as err:
print "Connection lost, waiting..."
time.sleep(5)
Yet another example:
try:
raise AttributeError
except AttributeError:
print 'Sorry'
#Sorry
Also take a look at here and here
I am trying to have a client connect to my server, and have a stream of communication between them. The only reason the connection should break is due to network errors, or unless the client wants to stop talking.
The issue I am running into is keeping the handler in a tight loop, and parsing the JSON.
My server code is :
#!/usr/bin/env python
import SocketServer
import socket
import json
import time
class MyTCPServer(SocketServer.ThreadingTCPServer):
allow_reuse_address = True
class MyTCPServerHandler(SocketServer.BaseRequestHandler):
def handle(self):
while 1:
try:
networkData = (self.request.recv(1024).strip())
try:
jsonInputData = json.loads(networkData)
print jsonInputData
try:
if jsonInputData['type'] == 'SAY_HI':
print "HI"
except Exception, e:
print "no hi"
pass
try:
if jsonInputData['type'] == 'GO_AWAY':
print "Going away!"
except Exception, e:
print "no go away"
pass
except Exception, e:
pass
#time.sleep(0.001)
#print "JSON Error", e
except Exception, e:
#time.sleep(0.001)
pass
#print "No message", e
server = MyTCPServer(('192.168.1.115', 13373), MyTCPServerHandler)
server.serve_forever()
My client code is simple :
#!/usr/bin/env python
import socket
import json
import time
import sys
hostname = '192.168.1.103'
port = 13373
try:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((hostname,port))
except Exception, e:
print "Error, could not open socket: ", e
data = {'type':'SAY_HI'}
sock.send(json.dumps(data))
data = {'type':'SAY_BYE'}
sock.send(json.dumps(data))
Sometimes I'll see the messages being sent, "SAY_HI" and "SAY_BYE", but most of the times, no data is being displayed on the server side.
This question is really not clear, but calling self.request.recv(1024) is very likely not what you want to do. You're eliminating all of the nice application-level handling that TCP will happily do for you. If you change that to self.request.recv(8) or a similarly very small number (such that recv() returns whenever it receives data, and doesn't try to fill your buffer), you may get better results.
Ultimately this is super-simplistic change, even if it works, that will not work in a larger context. You will need to be handling exceptions from your json parser on the server side and waiting for more data until an entire well-formed message is received.
This is a hopelessly more complex subject than will be handled generally in any SO answer. If you're going to be doing any amount of raw sockets programming, you absolutely must own a copy of Unix Network Programming, Volume 1.