We Keep Coding

How to Catch Exception in for loop and return exception only after loop is done?

Hi I have a function that parses the first 60 lines in a file and is supposed to alert a user if there are lines that are entirely whitespace. However these can occur anywhere in those 60 lines so I want the script to parse the entire 60 lines mainly because I need data from a couple of the lines for my error reporting. And we might want to know where those errors occur in the future. I wrote this:
def header_data(data):
dictionary = {}
datalen = len(data)
itrcntr = 0
try:
for line in data:
itrcntr += 1
if line.isspace():
raise Exception('File has badly formatted header data line(s)')
else:
linesplit = line.rstrip().split(":")
if len(linesplit) > 1:
dictionary[linesplit[0]] = linesplit[1].strip()
return dictionary
except Exception as e:
errmsg = str(e)
if itrcntr == datalen:
return (dictionary, errmsg)
else:
pass
With this function, I'd expect if it sees that itrcntr is not equal datalen, it would pass and go back to the try block and move on to the next line. But this doesn't happen. INstead it breaks out of the function and continues in the next line in the function caller. How can I make it continue looping till it reaches to the end of the loop in which it will then return the dictionary along with the error message? Or can this not be done with try catch exception handlers?

Unless you want to catch exceptions other than the situation of line.isspace, I wouldn't use a try block at all. Just collect your errors in a list, eg:
errors = []
for line in data:
itrcntr += 1
if line.isspace():
errors.append('File has badly formatted header data at line %d.' % itrcntr)
# then at the end:
if errors:
# do something about it...

If any exception occurred, try clause will be skipped and except clause
will run.
If you raise an exception anywhere in the Try, everything else will be skipped. So if you want the loop to continue, then just don't use Try Except.
Just collect every error message and then return it.

what is the correct way to handle a caught exception and raise it outside of try block

Im just starting my python learning journey and need some help with the correct way to raise an exception.
Consider a block of code which loops thru a list and performs a task. If an exception occurs,continue with the program execution. And execute the rest of the code. At the end of the program raise the exception and system the application with non zero code. The idea is the program should continue executing all the tasks, but exit with a non 0 code for an external application to track and report.
save_exc_info = None
def numMatcher(numbers):
try:
if numbers != 2:
print('number match ' + str(numbers))
else:
raise ValueError('Number not in list. Will be logged for troubleshooting') # raise exception and log it
except ValueError as veer: # exception 1 caught and saved
save_exc_info = sys.exc_info()
except (IOError, OSError) as ioerr: # exception 2 caught and saved
save_exc_info = sys.exc_info()
try:
print('Next step') # Perform rest of the tasks in the code
except Exception as excp: # exception 3 caught and saved
save_exc_info = sys.exc_info()
print('final step')
numlist = [1, 2, 3]
for numbers in numlist:
numMatcher(numbers)
if save_exc_info is not None:
traceback.print_exception(*save_exc_info) # how to return the right exception and print?
sys.exit(1) # At the end of the program, exit with non zero code as there was an exception in the program.

When handling the exception, you can assign it to a variable, e.g.:
except AssertionError as aerr:
saved_exception = aerr
Which you can later access e.g.:
print(saved_exception)
For you code, this gives you an option of not having two variable, and instead of isError just use saved_exception = None and later test if saved_exception is not None: ...
Not sure how useful saving exception for later (using it as general interface to pass information around) is. Perhaps it may be worth rethinking a bit more.
N Chauhan also made a good point int the comment about AssertionError not being very suitable exception to use to convey this type of information.
To your updated question. If you want to print the traceback is you'd see it when the exception was raised, probably the most straightforward would be to save exception information and use print_exception() (or its format_exception friend):
except ValueError:
save_exc_info = sys.exc_info()
...
traceback.print_exception(*save_exc_info)
You could have extracted the same information from / use the saved exception as well, *save_exc_info could have also been: type(saved_exception), saved_exception, saved_exception.__traceback__ (for the saved exception in the first example).

If condition check with python

if not ping_device(ip):
if not reboot_device(device_1)
logger.error("Unable to reboot.. Serious issue")
break
logger.info("HARD Rebooted")
if not ping_device(ip):
logger.error("Ping failed after hard reboot")
else:
logger.info("Pinged success after reboot")
else:
logger.info("Pinged")
logger.info("Reboot passed")
logger.info("Getting uptime")
sw_version_check = get_sw_version(ip)
check_upgrade()
:
:
:
:
Here is my piece of code. Now if the ping_device fn in the first line succeeds, I have no problem. The else part comes into the picture.
Now if my first line ping_device fails, I call a reboot_device fn. after reboot, I again call the ping function to check my device is online.
After the ping success after the reboot, I need to get going with the else part of the first if condition with sw_version_check, check_upgrade() and rest of code traverses as mentioned with :
Will my logic after ping failure continue with sw_version_chec and check_upgrade?

You could have some separate variable, say doUpgrade, initialized to False and set to True at any point(s) in the first section where you have determined you need an upgrade. Then test that variable to see if the upgrade code needs to be executed.

Try using exceptions, they help make this type of control flow easier to reason about and are more "Pythonic":
def reboot_device(device):
if not your_reboot_device_fn(device):
raise DeviceRebootError('Unrecoverable reboot error')
else:
pass # Or anything else you want to do on success.
def ping_device(ip, device, did_try_reboot=False):
if not your_ping_device_fn(ip):
if did_try_reboot:
raise DevicePingError('Ping failed after hard reboot')
else:
try:
reboot_device(device)
except DeviceRebootError as e:
logging.error(e)
raise
else:
logger.info("HARD Rebooted")
ping_device(ip, device, did_try_reboot=True)
else:
pass # Or anything else you want to do on success.
try:
ping_device(ip, device_1)
except DevicePingError as e:
logging.error(e)
# NOTE: DeviceRebootError isn't caught so it will end the program.
else:
logging.info('Pinged')
sw_version_check = get_sw_version(ip)
check_upgrade()

How do I exit program in try/except?

I have this try/except code:
document = raw_input ('Your document name is ')
try:
with open(document, 'r') as a:
for element in a:
print element
except:
print document, 'does not exist'
How do I exit the program after I print "[filename] does not exist"? break and pass obviously don't work, and I don't want to have any crashing errors, so sys.exit is not an option.
Please ignore the try part - it's just a dummy.

Use the sys.exit:
import sys
try:
# do something
except Exception, e:
print >> sys.stderr, "does not exist"
print >> sys.stderr, "Exception: %s" % str(e)
sys.exit(1)
A good practice is to print the Exception that occured so you can debug afterwards.
You can also print the stacktrace with the traceback module.
Note that the int you return in sys.exit will be the return code of your program. To see what exit code your program returned (which will give you information about what happens and can be automated), you can do:
echo $?

Using
sys.exit(1)
is not a crashing error, it's a perfectly normal way to exit a program. The exit code of 1 is a convention that means something went wrong (you would return 0 in the case of a successful run).

Just re-raise it. It's more friendly for developer
document = raw_input ('Your document name is ')
try:
with open(document, 'r') as a:
for element in a:
print element
except:
print document, 'does not exist'
raise
Check python document in the Raising Exceptions section about re-raise error in except.

You can also put your code in a function and issue a return. You may call it main which you can call from your script.
def main():
document = raw_input ('Your document name is ')
try:
with open(document, 'r') as a:
for element in a:
print element
except:
print document, 'does not exist'
return
if __name__ == "__main__":
main()

In case that you are using an if statement inside a try, you are going to need more than one sys.exit() to actually exit the program.
For example, you are parsing an argument when calling the execution of some file, e.g. $./do_instructions.py 821 such as:
import sys
# index number 1 is used to pass a set of instructions to parse
# allowed values are integer numbers from 1 to 4, maximum number of instructions is 3
arg_vector = "821" # <- pretending to be an example of sys.argv[1]
if len(arg_vector) > 3:
sys.exit(2) # <- this will take you out, but the following needs an extra step.
# for an invalid input (8).
for i in arg_vector:
# to validate that only numbers are passed as args.
try:
int(i) # <- 8 is valid so far
# value (8) is not valid, since is greater than 4
if (int(i) == 0) or (int(i) > 4):
print("Values must be 1-4")
# the following call does not takes you out from the program,
# but rise the SystemExit exception.
sys.exit(2)
except SystemExit: # <- needed to catch the previous as the first evaluation
# The following parameter "2" is just for this example
sys.exit(2) # <- needed to actually interrupt the execution of the program/script.
# if there is no "except SystemExit:", the following will be executed when the
# previous "if" statement evaluates to True and the sys.exit(2) is called.
#
# and the "print("Only num...") function will be called, even when the intention
# of it is to advice the *user* to use only numbers, since 8 is a number this
# shouldn't be executed.
except:
print("Only numbers are allowed.")
sys.exit(2)
otherwise, you want to use one try-except block for each evaluation.

Probably not the best practice, but it worked for me:
import sys
close = False
try:
if SomethingBadHappend:
close = True
except:
pass
if close:
sys.exit(1)
Close dose does not seem to work inside 'try'.

In Python epoll can I avoid the errno.EWOULDBLOCK, errno.EAGAIN?

I wrote a epoll wrapper in python, It works fine but recently I found the performance is not not ideal for large package sending. I look down into the code and found there's actually a LOT of error
Traceback (most recent call last):
File "/Users/dawn/Documents/workspace/work/dev/server/sandbox/single_point/tcp_epoll.py", line 231, in send_now
num_bytes = self.sock.send(self.response)
error: [Errno 35] Resource temporarily unavailable
and previously silent it as the document said, so my sending function was done this way:
def send_now(self):
'''send message at once'''
st = time.time()
times = 0
while self.response != '':
try:
num_bytes = self.sock.send(self.response)
l.info('msg wrote %s %d : %r size %r',self.ip,self.port,self.response[:num_bytes],num_bytes)
self.response = self.response[num_bytes:]
except socket.error,e:
if e[0] in (errno.EWOULDBLOCK,errno.EAGAIN):
#here I printed it, but I silent it in normal days
#print 'would block, again %r',tb.format_exc()
break
else:
l.warning('%r %r socket error %r',self.ip,self.port,tb.format_exc())
#must break or cause dead loop
break
except:
#other exceptions
l.warning('%r %r msg write error %r',self.ip,self.port,tb.format_exc())
break
times += 1
et = time.time()
I googled it, and says it caused by temporarily network buffer run out
So how can I manually and efficiently detect this error instead it goes to exception phase?
Because it cause to much time to rasie/handle the exception.

Use select to see if a socket is ready for reading or writing.