i need to identify who raise an exception to handle better str error, is there a way ?
look at my example:
try:
os.mkdir('/valid_created_dir')
os.listdir('/invalid_path')
except OSError, msg:
# here i want i way to identify who raise the exception
if is_mkdir_who_raise_an_exception:
do some things
if is_listdir_who_raise_an_exception:
do other things ..
how i can handle this, in python ?
If you have completely separate tasks to execute depending on which function failed, as your code seems to show, then separate try/exec blocks, as the existing answers suggest, may be better (though you may probably need to skip the second part if the first one has failed).
If you have many things that you need to do in either case, and only a little amount of work that depends on which function failed, then separating might create a lot of duplication and repetition so the form you suggested may well be preferable. The traceback module in Python's standard library can help in this case:
import os, sys, traceback
try:
os.mkdir('/valid_created_dir')
os.listdir('/invalid_path')
except OSError, msg:
tb = sys.exc_info()[-1]
stk = traceback.extract_tb(tb, 1)
fname = stk[0][2]
print 'The failing function was', fname
Of course instead of the print you'll use if checks to decide exactly what processing to do.
Wrap in "try/catch" each function individually.
try:
os.mkdir('/valid_created_dir')
except Exception,e:
## doing something,
## quite probably skipping the next try statement
try:
os.listdir('/invalid_path')
except OSError, msg:
## do something
This will help readability/comprehension anyways.
How about the simple solution:
try:
os.mkdir('/valid_created_dir')
except OSError, msg:
# it_is_mkdir_whow_raise_ane_xception:
do some things
try:
os.listdir('/invalid_path')
except OSError, msg:
# it_is_listdir_who_raise_ane_xception:
do other things ..
Here's the clean approach: attach additional information to the exception where it happens, and then use it in a unified place:
import os, sys
def func():
try:
os.mkdir('/dir')
except OSError, e:
if e.errno != os.errno.EEXIST:
e.action = "creating directory"
raise
try:
os.listdir('/invalid_path')
except OSError, e:
e.action = "reading directory"
raise
try:
func()
except Exception, e:
if getattr(e, "action", None):
text = "Error %s: %s" % (e.action, e)
else:
text = str(e)
sys.exit(text)
In practice, you'd want to create wrappers for functions like mkdir and listdir if you want to do this, rather than scattering small try/except blocks all over your code.
Normally, I don't find this level of detail in error messages so important (the Python message is usually plenty), but this is a clean way to do it.
Related
Im just starting my python learning journey and need some help with the correct way to raise an exception.
Consider a block of code which loops thru a list and performs a task. If an exception occurs,continue with the program execution. And execute the rest of the code. At the end of the program raise the exception and system the application with non zero code. The idea is the program should continue executing all the tasks, but exit with a non 0 code for an external application to track and report.
save_exc_info = None
def numMatcher(numbers):
try:
if numbers != 2:
print('number match ' + str(numbers))
else:
raise ValueError('Number not in list. Will be logged for troubleshooting') # raise exception and log it
except ValueError as veer: # exception 1 caught and saved
save_exc_info = sys.exc_info()
except (IOError, OSError) as ioerr: # exception 2 caught and saved
save_exc_info = sys.exc_info()
try:
print('Next step') # Perform rest of the tasks in the code
except Exception as excp: # exception 3 caught and saved
save_exc_info = sys.exc_info()
print('final step')
numlist = [1, 2, 3]
for numbers in numlist:
numMatcher(numbers)
if save_exc_info is not None:
traceback.print_exception(*save_exc_info) # how to return the right exception and print?
sys.exit(1) # At the end of the program, exit with non zero code as there was an exception in the program.
When handling the exception, you can assign it to a variable, e.g.:
except AssertionError as aerr:
saved_exception = aerr
Which you can later access e.g.:
print(saved_exception)
For you code, this gives you an option of not having two variable, and instead of isError just use saved_exception = None and later test if saved_exception is not None: ...
Not sure how useful saving exception for later (using it as general interface to pass information around) is. Perhaps it may be worth rethinking a bit more.
N Chauhan also made a good point int the comment about AssertionError not being very suitable exception to use to convey this type of information.
To your updated question. If you want to print the traceback is you'd see it when the exception was raised, probably the most straightforward would be to save exception information and use print_exception() (or its format_exception friend):
except ValueError:
save_exc_info = sys.exc_info()
...
traceback.print_exception(*save_exc_info)
You could have extracted the same information from / use the saved exception as well, *save_exc_info could have also been: type(saved_exception), saved_exception, saved_exception.__traceback__ (for the saved exception in the first example).
I am running the following try-except code:
try:
paths = file_system_client.get_paths("{0}/{1}/0/{2}/{3}/{4}".format(container_initial_folder, container_second_folder, chronological_date[0], chronological_date[1], chronological_date[2]), recursive=True)
list_of_paths=["abfss://{0}#{1}.dfs.core.windows.net/".format(storage_container_name, storage_account_name)+path.name for path in paths if ".avro" in path.name]
except Exception as e:
if e=="AccountIsDisabled":
pass
else:
print(e)
I want neither to print the following error if my try-except fells upon it nor to stop my program execution if I fell upon this error:
"(AccountIsDisabled) The specified account is disabled.
RequestId:3159a59e-d01f-0091-5f71-2ff884000000
Time:2020-05-21T13:09:03.3540242Z"
I just want to overpass it and print any other error/exception (eg. TypeError, ValueError, etc) that will occur.
Is this feasible in Python 3?
Please note that the .get_paths() method belongs to the azure.storage.filedatalake module which enables direct connection of Python with Azure Data Lake for path extraction.
I am giving the note to pinpoint that the Exception I am trying to bypass is not a built-in Exception.
[Update] In sort after following the proposed attached answers I modified my code to this:
import sys
from concurrent.futures import ThreadPoolExecutor
from azure.storage.filedatalake._models import StorageErrorException
from azure.storage.filedatalake import DataLakeServiceClient, DataLakeFileClient
storage_container_name="name1" #confidential
storage_account_name="name2" #confidential
storage_account_key="password" #confidential
container_initial_folder="name3" #confidential
container_second_folder="name4" #confidential
def datalake_connector(storage_account_name, storage_account_key):
global service_client
datalake_client = DataLakeServiceClient(account_url="{0}://{1}.dfs.core.windows.net".format("https", storage_account_name), credential=storage_account_key)
print("Client successfuly created!")
return datalake_client
def create_list_paths(chronological_date,
container_initial_folder="name3",
container_second_folder="name4",
storage_container_name="name1",
storage_account_name="name2"
):
list_of_paths=list()
print("1. success")
paths = file_system_client.get_paths("{0}/{1}/0/{2}/{3}/{4}".format(container_initial_folder, container_second_folder, chronological_date[0], chronological_date[1], chronological_date[2]), recursive=True)
print("2. success")
list_of_paths=["abfss://{0}#{1}.dfs.core.windows.net/".format(storage_container_name, storage_account_name)+path.name for path in paths if ".avro" in path.name]
print("3. success")
list_of_paths=functools.reduce(operator.iconcat, result, [])
return list_of_paths
service_client = datalake_connector(storage_account_name, storage_account_key)
file_system_client = service_client.get_file_system_client(file_system=storage_container_name)
try:
list_of_paths=[]
executor=ThreadPoolExecutor(max_workers=8)
print("Start path extraction!")
list_of_paths=[executor.submit(create_list_paths, i, container_initial_folder, storage_container_name, storage_account_name).result() for i in date_list]
except:
print("no success")
print(sys.exc_info())
Unfortunately the StorageErrorException cannot be handled for a reason, I am still getting the following stdout:
Listing [Python 3.Docs]: Compound statements - The try statement.
There are several ways of achieving this. Here's one:
try:
# ...
except StorageErrorException:
pass
except:
print(sys.exc_info()[1])
Note that except: is tricky because you might silently handle exceptions that you shouldn't. Another way would be to catch any exception the code could raise explicitly.
try:
# ...
except StorageErrorException:
pass
except (SomeException, SomeOtherException, SomeOtherOtherException) as e:
print(e)
Quickly browsing [MS.Docs]: filedatalake package and the sourcecode, revealed that StorageErrorException (which extends [MS.Docs]: HttpResponseError class) is the one that you need to handle.
Might want to check [SO]: About catching ANY exception.
Related to the failure of catching the exception, apparently there are 2 having the same name:
azure.storage.blob._generated.models._models_py3.StorageErrorException (currently imported)
azure.storage.filedatalake._generated.models._models_py3.StorageErrorException
I don't know the rationale (I didn't work with the package), but given the fact the package raises an exception defined in another package when it also defines one with the same name, seems lame. Anyway importing the right exception solves the problem.
As a side note, when dealing with this kind of situation, don't only import the base name, but work with the fully qualified one:
import azure.storage.filedatalake._generated.models.StorageErrorException
you want to compare the type of the exception, change your condition to:
if type(e)==AccountIsDisabled:
example:
class AccountIsDisabled(Exception):
pass
print("try #1")
try:
raise AccountIsDisabled
except Exception as e:
if type(e)==AccountIsDisabled:
pass
else:
print(e)
print("try #2")
try:
raise Exception('hi', 'there')
except Exception as e:
if type(e)==AccountIsDisabled:
pass
else:
print(e)
Output:
try #1
try #2
('hi', 'there')
I often read that in python "it is easier to ask for forgiveness then for permission", so it is sometimes considered better to use try except instead of if.
I often have statements like
if (not os.path.isdir(dir)):
os.mkdir(dir).
The likely replacement would be
try:
os.mkdir(dir)
except OSError:
pass.
However I would like to be more specific and only ignore the errno.EEXIST, as this is the only error that is expected to happen and I have no idea what could happen.
try:
os.mkdir(dir)
except OSError:
if(OSError.errno != errno.EEXIST):
raise
else:
pass.
Seems to do the trick. But this is really bulky and will 'pollute' my code and reduce readability if I need plenty of these code-blocks. Is there a pythonic way to do this in Python 2.X? What is the standard procedure to handle such cases?
edits:
use raise instead raise OSerror as pointed out by #Francisco Couzo
I use Python 2.7
I just stumbled across the probably most elegant solution: creating the ignored context manager:
import errno
from contextlib import contextmanager
#contextmanager
def ignorednr(exception, *errornrs):
try:
yield
except exception as e:
if e.errno not in errornrs:
raise
pass
with ignorednr(OSError, errno.EEXIST):
os.mkdir(dir)
This way I just have the ugly job of creating the context manager once, from then on the syntax is quite nice and readable.
The solution is taken from https://www.youtube.com/watch?v=OSGv2VnC0go.
If you are calling it multiple times with different args, put it in a function:
def catch(d, err):
try:
os.mkdir(d)
except OSError as e:
if e.errno != err:
raise
Then call the function passing in whatever args:
catch(, "foo", errno.EEXIST)
You could also allow the option of passing passing multiple errno's if you wanted more:
def catch(d, *errs):
try:
os.mkdir(d)
except OSError as e:
if e.errno not in errs:
raise
catch("foo", errno.EEXIST, errno.EPERM)
This example is for exception OSError : 17, 'File exists'
import sys
try:
value = os.mkdir("dir")
except:
e = sys.exc_info()[:2]
e = str(e)
if "17" in e:
raise OSError #Perform Action
else:
pass
Just change the number 17 to your exception number. You can get a better explanation at this link.
I'm a Python learner, trying to handle a few scenarios:
Reading a file.
Formatting Data.
Manipulating/Copying Data.
Writing a file.
So far, I have:
try:
# Do all
except Exception as err1:
print err1
#File Reading error/ File Not Present
except Exception as err2:
print err2
# Data Format is incorrect
except Exception as err3:
print err3
# Copying Issue
except Exception as err4:
print err4
# Permission denied for writing
The idea of implementing in this fashion is to catch the exact error for all different scenarios. I can do it in all separate try/except blocks.
Is this possible? And reasonable?
Your try blocks should be as minimal as possible, so
try:
# do all
except Exception:
pass
is not something you want to do.
The code in your example won't work as you expect it to, because in every except block you are catching the most general exception type Exception. In fact, only the first except block will ever be executed.
What you want to be doing is having multiple try/except blocks, each one responsible for as few things as possible and catching the most specific exception.
For example:
try:
# opening the file
except FileNotFoundException:
print('File does not exist')
exit()
try:
# writing to file
except PermissionError:
print('You do not have permission to write to this file')
exit()
However, sometimes it is appropriate to catch different types of exceptions, in the same except block or in several blocks.
try:
ssh.connect()
except (ConnectionRefused, TimeoutError):
pass
or
try:
ssh.connect()
except ConnectionRefused:
pass
except TimeoutError:
pass
As DeepSpace stated,
Your try blocks should be as minimal as possible.
If you want to achieve
try:
# do all
except Exception:
pass
Then you might as well do something like
def open_file(file):
retval = False
try:
# opening the file succesful?
retval = True
except FileNotFoundException:
print('File does not exist')
except PermissionError:
print('You have no permission.')
return retval
def crunch_file(file):
retval = False
try:
# conversion or whatever logical operation with your file?
retval = True
except ValueError:
print('Probably wrong data type?')
return retval
if __name__ == "__main__":
if open_file(file1):
open(file1)
if open_file(file2) and crunch_file(file2):
print('opened and crunched')
Yes this is possible.
Just say as example:
try:
...
except RuntimeError:
print err1
except NameError:
print err2
...
Just define the exact Error you want to intercept.
I created a class named Options. It works fine but not not with Python 2.
And I want it to work on both Python 2 and 3.
The problem is identified: FileNotFoundError doesn t exist in Python 2.
But if I use IOError it doesn t work in Python 3
Changed in version 3.3: EnvironmentError, IOError, WindowsError, VMSError, socket.error, select.error and mmap.error have been merged into OSError.
What should I do ???(Please do not discuss my choice of portability, I have reasons.)
Here s the code:
#!/usr/bin/python
#-*-coding:utf-8*
#option_controller.py
#Walle Cyril
#25/01/2014
import json
import os
class Options():
"""Options is a class designed to read, add and change informations in a JSON file with a dictionnary in it.
The entire object works even if the file is missing since it re-creates it.
If present it must respect the JSON format: e.g. keys must be strings and so on.
If something corrupted the file, just destroy the file or call read_file method to remake it."""
def __init__(self,directory_name="Cache",file_name="options.json",imported_default_values=None):
#json file
self.option_file_path=os.path.join(directory_name,file_name)
self.directory_name=directory_name
self.file_name=file_name
#self.parameters_json_file={'sort_keys':True, 'indent':4, 'separators':(',',':')}
#the default data
if imported_default_values is None:
DEFAULT_INDENT = 2
self.default_values={\
"translate_html_level": 1,\
"indent_size":DEFAULT_INDENT,\
"document_title":"Titre"}
else:
self.default_values=imported_default_values
def read_file(self,read_this_key_only=False):
"""returns the value for the given key or a dictionary if the key is not given.
returns None if it s impossible"""
try:
text_in_file=open(self.option_file_path,'r').read()
except FileNotFoundError:#not 2.X compatible
text_in_file=""#if the file is not there we re-make one with default values
if text_in_file=="":#same if the file is empty
self.__insert_all_default_values()
text_in_file=open(self.option_file_path,'r').read()
try:
option_dict=json.loads(text_in_file)
except ValueError:
#if the json file is broken we re-make one with default values
self.__insert_all_default_values()
text_in_file=open(self.option_file_path,'r').read()
option_dict=json.loads(text_in_file)
if read_this_key_only:
if read_this_key_only in option_dict:
return option_dict[read_this_key_only]#
else:
#if the value is not there it should be written for the next time
if read_this_key_only in self.default_values:
self.add_option_to_file(read_this_key_only,self.default_values[read_this_key_only])
return self.default_values[read_this_key_only]
else:
#impossible because there is not default value so the value isn t meant to be here
return None
else:
return option_dict
def add_option_to_file(self,key,value):#or update
"""Adds or updates an option(key and value) to the json file if the option exists in the default_values of the object."""
option_dict=self.read_file()
if key in self.default_values:
option_dict[key]=value
open(self.option_file_path,'w').write(\
json.dumps(option_dict,sort_keys=True, indent=4, separators=(',',':')))
def __insert_all_default_values(self):
"""Recreate json file with default values.
called if the document is empty or non-existing or corrupted."""
try:
open(self.option_file_path,'w').write(\
json.dumps(self.default_values,sort_keys=True, indent=4, separators=(',',':')))
except FileNotFoundError:
os.mkdir(self.directory_name)#Create the directory
if os.path.isdir(self.directory_name):#succes
self.__insert_all_default_values()
else:
print("Impossible to write in %s and file %s not found" % (os.getcwd(),self.option_file_path))
#demo
if __name__ == '__main__':
option_file_object=Options()
print(option_file_object.__doc__)
print(option_file_object.read_file())
option_file_object.add_option_to_file("","test")#this should have no effect
option_file_object.add_option_to_file("translate_html_level","0")#this should have an effect
print("value of translate_html_level:",option_file_object.read_file("translate_html_level"))
print(option_file_object.read_file())
If FileNotFoundError isn't there, define it:
try:
FileNotFoundError
except NameError:
FileNotFoundError = IOError
Now you can catch FileNotFoundError in Python 2 since it's really IOError.
Be careful though, IOError has other meanings. In particular, any message should probably say "file could not be read" rather than "file not found."
You can use the base class exception EnvironmentError and use the 'errno' attribute to figure out which exception was raised:
from __future__ import print_function
import os
import errno
try:
open('no file of this name') # generate 'file not found error'
except EnvironmentError as e: # OSError or IOError...
print(os.strerror(e.errno))
Or just use IOError in the same way:
try:
open('/Users/test/Documents/test') # will be a permission error
except IOError as e:
print(os.strerror(e.errno))
That works on Python 2 or Python 3.
Be careful not to compare against number values directly, because they can be different on different platforms. Instead, use the named constants in Python's standard library errno module which will use the correct values for the run-time platform.
The Python 2 / 3 compatible way to except a FileNotFoundError is this:
import errno
try:
with open('some_file_that_does_not_exist', 'r'):
pass
except EnvironmentError as e:
if e.errno != errno.ENOENT:
raise
Other answers are close, but don't re-raise if the error number doesn't match.
Using IOError is fine for most cases, but for some reason os.listdir() and friends raise OSError instead on Python 2. Since IOError inherits from OSError it's fine to just always catch OSError and check the error number.
Edit: The previous sentence is only true on Python 3. To be cross compatible, instead catch EnvironmentError and check the error number.
For what it's worth, although the IOError is hardly mentioned in Python 3's official document and does not even showed up in its official Exception hierarchy, it is still there, and it is the parent class of FileNotFoundError in Python 3. See python3 -c "print(isinstance(FileNotFoundError(), IOError))" giving you a True. Therefore, you can technically write your code in this way, which works for both Python 2 and Python 3.
try:
content = open("somefile.txt").read()
except IOError: # Works in both Python 2 & 3
print("Oops, we can not read this file")
It might be "good enough" in many cases. Although in general, it is not recommended to rely on an undocumented behavior. So, I'm not really suggesting this approach. I personally use Kindall's answer.