We Keep Coding

How to traceback a warning to get the line number where it occurred?

I have some codes written in Python and used pandas to make and manipulate some DataFrames. In a few of them, I get some warnings. for example settingwithcopywarning or performancewarning. I want to catch the line number that warning has occured and for that, I wrote the following code. I catch everything but the line number.
scripts = ['myfile.py', 'myotherfile.py']
with warnings.catch_warnings(record=True) as w:
# Cause all warnings to always be triggered.
warnings.simplefilter("default")
for s in scripts:
with open(s) as f:
try:
exec(f.read())
except Exception as e:
print('An Error happend during the execution', e)
finally:
f.close()
print(color.orange('There are {} error/s happend in {}.'.format(len(w), s)))
for i in range(0, len(w)):
print(color.green('LineNo: '), w[i].lineno)
print(color.green('Line: '), w[i].line)
print(color.green('warning category: '), w[i].category.__name__)
print(color.green('Warning: '), w[i].message)
print(color.green('filename: '), w[i].file)
print(color.cyan('-' * 40))
also I changed the warnings.simplefilter("default") to warnings.simplefilter("error") to trace them back however, it works just when the first exception happens.which is logical but I want to get the whole traceback for all warnings. by the way, that does not work when the warning time is PerformnceWarning
try:
exec(f.read())
except Exception:
lg.getLogger("error_logger").error(traceback.format_exc())
I expect to get the line number for the original file not the python core modules that the warnings logged

Searching for equivalent of FileNotFoundError in Python 2

I created a class named Options. It works fine but not not with Python 2.
And I want it to work on both Python 2 and 3.
The problem is identified: FileNotFoundError doesn t exist in Python 2.
But if I use IOError it doesn t work in Python 3
Changed in version 3.3: EnvironmentError, IOError, WindowsError, VMSError, socket.error, select.error and mmap.error have been merged into OSError.
What should I do ???(Please do not discuss my choice of portability, I have reasons.)
Here s the code:
#!/usr/bin/python
#-*-coding:utf-8*
#option_controller.py
#Walle Cyril
#25/01/2014
import json
import os
class Options():
"""Options is a class designed to read, add and change informations in a JSON file with a dictionnary in it.
The entire object works even if the file is missing since it re-creates it.
If present it must respect the JSON format: e.g. keys must be strings and so on.
If something corrupted the file, just destroy the file or call read_file method to remake it."""
def __init__(self,directory_name="Cache",file_name="options.json",imported_default_values=None):
#json file
self.option_file_path=os.path.join(directory_name,file_name)
self.directory_name=directory_name
self.file_name=file_name
#self.parameters_json_file={'sort_keys':True, 'indent':4, 'separators':(',',':')}
#the default data
if imported_default_values is None:
DEFAULT_INDENT = 2
self.default_values={\
"translate_html_level": 1,\
"indent_size":DEFAULT_INDENT,\
"document_title":"Titre"}
else:
self.default_values=imported_default_values
def read_file(self,read_this_key_only=False):
"""returns the value for the given key or a dictionary if the key is not given.
returns None if it s impossible"""
try:
text_in_file=open(self.option_file_path,'r').read()
except FileNotFoundError:#not 2.X compatible
text_in_file=""#if the file is not there we re-make one with default values
if text_in_file=="":#same if the file is empty
self.__insert_all_default_values()
text_in_file=open(self.option_file_path,'r').read()
try:
option_dict=json.loads(text_in_file)
except ValueError:
#if the json file is broken we re-make one with default values
self.__insert_all_default_values()
text_in_file=open(self.option_file_path,'r').read()
option_dict=json.loads(text_in_file)
if read_this_key_only:
if read_this_key_only in option_dict:
return option_dict[read_this_key_only]#
else:
#if the value is not there it should be written for the next time
if read_this_key_only in self.default_values:
self.add_option_to_file(read_this_key_only,self.default_values[read_this_key_only])
return self.default_values[read_this_key_only]
else:
#impossible because there is not default value so the value isn t meant to be here
return None
else:
return option_dict
def add_option_to_file(self,key,value):#or update
"""Adds or updates an option(key and value) to the json file if the option exists in the default_values of the object."""
option_dict=self.read_file()
if key in self.default_values:
option_dict[key]=value
open(self.option_file_path,'w').write(\
json.dumps(option_dict,sort_keys=True, indent=4, separators=(',',':')))
def __insert_all_default_values(self):
"""Recreate json file with default values.
called if the document is empty or non-existing or corrupted."""
try:
open(self.option_file_path,'w').write(\
json.dumps(self.default_values,sort_keys=True, indent=4, separators=(',',':')))
except FileNotFoundError:
os.mkdir(self.directory_name)#Create the directory
if os.path.isdir(self.directory_name):#succes
self.__insert_all_default_values()
else:
print("Impossible to write in %s and file %s not found" % (os.getcwd(),self.option_file_path))
#demo
if __name__ == '__main__':
option_file_object=Options()
print(option_file_object.__doc__)
print(option_file_object.read_file())
option_file_object.add_option_to_file("","test")#this should have no effect
option_file_object.add_option_to_file("translate_html_level","0")#this should have an effect
print("value of translate_html_level:",option_file_object.read_file("translate_html_level"))
print(option_file_object.read_file())

If FileNotFoundError isn't there, define it:
try:
FileNotFoundError
except NameError:
FileNotFoundError = IOError
Now you can catch FileNotFoundError in Python 2 since it's really IOError.
Be careful though, IOError has other meanings. In particular, any message should probably say "file could not be read" rather than "file not found."

You can use the base class exception EnvironmentError and use the 'errno' attribute to figure out which exception was raised:
from __future__ import print_function
import os
import errno
try:
open('no file of this name') # generate 'file not found error'
except EnvironmentError as e: # OSError or IOError...
print(os.strerror(e.errno))
Or just use IOError in the same way:
try:
open('/Users/test/Documents/test') # will be a permission error
except IOError as e:
print(os.strerror(e.errno))
That works on Python 2 or Python 3.
Be careful not to compare against number values directly, because they can be different on different platforms. Instead, use the named constants in Python's standard library errno module which will use the correct values for the run-time platform.

The Python 2 / 3 compatible way to except a FileNotFoundError is this:
import errno
try:
with open('some_file_that_does_not_exist', 'r'):
pass
except EnvironmentError as e:
if e.errno != errno.ENOENT:
raise
Other answers are close, but don't re-raise if the error number doesn't match.
Using IOError is fine for most cases, but for some reason os.listdir() and friends raise OSError instead on Python 2. Since IOError inherits from OSError it's fine to just always catch OSError and check the error number.
Edit: The previous sentence is only true on Python 3. To be cross compatible, instead catch EnvironmentError and check the error number.

For what it's worth, although the IOError is hardly mentioned in Python 3's official document and does not even showed up in its official Exception hierarchy, it is still there, and it is the parent class of FileNotFoundError in Python 3. See python3 -c "print(isinstance(FileNotFoundError(), IOError))" giving you a True. Therefore, you can technically write your code in this way, which works for both Python 2 and Python 3.
try:
content = open("somefile.txt").read()
except IOError: # Works in both Python 2 & 3
print("Oops, we can not read this file")
It might be "good enough" in many cases. Although in general, it is not recommended to rely on an undocumented behavior. So, I'm not really suggesting this approach. I personally use Kindall's answer.

Raise two errors at the same time

Is there any way to raise two errors at the same time by using try and except?
For example, ValueError and KeyError.
How do I do that?

The question asks how to RAISE multiple errors not catch multiple errors.
Strictly speaking you can't raise multiple exceptions but you could raise an object that contains multiple exceptions.
raise Exception(
[
Exception("bad"),
Exception("really bad"),
Exception("really really bad"),
]
)
Question: Why would you ever want to do this?
Answer: In a loop when you want to raise an error but process the loop to completion.
For example when unit-testing with unittest2 you might want to raise an exception and keep processing then raise all of the errors at the end. This way you can see all of the errors at once.
def test_me(self):
errors = []
for modulation in self.modulations:
logging.info('Testing modulation = {modulation}'.format(**locals()))
self.digitalModulation().set('value', modulation)
reply = self.getReply()
try:
self._test_nodeValue(reply, self.digitalModulation())
except Exception as e:
errors.append(e)
if errors:
raise Exception(errors)
Python 3.11
Starting with 3.11 you can use ExceptionGroup to raise multiple exceptions.
raise ExceptionGroup("this was bad",
[
Exception("bad"),
Exception("really bad"),
Exception("really really bad"),
]
)

You could raise an error which inherits from both ValueError and KeyError. It would get caught by a catch block for either.
class MyError(ValueError, KeyError):
...

Yes, you can handle more than one error, either using
try:
# your code here
except (ValueError, KeyError) as e:
# catch it, the exception is accessable via the variable e
Or, directly add two "ways" of handling different errors:
try:
# your code here
except ValueError as e:
# catch it, the exception is accessable via the variable e
except KeyError as e:
# catch it, the exception is accessable via the variable e
You may also leave out the "e" variable.
Checkout the documentation: http://docs.python.org/tutorial/errors.html#handling-exceptions

The solution from #shrewmouse still requires to choose an exception class to wrap the caught exceptions.
Following solution uses Exception Chaining via finally to execute code after one exception occurs
We don't need to know beforehand, what exceptions occur
Note that only the first exception that occurs can be detected from the caller via except
if this is a problem, use #Collin's solution above to inherit from all collected exceptions
You'll see the exceptions separated by:
"During handling of the above exception, another exception occurred:"
def raise_multiple(errors):
if not errors: # list emptied, recursion ends
return
try:
raise errors.pop() # pop removes list entries
finally:
raise_multiple(errors) # recursion
If you have a task that needs to be done for each element of a list, you don't need to collect the Exceptions beforehand. Here's an example for multiple file deletion with multiple error reporting:
def delete_multiple(files):
if not files:
return
try:
os.remove(files.pop())
finally:
delete_multiple(files)
PS:
Tested with Python 3.8.5
To print full traceback per exception have a look at traceback.print_exc
The original question is answered since years. But as this page is the top search result for "python raise multiple" I share my approach to fill an (IMHO relevant) gap in the solution spectrum.

try :
pass
except (ValueError,KeyError):
pass
read more about Handling exceptions

You can Raise more than one exception like this:
try:
i = 0
j = 1 / i
except ZeroDivisionError:
try:
i = 'j'
j = 4 + i
except TypeError:
raise ValueError
NOTE: it may be that only the ValueError is raised but this error message seems right:
Traceback (most recent call last):
File "<pyshell#9>", line 3, in <module>
j = 1 / i
ZeroDivisionError: division by zero
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<pyshell#9>", line 7, in <module>
j = 4 + i
TypeError: unsupported operand type(s) for +: 'int' and 'str'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<pyshell#9>", line 9, in <module>
raise ValueError
ValueError

Identify the exception to be used from the traceback

I want to catch an exception when user fails login due to wrong password .
So i make a function using imaplib .I enter a wrong password and get a traceback with error details.
Now my question is actually general.How do you identify the exception we have to mention in our "try and except" body from the error messages?
These is what I got->
>>> count("testarc31#gmail.com","Xbox#36")
Traceback (most recent call last):
File "<pyshell#2>", line 1, in <module>
count("testarc31#gmail.com","Xbox#36")
File "E:\Arindam\py_progs\Mail Notifier\0.0.19\Mail.py", line 24, in count
obj.login(m,p)
File "C:\Python27\lib\imaplib.py", line 500, in login
raise self.error(dat[-1])
error: [AUTHENTICATIONFAILED] Invalid credentials (Failure)
If i want to make a try and except,what will i mention in the exception part?
try:
login(mail,pass):
except ????:
something
Question :
1) What will be ???? here . Can it be deduced directly from the error report?
2) Is there a basic idea to identify what is the exception we have to use from each error we get ?

You want to use something like this:
try:
..code that might raise an exception...
except ExceptionType, e:
...do something...
In your case, that probably want this:
try:
login(mail,pass)
except imaplib.IMAP4.error, e:
print "Ouch -- an error from imaplib!"
To identify the type of an exception, you can look at its exception message. In this case it's just "error" -- unfortunately the module name is not included. You can get a better idea of exactly where it comes from by doing:
try:
login(mail,pass)
except Exception, e:
print type(e)

Writing Pythons Errors to a txt file

I am having trouble getting this to work correctly (obviously) - I am ALMOST there, and I have a good idea of WHY it is not working - just not sure how to make it work.
This is suppose to attempt to read a file into memory, if it fails it goes to the "except" clause of the block of code (that part is 'duh'). The error file prints: "<main.DebugOutput instance at 0x04021EB8>". What I want it to do is print the actual Error. Like a FileIOError or TraceBackError or whatever it is to that error file. This is just the beginning stages and I plan to add things like date stamps and to have it append, not write/create - I just need the actual error printed to the file. Advice?
import os, sys
try:
myPidFile = "Zeznadata.txt"
myOpenPID_File = open(myPidFile, "r") #Attempts to open the file
print "Sucessfully opened the file: \"" + myPidFile + "\"."
except:
print "This file, \"" + myPidFile + "\", does not exist. Please check the file name and try again. "
myFileErr = open("PIDErrorlog.txt", "w")
myStdError = str(sys.stderr)
myFileErr.write(myStdError)
myFileErr.close()
print "\nThis error was logged in the file (and stored in the directory): "

First, you should use a logging library. It will help you deal with different logging levels (info/warn/error), timestamps and more.
http://docs.python.org/library/logging.html
Second, you need to catch the error, and then you can log details about it. This example comes from the Python documentation.
import sys
try:
f = open('myfile.txt')
s = f.readline()
i = int(s.strip())
except IOError as (errno, strerror):
print "I/O error({0}): {1}".format(errno, strerror)
except ValueError:
print "Could not convert data to an integer."
except:
print "Unexpected error:", sys.exc_info()[0]
raise
See how it catches an IOError and assigns the error number and error message to variables? You can have an except block for each type of error you want to deal with. In the last (generic error) block, it uses sys.exc_info()[0] to get the error details.
http://docs.python.org/tutorial/errors.html

the problem is here:
myStdError = str(sys.stderr)
myFileErr.write(myStdError)
sys.stderr is a file-like interface defined in POSIX standards, called standard error, not "error text" that you can write to a file. So what you (probably) wanted to do is:
sys.stderr = myFileErr
This is the python equivalent to python your_python_sctipt.py 2> PIDErrorLog.txt in unix shell.

Use the logging module. The logging module have exception formatters that will help you print exceptions in a pretty way.
http://docs.python.org/library/logging.html

Like Kimvais says, your problem is:
myStdError = str(sys.stderr)
myFileErr.write(myStdError)
sys.stderr is a file handle to stderr (stdout is what print writes to).
You should do something like:
try:
...open file...
except IOError as (errno, strerror):
...open errfile...
errfile.write(strerror)
errfile.close()
Alison's answer is also very good and well written.