I'm still kinda new with Python, using Pandas, and I've got some issues debugging my Python script.
I've got the following warning message :
[...]\pandas\core\index.py:756: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
return self._engine.get_loc(key)
And can't find where it's from.
After some research, I tried to do that in the Pandas lib file (index.py):
try:
return self._engine.get_loc(key)
except UnicodeWarning:
warnings.warn('Oh Non', stacklevel=2)
But that didn't change anything about the warning message.
You can filter the warnings to raise which will enable you to debug (e.g. using pdb):
import warnings
warnings.filterwarnings('error')
*The warnings filter can be managed more finely (which is probably more appropriate) e.g.:
warnings.filterwarnings('error', category=UnicodeWarning)
warnings.filterwarnings('error', message='*equal comparison failed*')
Multiple filters will be looked up sequentially. ("Entries closer to the front of the list override entries later in the list, if both match a particular warning.")
You can also use the commandline to control the warnings:
python -W error::UnicodeWarning your_code.py
From the man page:
-W argument
[...] error to raise an exception instead of printing a warning message.
This will have the same effect as putting the following in your code:
import warnings
warnings.filterwarnings('error', category=UnicodeWarning)
As was already said in Andy's answer.
The most informative way to investigate a warning is to convert it into an error (Exception) so you can see its full stacktrace:
import warnings
warnings.simplefilter("error")
See warnings.
If you enable logging in python, then when an exception is received you can use the method logging.exception to log when an exception has been caught - this method will print out a nicely formatted stack trace that shows you exactly in the code where the exception originated. See the python document on logging for more information.
import logging
log = logging.getLogger('my.module.logger')
try:
return self._engine.get_loc(key)
except UnicodeWarning:
log.exception('A log message of your choosing')
Alternatively, you can get a tuple that contains details of the exception in your code by calling sys.exc_info() (this requires you to import the sys module).
Related
I'm importing some Python modules and they will raise exception with calls like raise TypeError(xyz). And now I want to change the line number of the reported exceptions if there is any.
I couldn't find the right solution on the site for my case, the one I found all required the raise() to be in a try and except block. With the warnings module I can do warnings.formatwarning to customize its format. Can I do the same with exception?
Does anyone have any experience using the python library Refextract, package index here. I'm using python 3.4 in Spyder 3.0.0. Pip install went fine, it said the installation was succesfull, in the correct folder (in the Libs/Site packages/ folder). But when I try to load it, it throws in a error message, and I can't really figure out what it means.
Here is my code snippet: from refextract import extract_journal_reference (as displayed in the manual), which gives the following error:
from refextract import extract_journal_reference
File "C:\path\to\python-3.4.3.amd64\lib\site-packages\refextract\references\api.py", line 96
raise FullTextNotAvailableError("URL not found: '{0}'".format(url)), None, sys.exc_info()[2]
^
SyntaxError: invalid syntax
This is just the importing, not yet the specifying of the link. Does anyone know what to do with this error?
The code that raises the exception is using syntax which is valid in Python2, but not in Python3.
In Python2, it is possible to associate an arbitrary traceback with an exception with this variation of the raise statement.
raise FooError, 'A foo has happened', a_traceback_object
or as in this case:
raise FooError('A foo has happened'), None, a_traceback_object.
In Python3, the traceback object must be explicitly assigned to the exception's __traceback__ attribute:
ex = FooError('A foo has happened')
ex.__traceback__ = a_traceback_object
raise ex
See PEP 3109 for discussion of this change (summary: reduce the number of different ways of using raise).
As far as I can see, the package does not claim to be python3-compliant, so you need to run it with Python2 (specifically, 2.7).
I am using this to issue warnings while parsing a configuration file. All sorts of errors could happen while doing this - some fatal, some not. All those non-fatal errors should not interrupt the parsing, but they must not escape user's attention either. This is where the warnings module comes in.
I am currently doing this (pseudo code):
while parsing:
try:
get dictionary["token"]
except KeyError:
warnings.warn("Looks like your config file don't have that token")
This all looks readable and cozy, but the message looks something like this:
C:\Users\Renae\Documents\test.py:3: UserWarning: Looks like your config file don't have that token
warnings.warn("Looks like your config file don't have that token")
Why is it printed twice? Should I be doing some sort of initialization before issuing warnings (like the logging module)? The standard docs doesn't have a tutorial on this (or does it?).
What differentiates warnings from print(), stdout or stderr?
When you are using warnings module, the second print is actually the stack, you can control what level you want to print using the stacklevel argument. Example -
import warnings
def warn():
warnings.warn("Blah",stacklevel=2)
warn()
This results -
a.py:6: UserWarning: Blah
warn()
If you set it to a non-existent level , in the above example lets say 3 , then it does not print the stack, Example -
def warn():
warnings.warn("Blah",stacklevel=3)
Result -
sys:1: UserWarning: Blah
Though as you can see, the file also changed to sys:1 . You might want to show a meaning stack over there (maybe something like stacklevel=2 for the caller of the function in which the warning was raised) .
Another way to suppress this would be to use warnings.warn_explicit() method and manually pass in the filename and linenumber (linenumber should not have any actual code in it, otherwise that code would be printed), though I do not advice this.
Also, yes when using warnings module, the data normally goes into sys.stderr , but you can also easily send warnings to a different file by using different functions like - warnings.showwarning()
I'm writing some code to manipulate the Windows clipboard. The first thing I do is to try and open the clipboard with OpenClipboard() function from the Windows API:
if OpenClipboard(None):
# Access the clipboard here
else:
# Handle failure
This function can fail. So if it does, I would like to raise an exception. My question is, which of the standard Python exceptions should I raise? I'm thinking WindowsError would be the right one, but not sure. Could someone please give me a suggestion?
It is better to avoid raising standard exceptions directly. Create your own exception class, inherit it from the most appropriate one (WindowsError is ok) and raise it. This way you'll avoid confusion between your own errors and system errors.
Raise the windows error and give it some extra infomation, for example
raise WindowsError("Clipboard can't be opened")
Then when its being debugged they can tell what your windows error means rather than just a random windowserror over nothing.
WindowsError seems a reasonable choice, and it will record extra error information for you. From the docs:
exception WindowsError
Raised when a Windows-specific error occurs or when the error number does not correspond to an errno value. The winerror and strerror values are created from the return values of the GetLastError() and FormatMessage() functions from the Windows Platform API. The errno value maps the winerror value to corresponding errno.h values. ...
In my python script I would like to trap a "Data truncated for column 'xxx'" warning durnig my query using MySql.
I saw some posts suggesting the code below, but it doesn' work.
Do you know if some specific module must be imported or if some option/flag should be called before using this code?
Thanks all
Afeg
import MySQLdb
try:
cursor.execute(some_statement)
# code steps always here: No Warning is trapped
# by the code below
except MySQLdb.Warning, e:
# handle warnings, if the cursor you're using raises them
except Warning, e:
# handle warnings, if the cursor you're using raises them
Warnings are just that: warnings. They get reported to (usually) stderr, but nothing else is done. You can't catch them like exceptions because they aren't being raised.
You can, however, configure what to do with warnings, and turn them off or turn them into exceptions, using the warnings module. For instance, warnings.filterwarnings('error', category=MySQLdb.Warning) to turn MySQLdb.Warning warnings into exceptions (in which case they would be caught using your try/except) or 'ignore' to not show them at all. You can (and probably should) have more fine-grained filters than just the category.
Raise MySQL Warnings as errors:
import warnings, MySQLdb
warnings.filterwarnings('error', category=MySQLdb.Warning)
To ignore instead of raising an error, replace "error" with "ignore".
Handle them in a try-except block like:
try:
# a MySQL DB operation that raises a warning
# for example: a data truncated warning
except Warning as a_warning:
# do something here
I would try first to set the sql_mode. You can do this at the session level. If you execute a:
SET ##sql_mode:=TRADITIONAL;
(you could also set it at the server level, but you need access to the server to do that. and, that setting an still be overridden at the session level, so the bottom line is, always set it at the session level, immediately after establishing the connection)
then many things that are normally warnings become errors. I don't know how those manifest themselves at the python level, but the clear advantage is that the changes are not stored in the database. See: http://dev.mysql.com/doc/refman/5.1/en/server-sql-mode.html#sqlmode_traditional
If you want to trap it to ignore it see "Temporarily suppressing warnings" in Python's documentation:
https://docs.python.org/2/library/warnings.html#temporarily-suppressing-warnings
import warnings
with warnings.catch_warnings():
warnings.simplefilter("ignore")
# Put here your query raising a warning
Else, just read the doc about "warnings" module of Python, you can transform them into exception if you want to catch them later, etc... it's all here: https://docs.python.org/2/library/warnings.html
Just to add to Thomas Wouters reply, there is no need to import the warnings module to turn them into errors. Just run your script with "-W error" (or ignore) as flag for Python.
Have you tried using MySQL's SHOW WARNINGS command?
Stop using MySQLdb. It has such stupid behavior as truncating data and issuing only a warning. Use oursql instead.