Breakpoint when encountering RunTimeWarning - python

I am having trouble getting the following code to work in Python:
for t in range(-5,5):
try:
a = np.log(t)
except RunTimeWarning:
breakpoint()
What I want is to execute the breakpoint statement, when the log function is evaluated from -5 to 0.


When using np.log(0), numpy won't throw an exception but a warning. These are significantly different from exceptions as an exception will stop the current code from running, whereas a warning just notifies you but keeps the code running.
Take this example:
>>> x = np.log(0)
Warning (from warnings module):
File "<pyshell#1>", line 1
RuntimeWarning: divide by zero encountered in log
>>> x
-inf
As you can see the variable x does get a value assigned -> the warning never kept log(...) from running and returning a value.
So to solve your problem, you'd have to check if the value passed to your code is 0 before calling np.log. Or if you don't want -inf to be a valid return value but still need to call np.log for some reason, you can check if the value is -inf after the call. However it's best practise not to write code that causes warnings so you should probably just check if it's 0 before calling log.
You can read more about warnings in the python docs.

Related

Raise an exception if RuntimeWarning found during pytest execution

I want to raise an error if RuntimeWarning's occurred during pytest call. The problem is, the solutions below are doing nothing - each time I call pytest, RuntimeWarnings are still warnings:
import warnings
warnings.filterwarnings("error", category=RuntimeWarning)
# setup.cfg
filterwarnings =
error::RuntimeWarning
default
pytest -W error::RuntimeWarning
Meanwhile, ignore::RuntimeWarning (and error::DeprecationWorning)works as expected. What's wrong with my code?
Warning example below:
/app/src/pbp_data_platfrom/tests/test_service.py:20: RuntimeWarning: coroutine 'AsyncMock.__call__' was never awaited
assert send_events_mock.called_count == 1
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
P.S. I know I can add the error option into filterwarning so it will raise an error each time any warning found (and it will work, I've tested it a couple of times). But this solution looks a bit clunky.

Python test runner - stestr return codes

Is there any documentation for stestr return codes. I tried to look into https://stestr.readthedocs.io/en/latest/MANUAL.html
but it only talks about non zero return codes but what are the typical return codes, it returns. I am trying to use stestr list

There currently isn't any documentation of the specific returns codes in every error case and honestly the actually value of the exit code is pretty ad-hoc throughout the code base. That being said all the stestr commands will return zero if the command was successful and the value will be greater than zero to indicate an error occurred (which includes a failed test). But adding documentation would be a very welcome addition. You could start by filing an issue about it: https://github.com/mtreinish/stestr/issues/new/choose
The documentation that currently exists that I think you're referring to is about the use of the --subunit flag on several of the commands. That is there because the behavior is different with subunit output enabled. Normally stestr will return an exit code > 0 when there are test failures that occurred during the run. But if you're using the --subunit flag it will always return with 0 even if there are test failures. The only case it will return > 0 is when there was an error generating subunit or another internal error. I wanted to make sure that this difference was clear for users since it might be unexpected.

Python still throwing exception I've explicitly excepted

I've got some code that I know will fail in the edge case where the list is empty so I added exception handling for IndexError.
But, despite handling the exception, it's still getting raised.
~/Code/foo.py in decode_tokens(tokens)
196 new_toks.append(new_tok)
197 else:
--> 198 try: new_toks[-1] += new_tok
199 except IndexError: pass
200 new_elem = True
IndexError: list index out of range
I don't understand how if I'm explicitly excepting IndexError why it's still getting raised and aborting script execution.
Edit: adding that this is Python 3.6 running in a Jupyter notebook. As it looked like a python error I didn't think that was relevant (but it sounds like it might be.)

Editors that support IPython, which include Jupyter, do not always reload modules even after they have been edited. I think this might be an unfortunate coincidence; the line that the error is thrown happens to coincide with the line of your exception handler after an edit. Likely the code that threw the error isn't what's at that line now. You may wish to force a reload of imported modules each time you run a script, or at least restart the underlying IPython kernel for now.

For this recursive code why does Python 2.7 not give a stack overflow error but 3.5 does?

Earlier today I read about this strange case in Python, Java & JS:
try:
return True
finally:
return False
Which returns False.
So, I decided to toy around with it:
def caseThree():
try:
caseThree()
except:
print("Error")
caseThree()
finally:
return False
print(caseThree())
In Python 2.7 this returns:
Error
False
However, in Python 3.5:
Error
Fatal Python error: Cannot recover from stack overflow.
Current thread 0x000025ec (most recent call first):
File "`<stdin>`", line 3 in caseThree
the last line is repeated until you eventually get: ...
Can anyone explain why 2.7's code doesn't result in a stack overflow, while 3.5 does?

It seems that the error you're encountering is actually expected, as it is explicitly tested for in Lib/test/test_sys.py function test_recursionlimit_fatalerror.
Now, without criticizing your colorful experiments, this is also the cause of a bug that causes a segfault (sometimes, see issue); there has already been one report of this to the Python bug tracker as issue 28179.
Keep an eye on that thread if you're curious as to what is causing this.

SPSS-Python Script stops with an Error when spss commands inside spss.Submit() would create a warning

Let's assume I have two lists of variables
list a: a1 a2 a3
list b: b1 b2 b3
which I want to process in a way like this:
TEMPORARY.
SELECT IF a1=b1.
FREQUENCY someVar.
TEMPORARY.
SELECT IF a2=b2.
FREQUENCY someVar.
TEMPORARY.
SELECT IF a2=b2.
FREQUENCY someVar.
I tried tried to do this within a python loop:
BEGIN PROGRAM.
import spss
la = ['a1', 'a2', 'a3']
lb = ['b1', 'b2', 'b3']
for a, b in zip(la, lb):
spss.Submit('''
TEMPORARY.
SELECT IF %s=%s.
FREQUENCY someVar.
''' % (a, b))
END PROGRAM.
So far so good. This works except when the SELECT IF command would create an empty dataset. Outside the Python program block this would cause the following warning message in the output viewer:
No cases were input to this procedure. Either there are none in the
working data file or all of them have been filtered out. Execution of
this command stops.
But inside a Python block it causes an Error and the python script to stop.
Traceback (most recent call last):
File "", line 7, in
File "C:\PROGRA~1\ibm\SPSS\STATIS~1\23\Python\Lib\site-packages\spss\spss.py", line 1527, in Submit
raise SpssError,error spss.errMsg.SpssError: [errLevel 3] Serious error.
Is there a way to run this loop (which might produce temporary empty data sets and therefore warnings) inside of python?

Yes, if you wrap the problematic function inside a try-except construct:
for a, b in zip(la, lb):
try:
spss.Submit('''
TEMPORARY.
SELECT IF %s=%s.
FREQUENCY someVar.
''' % (a, b))
except:
pass

You could also use Python APIs to calculate how many cases ai=bi and execute conditional blocks according to this.
So for example if only less than 5 valid cases remain you may not want to produce any output (or some output suggesting no output is being produced due to low base sizes). If under 50 cases remain then you may want to run frequencies and if more than 50 cases then run descriptives ect ect. There are a number of ways you get the case count, which approach is best and most efficient perhaps depends on your data set and end goal.
See for example:
spss.GetCaseCount
Here's another approach where you can get case count like statistics from active dataset to inspire further ideas.

Adding some explanation: Statistics syntax errors have an associated severity level between 1 (lowest) and 5. You will probably never see a 5, because that means that the system has gone down in flames. When running syntax from Python via Submit, level 1 and 2 errors, which are warnings that don't stop the syntax from running, are executed normally. Level 3 and higher raise exceptions. You can handle those in your Python code via the try/except mechanism as was suggested above.

I experimented with SET MXWARNS, however neither setting it to zero, nor setting it to a very high value (e.g. 1000000) worked. The warnings were still converted into errors. So I wrote this work-around:
import codecs
import re
import sys
import spss
from spssaux import getShow
def submit_syntax(sps_filename):
output_on = spss.IsOutputOn()
spss.SetOutput("off")
unicode_mode = getShow("unicode") == u"Yes"
encoding = "utf-8-sig" if unicode_mode else getShow("locale").split(u".")[-1]
if output_on:
spss.SetOutput("on")
with codecs.open(sps_filename, encoding=encoding) as f:
syntax = f.read()
statements = re.split(ur"\. *\r\n", syntax, flags=re.MULTILINE)
for stmtno, statement in enumerate(statements, 1):
if statement.startswith(u"*"):
continue
try:
spss.Submit(statement)
except spss.SpssError:
# "no cases were input" warnings are translated into errors.
if not spss.GetCaseCount() and spss.GetLastErrorLevel() <= 3:
continue
msg = u"ERROR in statement #%d: %s [%s]"
raise RuntimeError(msg % (stmtno, statement, sys.exc_info()[1]))

Categories