I pass an array of values as arguments to a function. This function divides somewhere by the values given as arguments. I want to bypass the calculation for zero-value values so that I don't have to divide by zero.
import numpy as np
def test(t):
e = np.where(t==0,0,10/t)
return e
i = np.arange(0, 5, 1)
print('in: ',i)
o = test(i)
print('out:',o)
Output is
in: [0 1 2 3 4]
out: [ 0. 10. 5. 3.33333333 2.5 ]
<ipython-input-50-321938d419be>:4: RuntimeWarning: divide by zero encountered in true_divide
e = np.where(t==0,0,10/t)
I thought np.where would be the appropriate function for this, but unfortunately I always get a runtime warning 'divide by zero'. So, it does the right thing, but the warning is annoying. I could of course suppress the warning, but I wonder if there is a cleaner solution to the problem?
Use np.divide(10, t, where=t!=0):
import numpy as np
def test(t):
e = np.divide(10, i, where=i!=0)
return e
i = np.arange(0, 5, 1)
print('in: ',i)
o = test(i)
print('out:',o)
Yes, you can test if the divisor is zero, like you do but consider testing if it is equal to a floating point zero 0.0 , or otherwise provide floating point arguments.
Change test
def test(t):
e = np.where(t==0.0,0,10.0/t) # Note : t == 0.0 (a float comparison)
return e
And/or use floating point arguments:
i = np.arange(0., 5., 1.)
Will give you the result with out an exception too.
Related
I have been trying to upgrade a library which has a bunch of geometric operations for scalars so they will work with numpy arrays as well. While doing this I noticed some strange behaviour with numpy divide.
In original code checks a normalised difference between to variables if neither variable is zero, swapping across to numpy this ended up looking something like:
import numpy as np
a = np.array([0, 1, 2, 3, 4])
b = np.array([1, 2, 3, 0, 4])
o = np.zeros(len(a))
o = np.divide(np.subtract(a, b), b, out=o, where=np.logical_and(a != 0, b != 0))
print(f'First implementation: {o}')
where I passed in a output buffer initialised to zero for instances which could not be calculated; this returns:
First implementation: [ 0. -0.5 -0.33333333 0. 0. ]
I had to slightly modify this for scalars as out required an array, but it seemed fine.
a = 0
b = 4
o = None if np.isscalar(a) else np.zeros(len(a))
o = np.divide(np.subtract(a, b), b, out=o, where=np.logical_and(b != 0, a != 0))
print(f'Modified out for scalar: {o}')
returns
Modified out for scalar: 0.0.
Then ran this through some test functions and found a lot of them failed. Digging into this, I found that the first time I call the divide with a scalar with where set to False the function returns zero, but if I called it again, the second time it returns something unpredictable.
a = 0
b = 4
print(f'First divide: {np.divide(b, a, where=False)}')
print(f'Second divide: {np.divide(b, a, where=False)}')
returns
First divide: 0.0
Second divide: 4.0
Looking at the documentation, it says "locations within it where the condition is False will remain uninitialized", so I guess numpy as some internal buffer which is initially set to zero then subsequently it ends up carrying over an earlier intermediate value.
I am struggling to see how I can use divide with or without a where clause; if I use where I get an unpredictable output and if I don't I can't protect against divide by zero. Am I missing something or do I just need to have a different code path in these cases? I realise I'm half way to a different code path already with the out variable.
I would be grateful for any advice.
It looks like a bug to me. But I think you'd want to short-circuit the calls to ufuncs in the case of scalars for performance reasons anyway, so its a question of trying to keep it from being too messy. Since either a or b could be scalars, you need to check them both. Put that check into a function that conditionally returns an output array or None, and you could do
def scalar_test_np_zeros(a, b):
"""Return np.zeros for the length of arguments unless both
arguments are scalar, then None."""
if a_is:=np.isscalar(a) and np.isscalar(b):
return None
else:
return np.zeros(len(a) if a_is else len(b))
a = 0
b = 4
if o := scalar_test_np_zeros(a, b) is None:
o = (a-b)/b if a and b else 0.0
else:
np.divide(np.subtract(a, b), b, out=o,
where=np.logical_and(b != 0, a != 0))
The scalar test would be useful in other code with similar problems.
For what it's worth, if I helps anyone I have come to the conclusion I need to wrap np.divide to use it safely in functions which can take arrays and scalars. This is my wrapping function:
import numpy as np
def divide_where(a, b, where, out=None, fill=0):
""" wraps numpy divide to safely handle where clause for both arrays and scalars
- a: dividend array or scalar
- b: divisor array or scalar
- where: locations where is True a/b will be set
- out: location where data is written to; if None, an output array will be created using fill value
- fill: defines fill value. if scalar and where True value will used; if out not set fill value is used creating output array
"""
if (a_is_scalar := np.isscalar(a)) and np.isscalar(b):
return fill if not where else a / b
if out is None:
out = np.full_like(b if a_is_scalar else a, fill)
return np.divide(a, b, out=out, where=where)
``I need to Write a function which calculates the following math equation and round your answer to 2 decimal places: z = π*ex2/4y.
There are the following contstraints: The input variables x and y are single values (that is, not a list/array). If a division by zero occurs, raise a ValueError. Output should be rounded to 2 decimal places. (hint) You can calculate ex using the NumPy function np.exp(x).
I have done the following but still fail the value error tests (ValueError
Inputs: [0.5, 0]
def test_question_3_ValueError(test_input)):
def custom_function(x, y):
# your code here
a = (np.pi*np.exp(x**2))
b = 4*y
z = a / b
if b < 0:
raise ValueError("Div by zero")
return round(z, 2)
First thing, I believe that the division by 0, will only happen if y = 0, as b = 4*y, and b can be only 0 if y = 0, thus, you should change the if statement for b == 0.
Anoter thing, that if statement should be before calculating the z value, because you want to raise the Error before any computations has been started, and thus stopping everything that follows afterwards.
I am using numpy.log10 to calculate the log of an array of probability values. There are some zeros in the array, and I am trying to get around it using
result = numpy.where(prob > 0.0000000001, numpy.log10(prob), -10)
However, RuntimeWarning: divide by zero encountered in log10 still appeared and I am sure it is this line caused the warning.
Although my problem is solved, I am confused why this warning appeared again and again?
You can turn it off with seterr
numpy.seterr(divide = 'ignore')
and back on with
numpy.seterr(divide = 'warn')
numpy.log10(prob) calculates the base 10 logarithm for all elements of prob, even the ones that aren't selected by the where. If you want, you can fill the zeros of prob with 10**-10 or some dummy value before taking the logarithm to get rid of the problem. (Make sure you don't compute prob > 0.0000000001 with dummy values, though.)
Just use the where argument in np.log10
import numpy as np
np.random.seed(0)
prob = np.random.randint(5, size=4) /4
print(prob)
result = np.where(prob > 0.0000000001, prob, -10)
# print(result)
np.log10(result, out=result, where=result > 0)
print(result)
Output
[1. 0. 0.75 0.75]
[ 0. -10. -0.12493874 -0.12493874]
I solved this by finding the lowest non-zero number in the array and replacing all zeroes by a number lower than the lowest :p
Resulting in a code that would look like:
def replaceZeroes(data):
min_nonzero = np.min(data[np.nonzero(data)])
data[data == 0] = min_nonzero
return data
...
prob = replaceZeroes(prob)
result = numpy.where(prob > 0.0000000001, numpy.log10(prob), -10)
Note that all numbers get a tiny fraction added to them.
Just specify where to calculate log10 as follows:
result = np.log10(prob,where=prob>0)
Here is a demo:
This solution worked for me, use numpy.sterr to turn warnings off followed by where
numpy.seterr(divide = 'ignore')
df_train['feature_log'] = np.where(df_train['feature']>0, np.log(df_train['feature']), 0)
I need to compute x in the following way (legacy code):
x = numpy.where(b == 0, a, 1/b)
I suppose it worked in python-2.x (as it was in a python-2.7 code), but it does not work in python-3.x (if b = 0 it returns an error).
How do I make it work in python-3.x?
EDIT: error message (Python 3.6.3):
ZeroDivisionError: division by zero
numpy.where is not conditional execution; it is conditional selection. Python function parameters are always completely evaluated before a function call, so there is no way for a function to conditionally or partially evaluate its parameters.
Your code:
x = numpy.where(b == 0, a, 1/b)
tells Python to invert every element of b and then select elements from a or 1/b based on elements of b == 0. Python never even reaches the point of selecting elements, because computing 1/b fails.
You can avoid this problem by only inverting the nonzero parts of b. Assuming a and b have the same shape, it could look like this:
x = numpy.empty_like(b)
mask = (b == 0)
x[mask] = a[mask]
x[~mask] = 1/b[~mask]
A old trick for handling 0 elements in an array division is to add a conditional value:
In [63]: 1/(b+(b==0))
Out[63]: array([1. , 1. , 0.5 , 0.33333333])
(I used this years ago in apl).
x = numpy.where(b == 0, a, 1/b) is evaluated in the same way as any other Python function. Each function argument is evaluated, and the value passed to the where function. There's no 'short-circuiting' or other method of bypassing bad values of 1/b.
So if 1/b returns a error you need to either change b so it doesn't do that, calculate it in context that traps traps the ZeroDivisionError, or skips the 1/b.
In [53]: 1/0
---------------------------------------------------------------------------
ZeroDivisionError Traceback (most recent call last)
<ipython-input-53-9e1622b385b6> in <module>()
----> 1 1/0
ZeroDivisionError: division by zero
In [54]: 1.0/0
---------------------------------------------------------------------------
ZeroDivisionError Traceback (most recent call last)
<ipython-input-54-99b9b9983fe8> in <module>()
----> 1 1.0/0
ZeroDivisionError: float division by zero
In [55]: 1/np.array(0)
/usr/local/bin/ipython3:1: RuntimeWarning: divide by zero encountered in true_divide
#!/usr/bin/python3
Out[55]: inf
What are a and b? Scalars, arrays of some size?
where makes most sense if b (and maybe a) is an array:
In [59]: b = np.array([0,1,2,3])
The bare division gives me a warning, and an inf element:
In [60]: 1/b
/usr/local/bin/ipython3:1: RuntimeWarning: divide by zero encountered in true_divide
#!/usr/bin/python3
Out[60]: array([ inf, 1. , 0.5 , 0.33333333])
I could use where to replace that inf with something else, for example a nan:
In [61]: np.where(b==0, np.nan, 1/b)
/usr/local/bin/ipython3:1: RuntimeWarning: divide by zero encountered in true_divide
#!/usr/bin/python3
Out[61]: array([ nan, 1. , 0.5 , 0.33333333])
The warning can be silenced as #donkopotamus shows.
An alternative to seterr is errstate in a with context:
In [64]: with np.errstate(divide='ignore'):
...: x = np.where(b==0, np.nan, 1/b)
...:
In [65]: x
Out[65]: array([ nan, 1. , 0.5 , 0.33333333])
How to suppress the error message when dividing 0 by 0 using np.divide (alongside other floats)?
If you wish to disable warnings in numpy while you divide by zero, then do something like:
>>> existing = numpy.seterr(divide="ignore")
>>> # now divide by zero in numpy raises no sort of exception
>>> 1 / numpy.zeros( (2, 2) )
array([[ inf, inf],
[ inf, inf]])
>>> numpy.seterr(*existing)
Of course this only governs division by zero in an array. It will not prevent an error when doing a simple 1 / 0.
In your particular case, if we wish to ensure that we work whether b is a scalar or a numpy type, do as follows:
# ignore division by zero in numpy
existing = numpy.seterr(divide="ignore")
# upcast `1.0` to be a numpy type so that numpy division will always occur
x = numpy.where(b == 0, a, numpy.float64(1.0) / b)
# restore old error settings for numpy
numpy.seterr(*existing)
I solved it using this:
x = (1/(np.where(b == 0, np.nan, b))).fillna(a)
The numpy.where documentation states:
If x and y are given and input arrays are 1-D, where is
equivalent to::
[xv if c else yv for (c,xv,yv) in zip(condition,x,y)]
So why do you see the error? Take this trivial example:
c = 0
result = (1 if c==0 else 1/c)
# 1
So far so good. if c==0 is checked first and the result is 1. The code does not attempt to evaluate 1/c. This is because the Python interpreter processes a lazy ternary operator and so only evaluates the appropriate expression.
Now let's translate this into numpy.where approach:
c = 0
result = (xv if c else yv for (c, xv, yv) in zip([c==0], [1], [1/c]))
# ZeroDivisionError
The error occurs in evaluating zip([c==0], [1], [1/c]) before even the logic is applied. The generator expression itself can't be evaluated. As a function, numpy.where does not, and indeed cannot, replicate the lazy computation of Python's ternary expression.
I wrote the following script:
import numpy
d = numpy.array([[1089, 1093]])
e = numpy.array([[1000, 4443]])
answer = numpy.exp(-3 * d)
answer1 = numpy.exp(-3 * e)
res = answer.sum()/answer1.sum()
print res
But I got this result and with the error occurred:
nan
C:\Users\Desktop\test.py:16: RuntimeWarning: invalid value encountered in double_scalars
res = answer.sum()/answer1.sum()
It seems to be that the input element were too small that python turned them to be zeros, but indeed the division has its result.
How to solve this kind of problem?
You can't solve it. Simply answer1.sum()==0, and you can't perform a division by zero.
This happens because answer1 is the exponential of 2 very large, negative numbers, so that the result is rounded to zero.
nan is returned in this case because of the division by zero.
Now to solve your problem you could:
go for a library for high-precision mathematics, like mpmath. But that's less fun.
as an alternative to a bigger weapon, do some math manipulation, as detailed below.
go for a tailored scipy/numpy function that does exactly what you want! Check out #Warren Weckesser answer.
Here I explain how to do some math manipulation that helps on this problem. We have that for the numerator:
exp(-x)+exp(-y) = exp(log(exp(-x)+exp(-y)))
= exp(log(exp(-x)*[1+exp(-y+x)]))
= exp(log(exp(-x) + log(1+exp(-y+x)))
= exp(-x + log(1+exp(-y+x)))
where above x=3* 1089 and y=3* 1093. Now, the argument of this exponential is
-x + log(1+exp(-y+x)) = -x + 6.1441934777474324e-06
For the denominator you could proceed similarly but obtain that log(1+exp(-z+k)) is already rounded to 0, so that the argument of the exponential function at the denominator is simply rounded to -z=-3000. You then have that your result is
exp(-x + log(1+exp(-y+x)))/exp(-z) = exp(-x+z+log(1+exp(-y+x))
= exp(-266.99999385580668)
which is already extremely close to the result that you would get if you were to keep only the 2 leading terms (i.e. the first number 1089 in the numerator and the first number 1000 at the denominator):
exp(3*(1089-1000))=exp(-267)
For the sake of it, let's see how close we are from the solution of Wolfram alpha (link):
Log[(exp[-3*1089]+exp[-3*1093])/([exp[-3*1000]+exp[-3*4443])] -> -266.999993855806522267194565420933791813296828742310997510523
The difference between this number and the exponent above is +1.7053025658242404e-13, so the approximation we made at the denominator was fine.
The final result is
'exp(-266.99999385580668) = 1.1050349147204485e-116
From wolfram alpha is (link)
1.105034914720621496.. × 10^-116 # Wolfram alpha.
and again, it is safe to use numpy here too.
You can use np.logaddexp (which implements the idea in #gg349's answer):
In [33]: d = np.array([[1089, 1093]])
In [34]: e = np.array([[1000, 4443]])
In [35]: log_res = np.logaddexp(-3*d[0,0], -3*d[0,1]) - np.logaddexp(-3*e[0,0], -3*e[0,1])
In [36]: log_res
Out[36]: -266.99999385580668
In [37]: res = exp(log_res)
In [38]: res
Out[38]: 1.1050349147204485e-116
Or you can use scipy.special.logsumexp:
In [52]: from scipy.special import logsumexp
In [53]: res = np.exp(logsumexp(-3*d) - logsumexp(-3*e))
In [54]: res
Out[54]: 1.1050349147204485e-116