This question already has answers here:
Is floating point math broken?
(31 answers)
Is floating point arbitrary precision available?
(5 answers)
Closed 7 years ago.
I don't know if this is an obvious bug, but while running a Python script for varying the parameters of a simulation, I realized the results with delta = 0.29 and delta = 0.58 were missing. On investigation, I noticed that the following Python code:
for i_delta in range(0, 101, 1):
delta = float(i_delta) / 100
(...)
filename = 'foo' + str(int(delta * 100)) + '.dat'
generated identical files for delta = 0.28 and 0.29, same with .57 and .58, the reason being that python returns float(29)/100 as 0.28999999999999998. But that isn't a systematic error, not in the sense it happens to every integer. So I created the following Python script:
import sys
n = int(sys.argv[1])
for i in range(0, n + 1):
a = int(100 * (float(i) / 100))
if i != a: print i, a
And I can't see any pattern in the numbers for which this rounding error happens. Why does this happen with those particular numbers?
Any number that can't be built from exact powers of two can't be represented exactly as a floating point number; it needs to be approximated. Sometimes the closest approximation will be less than the actual number.
Read What Every Computer Scientist Should Know About Floating-Point Arithmetic.
Its very well known due to the nature of floating point numbers.
If you want to do decimal arithmetic not floating point arithmatic there are libraries to do this.
E.g.,
>>> from decimal import Decimal
>>> Decimal(29)/Decimal(100)
Decimal('0.29')
>>> Decimal('0.29')*100
Decimal('29')
>>> int(Decimal('29'))
29
In general decimal is probably going overboard and still will have rounding errors in rare cases when the number does not have a finite decimal representation (for example any fraction where the denominator is not 1 or divisible by 2 or 5 - the factors of the decimal base (10)). For example:
>>> s = Decimal(7)
>>> Decimal(1)/s/s/s/s/s/s/s*s*s*s*s*s*s*s
Decimal('0.9999999999999999999999999996')
>>> int(Decimal('0.9999999999999999999999999996'))
0
So its best to always just round before casting floating points to ints, unless you want a floor function.
>>> int(1.9999)
1
>>> int(round(1.999))
2
Another alternative is to use the Fraction class from the fractions library which doesn't approximate. (It justs keeps adding/subtracting and multiplying the integer numerators and denominators as necessary).
Related
Interesting fact on Python 2.7.6 running on macosx:
You have a very small number like:
0.000000000000000000001
You can represent it as:
>>> 0.1 / (10 ** 20)
1.0000000000000001e-21
But you can see the floating pointing error at the end. What we really have is something like this:
0.0000000000000000000010000000000000001
So we got an error in the number, but thats ok. The problem is the following:
As expected:
>>> 0.1 / (10 ** 20) == 0
False
But, wait, whats this?
>>> 0.1 / (10 ** 20) + 1 == 1
True
>>> repr(0.1 / (10 ** 20) + 1)
'1.0'
Looks like python is using another data type to represent my number as this only happens when using the 16th decimal digit and so on. Also why python decided to automatically turn my number to 0 on the addition? Should not I be dealing with a decimal number with a very small decimal part and with a floating point error?
I know this problem can fall in the floating point errors umbrella, and the solution normally is do not trust floating point for this kinda of calculation, but I would like to understand more about whats happening under the hood.
Consider what happens when you attempt to add 7.00 and .001 and are only permitted to use three digits for the answer. 7.001 is not allowed. What do you do?
Of the values you can return, such as 6.99, 7.00, and 7.01, the closest to the correct answer is 7.00. So, when asked to add 7.00 and .001, you return 7.00.
The computer has the same issue when you ask it to add a number near 1e-21 and 1. It has only 53 bits to use for the fraction portion of the floating-point value, and 1e-21 is almost 70 bits below 1. So, it rounds the correct result to the closest value it can, 1, and returns that.
Floating point numbers work like scientific notation. 1.0000000000000001e-21 fits within the 53 bits of significand/mantissa a 64-bit float allows. Adding 1 to that, many orders of magnitude larger than the tiny fraction, causes the minor detail to be discarded, and storing exactly 1 instead
This question already has answers here:
How can I force division to be floating point? Division keeps rounding down to 0?
(11 answers)
Closed 9 years ago.
I'd like to pass numbers around between functions, while preserving the decimal places for the numbers.
I've discovered that if I pass a float like '10.00' in to a function, then the decimal places don't get used. This messes an operation like calculating percentages.
For example, x * (10 / 100) will always return 0.
But if I manage to preserve the decimal places, I end up doing x * (10.00 / 100). This returns an accurate result.
I'd like to have a technique that enables consistency when I'm working with numbers that decimal places that can hold zeroes.
When you write
10 / 100
you are performing integer division. That's because both operands are integers. The result is 0.
If you want to perform floating point division, make one of the operands be a floating point value. For instance:
10.0 / 100
or
float(10) / 100
Do beware also that
10.0 / 100
results in a binary floating point value and binary floating data types cannot represent the true result value of 0.1. So if you want to represent the result accurately you may need to use a decimal data type. The decimal module has the functionality needed for that.
Division in python for float and int works differently, take a look at this question and it's answers: Python division.
Moreover, if you are looking for a solution to format a decimal floating point of your figures into string, you might need to use %f.
Python
# '1.000000'
"%f" % (1.0)
# '1.00'
"%.2f" % (1.0)
# ' 1.00'
"%6.2f" % (1.0)
Python 2.x will use integer division when dividing two integers unless you explicitly tell it to do otherwise. Two integers in --> one integer out.
Python 3 onwards will return, to quote PEP 238 http://www.python.org/dev/peps/pep-0238/ a reasonable approximation of the result of the division approximation, i.e. it will perform a floating point division and return the result without rounding.
To enable this behaviour in earlier version of Python you can use:
from __future__ import division
At the very top of the module, this should get you the consistent results you want.
You should use the decimal module. Each number knows how many significant digits it has.
If you're trying to preserve significant digits, the decimal module is has everything you need. Example:
>>> from decimal import Decimal
>>> num = Decimal('10.00')
>>> num
Decimal('10.00')
>>> num / 10
Decimal('1.00')
This question already has answers here:
Floating Point Limitations [duplicate]
(3 answers)
Closed 9 years ago.
I spent an hour today trying to figure out why
return abs(val-desired) <= 0.1
was occasionally returning False, despite val and desired having an absolute difference of <=0.1. After some debugging, I found out that -13.2 + 13.3 = 0.10000000000000142. Now I understand that CPUs cannot easily represent most real numbers, but this is an exception, because you can subtract 0.00000000000000142 and get 0.1, so it can be represented in Python.
I am running Python 2.7 on Intel Core architecture CPUs (this is all I have been able to test it on). I'm curious to know how I can store a value of 0.1 despite not being able to apply arithmetic to particular floating point values. val and desired are float values.
Yes, this can be a bit surprising:
>>> +13.3
13.300000000000001
>>> -13.2
-13.199999999999999
>>> 0.1
0.10000000000000001
All these numbers can be represented with some 16 digits of accuracy. So why:
>>> 13.3-13.2
0.10000000000000142
Why only 14 digits of accuracy in that case?
Well, that's because 13.3 and -13.2 have 16 digits of accuracy, which means 14 decimal points, since there are two digits before the decimal point. So the result also have 14 decimal points of accuracy. Even though the computer can represent numbers with 16 digits.
If we make the numbers bigger, the accuracy of the result decreases further:
>>> 13000.3-13000.2
0.099999999998544808
>>> 1.33E10-13.2E10
-118700000000.0
In short, the accuracy of the result depends on the accuracy of the input.
"Now I understand that CPUs cannot easily represent most floating point numbers with high resolution", the fact you asked this question indicates that you don't understand. None of the real values 13.2, 13.3 nor 0.1 can be represented exactly as floating point numbers:
>>> "{:.20f}".format(13.2)
'13.19999999999999928946'
>>> "{:.20f}".format(13.3)
'13.30000000000000071054'
>>> "{:.20f}".format(0.1)
'0.10000000000000000555'
To directly address your question of "how do I store a value like 0.1 and do an exact comparison to it when I have imprecise floating-point numbers," the answer is to use a different type to represent your numbers. Python has a decimal module for doing decimal fixed-point and floating-point math instead of binary -- in decimal, obviously, 0.1, -13.2, and 13.3 can all be represented exactly instead of approximately; or you can set a specific level of precision when doing calculations using decimal and discard digits below that level of significance.
val = decimal.Decimal(some calculation)
desired = decimal.Decimal(some other calculation)
return abs(val-desired) <= decimal.Decimal('0.1')
The other common alternative is to use integers instead of floats by artificially multiplying by some power of ten.
return not int(abs(val-desired)*10)
The reason I'm asking this is because there is a validation in OpenERP that it's driving me crazy:
>>> round(1.2 / 0.01) * 0.01
1.2
>>> round(12.2 / 0.01) * 0.01
12.200000000000001
>>> round(122.2 / 0.01) * 0.01
122.2
>>> round(1222.2 / 0.01) * 0.01
1222.2
As you can see, the second round is returning an odd value.
Can someone explain to me why is this happening?
This has in fact nothing to with round, you can witness the exact same problem if you just do 1220 * 0.01:
>>> 1220*0.01
12.200000000000001
What you see here is a standard floating point issue.
You might want to read what Wikipedia has to say about floating point accuracy problems:
The fact that floating-point numbers cannot precisely represent all real numbers, and that floating-point operations cannot precisely represent true arithmetic operations, leads to many surprising situations. This is related to the finite precision with which computers generally represent numbers.
Also see:
Numerical analysis
Numerical stability
A simple example for numerical instability with floating-point:
the numbers are finite. lets say we save 4 digits after the dot in a given computer or language.
0.0001 multiplied with 0.0001 would result something lower than 0.0001, and therefore it is impossible to save this result!
In this case if you calculate (0.0001 x 0.0001) / 0.0001 = 0.0001, this simple computer will fail in being accurate because it tries to multiply first and only afterwards to divide. In javascript, dividing with fractions leads to similar inaccuracies.
The float type that you are using stores binary floating point numbers. Not every decimal number is exactly representable as a float. In particular there is no exact representation of 1.2 or 0.01, so the actual number stored in the computer will differ very slightly from the value written in the source code. This representation error can cause calculations to give slightly different results from the exact mathematical result.
It is important to be aware of the possibility of small errors whenever you use floating point arithmetic, and write your code to work well even when the values calculated are not exactly correct. For example, you should consider rounding values to a certain number of decimal places when displaying them to the user.
You could also consider using the decimal type which stores decimal floating point numbers. If you use decimal then 1.2 can be stored exactly. However, working with decimal will reduce the performance of your code. You should only use it if exact representation of decimal numbers is important. You should also be aware that decimal does not mean that you'll never have any problems. For example 0.33333... has no exact representation as a decimal.
There is a loss of accuracy from the division due to the way floating point numbers are stored, so you see that this identity doesn't hold
>>> 12.2 / 0.01 * 0.01 == 12.2
False
bArmageddon, has provided a bunch of links which you should read, but I believe the takeaway message is don't expect floats to give exact results unless you fully understand the limits of the representation.
Especially don't use floats to represent amounts of money! which is a pretty common mistake
Python also has the decimal module, which may be useful to you
Others have answered your question and mentioned that many numbers don't have an exact binary fractional representation. If you are accustomed to working only with decimal numbers, it can seem deeply weird that a nice, "round" number like 0.01 could be a non-terminating number in some other base. In the spirit of "seeing is believing," here's a little Python program that will print out a binary representation of any number to any desired number of digits.
from decimal import Decimal
n = Decimal("0.01") # the number to print the binary equivalent of
m = 1000 # maximum number of digits to print
p = -1
r = []
w = int(n)
n = abs(n) - abs(w)
while n and -p < m:
s = Decimal(2) ** p
if n >= s:
r.append("1")
n -= s
else:
r.append("0")
p -= 1
print "%s.%s%s" % ("-" if w < 0 else "", bin(abs(w))[2:],
"".join(r), "..." if n else "")
This question already has answers here:
Why does floating-point arithmetic not give exact results when adding decimal fractions?
(31 answers)
Is floating point arbitrary precision available?
(5 answers)
Closed 7 years ago.
I don't know if this is an obvious bug, but while running a Python script for varying the parameters of a simulation, I realized the results with delta = 0.29 and delta = 0.58 were missing. On investigation, I noticed that the following Python code:
for i_delta in range(0, 101, 1):
delta = float(i_delta) / 100
(...)
filename = 'foo' + str(int(delta * 100)) + '.dat'
generated identical files for delta = 0.28 and 0.29, same with .57 and .58, the reason being that python returns float(29)/100 as 0.28999999999999998. But that isn't a systematic error, not in the sense it happens to every integer. So I created the following Python script:
import sys
n = int(sys.argv[1])
for i in range(0, n + 1):
a = int(100 * (float(i) / 100))
if i != a: print i, a
And I can't see any pattern in the numbers for which this rounding error happens. Why does this happen with those particular numbers?
Any number that can't be built from exact powers of two can't be represented exactly as a floating point number; it needs to be approximated. Sometimes the closest approximation will be less than the actual number.
Read What Every Computer Scientist Should Know About Floating-Point Arithmetic.
Its very well known due to the nature of floating point numbers.
If you want to do decimal arithmetic not floating point arithmatic there are libraries to do this.
E.g.,
>>> from decimal import Decimal
>>> Decimal(29)/Decimal(100)
Decimal('0.29')
>>> Decimal('0.29')*100
Decimal('29')
>>> int(Decimal('29'))
29
In general decimal is probably going overboard and still will have rounding errors in rare cases when the number does not have a finite decimal representation (for example any fraction where the denominator is not 1 or divisible by 2 or 5 - the factors of the decimal base (10)). For example:
>>> s = Decimal(7)
>>> Decimal(1)/s/s/s/s/s/s/s*s*s*s*s*s*s*s
Decimal('0.9999999999999999999999999996')
>>> int(Decimal('0.9999999999999999999999999996'))
0
So its best to always just round before casting floating points to ints, unless you want a floor function.
>>> int(1.9999)
1
>>> int(round(1.999))
2
Another alternative is to use the Fraction class from the fractions library which doesn't approximate. (It justs keeps adding/subtracting and multiplying the integer numerators and denominators as necessary).