I have a random set of numbers in a SQL database:
1.2
0.4
5.1
0.0000000000232
1
7.54
0.000000000000006534
The decimals way below zero are displayed as scientific notation
num = 0.0000000000232
print(num)
> 2.23e-11
But that causes the rest of my code to bug out as the api behind it expects a decimal number. I checked it as I increased the precision with :.20f - that works fine.
Since the very small numbers are not constant with their precision, It would be unwise to simply set a static .20f.
What is a more elegant way to translate this to the correct decimal, always dynamic with the precision?
If Python provides a way to do this, they've hidden it very well. But a simple function can do it.
def float_to_str(x):
to_the_left = 1 + floor(log(x, 10))
to_the_right = sys.float_info.dig - to_the_left
if to_the_right <= 0:
s = str(int(x))
else:
s = format(x, f'0.{to_the_right}f').rstrip('0')
return s
>>> for num in [1.2, 0.4, 5.1, 0.0000000000232, 1, 7.54, 0.000000000000006534]:
print(float_to_str(num))
1.2
0.4
5.1
0.0000000000232
1.
7.54
0.000000000000006534
The first part uses the logarithm base 10 to figure out how many digits will be on the left of the decimal point, or the number of zeros to the right of it if the number is negative. To find out how many digits can be to the right, we take the total number of significant digits that a float can hold as given by sys.float_info.dig which should be 15 on most Python implementations, and subtract the digits on the left. If this number is negative there won't be anything but garbage after the decimal point, so we can rely on integer conversion instead - it never uses scientific notation. Otherwise we simply conjure up the proper string to use with format. For the final step we strip off the redundant trailing zeros.
Using integers for large numbers isn't perfect because we lose the rounding that naturally occurs with floating point string conversion. float_to_str(1e25) for example will return '10000000000000000905969664'. Since your examples didn't contain any such large numbers I didn't worry about it, but it could be fixed with a little more work. For the reasons behind this see Is floating point math broken?
I've got a class Simulator:
class Simulator: # we use a class just to hold variables between calls
def __init__(self):
# initialise accumulators
self.a_sum = 0
self.b_sum = 0
def update(self, a, b):
# increment
self.a_sum += a
self.b_sum += b
def results(self):
# return a pair of results
return self.a_sum, self.a_sum - self.b_sum
where if I print the result
a_error, d_error = simulate(Simulator())
print(f"Error in a_sum is {a_error} and {d_error} in d_sum")
I get Error in a_sum is 13.7734375 and 4.101232676410264e-08 in d_sum, which is caused by using floating point numbers. I know how the calculations work, but I was wondering whether it would be possible to fix the error without converting to decimal? I've tried rounding:
def update(self, a, b):
# increment
self.a_sum += round(a, 3)
self.b_sum += round(b, 3)
which got the error to slightly lower numbers: Error in a_sum is 5.375 and 4.101232676410264e-08 in d_sum but I couldn't get it any lower after that.
Any ideas?
How to fix Python rounding error in Floating Point numbers without using decimal, fractions or any other external libraries?
Floating Point Arithmetic: Issues and Limitations
(...)there are many different decimal numbers that share the same
nearest approximate binary fraction. For example, the numbers 0.1 and
0.10000000000000001 and
0.1000000000000000055511151231257827021181583404541015625 are all
approximated by 3602879701896397 / 2 ** 55. Since all of these
decimal values share the same approximation, any one of them could be
displayed while still preserving the invariant eval(repr(x)) == x.(...)this is in the very nature of binary floating-point: this is
not a bug in Python, and it is not a bug in your code either. You’ll
see the same kind of thing in all languages that support your
hardware’s floating-point arithmetic (although some languages may not
display the difference by default, or in all output modes).
emphasis added by me, so I suppose only way to repair float in this case is to not use float at all. Which is generally not feasible without decimal, fractions or any other external libraries.
NB: this question is about significant figures. It is not a question about "digits after the decimal point" or anything like that.
EDIT: This question is not a duplicate of Significant figures in the decimal module. The two questions are asking about entirely different problems. I want to know why the function about does not return the desired value for a specific input. None of the answers to Significant figures in the decimal module address this question.
The following function is supposed to return a string representation of a float with the specified number of significant figures:
import decimal
def to_sigfigs(value, sigfigs):
return str(decimal.Context(prec=sigfigs).create_decimal(value))
At first glance, it seems to work:
print to_sigfigs(0.000003141592653589793, 5)
# 0.0000031416
print to_sigfigs(0.000001, 5)
# 0.0000010000
print to_sigfigs(3.141592653589793, 5)
# 3.1416
...but
print to_sigfigs(1.0, 5)
# 1
The desired output for the last expression (IOW, the 5-significant figure representation of 1.0) is the string '1.0000'. The actual output is the string '1'.
Am I misunderstanding something or is this a bug in decimal?
The precision of a context is a maximum precision; if an operation would produce a Decimal with less digits than the context's precision, it is not padded out to the context's precision.
When you call to_sigfigs(0.000001, 5), 0.000001 already has some rounding error due to the conversion from source code to binary floating point. It's actually 9.99999999999999954748111825886258685613938723690807819366455078125E-7. Rounding that to 5 significant figures gives decimal.Decimal("0.0000010000").
On the other hand, 1 is exactly representable in binary floating point, so 1.0 is exactly 1. Since only 1 digit is needed to represent this in decimal, the context's precision doesn't require any rounding, and you get a 1-digit Decimal result.
Is it a bug? I don't know, I don't think the documentation is tight enough to make that determination. It certainly is a surprising result.
It is possible to fix your own function with a little more logic.
def to_sigfigs(value, sigfigs):
sign, digits, exponent = decimal.Context(prec=sigfigs).create_decimal(value).as_tuple()
if len(digits) < sigfigs:
missing = sigfigs - len(digits)
digits = digits + (0,) * missing
exponent -= missing
return str(decimal.Decimal((sign, digits, exponent)))
Is there a way to round a python float to x decimals? For example:
>>> x = roundfloat(66.66666666666, 4)
66.6667
>>> x = roundfloat(1.29578293, 6)
1.295783
I've found ways to trim/truncate them (66.666666666 --> 66.6666), but not round (66.666666666 --> 66.6667).
I feel compelled to provide a counterpoint to Ashwini Chaudhary's answer. Despite appearances, the two-argument form of the round function does not round a Python float to a given number of decimal places, and it's often not the solution you want, even when you think it is. Let me explain...
The ability to round a (Python) float to some number of decimal places is something that's frequently requested, but turns out to be rarely what's actually needed. The beguilingly simple answer round(x, number_of_places) is something of an attractive nuisance: it looks as though it does what you want, but thanks to the fact that Python floats are stored internally in binary, it's doing something rather subtler. Consider the following example:
>>> round(52.15, 1)
52.1
With a naive understanding of what round does, this looks wrong: surely it should be rounding up to 52.2 rather than down to 52.1? To understand why such behaviours can't be relied upon, you need to appreciate that while this looks like a simple decimal-to-decimal operation, it's far from simple.
So here's what's really happening in the example above. (deep breath) We're displaying a decimal representation of the nearest binary floating-point number to the nearest n-digits-after-the-point decimal number to a binary floating-point approximation of a numeric literal written in decimal. So to get from the original numeric literal to the displayed output, the underlying machinery has made four separate conversions between binary and decimal formats, two in each direction. Breaking it down (and with the usual disclaimers about assuming IEEE 754 binary64 format, round-ties-to-even rounding, and IEEE 754 rules):
First the numeric literal 52.15 gets parsed and converted to a Python float. The actual number stored is 7339460017730355 * 2**-47, or 52.14999999999999857891452847979962825775146484375.
Internally as the first step of the round operation, Python computes the closest 1-digit-after-the-point decimal string to the stored number. Since that stored number is a touch under the original value of 52.15, we end up rounding down and getting a string 52.1. This explains why we're getting 52.1 as the final output instead of 52.2.
Then in the second step of the round operation, Python turns that string back into a float, getting the closest binary floating-point number to 52.1, which is now 7332423143312589 * 2**-47, or 52.10000000000000142108547152020037174224853515625.
Finally, as part of Python's read-eval-print loop (REPL), the floating-point value is displayed (in decimal). That involves converting the binary value back to a decimal string, getting 52.1 as the final output.
In Python 2.7 and later, we have the pleasant situation that the two conversions in step 3 and 4 cancel each other out. That's due to Python's choice of repr implementation, which produces the shortest decimal value guaranteed to round correctly to the actual float. One consequence of that choice is that if you start with any (not too large, not too small) decimal literal with 15 or fewer significant digits then the corresponding float will be displayed showing those exact same digits:
>>> x = 15.34509809234
>>> x
15.34509809234
Unfortunately, this furthers the illusion that Python is storing values in decimal. Not so in Python 2.6, though! Here's the original example executed in Python 2.6:
>>> round(52.15, 1)
52.200000000000003
Not only do we round in the opposite direction, getting 52.2 instead of 52.1, but the displayed value doesn't even print as 52.2! This behaviour has caused numerous reports to the Python bug tracker along the lines of "round is broken!". But it's not round that's broken, it's user expectations. (Okay, okay, round is a little bit broken in Python 2.6, in that it doesn't use correct rounding.)
Short version: if you're using two-argument round, and you're expecting predictable behaviour from a binary approximation to a decimal round of a binary approximation to a decimal halfway case, you're asking for trouble.
So enough with the "two-argument round is bad" argument. What should you be using instead? There are a few possibilities, depending on what you're trying to do.
If you're rounding for display purposes, then you don't want a float result at all; you want a string. In that case the answer is to use string formatting:
>>> format(66.66666666666, '.4f')
'66.6667'
>>> format(1.29578293, '.6f')
'1.295783'
Even then, one has to be aware of the internal binary representation in order not to be surprised by the behaviour of apparent decimal halfway cases.
>>> format(52.15, '.1f')
'52.1'
If you're operating in a context where it matters which direction decimal halfway cases are rounded (for example, in some financial contexts), you might want to represent your numbers using the Decimal type. Doing a decimal round on the Decimal type makes a lot more sense than on a binary type (equally, rounding to a fixed number of binary places makes perfect sense on a binary type). Moreover, the decimal module gives you better control of the rounding mode. In Python 3, round does the job directly. In Python 2, you need the quantize method.
>>> Decimal('66.66666666666').quantize(Decimal('1e-4'))
Decimal('66.6667')
>>> Decimal('1.29578293').quantize(Decimal('1e-6'))
Decimal('1.295783')
In rare cases, the two-argument version of round really is what you want: perhaps you're binning floats into bins of size 0.01, and you don't particularly care which way border cases go. However, these cases are rare, and it's difficult to justify the existence of the two-argument version of the round builtin based on those cases alone.
Use the built-in function round():
In [23]: round(66.66666666666,4)
Out[23]: 66.6667
In [24]: round(1.29578293,6)
Out[24]: 1.295783
help on round():
round(number[, ndigits]) -> floating point number
Round a number to a given precision in decimal digits (default 0
digits). This always returns a floating point number. Precision may
be negative.
Default rounding in python and numpy:
In: [round(i) for i in np.arange(10) + .5]
Out: [0, 2, 2, 4, 4, 6, 6, 8, 8, 10]
I used this to get integer rounding to be applied to a pandas series:
import decimal
and use this line to set the rounding to "half up" a.k.a rounding as taught in school:
decimal.getcontext().rounding = decimal.ROUND_HALF_UP
Finally I made this function to apply it to a pandas series object
def roundint(value):
return value.apply(lambda x: int(decimal.Decimal(x).to_integral_value()))
So now you can do roundint(df.columnname)
And for numbers:
In: [int(decimal.Decimal(i).to_integral_value()) for i in np.arange(10) + .5]
Out: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
Credit: kares
The Mark Dickinson answer, although complete, didn't work with the float(52.15) case. After some tests, there is the solution that I'm using:
import decimal
def value_to_decimal(value, decimal_places):
decimal.getcontext().rounding = decimal.ROUND_HALF_UP # define rounding method
return decimal.Decimal(str(float(value))).quantize(decimal.Decimal('1e-{}'.format(decimal_places)))
(The conversion of the 'value' to float and then string is very important, that way, 'value' can be of the type float, decimal, integer or string!)
Hope this helps anyone.
I coded a function (used in Django project for DecimalField) but it can be used in Python project :
This code :
Manage integers digits to avoid too high number
Manage decimals digits to avoid too low number
Manage signed and unsigned numbers
Code with tests :
def convert_decimal_to_right(value, max_digits, decimal_places, signed=True):
integer_digits = max_digits - decimal_places
max_value = float((10**integer_digits)-float(float(1)/float((10**decimal_places))))
if signed:
min_value = max_value*-1
else:
min_value = 0
if value > max_value:
value = max_value
if value < min_value:
value = min_value
return round(value, decimal_places)
value = 12.12345
nb = convert_decimal_to_right(value, 4, 2)
# nb : 12.12
value = 12.126
nb = convert_decimal_to_right(value, 4, 2)
# nb : 12.13
value = 1234.123
nb = convert_decimal_to_right(value, 4, 2)
# nb : 99.99
value = -1234.123
nb = convert_decimal_to_right(value, 4, 2)
# nb : -99.99
value = -1234.123
nb = convert_decimal_to_right(value, 4, 2, signed = False)
# nb : 0
value = 12.123
nb = convert_decimal_to_right(value, 8, 4)
# nb : 12.123
def trim_to_a_point(num, dec_point):
factor = 10**dec_point # number of points to trim
num = num*factor # multiple
num = int(num) # use the trimming of int
num = num/factor #divide by the same factor of 10s you multiplied
return num
#test
a = 14.1234567
trim_to_a_point(a, 5)
output
========
14.12345
multiple by 10^ decimal point you want
truncate with int() method
divide by the same number you multiplied before
done!
Just posted this for educational reasons i think it is correct though :)
The reason I'm asking this is because there is a validation in OpenERP that it's driving me crazy:
>>> round(1.2 / 0.01) * 0.01
1.2
>>> round(12.2 / 0.01) * 0.01
12.200000000000001
>>> round(122.2 / 0.01) * 0.01
122.2
>>> round(1222.2 / 0.01) * 0.01
1222.2
As you can see, the second round is returning an odd value.
Can someone explain to me why is this happening?
This has in fact nothing to with round, you can witness the exact same problem if you just do 1220 * 0.01:
>>> 1220*0.01
12.200000000000001
What you see here is a standard floating point issue.
You might want to read what Wikipedia has to say about floating point accuracy problems:
The fact that floating-point numbers cannot precisely represent all real numbers, and that floating-point operations cannot precisely represent true arithmetic operations, leads to many surprising situations. This is related to the finite precision with which computers generally represent numbers.
Also see:
Numerical analysis
Numerical stability
A simple example for numerical instability with floating-point:
the numbers are finite. lets say we save 4 digits after the dot in a given computer or language.
0.0001 multiplied with 0.0001 would result something lower than 0.0001, and therefore it is impossible to save this result!
In this case if you calculate (0.0001 x 0.0001) / 0.0001 = 0.0001, this simple computer will fail in being accurate because it tries to multiply first and only afterwards to divide. In javascript, dividing with fractions leads to similar inaccuracies.
The float type that you are using stores binary floating point numbers. Not every decimal number is exactly representable as a float. In particular there is no exact representation of 1.2 or 0.01, so the actual number stored in the computer will differ very slightly from the value written in the source code. This representation error can cause calculations to give slightly different results from the exact mathematical result.
It is important to be aware of the possibility of small errors whenever you use floating point arithmetic, and write your code to work well even when the values calculated are not exactly correct. For example, you should consider rounding values to a certain number of decimal places when displaying them to the user.
You could also consider using the decimal type which stores decimal floating point numbers. If you use decimal then 1.2 can be stored exactly. However, working with decimal will reduce the performance of your code. You should only use it if exact representation of decimal numbers is important. You should also be aware that decimal does not mean that you'll never have any problems. For example 0.33333... has no exact representation as a decimal.
There is a loss of accuracy from the division due to the way floating point numbers are stored, so you see that this identity doesn't hold
>>> 12.2 / 0.01 * 0.01 == 12.2
False
bArmageddon, has provided a bunch of links which you should read, but I believe the takeaway message is don't expect floats to give exact results unless you fully understand the limits of the representation.
Especially don't use floats to represent amounts of money! which is a pretty common mistake
Python also has the decimal module, which may be useful to you
Others have answered your question and mentioned that many numbers don't have an exact binary fractional representation. If you are accustomed to working only with decimal numbers, it can seem deeply weird that a nice, "round" number like 0.01 could be a non-terminating number in some other base. In the spirit of "seeing is believing," here's a little Python program that will print out a binary representation of any number to any desired number of digits.
from decimal import Decimal
n = Decimal("0.01") # the number to print the binary equivalent of
m = 1000 # maximum number of digits to print
p = -1
r = []
w = int(n)
n = abs(n) - abs(w)
while n and -p < m:
s = Decimal(2) ** p
if n >= s:
r.append("1")
n -= s
else:
r.append("0")
p -= 1
print "%s.%s%s" % ("-" if w < 0 else "", bin(abs(w))[2:],
"".join(r), "..." if n else "")