I am trying figure out the best way to robustly round floats in python using the round half up algorithm. It seems the best way to do this is using the decimal library. However I would expect this method to carry over the rounding up of a 5 across a float. For example:
from decimal import *
Decimal('3.445').quantize(Decimal('0.1'), rounding=ROUND_HALF_UP)
The result is 3.4. What I would expect the algorithm to do is carry over the round up of the 5 such that 3.445 = 3.45 = 3.5.
Does anybody know how to do this in python? I cannot seem to find a robust way of doing this.
Rounding half up doesn't work by carrying over the round-ups from lower digits, but simply determines the half point at the given exponent. Since 3.445 % 0.1 == 0.045, which is less than half of 0.1, it would correctly round down to 3.4.
You can instead implement the desired rounding logic by rounding half up the given decimal number from the second-least significant digit to the given target exponent in a loop:
def round_half_up_carryover(d, target_exp):
exp = Decimal(10) ** (d.as_tuple()[2] + 1)
while exp <= target_exp:
d = d.quantize(exp.normalize(), rounding=ROUND_HALF_UP)
exp *= 10
return d
so that:
print(round_half_up_carryover(Decimal('3.445'), Decimal('0.1')))
would output:
3.5
Related
I have a random set of numbers in a SQL database:
1.2
0.4
5.1
0.0000000000232
1
7.54
0.000000000000006534
The decimals way below zero are displayed as scientific notation
num = 0.0000000000232
print(num)
> 2.23e-11
But that causes the rest of my code to bug out as the api behind it expects a decimal number. I checked it as I increased the precision with :.20f - that works fine.
Since the very small numbers are not constant with their precision, It would be unwise to simply set a static .20f.
What is a more elegant way to translate this to the correct decimal, always dynamic with the precision?
If Python provides a way to do this, they've hidden it very well. But a simple function can do it.
def float_to_str(x):
to_the_left = 1 + floor(log(x, 10))
to_the_right = sys.float_info.dig - to_the_left
if to_the_right <= 0:
s = str(int(x))
else:
s = format(x, f'0.{to_the_right}f').rstrip('0')
return s
>>> for num in [1.2, 0.4, 5.1, 0.0000000000232, 1, 7.54, 0.000000000000006534]:
print(float_to_str(num))
1.2
0.4
5.1
0.0000000000232
1.
7.54
0.000000000000006534
The first part uses the logarithm base 10 to figure out how many digits will be on the left of the decimal point, or the number of zeros to the right of it if the number is negative. To find out how many digits can be to the right, we take the total number of significant digits that a float can hold as given by sys.float_info.dig which should be 15 on most Python implementations, and subtract the digits on the left. If this number is negative there won't be anything but garbage after the decimal point, so we can rely on integer conversion instead - it never uses scientific notation. Otherwise we simply conjure up the proper string to use with format. For the final step we strip off the redundant trailing zeros.
Using integers for large numbers isn't perfect because we lose the rounding that naturally occurs with floating point string conversion. float_to_str(1e25) for example will return '10000000000000000905969664'. Since your examples didn't contain any such large numbers I didn't worry about it, but it could be fixed with a little more work. For the reasons behind this see Is floating point math broken?
In Python, round can be used to round to n decimal digits. Now, round(0.229, 1) gives 0.2 and round(0.251, 1) gives 0.3. Is there a way in Python, to round them both in one direction, say, to 0.2? In other words, is there something analogous to floor or ceil for rounding to an integer, in the context of rounding to a certain number of decimal places?
Update: Accepting the solution based on decimal because it throws light on such a feature available in Python. The solutions based on dividing (and later multiplying) by a multiple of 10 are good ones and straightforward to use.
You can use the decimal library:
from decimal import *
print(Decimal(0.229).quantize(Decimal('.1'), rounding=ROUND_DOWN))
print(Decimal(0.251).quantize(Decimal('.1'), rounding=ROUND_DOWN))
Output:
0.2
0.2
You can also convert from Decimal to float:
f = float(Decimal(0.229).quantize(Decimal('.1'), rounding=ROUND_DOWN)))
You could use floor as follows:
def round_down(x):
return math.floor(10*x) / 10
print(round_down(0.229))
print(round_down(0.251))
This prints:
0.2
0.2
The logic here is to first multiply up by 10, which then lets floor round down all decimal components other than the first one. Then, we divide again by 10 to generate the rounded-down number.
There are already a couple of good answers to your question, however both solutions rely on external libraries. It is also possible to achieve what you desire with no imports:
def round_down(x):
return int(10 * x) / 10
x = 0.251
round_down(x)
# 0.2
x = 0.229
round_down(x)
# 0.2
If you wanted something more similar to Python's round (where you can specify the number n of digits to round to), you could use:
def round_down(x, n=1):
return int(10**n * x) / 10**n
x = 0.229
round_down(x)
# 0.2
round_down(x, 2)
# 0.22
Depending how you'd like the rounding to behave for negative numbers, you may instead wish to define round_down(x) as:
def round_down(x):
return 10 * x // 1 / 10
I was investigating different rounding method using Python built-in solution and some other external libraries such SymPy and while doing so I stumbled upon some cases that I need help with understanding the reason behind it.
Ex-1:
print(round(1.0065,3))
output:
1.006
In the first case, using the Python built-in rounding function the output was 1.006 instead of 1.007 and I can understand that this is not a mistake as Python rounds to the nearest even and that's known as Bankers rounding.
And this is why I from the beginning started searching for another way to control the rounding behaviour. With a quick search, I've found decimal.Decimal module which can easily handle decimal values and efficiently round is using quantize() as in this example:
from decimal import Decimal, getcontext, ROUND_HALF_UP
context= getcontext()
context.rounding='ROUND_HALF_UP'
print(Decimal('1.0065').quantize(Decimal('.001')))
output:1.007
This is a very good solution but the only problem is it is not easy to be hardcoded in long math expressions as I'll need to convert every number to string then after using decimal I will pass it the precession as in the form of "0.001" instead of writing '3' directly as in the case of built-in round.
While searching for another solution I found that SymPy, which I already use a lot in my scripts, offers some very powerful functions that might help but when I tried it the output was not as I expected.
Ex-1 using SymPy sympify():
print(sympify(1.0065).evalf(3))
output: 1.01
Ex-2 using SymPy N (normalize):
print(N(1.0065,3))
output: 1.01
Af first the output was a little bit weird but after investigating I realized that N and sympify already performing round right but rounding to significant figures, not to decimal places.
And here the questions come:
As I can use with Decimal objects getcontext().rounding='ROUND_HALF_UP' to change the rounding behaviour, is there a way to change the N and sympify rounding behaviour to decimal places instead of significant figures?
Instead of re-implementing decimal rounding in SymPy, perhaps use decimal to do the rounding, but hide the calculation in a utility function:
import sympy as sym
import decimal
from decimal import Decimal as D
def dround(d, ndigits, rounding=decimal.ROUND_HALF_UP):
result = D(str(d)).quantize(D('0.1')**ndigits, rounding=rounding)
# result = sym.sympify(result) # if you want a SymPy Float
return result
for x in [0.0065, 1.0065, 10.0065, 100.0065]:
print(dround(x, 3))
prints
0.007
1.007
10.007
100.007
The n of evalf gives the first n significant digits of x (measured from the left). If you use x.round(3) it will round x to the nth digit from the decimal point and can be positive (right of decimal pt) or negative (left of decimal pt).
>>> for x in '0.0065, 1.0065, 10.0065, 100.0065'.split(', '):
... print S(x).round(3)
0.006
1.006
10.007
100.007
>>> int(S(12345).round(-2))
12300
First of all, N and evalf are essentially the same thing; N(x, n) amounts to sympify(x).evalf(n). In your case, since x is a Python float, it's easier to use N because it sympifies the input.
To get three digits after decimal dot, use N(x, 3 + log(x, 10) + 1). The adjustment log(x, 10) + 1 is 0 when x is between 0.1 and 1; in this case the number of significant digits is the same as the number of digits after the decimal dot. If x is larger, we get more significant digits.
Example:
for x in [0.0065, 1.0065, 10.0065, 100.0065]:
print(N(x, 3 + log(x, 10) + 1))
prints
0.006
1.007
10.007
100.007
The transition from 6 to 7 is curious, but not entirely surprising. These numbers are not exactly represented in binary system, so the truncation to nearest double-precision float may be a factor here. I've made a few additional observation on this effect on my blog.
If I tell Python v. 3.4.3, round(2.5), then it outputs 2. If I tell it round(1.5) then it outputs 2 as well, though. Similarly, round(3.5) gives 4, while round(4.5) gives 4 as well. I need Python to round with consistency, though. Specifically, it needs to round anytime I input a number halfway between two integers. So round(1.5) = 1 and round(2.5) = 2, while round(1.6) = 2 and such, as usual.
How can I resolve this?
EDIT: I've already read the documentation for the round function and understand that this is its intended behavior. My question is, how can I alter this behavior, because for my purposes I need 1.5 round down.
Python 3 uses a different rounding behaviour compared to Python 2: it now uses so-called "banker's rounding" (Wikipedia): when the integer part is odd, the number is rounded away from zero; when the integer part is even, is it rounded towards zero.
The reason for this is to avoid a bias, when all values at .5 are rounded away from zero (and then e.g. summed).
This is the behaviour you are seeing, and it is in fact consistent. It's perhaps just different than what you are used to.
The round docs do address the peculiaries of rounding floating point numbers.
You can use the decimal library to achieve what you want.
from decimal import Decimal, ROUND_HALF_UP, ROUND_HALF_DOWN
round(2.675, 2)
# output: 2.67
Decimal('2.675').quantize(Decimal('1.11'), rounding=ROUND_HALF_UP)
# output: 2.68
Decimal('2.5').quantize(Decimal('1.'), rounding=ROUND_HALF_DOWN)
# output: 2
Your want "round down", and you are getting "round to even". Just do it manually by doing
ceil(x - 0.5)
This is documented pretty well. According to the Python docs for round:
Note The behavior of round() for floats can be surprising: for example, round(2.675, 2) gives 2.67 instead of the expected 2.68. This is not a bug: it’s a result of the fact that most decimal fractions can’t be represented exactly as a float. See Floating Point Arithmetic: Issues and Limitations for more information
In specific, this is a side-effect of how computers handle floating-point numbers in general.
If you need more precision, including different rounding, I suggest you check out the Python Decimal module. Specifically of interest, they have the ability to control rounding modes. Looks like you might want decimal.ROUND_HALF_DOWN.
Python 3 provides rounding methods defined in the IEEE Standard for Floating-Point Arithmetic (IEEE 754), the default rounding[1] is directed to the nearest number and minimizing cumulative errors.
In IEEE 754, there are 5 methods defined, two for rounding to nearest (Python provides the first one by round) and three methods that are explicitly directed (Python has trunc, ceil, and floor in its Math module).
You obviously need a directed rounding and there is a way to tell this Python, you have just to choose.
[1] Since the representation of floating point numbers in computers is limited, rounding is not as trivial as you might think, you'll be surprised! I recommend a careful read of 15. Floating Point Arithmetic: Issues and Limitations in the python 3 documentation.
I believe I have the answer to all the rounding errors people have been encountering. I have wrote my own method, which functions same as the "round" but actually looks at the last digit and rounds from there case by case. There is no converting a decimal to binary. It can handle any amount of numbers behind the decimal and it also takes scientific notation (as outputted by floats). It also doesn't require any imports! Let me know if you catch any cases that don't work!
def Round(Number,digits = 0):
Number_Str = str(Number)
if "e" in Number_Str:
Number_Str = "%.10f" % float(Number_Str)
if "." in Number_Str: #If not given an integer
try:
Number_List = list(Number_Str) #All the characters in Number in a list
Number_To_Check = Number_List[Number_List.index(".") + 1 + digits] #Gets value to be looked at for rounding.
if int(Number_To_Check) >= 5:
Index_Number_To_Round = Number_List.index(".") + digits
if Number_List[Index_Number_To_Round] == ".":
Index_Number_To_Round -= 1
if int(Number_List[Index_Number_To_Round]) == 9:
Number_List_Spliced = Number_List[:Number_List.index(".")+digits]
for index in range(-1,-len(Number_List_Spliced) - 1,-1):
if Number_List_Spliced[index] == ".":
continue
elif int(Number_List_Spliced[index]) == 9:
Number_List_Spliced[index] = "0"
try:
Number_List_Spliced[index-1]
continue
except IndexError:
Number_List_Spliced.insert(0,"1")
else:
Number_List_Spliced[index] = str(int(Number_List_Spliced[index])+1)
break
FinalNumber = "".join(Number_List_Spliced)
else:
Number_List[Index_Number_To_Round] = str(int(Number_List[Index_Number_To_Round])+1)
FinalNumber = "".join(Number_List[:Index_Number_To_Round + 1])
return float(FinalNumber)
else:
FinalNumber = "".join(Number_List[:Number_List.index(".") + 1 + digits])
return float(FinalNumber)
except IndexError:
return float(Number)
else: #If given an integer
return float(Number)
The reason I'm asking this is because there is a validation in OpenERP that it's driving me crazy:
>>> round(1.2 / 0.01) * 0.01
1.2
>>> round(12.2 / 0.01) * 0.01
12.200000000000001
>>> round(122.2 / 0.01) * 0.01
122.2
>>> round(1222.2 / 0.01) * 0.01
1222.2
As you can see, the second round is returning an odd value.
Can someone explain to me why is this happening?
This has in fact nothing to with round, you can witness the exact same problem if you just do 1220 * 0.01:
>>> 1220*0.01
12.200000000000001
What you see here is a standard floating point issue.
You might want to read what Wikipedia has to say about floating point accuracy problems:
The fact that floating-point numbers cannot precisely represent all real numbers, and that floating-point operations cannot precisely represent true arithmetic operations, leads to many surprising situations. This is related to the finite precision with which computers generally represent numbers.
Also see:
Numerical analysis
Numerical stability
A simple example for numerical instability with floating-point:
the numbers are finite. lets say we save 4 digits after the dot in a given computer or language.
0.0001 multiplied with 0.0001 would result something lower than 0.0001, and therefore it is impossible to save this result!
In this case if you calculate (0.0001 x 0.0001) / 0.0001 = 0.0001, this simple computer will fail in being accurate because it tries to multiply first and only afterwards to divide. In javascript, dividing with fractions leads to similar inaccuracies.
The float type that you are using stores binary floating point numbers. Not every decimal number is exactly representable as a float. In particular there is no exact representation of 1.2 or 0.01, so the actual number stored in the computer will differ very slightly from the value written in the source code. This representation error can cause calculations to give slightly different results from the exact mathematical result.
It is important to be aware of the possibility of small errors whenever you use floating point arithmetic, and write your code to work well even when the values calculated are not exactly correct. For example, you should consider rounding values to a certain number of decimal places when displaying them to the user.
You could also consider using the decimal type which stores decimal floating point numbers. If you use decimal then 1.2 can be stored exactly. However, working with decimal will reduce the performance of your code. You should only use it if exact representation of decimal numbers is important. You should also be aware that decimal does not mean that you'll never have any problems. For example 0.33333... has no exact representation as a decimal.
There is a loss of accuracy from the division due to the way floating point numbers are stored, so you see that this identity doesn't hold
>>> 12.2 / 0.01 * 0.01 == 12.2
False
bArmageddon, has provided a bunch of links which you should read, but I believe the takeaway message is don't expect floats to give exact results unless you fully understand the limits of the representation.
Especially don't use floats to represent amounts of money! which is a pretty common mistake
Python also has the decimal module, which may be useful to you
Others have answered your question and mentioned that many numbers don't have an exact binary fractional representation. If you are accustomed to working only with decimal numbers, it can seem deeply weird that a nice, "round" number like 0.01 could be a non-terminating number in some other base. In the spirit of "seeing is believing," here's a little Python program that will print out a binary representation of any number to any desired number of digits.
from decimal import Decimal
n = Decimal("0.01") # the number to print the binary equivalent of
m = 1000 # maximum number of digits to print
p = -1
r = []
w = int(n)
n = abs(n) - abs(w)
while n and -p < m:
s = Decimal(2) ** p
if n >= s:
r.append("1")
n -= s
else:
r.append("0")
p -= 1
print "%s.%s%s" % ("-" if w < 0 else "", bin(abs(w))[2:],
"".join(r), "..." if n else "")