I was under the impression that Python uses double precision arithmetic, with "nearest to" rounding. However, consider the following:
In the double precision system, the next number after 1.0 is 1.00...01 = 1 + 2**(-52). Now, if Python uses the "nearest to" rounding, the number 1 + 2**(-53) should round to 1 + 2**(-52). However, it turns out that 1 + 2**(-53) == 1. This would make sense if Python uses the "chop" rounding rule, but I was under the impression that nobody uses that since it biases calculations towards lower results.
The Python documentation is not strict about how floating-point arithmetic is handled. Some Python implementations use IEEE 754 with round-to-nearest-ties-to-even.
In the IEEE-754 binary64 format, also known as the “double precision” format, 1+2−53 is the midpoint between 1 and the next representable number, 1+2−52. So this is a tie, and the round-to-even rule applies. The significand for 1 is 1.000…0002, and the significand for 1+2−52 is 1.000…0012. The former is even, so the tie-breaking rule chooses it, and the result is 1.
Consider instead 1+3•2−54. This is three-quarters of the way from 1 to 1+2−52. So rounding it to the nearest representable value will produce 1+2−52. For print(1 == 1+3*2**-54), your Python implementation will likely print “False”.
Related
I wrote a simple code in python that gives me the same value of machine epsilon using the numpy command:np.finfo(float).eps
The code is:
eps=1
while eps+1 != 1:
eps /= 2
print(eps)
But I didn't want to stop here ! I used smaller and smaller numbers to divide eps, for example:
eps=1
while eps+1 != 1:
eps /= 1.1
print (eps)
With this, I got a value of 1.158287085355336e-16 for epsilon. I noticed that epsilon was converging to a number, my last attempt at 1.0000001 gave me the value of 1.1102231190697707e-16.
Is this value closer to the real value of epsilon for my Pc? I think I'm not considering something important and my line of thinking is wrong.
Thank you in advance for the help !
The term “machine epsilon” is not consistently defined, so it's better to avoid that word and instead to say what specifically you're talking about:
ulp(1), the magnitude of the least significant digit, or Unit in the Last Place, of 1.
This is the distance from 1 to the next larger floating-point number.
More generally, ulp(𝑥) is the distance from 𝑥 to the next larger floating-point number in magnitude.
In binary64 floating-point, with 53 bits of precision, ulp(1) is 2⁻⁵² ≈ 2.220446049250313 × 10⁻¹⁶.
In decimal64 floating-point, with 16 digits of precision, ulp(1) is 10⁻¹⁵.
In general, for floating-point in radix 𝛽 with 𝑝 digits of precision (including the implicit 1 digit), ulp(1) = 𝛽1 − 𝑝.
The relative error bound, sometimes also called unit roundoff or u.
A floating-point operation may round the outcome of a mathematical function such as 𝑥 + 𝑦, giving fl(𝑥 + 𝑦) = (𝑥 + 𝑦)⋅(1 + 𝛿) for some relative error 𝛿.
For basic arithmetic operations in IEEE 754 (+, −, *, /, sqrt), the result of computing the floating-point operation is guaranteed to be correctly rounded, which in the default rounding mode means it yields the nearest floating-point number, or one of the two nearest such numbers if 𝑥 + 𝑦 lies exactly halfway between them.
In binary64 floating-point, with 53 bits of precision, the relative error of an operation correctly rounded to nearest is at most 2⁻⁵³ ≈ 1.1102230246251565 × 10⁻¹⁶.
In decimal64 floating-point, with 16 digits of precision, the relative error of an operation correctly rounded to nearest is at most 5 × 10⁻¹⁶.
In general, when floating-point arithmetic in radix 𝛽 with 𝑝 digits of precision is correctly rounded to nearest, 𝛿 is bounded in magnitude by the relative error bound (𝛽/2) 𝛽−𝑝.
What the Python iteration while 1 + eps != 1: eps /= 2 computes, starting with eps = 1., is the relative error bound in binary64 floating-point, since that's the floating-point that essentially all Python implementations use.
If you had a version of Python that worked in a different radix, say b, you would instead want to use while 1 + eps != 1: eps /= b.
If you do eps /= 1.1 or eps /= 1.0001, you will get an approximation to the relative error bound erring on the larger side, with no particular significance to the result.
Note that sys.float_info.epsilon is ulp(1), rather than the relative error bound.
They are always related: ulp(1)/2 is the relative error bound in every floating-point format.
If you want the actual machine epsilon for a Python float on your PC, you can get it from the epsilon attribute of sys.float_info. By the way, on my x86-64 machine, numpy.finfo(float) gives me 2.220446049250313e-16, which is the expected machine epsilon for a 64-bit float.
Your intuition for trying to find the value eps such that 1 + eps != 1 is True is good, but machine epsilon is an upper bound on the relative error due to rounding in floating-point arithmetic. Not to mention, the inexact nature of floating-point arithmetic can be mystifying sometimes: note that 0.1 + 0.1 + 0.1 != 0.3 evaluates to True. Also, if I modify your code to
eps = 1
while eps + 9 != 9:
eps = eps / 1.0000001
print(eps)
I get, after around maybe half a minute,
8.881783669690459e-16
The relative error in this case is 8.881783669690459e-16 / 9 = 9.868648521878288e-17.
Your program is almost fine.
IT should be
eps=1
while eps+1 != 1:
eps /= 2
print(2*eps)
The result is
2.220446049250313e-16
Which is the epsilon of the machine. You can check this with this piece of
code:
import sys
sys.float_info.epsilon
The reason we should multiply by 2 in the last line is because you went 1 division too far inside the while loop.
I'm new to programming and studying the basic now. I'm wondering that does the float type in python only represent approximations to real number? I know float uses binary fractions but are the floats 0.5, 0.25, 0.125, etc still the approximations? I tried:
sum([0.1] * 10) == 1
it returned False.
But
sum([0.5] * 10) == 5
It returned True.
Finally I tried:
for i in range(1, 8):
answer = sum([1 / 2 ** i] * 10)
print(answer == 1 / 2 ** i * 10)
The answer is all True.
It's that means some floats in python are exactly the real number not the approximations?
Each floating-point object represents one number (or special value such as NaN) exactly. Floating-point objects do not represent approximations.
The correct way to think about floating-point is that floating-point values are exact numbers, but floating-point operations approximate real arithmetic.
Python does not specify floating-point arithmetic precisely; each Python implementation may use the underlying arithmetic of the platform it is implemented on. Commonly, IEEE 754 formats are used, although the operations may not conform to IEEE 754 completely. To illustrate what is happening with your code, I will use IEEE-754 basic 64-bit binary floating-point.
When the source text 0.5 is processed, it is converted to floating-point. Note that conversion is an operation, just as addition or multiplication are operations. The characters are interpreted as a decimal numeral, and the conversion produces the floating-point number that is closest to the number represented by the decimal numeral. In this case, 0.5 represents one-half, and that is exactly representable in binary floating-point, so the result is exactly 0.5.
Then [0.5] * 10 produces a list containing ten copies of 0.5, and sum adds those. All of the additions performed in this summation are exact, because the floating-point format can exactly represent 0.5, 1, 1.5, 2, and so on. So the result is 5, exactly, and comparing this to 5 produces true.
On the other hand, when the source text 0.1 is processed, that decimal numeral represents one-tenth, which cannot be represented exactly. The conversion produces the nearest representable value, which is 0.1000000000000000055511151231257827021181583404541015625.
When sum adds the ten copies of this, the addition cannot always be performed exactly. Adding the first two is exact, adding 0.1000000000000000055511151231257827021181583404541015625 to 0.1000000000000000055511151231257827021181583404541015625 produces 0.200000000000000011102230246251565404236316680908203125. However, when 0.200000000000000011102230246251565404236316680908203125 is added to 0.1000000000000000055511151231257827021181583404541015625, the result is 0.3000000000000000444089209850062616169452667236328125. During this addition, the bits in the addition carried to a new position (the operands are under ¼, but the result is over ¼—the addition carried into the ¼ position. Since the floating-point format has only a fixed number of bits (53) available for the value, the operation had to discard the low bit. In doing so, it changed the result slightly. So this addition is only approximate.
As these additions go on, the final value is 0.99999999999999988897769753748434595763683319091796875. When this is compared to 1, the result is false.
Python represents floating point numbers as binary fractions. Therefore numbers like 0.5 can be represented accurately, whereas 0.1 for example can not.
Floating point numbers in Python are just approximations, if they can not be exactly represented using binary fractions.
If you need more accuracy when dealing with floating point arithmetic, I would suggest taking a look at decimals: https://docs.python.org/3/library/decimal.html
Additionally, a good resource on floating point numbers in Python can be found here: https://docs.python.org/3/tutorial/floatingpoint.html
Using Python 3, how does the following return True ?
a = 2/3
b = 4/6
print(a == b)
I have an algorithm that requires sorting a list of numbers which are each of the form x/y where x and y are integers. (y != 0).
I was concerned that the numerical precision of the division would result in instability and arbitrary ordering of cases such as above. This being an example of relevant comments.But, as per the example and for larger integers as well, it does not seem to be an issue.
Does Python remove the common factor of 2 from the numerator and denominator of b, and retain information that a and b are not just floats?
Python follows the IEEE 754 floating point specification.* (64-bit) IEEE floats are essentially a form of base 2 scientific notation, broken down as follows:
One bit for the sign (positive or negative)
53 bits for the mantissa or significand, including the implied leading one.
11 bits for the exponent.
Multiplying or dividing a floating point value by two, or any power of two, only affects the exponent, and not the mantissa.** As a result, it is normally a fairly "stable" operation by itself, so 2/3 should yield the same result as 4/6. However, IEEE floats still have the following problems:
Most operations are not associative (e.g. (a * b) * c != a * (b * c) in the general case).
More complicated operations are not required to be correctly rounded (however, as Tim Peters points out, division certainly is not a "more complicated" operation and will be correctly rounded).***
Intermediate results are always rounded to 53 bits.
You should be prepared to handle these issues and assume that most mathematically-equivalent floating point expressions will not result in identical values. In Python specifically, you can use math.isclose() to estimate whether two floats are "close enough" to be "probably the same value."
* Actually, this is a lie. Python follows C's double, which nearly always follows IEEE 754 in some fashion, but might deviate from it on sufficiently exotic architectures. In such cases the C standard provides few or no guarantees, so you will have to look to your architecture or compiler's floating point documentation.
** Provided the exponent does not overflow or underflow. If it does, then you will typically land on an appropriately-signed infinity or zero, respectively, or you might underflow to a denormal number depending on architecture and/or how Python was compiled.
*** The exact set of "more complicated" operations varies somewhat because IEEE 754 made a lot of operations optional while still demanding precision. As a result, it is seldom obvious whether a given operation conforms to IEEE 754 or only conforms to the notoriously lax C standard. In some cases, an operation may conform to no standard whatsoever.
Just noting that so long as integers x and y are exactly representable as Python floats, x / y is - on all current machines - the correctly rounded value of the infinitely precise quotient. That's what the IEEE 754 floating-point standard requires, and all current machines support that.
So the important part in your specific example isn't that the numerator and denominator in b = 4/6 have a factor of (specifically!) 2 in common, it's that (a) they have some factor in common; and, (b) 4 and 6 are both exactly representable as Python floats.
So, for example, it's guaranteed that
(2 * 9892837) / (3 * 9892837) == 2 / 3
is also true. Because the infinitely precise value of (2 * 9892837) / (3 * 9892837) is the same as the infinitely precisely value of 2/3, and IEEE 754 division acts as if the infinitely precise quotient were computed. And you can replace 9892837 with any other non-zero integer in that, provided that the products remain exactly representable as Python floats.
2/3 is the same as 4/6. (2/3)*(2/2) = 2/2 = 1, the identity element. The response is correct.
If I tell Python v. 3.4.3, round(2.5), then it outputs 2. If I tell it round(1.5) then it outputs 2 as well, though. Similarly, round(3.5) gives 4, while round(4.5) gives 4 as well. I need Python to round with consistency, though. Specifically, it needs to round anytime I input a number halfway between two integers. So round(1.5) = 1 and round(2.5) = 2, while round(1.6) = 2 and such, as usual.
How can I resolve this?
EDIT: I've already read the documentation for the round function and understand that this is its intended behavior. My question is, how can I alter this behavior, because for my purposes I need 1.5 round down.
Python 3 uses a different rounding behaviour compared to Python 2: it now uses so-called "banker's rounding" (Wikipedia): when the integer part is odd, the number is rounded away from zero; when the integer part is even, is it rounded towards zero.
The reason for this is to avoid a bias, when all values at .5 are rounded away from zero (and then e.g. summed).
This is the behaviour you are seeing, and it is in fact consistent. It's perhaps just different than what you are used to.
The round docs do address the peculiaries of rounding floating point numbers.
You can use the decimal library to achieve what you want.
from decimal import Decimal, ROUND_HALF_UP, ROUND_HALF_DOWN
round(2.675, 2)
# output: 2.67
Decimal('2.675').quantize(Decimal('1.11'), rounding=ROUND_HALF_UP)
# output: 2.68
Decimal('2.5').quantize(Decimal('1.'), rounding=ROUND_HALF_DOWN)
# output: 2
Your want "round down", and you are getting "round to even". Just do it manually by doing
ceil(x - 0.5)
This is documented pretty well. According to the Python docs for round:
Note The behavior of round() for floats can be surprising: for example, round(2.675, 2) gives 2.67 instead of the expected 2.68. This is not a bug: it’s a result of the fact that most decimal fractions can’t be represented exactly as a float. See Floating Point Arithmetic: Issues and Limitations for more information
In specific, this is a side-effect of how computers handle floating-point numbers in general.
If you need more precision, including different rounding, I suggest you check out the Python Decimal module. Specifically of interest, they have the ability to control rounding modes. Looks like you might want decimal.ROUND_HALF_DOWN.
Python 3 provides rounding methods defined in the IEEE Standard for Floating-Point Arithmetic (IEEE 754), the default rounding[1] is directed to the nearest number and minimizing cumulative errors.
In IEEE 754, there are 5 methods defined, two for rounding to nearest (Python provides the first one by round) and three methods that are explicitly directed (Python has trunc, ceil, and floor in its Math module).
You obviously need a directed rounding and there is a way to tell this Python, you have just to choose.
[1] Since the representation of floating point numbers in computers is limited, rounding is not as trivial as you might think, you'll be surprised! I recommend a careful read of 15. Floating Point Arithmetic: Issues and Limitations in the python 3 documentation.
I believe I have the answer to all the rounding errors people have been encountering. I have wrote my own method, which functions same as the "round" but actually looks at the last digit and rounds from there case by case. There is no converting a decimal to binary. It can handle any amount of numbers behind the decimal and it also takes scientific notation (as outputted by floats). It also doesn't require any imports! Let me know if you catch any cases that don't work!
def Round(Number,digits = 0):
Number_Str = str(Number)
if "e" in Number_Str:
Number_Str = "%.10f" % float(Number_Str)
if "." in Number_Str: #If not given an integer
try:
Number_List = list(Number_Str) #All the characters in Number in a list
Number_To_Check = Number_List[Number_List.index(".") + 1 + digits] #Gets value to be looked at for rounding.
if int(Number_To_Check) >= 5:
Index_Number_To_Round = Number_List.index(".") + digits
if Number_List[Index_Number_To_Round] == ".":
Index_Number_To_Round -= 1
if int(Number_List[Index_Number_To_Round]) == 9:
Number_List_Spliced = Number_List[:Number_List.index(".")+digits]
for index in range(-1,-len(Number_List_Spliced) - 1,-1):
if Number_List_Spliced[index] == ".":
continue
elif int(Number_List_Spliced[index]) == 9:
Number_List_Spliced[index] = "0"
try:
Number_List_Spliced[index-1]
continue
except IndexError:
Number_List_Spliced.insert(0,"1")
else:
Number_List_Spliced[index] = str(int(Number_List_Spliced[index])+1)
break
FinalNumber = "".join(Number_List_Spliced)
else:
Number_List[Index_Number_To_Round] = str(int(Number_List[Index_Number_To_Round])+1)
FinalNumber = "".join(Number_List[:Index_Number_To_Round + 1])
return float(FinalNumber)
else:
FinalNumber = "".join(Number_List[:Number_List.index(".") + 1 + digits])
return float(FinalNumber)
except IndexError:
return float(Number)
else: #If given an integer
return float(Number)
I am new to Python and while experimenting with operators, I came across this:
>>> 7.0 / 3
2.3333333333333335
Shouldn't the result be 2.3333333333333333 or maybe 2.3333333333333334. Why is it rounding the number in such a way?
Also, with regard to floor division in Python 2.7 my results were:
>>> 5 / 2
2
>>> 5 // 2
2
>>> 5.0 / 2
2.5
>>> 5.0 // 2
2.0
So my observation is that floor division returns the integer quotient even in case of floating numbers, while normal division return the decimal value. Is this true?
Take a look at this 0.30000000000000004.com
Your language isn't broken, it's doing floating point math. Computers can only natively store integers, so they need some way of representing decimal numbers. This representation comes with some degree of inaccuracy. That's why, more often than not, .1 + .2 != .3.
Shouldn't the result be 2.3333333333333333 or maybe 2.3333333333333334. Why is it rounding the number in such a way?
The key is the number is being rounded twice.
The first rounding is part of the division operation, rounding the number to the nearest double-precision floating point value. This is a binary operation not a decimal one.
The second rounding is part of converting the floating point number to a decimal representation for display. It is possible to represent the exact value of any binary fraction in decimal, but it is usually not desirable as in most applications doing so will simply result in many digits of false-precision. Python instead outputs the shortest decimal approximation that will round-trip to the correct floating point value.
We can better see what is going on by using the Fraction and Decimal types, unlike converting directly to a string converting a floating point number to a Fraction or Decimal will give the exact value. We can also use the Fraction type to determine the error in our calculation.
>>> from fractions import Fraction
>>> from decimal import Decimal
>>> 7.0 / 3
2.3333333333333335
>>> Decimal(7.0 / 3)
Decimal('2.333333333333333481363069950020872056484222412109375')
>>> Fraction(7.0 / 3)
Fraction(5254199565265579, 2251799813685248)
>>> Fraction(7,3) - Fraction(7.0 / 3)
Fraction(-1, 6755399441055744)
The conversion via type Decimal shows us the exact value of the floating point number and demonstrates the many digits of false-precision that typically result from exact conversion of a floating point value to decimal.
The conversion to a Fraction is also interesting, the denominator is 2251799813685248 which is equivalent to 251. This makes perfect sense, a Double precision floating point has 53 effective bits of mantissa and we need two of those for the integral part of the result leaving 51 for the fractional part.
The error in our floating point calculation is 1/6755399441055744 or ⅓ * 2-51. This error is less than half our precision step of 2-51 so the answer was indeed correctly rounded to a double precision floating point value.