We Keep Coding

Does Python use "chop" instead of "nearest number" rounding?

I was under the impression that Python uses double precision arithmetic, with "nearest to" rounding. However, consider the following:
In the double precision system, the next number after 1.0 is 1.00...01 = 1 + 2**(-52). Now, if Python uses the "nearest to" rounding, the number 1 + 2**(-53) should round to 1 + 2**(-52). However, it turns out that 1 + 2**(-53) == 1. This would make sense if Python uses the "chop" rounding rule, but I was under the impression that nobody uses that since it biases calculations towards lower results.

The Python documentation is not strict about how floating-point arithmetic is handled. Some Python implementations use IEEE 754 with round-to-nearest-ties-to-even.
In the IEEE-754 binary64 format, also known as the “double precision” format, 1+2−53 is the midpoint between 1 and the next representable number, 1+2−52. So this is a tie, and the round-to-even rule applies. The significand for 1 is 1.000…0002, and the significand for 1+2−52 is 1.000…0012. The former is even, so the tie-breaking rule chooses it, and the result is 1.
Consider instead 1+3•2−54. This is three-quarters of the way from 1 to 1+2−52. So rounding it to the nearest representable value will produce 1+2−52. For print(1 == 1+3*2**-54), your Python implementation will likely print “False”.

Python: Why does have 2^-n work for n>52 and not 1+2^-n-1?

I'm pretty new to python, and I've made a table which calculates T=1+2^-n-1 and C=2^n, which both give the same values from n=40 to n=52, but for n=52 to n=61 I get 0.0 for T, whereas C gives me progressively smaller decimals each time - why is this?
I think I understand why T becomes 0.0, because of python using binary floating point and because of the machine epsilon value - but I'm slightly confused as to why C doesn't also become 0.0.
import numpy as np
import math
t=np.zeros(21)
c=np.zeros(21)
for n in range(40,61):
m=n-40
t[m]=1+2**(-n)-1
c[m]=2**(-n)
print (n,t[m],c[m])

The "floating" in floating point means that values are represented by storing a fixed number of leading digits and a scale factor, rather than assuming a fixed scale (which would be fixed point).
2**-53 only takes one (binary) digit to represent (not including the scale), but 1+2**-53 would take 54 to represent exactly. Python floats only have 53 binary digits of precision; 2**-53 can be represented exactly, but 1+2**-53 gets rounded to exactly 1, and subtracting 1 from that gives exactly 0. Thus, we have
>>> 2**-53
1.1102230246251565e-16
>>> 1+(2**-53)-1
0.0
Postscript: you might wonder why 2**-53 displays as a value not equal to the exact mathematical value when I said it was exact. That's due to the float->string conversion logic, which only keeps enough decimal digits to reconstruct the original float (instead of printing a bunch of digits at the end that are usually just noise).

The difference between both is indeed due to floating-point representation. Indeed, if you perform 1 + X where X is a very very small number, then the floating-point representation sets its exponent value to 0 and the precision is ensured by the mantissa, which is 52-bit on a 64-bit computer. Therefore, 1 + 2^(-X) if X > 52 is equal to 1. However, even 2^-100 can be represented in double-precision floating-point, so you can see C decrease for a larger number of samples.

why (0.0006*100000)%10 is 10

When I did (0.0006*100000)%10 and (0.0003*100000)%10 in python it returned 9.999999999999993 respectively, but actually it has to be 0.
Similarly in c++ fmod(0.0003*100000,10) gives the value as 10. Can someone help me out where i'm getting wrong.

The closest IEEE 754 64-bit binary number to 0.0003 is 0.0002999999999999999737189393389513725196593441069126129150390625. The closest representable number to the result of multiplying it by 100000 is 29.999999999999996447286321199499070644378662109375.
There are a number of operations, such as floor and mod, that can make very low significance differences very visible. You need to be careful using them in connection with floating point numbers - remember that, in many cases, you have a very, very close approximation to the infinite precision value, not the infinite precision value itself. The actual value can be slightly high or, as in this case, slightly low.

Just to give the obvious answer: 0.0006 and 0.0003 are not representable in a machine double (at least on modern machines). So you didn't actually multiply by those values, but by some value very close. Slightly more, or slightly less, depending on how the compiler rounded them.

May I suggest using the remainder function in C?
It will compute the remainder after rounding the quotient to nearest integer, with exact computation (no rounding error):
remainder = dividend - round(dividend/divisor)*divisor
This way, your result will be in [-divisor/2,+divisor/2] interval.
This will still emphasize the fact that you don't get a float exactly equal to 6/10,000 , but maybe in a less surprising way when you expect a null remainder:
remainder(0.0006*100000,10.0) -> -7.105427357601002e-15
remainder(0.0003*100000,10.0) -> -3.552713678800501e-15
I don't know of such remainder function support in python, but there seems to be a match in gnulib-python module (to be verified...)
https://github.com/ghostmansd/gnulib-python/blob/master/modules/remainder
EDIT
Why does it apparently work with every other N/10,000 in [1,9] interval but 3 and 6?
It's not completely lucky, this is somehow good properties of IEEE 754 in default rounding mode (round to nearest, tie to even).
The result of a floating point operation is rounded to nearest floating point value.
Instead of N/D you thus get (N/D+err) where the absolute error err is given by this snippet (I'm more comfortable in Smalltalk, but I'm sure you will find equivalent in Python):
| d |
d := 10000.
^(1 to: 9) collect: [:n | ((n/d) asFloat asFraction - (n/d)) asFloat]
It gives you something like:
#(4.79217360238593e-21 9.58434720477186e-21 -2.6281060661048628e-20 1.916869440954372e-20 1.0408340855860843e-20 -5.2562121322097256e-20 -7.11236625150491e-21 3.833738881908744e-20 -2.4633073358870662e-20)
Changing the last bit of a floating point significand leads to a small difference named the unit of least precision (ulp), and it might be good to express the error in term of ulp:
| d |
d := 10000.
^(1 to: 9) collect: [:n | ((n/d) asFloat asFraction - (n/d)) / (n/d) asFloat ulp]
the number of ulp off the exact fraction is thus:
#(0.3536 0.3536 -0.4848 0.3536 0.096 -0.4848 -0.0656 0.3536 -0.2272)
The error is the same for N=1,2,4,8 because they are essentially the same floating point - same significand, just the exponent changes.
It's also the same for N=3 and 6 for same reason, but very near the maximum error for a single operation which is 0.5 ulp (unluckily the number can be half way between two floats).
For N=9, the relative error is smaller than for N=1, and for 5 and 7, the error is very small.
Now when we multiply these approximation by 10000 which is exactly representable as a float, (N/D+err)D is N+Derr, and it's then rounded to nearest float. If D*err is less than half distance to next float, then this is rounded to N and the rounding error vanishes.
| d |
d := 10000.
^(1 to: 9) collect: [:n | ((n/d) asFloat asFraction - (n/d)) * d / n asFloat ulp]
OK, we were unlucky for N=3 and 6, the already high rounding error magnitude has become greater than 0.5 ulp:
#(0.2158203125 0.2158203125 -0.591796875 0.2158203125 0.1171875 -0.591796875 -0.080078125 0.2158203125 -0.138671875)
Beware, the distance is not symmetric for exact powers of two, the next float after 1.0 is 1.0+2^-52, but before 1.0 it's 1.0-2^-53.
Nonetheless, what we see here, is that after the second rounding operation, the error did annihilate in four cases, and did cumulate only in a single case (counting only the cases with different significands).
We can generalize that result. As long as we do not sum numbers with very different exponents, but just use muliply/divide operations, while the error bound can be high after P operations, the statistical distribution of cumulated errors has a remarkably narrow peak compared to this bound, and the result are somehow surprisingly good w.r.t. what we regularly read about float imprecision. See my answer to The number of correct decimal digits in a product of doubles with a large number of terms for example.
I just wanted to mention that yes, float are inexact, but they sometimes do such a decent job, that they are fostering the illusion of exactness. Finding a few outliers like mentionned in this post is then surprising. The sooner surprise, the least surprise. Ah, if only float were implemented less carefully, there would be less questions in this category...

A "round"ed number multiplied by 0.01 results in x.y00000000000001 and not x.y?

The reason I'm asking this is because there is a validation in OpenERP that it's driving me crazy:
>>> round(1.2 / 0.01) * 0.01
1.2
>>> round(12.2 / 0.01) * 0.01
12.200000000000001
>>> round(122.2 / 0.01) * 0.01
122.2
>>> round(1222.2 / 0.01) * 0.01
1222.2
As you can see, the second round is returning an odd value.
Can someone explain to me why is this happening?

This has in fact nothing to with round, you can witness the exact same problem if you just do 1220 * 0.01:
>>> 1220*0.01
12.200000000000001
What you see here is a standard floating point issue.
You might want to read what Wikipedia has to say about floating point accuracy problems:
The fact that floating-point numbers cannot precisely represent all real numbers, and that floating-point operations cannot precisely represent true arithmetic operations, leads to many surprising situations. This is related to the finite precision with which computers generally represent numbers.
Also see:
Numerical analysis
Numerical stability
A simple example for numerical instability with floating-point:
the numbers are finite. lets say we save 4 digits after the dot in a given computer or language.
0.0001 multiplied with 0.0001 would result something lower than 0.0001, and therefore it is impossible to save this result!
In this case if you calculate (0.0001 x 0.0001) / 0.0001 = 0.0001, this simple computer will fail in being accurate because it tries to multiply first and only afterwards to divide. In javascript, dividing with fractions leads to similar inaccuracies.

The float type that you are using stores binary floating point numbers. Not every decimal number is exactly representable as a float. In particular there is no exact representation of 1.2 or 0.01, so the actual number stored in the computer will differ very slightly from the value written in the source code. This representation error can cause calculations to give slightly different results from the exact mathematical result.
It is important to be aware of the possibility of small errors whenever you use floating point arithmetic, and write your code to work well even when the values calculated are not exactly correct. For example, you should consider rounding values to a certain number of decimal places when displaying them to the user.
You could also consider using the decimal type which stores decimal floating point numbers. If you use decimal then 1.2 can be stored exactly. However, working with decimal will reduce the performance of your code. You should only use it if exact representation of decimal numbers is important. You should also be aware that decimal does not mean that you'll never have any problems. For example 0.33333... has no exact representation as a decimal.

There is a loss of accuracy from the division due to the way floating point numbers are stored, so you see that this identity doesn't hold
>>> 12.2 / 0.01 * 0.01 == 12.2
False
bArmageddon, has provided a bunch of links which you should read, but I believe the takeaway message is don't expect floats to give exact results unless you fully understand the limits of the representation.
Especially don't use floats to represent amounts of money! which is a pretty common mistake
Python also has the decimal module, which may be useful to you

Others have answered your question and mentioned that many numbers don't have an exact binary fractional representation. If you are accustomed to working only with decimal numbers, it can seem deeply weird that a nice, "round" number like 0.01 could be a non-terminating number in some other base. In the spirit of "seeing is believing," here's a little Python program that will print out a binary representation of any number to any desired number of digits.
from decimal import Decimal
n = Decimal("0.01") # the number to print the binary equivalent of
m = 1000 # maximum number of digits to print
p = -1
r = []
w = int(n)
n = abs(n) - abs(w)
while n and -p < m:
s = Decimal(2) ** p
if n >= s:
r.append("1")
n -= s
else:
r.append("0")
p -= 1
print "%s.%s%s" % ("-" if w < 0 else "", bin(abs(w))[2:],
"".join(r), "..." if n else "")

Why fmod(1.0,0.1) == .1?

I experienced this phenomenon in Python first, but it turned out that it is the common answer, for example MS Excel gives this. Wolfram Alpha gives an interesting schizoid answer, where it states that the rational approximation of zero is 1/5. ( 1.0 mod 0.1 )
On the other hand, if I implement the definition by hand it gives me the 'right' answer (0).
def myFmod(a,n):
return a - floor(a/n) * n
What is going on here. Do I miss something?

Because 0.1 isn't 0.1; that value isn't representable in double precision, so it gets rounded to the nearest double-precision number, which is exactly:
0.1000000000000000055511151231257827021181583404541015625
When you call fmod, you get the remainder of division by the value listed above, which is exactly:
0.0999999999999999500399638918679556809365749359130859375
which rounds to 0.1 (or maybe 0.09999999999999995) when you print it.
In other words, fmod works perfectly, but you're not giving it the input that you think you are.
Edit: Your own implementation gives you the correct answer because it is less accurate, believe it or not. First off, note that fmod computes the remainder without any rounding error; the only source of inaccuracy is the representation error introduced by using the value 0.1. Now, let's walk through your implementation, and see how the rounding error that it incurs exactly cancels out the representation error.
Evaluate a - floor(a/n) * n one step at a time, keeping track of the exact values computed at each stage:
First we evaluate 1.0/n, where n is the closest double-precision approximation to 0.1 as shown above. The result of this division is approximately:
9.999999999999999444888487687421760603063276150363492645647081359...
Note that this value is not a representable double precision number -- so it gets rounded. To see how this rounding happens, let's look at the number in binary instead of decimal:
1001.1111111111111111111111111111111111111111111111111 10110000000...
The space indicates where the rounding to double precision occurs. Since the part after the round point is larger than the exact half-way point, this value rounds up to exactly 10.
floor(10.0) is, predictably, 10.0. So all that's left is to compute 1.0 - 10.0*0.1.
In binary, the exact value of 10.0 * 0.1 is:
1.0000000000000000000000000000000000000000000000000000 0100
again, this value is not representable as a double, and so is rounded at the position indicated by a space. This time it rounds down to exactly 1.0, and so the final computation is 1.0 - 1.0, which is of course 0.0.
Your implementation contains two rounding errors, which happen to exactly cancel out the representation error of the value 0.1 in this case. fmod, by contrast, is always exact (at least on platforms with a good numerics library), and exposes the representation error of 0.1.

This result is due to machine floating point representation. In your method, you are 'casting' (kinda) the float to an int and do not have this issue. The 'best' way to avoid such issues (esp for mod) is to multiply by a sufficiently large enough int (only 10 is needed in your case) and perform the operation again.
fmod(1.0,0.1)
fmod(10.0,1.0) = 0

From man fmod:
The fmod() function computes the floating-point remainder of dividing x
by y. The return value is x - n * y, where n is the quotient of x / y,
rounded towards zero to an integer.
So what happens is:
In fmod(1.0, 0.1), the 0.1 is actually slightly larger than 0.1 because the value cannot be exactly represented as a float.
So n = x / y = 1.0 / 0.1000something = 9.9999something
When rounded towards 0, n actually becomes 9
x - n * y = 1.0 - 9 * 0.1 = 0.1
Edit: As for why it works with floor(x/y), as far as I can tell this seems to be an FPU quirk. On x86, fmod uses the fprem instruction, whereas x/y will use fdiv. Curiously 1.0/0.1 seems to return exactly 10.0:
>>> struct.pack('d', 1.0/0.1) == struct.pack('d', 10.0)
True
I suppose fdiv uses a more precise algorithm than fprem. Some discussion can be found here: http://www.rapideuphoria.com/cgi-bin/esearch.exu?thread=1&fromMonth=A&fromYear=8&toMonth=C&toYear=8&keywords=%22Remainder%22

fmod returns x-i*y, which is less than y, and i is an integer. 0.09.... is because of floating point precision. try fmod(0.3, 0.1) -> 0.09... but fmod(0.4, 0.1) -> 0.0 because 0.3 is 0.2999999... as a float.
fmod(1/(2.**n), 1/(2.**m) will never produce anything but 0.0 for integer n>=m.

This gives the right answer:
a = 1.0
b = 0.1
a1,a2 = a.as_integer_ratio()
b1,b2 = b.as_integer_ratio()
div = float(a1*b2) / float(a2*b1)
mod = a - b*div
print mod
# 0.0
I think it works because by it uses rational equivalents of the two floating point numbers which provides a more accurate answer.

The Python divmod function is instructive here. It tells you both the quotient and remainder of a division operation.
$ python
>>> 0.1
0.10000000000000001
>>> divmod(1.0, 0.1)
(9.0, 0.09999999999999995)
When you type 0.1, the computer can't represent that exact value in binary floating-point arithmetic, so it chooses the closest number that it can represent, 0.10000000000000001. Then when you perform the division operation, floating-point arithmetic decides that the quotient has to be 9, since 0.10000000000000001 * 10 is larger than 1.0. This leaves you with a remainder that is slightly less than 0.1.
I would want to use the new Python fractions module to get exact answers.
>>> from fractions import Fraction
>>> Fraction(1, 1) % Fraction(1, 10)
Fraction(0, 1)
IOW, (1/1) mod (1/10) = (0/1), which is equivalent to 1 mod 0.1 = 0.
Another option is to implement the modulus operator yourself, allowing you to specify your own policy.
>>> x = 1.0
>>> y = 0.1
>>> x / y - math.floor(x / y)
0.0