I experienced this phenomenon in Python first, but it turned out that it is the common answer, for example MS Excel gives this. Wolfram Alpha gives an interesting schizoid answer, where it states that the rational approximation of zero is 1/5. ( 1.0 mod 0.1 )
On the other hand, if I implement the definition by hand it gives me the 'right' answer (0).
def myFmod(a,n):
return a - floor(a/n) * n
What is going on here. Do I miss something?
Because 0.1 isn't 0.1; that value isn't representable in double precision, so it gets rounded to the nearest double-precision number, which is exactly:
0.1000000000000000055511151231257827021181583404541015625
When you call fmod, you get the remainder of division by the value listed above, which is exactly:
0.0999999999999999500399638918679556809365749359130859375
which rounds to 0.1 (or maybe 0.09999999999999995) when you print it.
In other words, fmod works perfectly, but you're not giving it the input that you think you are.
Edit: Your own implementation gives you the correct answer because it is less accurate, believe it or not. First off, note that fmod computes the remainder without any rounding error; the only source of inaccuracy is the representation error introduced by using the value 0.1. Now, let's walk through your implementation, and see how the rounding error that it incurs exactly cancels out the representation error.
Evaluate a - floor(a/n) * n one step at a time, keeping track of the exact values computed at each stage:
First we evaluate 1.0/n, where n is the closest double-precision approximation to 0.1 as shown above. The result of this division is approximately:
9.999999999999999444888487687421760603063276150363492645647081359...
Note that this value is not a representable double precision number -- so it gets rounded. To see how this rounding happens, let's look at the number in binary instead of decimal:
1001.1111111111111111111111111111111111111111111111111 10110000000...
The space indicates where the rounding to double precision occurs. Since the part after the round point is larger than the exact half-way point, this value rounds up to exactly 10.
floor(10.0) is, predictably, 10.0. So all that's left is to compute 1.0 - 10.0*0.1.
In binary, the exact value of 10.0 * 0.1 is:
1.0000000000000000000000000000000000000000000000000000 0100
again, this value is not representable as a double, and so is rounded at the position indicated by a space. This time it rounds down to exactly 1.0, and so the final computation is 1.0 - 1.0, which is of course 0.0.
Your implementation contains two rounding errors, which happen to exactly cancel out the representation error of the value 0.1 in this case. fmod, by contrast, is always exact (at least on platforms with a good numerics library), and exposes the representation error of 0.1.
This result is due to machine floating point representation. In your method, you are 'casting' (kinda) the float to an int and do not have this issue. The 'best' way to avoid such issues (esp for mod) is to multiply by a sufficiently large enough int (only 10 is needed in your case) and perform the operation again.
fmod(1.0,0.1)
fmod(10.0,1.0) = 0
From man fmod:
The fmod() function computes the floating-point remainder of dividing x
by y. The return value is x - n * y, where n is the quotient of x / y,
rounded towards zero to an integer.
So what happens is:
In fmod(1.0, 0.1), the 0.1 is actually slightly larger than 0.1 because the value cannot be exactly represented as a float.
So n = x / y = 1.0 / 0.1000something = 9.9999something
When rounded towards 0, n actually becomes 9
x - n * y = 1.0 - 9 * 0.1 = 0.1
Edit: As for why it works with floor(x/y), as far as I can tell this seems to be an FPU quirk. On x86, fmod uses the fprem instruction, whereas x/y will use fdiv. Curiously 1.0/0.1 seems to return exactly 10.0:
>>> struct.pack('d', 1.0/0.1) == struct.pack('d', 10.0)
True
I suppose fdiv uses a more precise algorithm than fprem. Some discussion can be found here: http://www.rapideuphoria.com/cgi-bin/esearch.exu?thread=1&fromMonth=A&fromYear=8&toMonth=C&toYear=8&keywords=%22Remainder%22
fmod returns x-i*y, which is less than y, and i is an integer. 0.09.... is because of floating point precision. try fmod(0.3, 0.1) -> 0.09... but fmod(0.4, 0.1) -> 0.0 because 0.3 is 0.2999999... as a float.
fmod(1/(2.**n), 1/(2.**m) will never produce anything but 0.0 for integer n>=m.
This gives the right answer:
a = 1.0
b = 0.1
a1,a2 = a.as_integer_ratio()
b1,b2 = b.as_integer_ratio()
div = float(a1*b2) / float(a2*b1)
mod = a - b*div
print mod
# 0.0
I think it works because by it uses rational equivalents of the two floating point numbers which provides a more accurate answer.
The Python divmod function is instructive here. It tells you both the quotient and remainder of a division operation.
$ python
>>> 0.1
0.10000000000000001
>>> divmod(1.0, 0.1)
(9.0, 0.09999999999999995)
When you type 0.1, the computer can't represent that exact value in binary floating-point arithmetic, so it chooses the closest number that it can represent, 0.10000000000000001. Then when you perform the division operation, floating-point arithmetic decides that the quotient has to be 9, since 0.10000000000000001 * 10 is larger than 1.0. This leaves you with a remainder that is slightly less than 0.1.
I would want to use the new Python fractions module to get exact answers.
>>> from fractions import Fraction
>>> Fraction(1, 1) % Fraction(1, 10)
Fraction(0, 1)
IOW, (1/1) mod (1/10) = (0/1), which is equivalent to 1 mod 0.1 = 0.
Another option is to implement the modulus operator yourself, allowing you to specify your own policy.
>>> x = 1.0
>>> y = 0.1
>>> x / y - math.floor(x / y)
0.0
Related
I wrote a simple code in python that gives me the same value of machine epsilon using the numpy command:np.finfo(float).eps
The code is:
eps=1
while eps+1 != 1:
eps /= 2
print(eps)
But I didn't want to stop here ! I used smaller and smaller numbers to divide eps, for example:
eps=1
while eps+1 != 1:
eps /= 1.1
print (eps)
With this, I got a value of 1.158287085355336e-16 for epsilon. I noticed that epsilon was converging to a number, my last attempt at 1.0000001 gave me the value of 1.1102231190697707e-16.
Is this value closer to the real value of epsilon for my Pc? I think I'm not considering something important and my line of thinking is wrong.
Thank you in advance for the help !
The term βmachine epsilonβ is not consistently defined, so it's better to avoid that word and instead to say what specifically you're talking about:
ulp(1), the magnitude of the least significant digit, or Unit in the Last Place, of 1.
This is the distance from 1 to the next larger floating-point number.
More generally, ulp(π₯) is the distance from π₯ to the next larger floating-point number in magnitude.
In binary64 floating-point, with 53 bits of precision, ulp(1) is 2β»β΅Β² β 2.220446049250313βΓβ10β»ΒΉβΆ.
In decimal64 floating-point, with 16 digits of precision, ulp(1) is 10β»ΒΉβ΅.
In general, for floating-point in radix π½ with π digits of precision (including the implicit 1 digit), ulp(1) = π½1 β π.
The relative error bound, sometimes also called unit roundoff or u.
A floating-point operation may round the outcome of a mathematical function such as π₯ + π¦, giving fl(π₯ + π¦) = (π₯ + π¦)β
(1 + πΏ) for some relative error πΏ.
For basic arithmetic operations in IEEE 754 (+, β, *, /, sqrt), the result of computing the floating-point operation is guaranteed to be correctly rounded, which in the default rounding mode means it yields the nearest floating-point number, or one of the two nearest such numbers if π₯ + π¦ lies exactly halfway between them.
In binary64 floating-point, with 53 bits of precision, the relative error of an operation correctly rounded to nearest is at most 2β»β΅Β³ β 1.1102230246251565βΓβ10β»ΒΉβΆ.
In decimal64 floating-point, with 16 digits of precision, the relative error of an operation correctly rounded to nearest is at most 5βΓβ10β»ΒΉβΆ.
In general, when floating-point arithmetic in radix π½ with π digits of precision is correctly rounded to nearest, πΏ is bounded in magnitude by the relative error bound (π½/2) π½βπ.
What the Python iteration while 1 + eps != 1: eps /= 2 computes, starting with eps = 1., is the relative error bound in binary64 floating-point, since that's the floating-point that essentially all Python implementations use.
If you had a version of Python that worked in a different radix, say b, you would instead want to use while 1 + eps != 1: eps /= b.
If you do eps /= 1.1 or eps /= 1.0001, you will get an approximation to the relative error bound erring on the larger side, with no particular significance to the result.
Note that sys.float_info.epsilon is ulp(1), rather than the relative error bound.
They are always related: ulp(1)/2 is the relative error bound in every floating-point format.
If you want the actual machine epsilon for a Python float on your PC, you can get it from the epsilon attribute of sys.float_info. By the way, on my x86-64 machine, numpy.finfo(float) gives me 2.220446049250313e-16, which is the expected machine epsilon for a 64-bit float.
Your intuition for trying to find the value eps such that 1 + eps != 1 is True is good, but machine epsilon is an upper bound on the relative error due to rounding in floating-point arithmetic. Not to mention, the inexact nature of floating-point arithmetic can be mystifying sometimes: note that 0.1 + 0.1 + 0.1 != 0.3 evaluates to True. Also, if I modify your code to
eps = 1
while eps + 9 != 9:
eps = eps / 1.0000001
print(eps)
I get, after around maybe half a minute,
8.881783669690459e-16
The relative error in this case is 8.881783669690459e-16 / 9 = 9.868648521878288e-17.
Your program is almost fine.
IT should be
eps=1
while eps+1 != 1:
eps /= 2
print(2*eps)
The result is
2.220446049250313e-16
Which is the epsilon of the machine. You can check this with this piece of
code:
import sys
sys.float_info.epsilon
The reason we should multiply by 2 in the last line is because you went 1 division too far inside the while loop.
In Python, round can be used to round to n decimal digits. Now, round(0.229, 1) gives 0.2 and round(0.251, 1) gives 0.3. Is there a way in Python, to round them both in one direction, say, to 0.2? In other words, is there something analogous to floor or ceil for rounding to an integer, in the context of rounding to a certain number of decimal places?
Update: Accepting the solution based on decimal because it throws light on such a feature available in Python. The solutions based on dividing (and later multiplying) by a multiple of 10 are good ones and straightforward to use.
You can use the decimal library:
from decimal import *
print(Decimal(0.229).quantize(Decimal('.1'), rounding=ROUND_DOWN))
print(Decimal(0.251).quantize(Decimal('.1'), rounding=ROUND_DOWN))
Output:
0.2
0.2
You can also convert from Decimal to float:
f = float(Decimal(0.229).quantize(Decimal('.1'), rounding=ROUND_DOWN)))
You could use floor as follows:
def round_down(x):
return math.floor(10*x) / 10
print(round_down(0.229))
print(round_down(0.251))
This prints:
0.2
0.2
The logic here is to first multiply up by 10, which then lets floor round down all decimal components other than the first one. Then, we divide again by 10 to generate the rounded-down number.
There are already a couple of good answers to your question, however both solutions rely on external libraries. It is also possible to achieve what you desire with no imports:
def round_down(x):
return int(10 * x) / 10
x = 0.251
round_down(x)
# 0.2
x = 0.229
round_down(x)
# 0.2
If you wanted something more similar to Python's round (where you can specify the number n of digits to round to), you could use:
def round_down(x, n=1):
return int(10**n * x) / 10**n
x = 0.229
round_down(x)
# 0.2
round_down(x, 2)
# 0.22
Depending how you'd like the rounding to behave for negative numbers, you may instead wish to define round_down(x) as:
def round_down(x):
return 10 * x // 1 / 10
I know that most decimals don't have an exact floating point representation (Is floating point math broken?).
But I don't see why 4*0.1 is printed nicely as 0.4, but 3*0.1 isn't, when
both values actually have ugly decimal representations:
>>> 3*0.1
0.30000000000000004
>>> 4*0.1
0.4
>>> from decimal import Decimal
>>> Decimal(3*0.1)
Decimal('0.3000000000000000444089209850062616169452667236328125')
>>> Decimal(4*0.1)
Decimal('0.40000000000000002220446049250313080847263336181640625')
The simple answer is because 3*0.1 != 0.3 due to quantization (roundoff) error (whereas 4*0.1 == 0.4 because multiplying by a power of two is usually an "exact" operation). Python tries to find the shortest string that would round to the desired value, so it can display 4*0.1 as 0.4 as these are equal, but it cannot display 3*0.1 as 0.3 because these are not equal.
You can use the .hex method in Python to view the internal representation of a number (basically, the exact binary floating point value, rather than the base-10 approximation). This can help to explain what's going on under the hood.
>>> (0.1).hex()
'0x1.999999999999ap-4'
>>> (0.3).hex()
'0x1.3333333333333p-2'
>>> (0.1*3).hex()
'0x1.3333333333334p-2'
>>> (0.4).hex()
'0x1.999999999999ap-2'
>>> (0.1*4).hex()
'0x1.999999999999ap-2'
0.1 is 0x1.999999999999a times 2^-4. The "a" at the end means the digit 10 - in other words, 0.1 in binary floating point is very slightly larger than the "exact" value of 0.1 (because the final 0x0.99 is rounded up to 0x0.a). When you multiply this by 4, a power of two, the exponent shifts up (from 2^-4 to 2^-2) but the number is otherwise unchanged, so 4*0.1 == 0.4.
However, when you multiply by 3, the tiny little difference between 0x0.99 and 0x0.a0 (0x0.07) magnifies into a 0x0.15 error, which shows up as a one-digit error in the last position. This causes 0.1*3 to be very slightly larger than the rounded value of 0.3.
Python 3's float repr is designed to be round-trippable, that is, the value shown should be exactly convertible into the original value (float(repr(f)) == f for all floats f). Therefore, it cannot display 0.3 and 0.1*3 exactly the same way, or the two different numbers would end up the same after round-tripping. Consequently, Python 3's repr engine chooses to display one with a slight apparent error.
repr (and str in Python 3) will put out as many digits as required to make the value unambiguous. In this case the result of the multiplication 3*0.1 isn't the closest value to 0.3 (0x1.3333333333333p-2 in hex), it's actually one LSB higher (0x1.3333333333334p-2) so it needs more digits to distinguish it from 0.3.
On the other hand, the multiplication 4*0.1 does get the closest value to 0.4 (0x1.999999999999ap-2 in hex), so it doesn't need any additional digits.
You can verify this quite easily:
>>> 3*0.1 == 0.3
False
>>> 4*0.1 == 0.4
True
I used hex notation above because it's nice and compact and shows the bit difference between the two values. You can do this yourself using e.g. (3*0.1).hex(). If you'd rather see them in all their decimal glory, here you go:
>>> Decimal(3*0.1)
Decimal('0.3000000000000000444089209850062616169452667236328125')
>>> Decimal(0.3)
Decimal('0.299999999999999988897769753748434595763683319091796875')
>>> Decimal(4*0.1)
Decimal('0.40000000000000002220446049250313080847263336181640625')
>>> Decimal(0.4)
Decimal('0.40000000000000002220446049250313080847263336181640625')
Here's a simplified conclusion from other answers.
If you check a float on Python's command line or print it, it goes through function repr which creates its string representation.
Starting with version 3.2, Python's str and repr use a complex rounding scheme, which prefers
nice-looking decimals if possible, but uses more digits where
necessary to guarantee bijective (one-to-one) mapping between floats
and their string representations.
This scheme guarantees that value of repr(float(s)) looks nice for simple
decimals, even if they can't be
represented precisely as floats (eg. when s = "0.1").
At the same time it guarantees that float(repr(x)) == x holds for every float x
Not really specific to Python's implementation but should apply to any float to decimal string functions.
A floating point number is essentially a binary number, but in scientific notation with a fixed limit of significant figures.
The inverse of any number that has a prime number factor that is not shared with the base will always result in a recurring dot point representation. For example 1/7 has a prime factor, 7, that is not shared with 10, and therefore has a recurring decimal representation, and the same is true for 1/10 with prime factors 2 and 5, the latter not being shared with 2; this means that 0.1 cannot be exactly represented by a finite number of bits after the dot point.
Since 0.1 has no exact representation, a function that converts the approximation to a decimal point string will usually try to approximate certain values so that they don't get unintuitive results like 0.1000000000004121.
Since the floating point is in scientific notation, any multiplication by a power of the base only affects the exponent part of the number. For example 1.231e+2 * 100 = 1.231e+4 for decimal notation, and likewise, 1.00101010e11 * 100 = 1.00101010e101 in binary notation. If I multiply by a non-power of the base, the significant digits will also be affected. For example 1.2e1 * 3 = 3.6e1
Depending on the algorithm used, it may try to guess common decimals based on the significant figures only. Both 0.1 and 0.4 have the same significant figures in binary, because their floats are essentially truncations of (8/5)(2^-4) and (8/5)(2^-6) respectively. If the algorithm identifies the 8/5 sigfig pattern as the decimal 1.6, then it will work on 0.1, 0.2, 0.4, 0.8, etc. It may also have magic sigfig patterns for other combinations, such as the float 3 divided by float 10 and other magic patterns statistically likely to be formed by division by 10.
In the case of 3*0.1, the last few significant figures will likely be different from dividing a float 3 by float 10, causing the algorithm to fail to recognize the magic number for the 0.3 constant depending on its tolerance for precision loss.
Edit:
https://docs.python.org/3.1/tutorial/floatingpoint.html
Interestingly, there are many different decimal numbers that share the same nearest approximate binary fraction. For example, the numbers 0.1 and 0.10000000000000001 and 0.1000000000000000055511151231257827021181583404541015625 are all approximated by 3602879701896397 / 2 ** 55. Since all of these decimal values share the same approximation, any one of them could be displayed while still preserving the invariant eval(repr(x)) == x.
There is no tolerance for precision loss, if float x (0.3) is not exactly equal to float y (0.1*3), then repr(x) is not exactly equal to repr(y).
Can someone help me unpack what exactly is going on under the hood here?
>>> 1e16 + 1.
1e+16
>>> 1e16 + 1.1
1.0000000000000002e+16
I'm on 64-bit Python 2.7. For the first, I would assume that since there is only a precision of 15 for float that it's just round-off error. The true floating-point answer might be something like
10000000000000000.999999....
And the decimal just gets lopped of. But the second result makes me question this understanding and can't 1 be represented exactly? Any thoughts?
[Edit: Just to clarify. I'm not in any way suggesting that the answers are "wrong." Clearly, they're right, because, well they are. I'm just trying to understand why.]
It's just rounding as close as it can.
1e16 in floating hex is 0x4341c37937e08000.
1e16+2 is 0x4341c37937e08001.
At this level of magnitude, the smallest difference in precision that you can represent is 2. Adding 1.0 exactly rounds down (because typically IEEE floating point math will round to an even number). Adding values larger than 1.0 will round up to the next representable value.
10^16 = 0x002386f26fc10000 is exactly representable as a double precision floating point number. The next representable number is 1e16+2. 1e16+1 is correctly rounded to 1e16, and 1e16+1.1 is correctly rounded to 1e16+2. Check the output of this C program:
#include <stdio.h>
#include <math.h>
#include <stdint.h>
int main()
{
uint64_t i = 10000000000000000ULL;
double a = (double)i;
double b = nextafter(a,1.0e20); // next representable number
printf("I=0x%016llx\n",i); // 10^16 in hex
printf("A=%a (%.4f)\n",a,a); // double representation
printf("B=%a (%.4f)\n",b,b); // next double
}
Output:
I=0x002386f26fc10000
A=0x1.1c37937e08p+53 (10000000000000000.0000)
B=0x1.1c37937e08001p+53 (10000000000000002.0000)
Let's decode some floats, and see what's actually going on! I'm going to use Common Lisp, which has a handy function for getting at the significand (a.k.a mantissa) and exponent of a floating-point number without needing to twiddle any bits. All floats used are IEEE double-precision floats.
> (integer-decode-float 1.0d0)
4503599627370496
-52
1
That is, if we consider the value stored in the significand as an integer, it is the maximum power of 2 available (4503599627370496 = 2^52), scaled down (2^-52). (It isn't stored as 1 with an exponent of 0 because it's simpler for the significand to never have zeros on the left, and this allows us to skip representing the leftmost 1 bit and have more precision. Numbers not in this form are called denormal.)
Let's look at 1e16.
> (integer-decode-float 1d16)
5000000000000000
1
1
Here we have the representation (5000000000000000) * 2^1. Note that the significand, despite being a nice round decimal number, is not a power of 2; this is because 1e16 is not a power of 2. Every time you multiply by 10, you are multiplying by 2 and 5; multiplying by 2 is just incrementing the exponent, but multiplying by 5 is an "actual" multiplication, and here we've multiplied by 5 16 times.
5000000000000000 = 10001110000110111100100110111111000001000000000000000 (base 2)
Observe that this is a 53-bit binary number, as it should be since double floats have a 53-bit significand.
But the key to understanding the situation is that the exponent is 1. (The exponent being small is an indication that we are getting close to the limits of precision.) This means that the float value is 2^1 = 2 times this significand.
Now, what happens when we try to represent adding 1 to this number? Well, we need to represent 1 at the same scale. But the smallest change we can make in this number is exactly 2, because the least significant bit of the significand has value 2!
That is, if we increment the significand, making the smallest possible change, we get
5000000000000001 = 10001110000110111100100110111111000001000000000000001 (base 2)
and when we apply the exponent, we get 2 * 5000000000000001 = 10000000000000002, which is exactly the value you observed. You can only have either 10000000000000000 or 10000000000000002, and 10000000000000001.1 is closer to the latter.
(Note that the issue here isn't even that decimal numbers aren't exact in binary! There's no binary "repeating decimals" here, and there's plenty of 0 bits on the right end of the significand β it's just that your input neatly falls just beyond the lowest bit.)
With numpy, you can see the next larger and smaller representable IEEE floating point number:
>>> import numpy as np
>>> huge=1e100
>>> tiny=1e-100
>>> np.nextafter(1e16,huge)
10000000000000002.0
>>> np.nextafter(1e16,tiny)
9999999999999998.0
So:
>>> (np.nextafter(1e16,huge)-np.nextafter(1e16,tiny))/2.0
2.0
And:
>>> 1.1>2.0/2
True
Therefore 1e16 + 1.1 is correctly rounded to the next larger IEEE representable number of 10000000000000002.0
As is:
>>> 1e16+1.0000000000000005
1.0000000000000002e+16
and 1e16-(something slightly larger than 1) is rounded down by 2 to the next smaller IEEE number:
>>> 1e16-1.0000000000000005
9999999999999998.0
Keep in mind that 32 bit vs 64 bit Python is irrelevant. It is the size of the IEEE format used that matters. Also keep in mind that the larger the magnitude of the number, the epsilon value (the spread between the two next larger and smaller IEEE values basically) changes.
You can see this in bits as well:
>>> def f_to_bits(f): return struct.unpack('<Q', struct.pack('<d', f))[0]
...
>>> def bits_to_f(bits): return struct.unpack('<d', struct.pack('<Q', bits))[0]
...
>>> bits_to_f(f_to_bits(1e16)+1)
1.0000000000000002e+16
>>> bits_to_f(f_to_bits(1e16)-1)
9999999999999998.0
The reason I'm asking this is because there is a validation in OpenERP that it's driving me crazy:
>>> round(1.2 / 0.01) * 0.01
1.2
>>> round(12.2 / 0.01) * 0.01
12.200000000000001
>>> round(122.2 / 0.01) * 0.01
122.2
>>> round(1222.2 / 0.01) * 0.01
1222.2
As you can see, the second round is returning an odd value.
Can someone explain to me why is this happening?
This has in fact nothing to with round, you can witness the exact same problem if you just do 1220 * 0.01:
>>> 1220*0.01
12.200000000000001
What you see here is a standard floating point issue.
You might want to read what Wikipedia has to say about floating point accuracy problems:
The fact that floating-point numbers cannot precisely represent all real numbers, and that floating-point operations cannot precisely represent true arithmetic operations, leads to many surprising situations. This is related to the finite precision with which computers generally represent numbers.
Also see:
Numerical analysis
Numerical stability
A simple example for numerical instability with floating-point:
the numbers are finite. lets say we save 4 digits after the dot in a given computer or language.
0.0001 multiplied with 0.0001 would result something lower than 0.0001, and therefore it is impossible to save this result!
In this case if you calculate (0.0001 x 0.0001) / 0.0001 = 0.0001, this simple computer will fail in being accurate because it tries to multiply first and only afterwards to divide. In javascript, dividing with fractions leads to similar inaccuracies.
The float type that you are using stores binary floating point numbers. Not every decimal number is exactly representable as a float. In particular there is no exact representation of 1.2 or 0.01, so the actual number stored in the computer will differ very slightly from the value written in the source code. This representation error can cause calculations to give slightly different results from the exact mathematical result.
It is important to be aware of the possibility of small errors whenever you use floating point arithmetic, and write your code to work well even when the values calculated are not exactly correct. For example, you should consider rounding values to a certain number of decimal places when displaying them to the user.
You could also consider using the decimal type which stores decimal floating point numbers. If you use decimal then 1.2 can be stored exactly. However, working with decimal will reduce the performance of your code. You should only use it if exact representation of decimal numbers is important. You should also be aware that decimal does not mean that you'll never have any problems. For example 0.33333... has no exact representation as a decimal.
There is a loss of accuracy from the division due to the way floating point numbers are stored, so you see that this identity doesn't hold
>>> 12.2 / 0.01 * 0.01 == 12.2
False
bArmageddon, has provided a bunch of links which you should read, but I believe the takeaway message is don't expect floats to give exact results unless you fully understand the limits of the representation.
Especially don't use floats to represent amounts of money! which is a pretty common mistake
Python also has the decimal module, which may be useful to you
Others have answered your question and mentioned that many numbers don't have an exact binary fractional representation. If you are accustomed to working only with decimal numbers, it can seem deeply weird that a nice, "round" number like 0.01 could be a non-terminating number in some other base. In the spirit of "seeing is believing," here's a little Python program that will print out a binary representation of any number to any desired number of digits.
from decimal import Decimal
n = Decimal("0.01") # the number to print the binary equivalent of
m = 1000 # maximum number of digits to print
p = -1
r = []
w = int(n)
n = abs(n) - abs(w)
while n and -p < m:
s = Decimal(2) ** p
if n >= s:
r.append("1")
n -= s
else:
r.append("0")
p -= 1
print "%s.%s%s" % ("-" if w < 0 else "", bin(abs(w))[2:],
"".join(r), "..." if n else "")