Why happen this conversion to a number with 2 decimals?
x = 369.69
y=decimal.Decimal(x)
Decimal('369.68999999999999772626324556767940521240234375')
even if I've declared
getcontext().prec = 2
?
Then why if I try to get the roundup to get 370.00:
y.quantize(decimal.Decimal('0.01'),rounding=decimal.ROUND_UP)
end up with this error:
InvalidOperation: quantize result has too many digits for current
context
quantize result has too many digits for current context
The issue is that x is a float, and so you've lost precision as soon as you assign to x. If you want to work round this, you could make x a string "369.69". A Decimal built from a string will have the exact precision.
When you create a Decimal object, the prec is ignored. From the documentation:
The significance of a new Decimal is determined solely by the number of digits input. Context precision and rounding only come into play during arithmetic operations.
The error you're getting in the quantize is because the number of digits in the result is greater than the set precision. prec sets the total number of digits, not the digits after the decimal point.
>>> y = decimal.Decimal(69.69)
>>> y
Decimal('69.68999999999999772626324556767940521240234375')
>>> y.quantize(decimal.Decimal('1'), rounding=decimal.ROUND_UP)
Decimal('70')
Related
When I convert a float to decimal.Decimal in Python and afterwards call Decimal.shift it may give me completely wrong and unexpected results depending on the float. Why is this the case?
Converting 123.5 and shifting it:
from decimal import Decimal
a = Decimal(123.5)
print(a.shift(1)) # Gives expected result
The code above prints the expected result of 1235.0.
If I instead convert and shift 123.4:
from decimal import Decimal
a = Decimal(123.4)
print(a.shift(1)) # Gives UNexpected result
it gives me 3.418860808014869689941406250E-18 (approx. 0) which is completely unexpected and wrong.
Why is this the case?
Note:
I understand the floating-point imprecision because of the representation of floats in memory. However, I can't explain why this should give me such a completely wrong result.
Edit:
Yes, in general it would be best to not convert the floats to decimals but convert strings to decimals instead. However, this is not the point of my question. I want to understand why the shifting after float conversion gives such a completely wrong result. So if I print(Decimal(123.4)) it gives 123.40000000000000568434188608080148696899414062 so after shifting I would expect it to be 1234.0000000000000568434188608080148696899414062 and not nearly zero.
You need to change the Decimal constructor input to use strings instead of floats.
a = Decimal('123.5')
print(a.shift(1))
a = Decimal('123.4')
print(a.shift(1))
or
a = Decimal(str(123.5))
print(a.shift(1))
a = Decimal(str(123.4))
print(a.shift(1))
The output will be as expected.
>>> 1235.0
>>> 1234.0
Decimal instances can be constructed from integers, strings, floats, or tuples. Construction from an integer or a float performs an exact conversion of the value of that integer or float.
For floats, Decimal calls Decimal.from_float()
Note that Decimal.from_float(0.1) is not the same as Decimal('0.1'). Since 0.1 is not exactly representable in binary floating point, the value is stored as the nearest representable value which is 0x1.999999999999ap-4. The exact equivalent of the value in decimal is 0.1000000000000000055511151231257827021181583404541015625.
Internally, the Python decimal library converts a float into two integers representing the numerator and denominator of a fraction that yields the float.
n, d = abs(123.4).as_integer_ratio()
It then calculates the bit length of the denominator, which is the number of bits required to represent the number in binary.
k = d.bit_length() - 1
And then from there the bit length k is used to record the coefficient of the decimal number by multiplying the numerator * 5 to the power of the bit length of the denominator.
coeff = str(n*5**k)
The resulting values are used to create a new Decimal object with constructor arguments of sign, coefficient, and exponent using this values.
For the float 123.5 these values are
>>> 1 1235 -1
and for the float 123.4 these values are
1 123400000000000005684341886080801486968994140625 -45
So far, nothing is amiss.
However when you call shift, the Decimal library has to calculate how much to pad the number with zeroes based on the shift you've specified. To do this internally it takes the precision subtracted by length of the coefficient.
amount_to_pad = context.prec - len(coeff)
The default precision is only 28 and with a float like 123.4 the coefficient becomes much longer than the default precision as noted above. This creates a negative amount to pad with zeroes and makes the number very tiny as you noted.
A way around this is to increase the precision to the length of the exponent + the length of the number you started with (45 + 4).
from decimal import Decimal, getcontext
getcontext().prec = 49
a = Decimal(123.4)
print(a)
print(a.shift(1))
>>> 123.400000000000005684341886080801486968994140625
>>> 1234.000000000000056843418860808014869689941406250
The documentation for shift hints that the precision is important for this calculation:
The second operand must be an integer in the range -precision through precision.
However it does not explain this caveat for floats that don't play nice with memory limitations.
I would expect this to raise some kind of error and prompt you to change your input or increase the precision, but at least you know!
#MarkDickinson noted in a comment above that you can view this Python bug tracker for more information: https://bugs.python.org/issue7233
from decimal import *
getcontext().prec = 8
print(getcontext(),"\n")
x_amount = Decimal(0.025)
y_amount = Decimal(0.005)
test3 = x_amount - y_amount
print("test3",test3)
Output:
Context(prec=8, rounding=ROUND_HALF_EVEN, Emin=-999999, Emax=999999, capitals=1, clamp=0, flags=[], traps=[InvalidOperation, DivisionByZero, Overflow])
test3 0.020000000
Why does this return value of 'test3' up to 9 decimal places if the precision is set to 8 according to example mentioned here?
And it changes to 3 decimal places if I replace Line 6 and 7 in the above code with:
x_amount = Decimal('0.025')
y_amount = Decimal('0.005')
I am using decimals in a financial application and find it very confusing about conversions, operations, definition, precision etc. Is there a link which I can refer to know the details of using decimals in python?
Your result is correct. prec says how many digits to keep starting from the most significant non-zero digit. So in your result:
test3 0.020000000
^^^^^^^^
the digits pointed to by carets are the expected eight digits covered by the precision. The reason you get all of them is that using float to initialize Decimal is always wrong. Never initialize a Decimal from a float; it just reproduces the inaccuracy of float in the Decimal (try printing x_value and y_value to see the garbage). Passing a str is the only safe way to do this.
When you do this with str, the individual Decimals "know" the last digit of meaningful precision they possess, and it only goes to the third decimal place, so the result doesn't include precision beyond that point. If you want prec capping to kick in, try initializing one of the arguments with more precision than prec, and the result will be rounded to prec.
NB: this question is about significant figures. It is not a question about "digits after the decimal point" or anything like that.
EDIT: This question is not a duplicate of Significant figures in the decimal module. The two questions are asking about entirely different problems. I want to know why the function about does not return the desired value for a specific input. None of the answers to Significant figures in the decimal module address this question.
The following function is supposed to return a string representation of a float with the specified number of significant figures:
import decimal
def to_sigfigs(value, sigfigs):
return str(decimal.Context(prec=sigfigs).create_decimal(value))
At first glance, it seems to work:
print to_sigfigs(0.000003141592653589793, 5)
# 0.0000031416
print to_sigfigs(0.000001, 5)
# 0.0000010000
print to_sigfigs(3.141592653589793, 5)
# 3.1416
...but
print to_sigfigs(1.0, 5)
# 1
The desired output for the last expression (IOW, the 5-significant figure representation of 1.0) is the string '1.0000'. The actual output is the string '1'.
Am I misunderstanding something or is this a bug in decimal?
The precision of a context is a maximum precision; if an operation would produce a Decimal with less digits than the context's precision, it is not padded out to the context's precision.
When you call to_sigfigs(0.000001, 5), 0.000001 already has some rounding error due to the conversion from source code to binary floating point. It's actually 9.99999999999999954748111825886258685613938723690807819366455078125E-7. Rounding that to 5 significant figures gives decimal.Decimal("0.0000010000").
On the other hand, 1 is exactly representable in binary floating point, so 1.0 is exactly 1. Since only 1 digit is needed to represent this in decimal, the context's precision doesn't require any rounding, and you get a 1-digit Decimal result.
Is it a bug? I don't know, I don't think the documentation is tight enough to make that determination. It certainly is a surprising result.
It is possible to fix your own function with a little more logic.
def to_sigfigs(value, sigfigs):
sign, digits, exponent = decimal.Context(prec=sigfigs).create_decimal(value).as_tuple()
if len(digits) < sigfigs:
missing = sigfigs - len(digits)
digits = digits + (0,) * missing
exponent -= missing
return str(decimal.Decimal((sign, digits, exponent)))
>>> '%f'%0.8407745
'0.840774'
>>> '%f'%0.8407755
'0.840776'
>>> '%f'%-0.8407755
'-0.840776'
>>> '%f'%-0.8407745
'-0.840774'
The results look weird. It is sometimes floor and sometimes ceil.
What is the default rounding mode of string formatter in Python?
Floating point numbers are always an approximation. 0.8407745 is not exactly 0.8407745:
>>> '%.53f' % 0.8407745
'0.84077449999999998020427938172360882163047790527343750'
So the default formatter, rounding to 6 decimals, correctly rounds that value to 0.840774.
0.8407755 on the other hand is:
>>> '%.53f' % 0.8407755
'0.84077550000000000895994389793486334383487701416015625'
and should thus be rounded up.
See The Floating Point Guide for a good introduction as to why that is. (Summary: floating point numbers are represented by the sum of binary fractions).
As per this table here, by default %f will have only 6 digit precision. So, the data is rounded off to 6 digits after the decimal point.
Quoting from the notes section of that table
The alternate form causes the result to always contain a decimal
point, even if no digits follow it.
The precision determines the number of digits after the decimal point
and defaults to 6.
I have a list and it contains a certain number '5.74536541' in it which I convert to a float.
I am printing it out in Python 3 using ("%0.2f" % (variable)) but it always prints out 5.75 instead of 5.74.
I know you're thinking who cares, but it is for a currency converter program and I don't want the currencies to round up/down but to be exact.
How can I keep it from rounding but also keep the 2 decimal places?
You shouldn't use floating point numbers for currency, due to rounding errors like you mentioned.
Your best bet is to use a fixed-precision decimal where you also have full control over how rounding and truncation works. From the docs:
>>> from decimal import *
>>> getcontext()
Context(prec=28, rounding=ROUND_HALF_EVEN, Emin=-999999999, Emax=999999999,
capitals=1, flags=[], traps=[Overflow, DivisionByZero,
InvalidOperation])
>>> getcontext().prec = 6
>>> Decimal('3.0')
Decimal('3.0')
>>> Decimal('3.1415926535')
Decimal('3.1415926535')
>>> Decimal('3.1415926535') + Decimal('2.7182818285')
Decimal('5.85987')
>>> getcontext().rounding = ROUND_UP
>>> Decimal('3.1415926535') + Decimal('2.7182818285')
Decimal('5.85988')
You should represent all currency-based values internally as Decimals with a high precision (the standard level of precision should be fine in your case - just leave the prec alone!). If you want to print a nicely formatted dollars and cents value to the user, using the locale module is a straightforward way to do this.
Be careful when printing as you will have to quantize the Decimal down to the correct number of places for display or the rounding will not be based on your Decimal context! You should only perform the quantize step for final display or for a single, final value - all intermediate steps should use high-precision Decimals to make any operations as accurate as possible.
>>> from decimal import *
>>> import locale
>>> locale.setlocale(locale.LC_ALL, '')
'en_AU.UTF-8'
>>> getcontext().rounding = ROUND_DOWN
>>> TWOPLACES = Decimal(10) ** -2
>>> var = Decimal('5.74536541')
Decimal('5.74536541')
>>> var.quantize(TWOPLACES)
Decimal('5.74')
>>> locale.currency(var.quantize(TWOPLACES))
'$5.74'
If you're dealing with currency and accuracy matters, don't use float, use decimal.
Take away the number mod 0.01
i.e.
rounded = number - (number % 0.01)
then print it the same as before.
This said, rounding down is not more accurate. Are you trying the old steal money from a bank by exploiting rounding errors scheme?
Floating point values are known as "useful approximations". Whatever you do to a floating point number—round it, truncate it, whatever—if the result is a floating point value, you don't get to decide how many digits to the right of the decimal point it has.
Never use floating point values for currency. See pydoc decimal, for example. Python's decimal module supports decimal fixed point and decimal floating point arithmetic.
Python docs warn about rounding floats.
Note The behavior of round() for floats can be surprising: for
example, round(2.675, 2) gives 2.67 instead of the expected 2.68. This
is not a bug: it’s a result of the fact that most decimal fractions
can’t be represented exactly as a float.
If you're not careful, you'll be misled by the value that appears at the interpreter prompt.
Python only prints a decimal approximation to the true decimal value
of the binary approximation stored by the machine.
And
It’s important to realize that this is, in a real sense, an illusion:
the value in the machine is not exactly 1/10, you’re simply rounding
the display of the true machine value. This fact becomes apparent as
soon as you try to do arithmetic with these values
If the number is a string then truncate the string to only 2 characters after the decimal and then convert it to a float.
Otherwise multiply it with 10^n where n is the number of digits after the decimal and then divide your float by 10^n.