We Keep Coding

decimal.InvalidOperation, DivisionImpossible for very large numbers

Using python 3.5.2
>>> from decimal import Decimal
>>> Decimal('12') % Decimal('0.01')
Decimal('0.00')
>>> Decimal('234567') % Decimal('0.01')
Decimal('0.00')
Works as expected. But...
>>> Decimal('7316717653133062491922511967442657474235534919493496983520312774506326239578318016984801869478851843858615607891129494954595017379583319528532088055111254069874715852386305071569329096329522744304355766896648950445244523161731856403098711121722383113622298934233803081353362766142828064444866452387493035890729629049156044077239071381051585930796086670172427121883998797908792274921901699720888093776657273330010533678812202354218097512545405947522435258490771167055601360483958644670632441572215539753697817977846174064955149290862569321978468622482839722413756570560574902614079729686524145351004748216637048440319989000889524345065854122758866688116427171479924442928230863465674813919123162824586178664583591245665294765456828489128831426076900422421902267105562632111110937054421750694165896040807198403850962455444362981230987879927244284909188845801561660979191338754992005240636899125607176060588611646710940507754100225698315520005593572972571636269561882670428252483600823257530420752963450') % Decimal('0.01')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
decimal.InvalidOperation: [<class 'decimal.DivisionImpossible'>]
EDIT: This is the smallest number I found that can cause this error:
>>> Decimal(10**26) % Decimal('0.01')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
decimal.InvalidOperation: [<class 'decimal.DivisionImpossible'>]
Why does Decimal(very_large_int) % Decimal('0.01') give this error? I thought that Decimal is able to handle very large numbers?

As L3viathan answered, the problem is that a result (not the result—this is the "hidden part" I mention in a comment) has overrun the available precision.
The hidden part is more obvious if we use Python2:
Traceback (most recent call last):
File "/tmp/d.py", line 24, in <module>
print(big % Decimal('0.01'))
File "/usr/local/lib/python2.7/decimal.py", line 1460, in __mod__
remainder = self._divide(other, context)[1]
File "/usr/local/lib/python2.7/decimal.py", line 1381, in _divide
'quotient too large in //, % or divmod')
File "/usr/local/lib/python2.7/decimal.py", line 3873, in _raise_error
raise error(explanation)
InvalidOperation: quotient too large in //, % or divmod
Essentially, a % b is implemented by doing both division and modulus together (a la Algorithm D in Knuth vol 2; for a C implementation restricted to two fullwords, see the qdivrem.c code I wrote in the early 2000s). The library code therefore needs two extra digits (the number of digits to the right of the decimal point in Decimal('0.01')—calculating the actual number of digits needed is not as simple as for big below as we have to look at the exponents) to compute the intermediate quotient.
The decimal library was reimplemented directly in C for Python3, which hides the detail, but the cure is the same for both: extend the precision. Here's an example source routine that catches the exception and tries again, though with magic constant 2.
from __future__ import print_function
import decimal
Decimal = decimal.Decimal
import traceback
big = Decimal(
'731671765313306249192251196744265747423553491949349698352031277'
'4506326239578318016984801869478851843858615607891129494954595017379'
'5833195285320880551112540698747158523863050715693290963295227443043'
'5576689664895044524452316173185640309871112172238311362229893423380'
'3081353362766142828064444866452387493035890729629049156044077239071'
'3810515859307960866701724271218839987979087922749219016997208880937'
'7665727333001053367881220235421809751254540594752243525849077116705'
'5601360483958644670632441572215539753697817977846174064955149290862'
'5693219784686224828397224137565705605749026140797296865241453510047'
'4821663704844031998900088952434506585412275886668811642717147992444'
'2928230863465674813919123162824586178664583591245665294765456828489'
'1288314260769004224219022671055626321111109370544217506941658960408'
'0719840385096245544436298123098787992724428490918884580156166097919'
'1338754992005240636899125607176060588611646710940507754100225698315'
'520005593572972571636269561882670428252483600823257530420752963450')
try:
print(big % Decimal('0.01'))
except decimal.DecimalException:
traceback.print_exc()
print('')
ctx = decimal.getcontext()
print('failed because precision was', ctx.prec, 'and big is',
len(big.as_tuple().digits), 'digits long')
print('trying again with 2 more digits')
with decimal.localcontext() as ctx:
ctx.prec = len(big.as_tuple().digits) + 2
try:
print(big % Decimal('0.01'))
except decimal.DecimalException:
traceback.print_exc()
With Python2:
$ python2 /tmp/d.py
Traceback (most recent call last):
File "/tmp/d.py", line 24, in <module>
print(big % Decimal('0.01'))
File "/usr/local/lib/python2.7/decimal.py", line 1460, in __mod__
remainder = self._divide(other, context)[1]
File "/usr/local/lib/python2.7/decimal.py", line 1381, in _divide
'quotient too large in //, % or divmod')
File "/usr/local/lib/python2.7/decimal.py", line 3873, in _raise_error
raise error(explanation)
InvalidOperation: quotient too large in //, % or divmod
failed because precision was 28 and big is 1000 digits long
trying again with 2 more digits
0.00
With Python3:
$ python3 /tmp/d.py
Traceback (most recent call last):
File "/tmp/d.py", line 24, in <module>
print(big % Decimal('0.01'))
decimal.InvalidOperation: [<class 'decimal.DivisionImpossible'>]
failed because precision was 28 and big is 1000 digits long
trying again with 2 more digits
0.00
Note that dividing by a very large number is actually easier: it's the division by 0.01 that is causing problems here. If the exponent on the divisor were at least 1000 - 28 (1e972 or larger), we would not have the problem.

Decimal is based on the Decimal Arithmetic specification. You can see here that "Division impossible" means that
the integer result of a divide-integer or remainder operation had too many digits (would be longer than precision).
This precision is something you can adjust:
>>> decimal.getcontext().prec=10000
>>> Decimal('7316717653133062491922511967442657474235534919493496983520312774506326239578318016984801869478851843858615607891129494954595017379583319528532088
... 0551112540698747158523863050715693290963295227443043557668966489504452445231617318564030987111217223831136222989342338030813533627661428280644448664523874
... 9303589072962904915604407723907138105158593079608667017242712188399879790879227492190169972088809377665727333001053367881220235421809751254540594752243525
... 8490771167055601360483958644670632441572215539753697817977846174064955149290862569321978468622482839722413756570560574902614079729686524145351004748216637
... 0484403199890008895243450658541227588666881164271714799244429282308634656748139191231628245861786645835912456652947654568284891288314260769004224219022671
... 0556263211111093705442175069416589604080719840385096245544436298123098787992724428490918884580156166097919133875499200524063689912560717606058861164671094
... 0507754100225698315520005593572972571636269561882670428252483600823257530420752963450') % Decimal('0.01')
Decimal('0.00')

Is there any way to make Python .format() thow an exception if the data won't fit the field?

I want to normalize floating-point numbers to nn.nn strings, and to do some special handling if the number is out of range.
try:
norm = '{:5.2f}'.format(f)
except ValueError:
norm = 'BadData' # actually a bit more complex than this
except it doesn't work: .format silently overflows the 5-character width. Obviously I could length-check norm and raise my own ValueError, but have I missed any way to force format (or the older % formatting) to raise an exception on field-width overflow?

You can not achieve this with format(). You have to create your custom formatter which raises the exception. For example:
def format_float(num, max_int=5, decimal=2):
if len(str(num).split('.')[0])>max_int:
raise ValueError('Integer part of float can have maximum {} digits'.format(max_int))
return "{:.2f}".format(num)
Sample run:
>>> format_float(123.456)
'123.46'
>>> format_float(123.4)
'123.40'
>>> format_float(123789.456) # Error since integer part is having length more than 5
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in format_float
ValueError: Integer part of float can have maximum 5 digits

Trouble finding certain tag values

I'm trying to find the value of several tags using pydicom. For some reason only certain tags work while others do not. Below is a traceback explaining my problem. Can anyone see a way around the int() base 16 problem?
>>> ds['0x18','0x21'].value
'SP'
>>> ds['0x18','13x14'].value
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/space/jazz/1/users/gwarner/anaconda/lib/python2.7/site-packages/pydicom-0.9.9-py2.7.egg/dicom/dataset.py", line 276, in __getitem__
tag = Tag(key)
File "/space/jazz/1/users/gwarner/anaconda/lib/python2.7/site-packages/pydicom-0.9.9-py2.7.egg/dicom/tag.py", line 27, in Tag
arg = (int(arg[0], 16), int(arg[1], 16))
ValueError: invalid literal for int() with base 16: '13x14'

'13x14' is not a valid representation of base 16 number.
In python, base 16 numbers are represented with '0x' as prefix and then the number in base 16.
For example:
0x0, 0x1, 0x001, 0x235, 0xA5F, ..., are all valid base 16 number representations.
This:
ds['0x18','13x14'].value
Could be, for example, this:
ds['0x18','0x14'].value
And it should execute fine.

How can I get a big number in non-scientific notation?

I just tried
>>> 2.17 * 10**27
2.17e+27
>>> str(2.17 * 10**27)
'2.17e+27'
>>> "%i" % 2.17 * 10**27
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OverflowError: cannot fit 'long' into an index-sized integer
>>> "%f" % 2.17 * 10**27
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OverflowError: cannot fit 'long' into an index-sized integer
>>> "%l" % 2.17 * 10**27
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: incomplete format
Now I ran out of ideas. I want to get
2170000000000000000000000000
How can I print such big numbers? (I don't care if it's a Python 2.7+ solution or a Python 3.X solution)

You are getting your operator precedence wrong. You are formatting 2.17, then multiplying that by a long integer:
>>> r = "%f" % 2.17
>>> r
'2.170000'
>>> r * 10 ** 27
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OverflowError: cannot fit 'long' into an index-sized integer
Put parentheses around the multiplication:
>>> "%f" % (2.17 * 10**27)
'2169999999999999971109634048.000000'
This is one of the drawbacks of overloading the modulus operator for string formatting; the newer Format String syntax used by the str.format() method and the Format Specification Mini-Language it employs (and can be used with the format() function) neatly skirt around that issue. I'd use format() for this case:
>>> format(2.17 * 10**27, 'f')
'2169999999999999971109634048.000000'

Float value is equal -1.#IND

A function returns a list which contains of float values. If I plot this list, I see that some of the float values are equal -1.#IND. I also checked the type of those -1.#IND values. And they are also of float type.
But how can I understand this -1.#IND values? What do they represent or stand for?

-1.#IND means indefinite, the result of a floating point equation that doesn't have a solution. On other platforms, you'd get NaN instead, meaning 'not a number', -1.#IND is specific to Windows. On Python 2.5 on Linux I get:
>>> 1e300 * 1e300 * 0
-nan
You'll only find this on python versions 2.5 and before, on Windows platforms. The float() code was improved in python 2.6 and consistently uses float('nan') for such results; mostly because there was no way to turn 1.#INF and -1.#IND back into an actual float() instance again:
>>> repr(inf)
'1.#INF'
>>> float(_)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: invalid literal for float(): 1.#INF
>>> repr(nan)
'-1.#IND'
>>> float(_)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: invalid literal for float(): -1.#IND
On versions 2.6 and newer this has all been cleaned up and made consistent:
>>> 1e300 * 1e300 * 0
nan
>>> 1e300 * 1e300
inf
>>> 1e300 * 1e300 * -1
-inf