I am working on a analysis problem in python in which I have to work on big floating point numbers and was performing some operations on it. My codes seems to break in between; looking carefully I found that sometimes a simple add operation would return inf it must be an overflow.
>>> a = float(2**1022) + float(2**1023)
>>> print a
1.348269851146737e+308
>>> a = float(2**1023) + float(2**1023)
>>> print a
inf
>>>
How do we check for overflow in python while floating point operation since instead of giving a OverflowError it silently gives a value inf.
I can only imagine checking if abs(a)==float('inf'): raise OverflowError()…
Use an arbitrary precision library like GMPY and you won't need to worry.
Disclaimer: I maintain gmpy2.
The gmpy2 library supports both arbitrary precision (to decrease the occurrences of overflow) and the ability to trap on floating point events. Here is an example of modify the context to automatically raise an exception when an overflow occurs.
>>> import gmpy2
>>> from gmpy2 import get_context,set_context, ieee, mpfr
>>> set_context(ieee(64))
>>> get_context()
context(precision=53, real_prec=Default, imag_prec=Default,
round=RoundToNearest, real_round=Default, imag_round=Default,
emax=1024, emin=-1073,
subnormalize=True,
trap_underflow=False, underflow=False,
trap_overflow=False, overflow=False,
trap_inexact=False, inexact=False,
trap_invalid=False, invalid=False,
trap_erange=False, erange=False,
trap_divzero=False, divzero=False,
trap_expbound=False,
allow_complex=False)
>>> get_context().trap_overflow=True
>>> get_context()
context(precision=53, real_prec=Default, imag_prec=Default,
round=RoundToNearest, real_round=Default, imag_round=Default,
emax=1024, emin=-1073,
subnormalize=True,
trap_underflow=False, underflow=False,
trap_overflow=True, overflow=False,
trap_inexact=False, inexact=False,
trap_invalid=False, invalid=False,
trap_erange=False, erange=False,
trap_divzero=False, divzero=False,
trap_expbound=False,
allow_complex=False)
>>> mpfr(2**1023) + mpfr(2**1023)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
gmpy2.OverflowResultError: 'mpfr' overflow in "addition"
>>>
Related
I am trying to reproduce R codes in Python and I have to generate same random numbers in both languages. I know that using the same seed is not enough to get same random numbers and reading one of the answers on this platform regarding this topic I have discovered that there exists: SyncRNG library, which generates same random numbers between R and Python. Everything looks fine as long as I have discovered that on Python 3.7.3 I can generate via SyncRNG just one number because as soon as you iterate the procedure, for instance, with a for loop you get this error:
OverflowError: Python int too large to convert to C long.
As I was mentioning:
>>> from SyncRNG import SyncRNG
>>> s = SyncRNG(seed=123)
>>> r = s.rand()
>>> r
0.016173338983207965
and as we can see it works. The method ".rand()" generates random numbers between zero and one.
But if I try to iterate:
>>> from SyncRNG import SyncRNG
>>> s = SyncRNG(seed=123)
>>> b = []
>>> for i in range(5):
temp = s.rand()
b.append(temp)
and I get this:
OverflowError: Python int too large to convert to C long
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<pyshell#41>", line 2, in <module>
temp = s.rand()
File "C:\Users\Stefano\AppData\Local\Programs\Python\Python37\lib\site-packages\SyncRNG\__init__.py", line 27, in rand
return self.randi() * 2.3283064365387e-10
File "C:\Users\Stefano\AppData\Local\Programs\Python\Python37\lib\site-packages\SyncRNG\__init__.py", line 22, in randi
tmp = syncrng.rand(self.state)
SystemError: <built-in function rand> returned a result with an error set
So, I humbly ask if someone is able to solve this problem. If I lost old answers about this topic I'm sorry, please link them in the answer section.
Thank you!
This is not a solution, but a workaround. The overflow bug is not reproducible on my system. The workaround:
from SyncRNG import SyncRNG
s = SyncRNG(seed=123)
ct = 0
b = []
while ct < 5:
try:
temp = s.rand()
b.append(temp)
ct += 1
except OverflowError:
pass
Performance will be compromised due to the try/except on each loop.
I have a sympy poly that looks like:
Poly(0.764635937801645*I**4 + 7.14650839258644*I**3 - 0.667712176660315*I**2 - 2.81663805543677*I - 0.623299856233272, I, domain='RR')
I'm converting to mpc using the following code:
a = val.subs('I',1.0j)
b = sy.re(a)
c = sy.im(a)
d = mpmath.mpc(b,c)
Two questions.
Assuming my mpc and sympy type have equal precision (of eg 100 dps) is there a precision loss using this conversion from a to d?
Is there a better way to convert?
Aside: sympy seems to treat I just like a symbol here. How do I get sympy to simplify this polynomial?
Edit: Ive also noticed that the following works in place of a above:
a = val.args[0]
Strings and expressions
Root cause of the issue is seen in val.subs('I', 1.0j) -- you appear to pass strings as arguments to SymPy functions. There are some valid uses for this (such as creation of high-precision floats), but when symbols are concerned, using strings is a recipe for confusion. The string 'I' gets implicitly converted to SymPy expression Symbol('I'), which is different from SymPy expression I. So the answer to
How do I get sympy to simplify this polynomial?
is to revisit the process of creation of that polynomial, and fix that. If you really need to create it from a string, then use locals parameter:
>>> S('3.3*I**2 + 2*I', locals={'I': I})
-3.3 + 2*I
Polynomials and expressions
If the Poly structure is not needed, use the method as_expr() of Poly to get an expression from it.
Conversion to mpmath and precision loss
is there a precision loss using this conversion from a to d?
Yes, splitting into real and imaginary and then recombining can lead to precision loss. Pass a SymPy object directly to mpc if you know it's a complex number. Or to mpmathify if you want mpmath to decide what type it should have. An example:
>>> val = S('1.111111111111111111111111111111111111111111111111')*I**3 - 2
>>> val
-2 - 1.111111111111111111111111111111111111111111111111*I
>>> import mpmath
>>> mpmath.mp.dps = 40
>>> mpmath.mpc(val)
mpc(real='-2.0', imag='-1.111111111111111111111111111111111111111111')
>>> mpmath.mpmathify(val)
mpc(real='-2.0', imag='-1.111111111111111111111111111111111111111111')
>>> mpmath.mpc(re(val), im(val))
mpc(real='-2.0', imag='-1.111111111111111111111111111111111111111114')
Observations:
When I is actual imaginary unit, I**3 evaluates fo -I, you don't have to do anything for it to happen.
A string representation of high-precision decimal is used to create such a float in SymPy. Here S stands for sympify. One can also be more direct and use Float('1.1111111111111111111111111')
Direct conversion of a SymPy complex number to an mpmath complex number is preferable to splitting in real/complex and recombining.
Conclusion
Most of the above is just talking around an XY problem. Your expression with I was not what you think it was, so you tried to do strange things that were not needed, and my answer is mostly a waste of time.
I'm adding my own answer here, as FTP's answer, although relevant and very helpful, did not (directly) resolve my issue (which wasn't that clear from the question tbh). When I ran the code in his example I got the following:
>>> from sympy import *
>>> import mpmath
>>> val = S('1.111111111111111111111111111111111111111111111111')*I**3 - 2
>>> val
-2 - 1.111111111111111111111111111111111111111111111111*I
>>> mpmath.mp.dps = 40
>>> mpmath.mpc(val)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\site-packages\mpmath\ctx_mp_python.py", line 373, in __new__
real = cls.context.mpf(real)
File "C:\Python27\lib\site-packages\mpmath\ctx_mp_python.py", line 77, in __new__
v._mpf_ = mpf_pos(cls.mpf_convert_arg(val, prec, rounding), prec, rounding)
File "C:\Python27\lib\site-packages\mpmath\ctx_mp_python.py", line 96, in mpf_convert_arg
raise TypeError("cannot create mpf from " + repr(x))
TypeError: cannot create mpf from -2 - 1.111111111111111111111111111111111111111111111111*I
>>> mpmath.mpmathify(val)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\site-packages\mpmath\ctx_mp_python.py", line 662, in convert
return ctx._convert_fallback(x, strings)
File "C:\Python27\lib\site-packages\mpmath\ctx_mp.py", line 614, in _convert_fallback
raise TypeError("cannot create mpf from " + repr(x))
TypeError: cannot create mpf from -2 - 1.111111111111111111111111111111111111111111111111*I
>>> mpmath.mpc(re(val), im(val))
mpc(real='-2.0', imag='-1.111111111111111111111111111111111111111114')
>>> mpmath.mpmathify(val)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\site-packages\mpmath\ctx_mp_python.py", line 662, in convert
return ctx._convert_fallback(x, strings)
File "C:\Python27\lib\site-packages\mpmath\ctx_mp.py", line 614, in _convert_fallback
raise TypeError("cannot create mpf from " + repr(x))
TypeError: cannot create mpf from -2 - 1.111111111111111111111111111111111111111111111111*I
Updating my sympy (1.0->1.1.1) and mpmath (0.19->1.0.0) fixed the exceptions. I did not test which of these upgrades actually resolved the issue.
I was getting this error:
> float() argument must be a string or a number
So, why does this happen?(I tried commands like np.asarray() but it keeps failing).
mp.mpc(cmath.rect(a,b)))
The items in raizes are actually mpmath.mpc instances rather than native Python complex floats. numpy doesn't know how to deal with mpmath types, hence the TypeError.
You didn't mention mpmath at all in your original question. The problem would still have been easy to diagnose if you had posted the full traceback, rather than cutting off the most important part at the end:
In [10]: np.roots(Q)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-10-f3a270c7e8c0> in <module>()
----> 1 np.roots(Q)
/home/alistair/.venvs/mpmath/lib/python3.6/site-packages/numpy/lib/polynomial.py in roots(p)
220 # casting: if incoming array isn't floating point, make it floating point.
221 if not issubclass(p.dtype.type, (NX.floating, NX.complexfloating)):
--> 222 p = p.astype(float)
223
224 N = len(p)
TypeError: float() argument must be a string or a number, not 'mpc'
Whenever you ask for help with debugging on this site, please always post the whole traceback rather than just (part of) the last line - it contains a lot of information that can be helpful for diagnosing the problem.
The solution is simple enough - just don't convert the native Python complex floats returned by cmath.rect to mpmath.mpc complex floats:
raizes = []
for i in range(2*n):
a, f = cmath.polar(l[i])
if((f>np.pi/2) or (f<-np.pi/2)):
raizes.append(cmath.rect(a*r,f))
Q = np.poly(raizes)
print(np.roots(Q))
# [-0.35372430 +1.08865146e+00j -0.92606224 +6.72823602e-01j
# -0.35372430 -1.08865146e+00j -1.14467588 -9.11902316e-16j
# -0.92606224 -6.72823602e-01j]
I found the following answer here on Stackoverflow:
https://stackoverflow.com/a/356187/1829329
But it only works for integers as n in nth root:
import gmpy2 as gmpy
result = gmpy.root((1/0.213), 31.5).real
print('result:', result)
results in:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-14-eb4628226deb> in <module>()
8
----> 9 result = gmpy.root((1/0.213), 31.5).real
10
11 print('result:', result)
TypeError: root() requires 'mpfr','int' arguments
What is a good and precise way to calculate such a root?
(This is the python code representation of some formular, which I need to use to calculate in a lecture.)
EDIT#1
Here is my solution based on Spektre's answer and information from the people over here at http://math.stackexchange.com.
import numpy as np
def naive_root(nth, a, datatype=np.float128):
"""This function can only calculate the nth root, if the operand a is positive."""
logarithm = np.log2(a, dtype=datatype)
exponent = np.multiply(np.divide(1, nth, dtype=datatype), logarithm, dtype=datatype)
result = np.exp2(exponent, dtype=datatype)
return result
def nth_root(nth, a, datatype=np.float128):
if a == 0:
print('operand is zero')
return 0
elif a > 0:
print('a > 0')
return naive_root(nth, a, datatype=datatype)
elif a < 0:
if a % 2 == 1:
print('a is odd')
return -naive_root(nth, np.abs(a))
else:
print('a is even')
return naive_root(nth, np.abs(a))
see Power by squaring for negative exponents
anyway as I do not code in python or gmpy here some definitions first:
pow(x,y) means x powered by y
root(x,y) means x-th root of y
As these are inverse functions we can rewrite:
pow(root(x,y),x)=y
You can use this to check for correctness. As the functions are inverse you can write also this:
pow(x,1/y)=root(y,x)
root(1/x,y)=pow(y,x)
So if you got fractional (rational) root or power you can compute it as integer counterpart with inverse function.
Also if you got for example something like root(2/3,5) then you need to separate to integer operands first:
root(2/3,5)=pow(root(2,5),3)
~11.18034 = ~2.236068 ^3
~11.18034 = ~11.18034
For irational roots and powers you can not obtain precise result. Instead you round the root or power to nearest possible representation you can to minimize the error or use pow(x,y) = exp2(y*log2(x)) approach. If you use any floating point or fixed point decimal numbers then you can forget about precise results and go for pow(x,y) = exp2(y*log2(x)) from the start ...
[Notes]
I assumed only positive operand ... if you got negative number powered or rooted then you need to handle the sign for integer roots and powers (odd/even). For irational roots and powers have the sign no meaning (or at least we do not understand any yet).
If you are willing to use Python 3.x, the native pow() will do exactly what you want by just using root(x,y) = pow(x,1/y). It will automatically return a complex result if that is appropriate.
Python 3.4.3 (default, Sep 27 2015, 20:37:11)
[GCC 5.2.1 20150922] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> pow(1/0.213, 1/31.5)
1.0503191465568489
>>> pow(1/0.213, -1/31.5)
0.952091565004975
>>> pow(-1/0.213, -1/31.5)
(0.9473604081457588-0.09479770688958634j)
>>> pow(-1/0.213, 1/31.5)
(1.045099874779588+0.10457801566102139j)
>>>
Returning a complex result instead of raising a ValueError is one of changes in Python 3. If you want the same behavior with Python 2, you can use gmpy2 and enable complex results.
>>> import gmpy2
>>> gmpy2.version()
'2.0.5'
>>> gmpy2.get_context().allow_complex=True
>>> pow(1/gmpy2.mpfr("0.213"), 1/gmpy2.mpfr("31.5"))
mpfr('1.0503191465568489')
>>> pow(-1/gmpy2.mpfr("0.213"), 1/gmpy2.mpfr("31.5"))
mpc('1.0450998747795881+0.1045780156610214j')
>>> pow(-1/gmpy2.mpfr("0.213"), -1/gmpy2.mpfr("31.5"))
mpc('0.94736040814575884-0.094797706889586358j')
>>> pow(1/gmpy2.mpfr("0.213"), -1/gmpy2.mpfr("31.5"))
mpfr('0.95209156500497505')
>>>
Here is something I use that seems to work with any number just fine:
root = number**(1/nthroot)
print(root)
It works with any number data type.
I had to compute 2 to the power of 8635. I came across this error when i was computing 2^8635. Any suggestion how to solve this in python. Using Decimal module also didn't help.
math.exp(2**8635)
Traceback (most recent call last):
File "<pyshell#50>", line 1, in <module>
long(math.exp(2**8635))
OverflowError: long int too large to convert to float
You can calculate exp(2**8635) using the mpmath arbitrary precision mathematics module:
>>> from mpmath import mp
>>> mp.exp(2**mp.mpf(8635))
The result is rather large, even in scientific notation, so I did a little bit of string processing on it, by copy & pasting the exponent into a string s and then reformatting it with:
>>> ' '.join([''.join(b) for b in zip(*[iter(s)]*10)])
The result, using a precision of mp.dps=20 is
4.2745087809955028863988E+ 1075961842 0748495519 2355783856 9202899773
5836295555 7242537851 6693465178 9188332936 7912863215 8269919370
9351573282 2091597135 7873818954 8423038425 5471700446 0909815025
4915349824 6380686906 6065142412 7999166978 5222679350 6349971415
2079241766 6069616716 5214806906 8965805654 9049342395 4257087497
9202056938 3115899033 9617474292 7677302965 5753590474 3880492978
3310677063 3380119492 7251778893 3532333942 0392908196 6724211094
8421111629 7159351144 1795137181 2264799291 0856206603 1305457648
0092179442 8275386674 9796135348 4212345607 2596906933 7416259320
1466097812 2578572779 3349115312 8876785712 0147507083 6261226991
0422767260 2197316370 2566751899 4709067324 2810166509 7120800169
7485704027 5902848998 8322436088 0980908767 3758921061 0035161163
7079806523 1330002249 6901129466 0981598858 6254175013 7068344219
8144416569 4188190080 3725722501 2061326523 2288670566 9409102206
8066363145 9074202764 8831634730 7112255017 0796958442 3491574250
3284769002 8003076564 8628481483 0439060173 9895022099 1254178687
3665797407 1814255571 0142720172 8605229862 9458410467 4672777325
9445377886 5938198959 1752282268 4759702578 8305033658 4673474227
0701134707 7350267519 5446056950 6855231007 2274227123 3745529068
2622945712 5542911110 4405852782 0415855498 0153628764 5474618275
7401652756 8947182158 7839446628 2805776521 9227970761 2076840252
6317298197 2928759787 3166056075 4718446805 4085909927 3696546013
4104199677 6666307004 0490874208 2916409726 3519895965 9647475416
4225305349 6334839659 5665063673 3375427908 5570705561 0968733246
7656866381 6185956211 6475910345 4507247539 2187977091 2735607271
0209770526 5707982249 7378999875 8729821041 8908550791 3750558609
2896112135 9430559215 0044675914 5643599469 1254786634 5242493728
3502270440 3874100770 4799209491 3826434565 6967844371 7035989909
7698785578 9911981690 6907193705 6149599295 7385801544 7008446783
4130808551 5517628873 4403074667 6152264003 5246435484 0092524145
7903102870 5636975728 9853241717 1112914752 3204974235 3214176498
6118461362 7598845054 2855525271 2496489615 8680550278 4583386686
7070831392 2705583569 2224816732 3325036729 8106736491 3995562245
2081730125 0118717575 0946706166 8838482200 0110510511 5193674369
7359353196 7070446633 3514990982 6428484539 3692837968 6448096531
0406301917 5207050949 5578924420 0588400753 2802517808 1953501443
3445927237 8496133188 6299359291 6240744112 8041383838 2523556917
1966060664 8100342007 9597089073 1150067540 7733669287 2770028938
5557716323 9545436149 8913712891 0452403558 6380308865 8375826610
3861873050 5349593724 7443661231 6344789431 9248320333 3901279921
2985101147 6464375714 1302826973 5468251818 0411981792 4830271478
1578285904 7739571097 8513979667 0471302144 9500998430 2500602635
1255157768 2597448484 2711867118 3605984621 3175011866 9369283240
7857863664 6903750271 5353454119 0302118673