Is there a simple way to calculate (especially powers/exponetiation) with matrices whose elements are integers from finite field, or at least arbitrary integer precision matrices with support of % operator?
For example let's say we have a matrix
A = 1 1
1 0
and want to compute something like (A**100) % 1000, how to achieve this?
I have tried numpy, but problem is that it uses fixed precision data types so it overflows quickly... Then I tried sympy since it supports arbitrary integer precision, but it does not seem to have support for finite fields operations (except for inverse)...
It might be an overkill, but Sage has everything you want (and much more). It is a python based software, but is very large (~1.2GB download). You can use sage --preparse script.sage to create a python file.
There is even SO-like QA site https://ask.sagemath.org/questions/ which is specialized to sage.
Example of your code might be:
m = Matrix(GF(5), [[1, 1], [1, 0]])
power = m^100
I have used GF(5) as GF(1000) is not a finite field. Also there are some differences, for instance the exponentiation can be done either by x**y or equivalently x^y.
Related
import numpy as np
v = np.zeros((3,10000), dtype=np.float32)
mat = np.zeros((10000,10000000), dtype=np.int8)
w = np.matmul(v, mat)
yields
Traceback (most recent call last):
File "int_mul_test.py", line 6, in <module>
w = np.matmul(v, mat)
numpy.core._exceptions.MemoryError: Unable to allocate 373. GiB
for an array with shape (10000, 10000000) and data type float32
Apparently, numpy is trying to convert my 10k x 10m int8 matrix to dtype float32. Why does it need to do this? It seems extremely wasteful, and if matrix multiplication must work with float numbers in memory, it could convert say 1m columns at a time (which shouldn't sacrifice speed too much), instead of converting all 10m columns all at once.
My current solution is to use a loop to break the matrix into 10 pieces and reduce temporary memory allocation to 1/10 of the 373 GiB:
w = np.empty((v.shape[0],mat.shape[1]),dtype=np.float32)
start = 0
block = 1000000
for i in range(mat.shape[1]//block):
end = start + block
w[:,start:end] = np.matmul(v, mat[:,start:end])
start = end
w[:,start:] = np.matmul(v, mat[:,start:])
# runs in 396 seconds
Is there a numpy-idiomatic way to multiply "piece by piece" without manually coding a loop?
The semantic of Numpy operations force the inputs of a binary operation to be casted when the left/right types are different. In fact, this is the case in almost all statically typed language including C, C++, Java, Rust, but also many dynamically-typed languages (the semantic rules are applied at runtime in this case). Python also (partially) applies such a well defined semantic rule. For example, when you evaluate the expression True * 1.7, the interpreter evaluates the type of both operands (bool and float here) and then applies multiple semantic rules until the type of both operand are the same before performing the actual multiplication. In this case, True of type bool is casted to 1 of type int which is then casted to 1.0 of type float. Such semantic rules are generally defined in a way that is both relatively unambiguous and safe. For example, you do not expect 2 * 1.7 to be equal to 3. Numpy use semantic rules similar to the ones of the C language because it is written in C and provide native types. The semantic rules should be defined independently of a given implementation. That being said performance and ease-of-use matters a lot when designing it. Unfortunately, in your case, this means a huge array has to be allocated.
Note that Numpy could theoretically bypass casting and implement the N * N possible versions for the N different types for each binary operations in order to make them faster (like the "as-if" semantic rule of the C language). However, this would be insane to implement for developers and it would result in a more bug-prone code (ie. less stable and slower development) and a huge code bloat (bigger binaries). This is especially true since other parameters should be taken into account like the shape of the array and the memory layout (eg. alignment) or event the target architecture. The current main casting generative function of Numpy is already quite complex and already results in 4 x 18 x 18 = 1296 different C functions to be compiled and stored in Numpy binaries!
In your case, you can use Cython or Numba to generate a memory-efficient (and possibly faster) implementation dedicated to your specific needs. Be careful about possible overflows though.
I am facing a problem during my work with MATLAB, to compute sinh(a + b * i) (e.g., 1000+1i), where i is the imaginary unit and a, b are quite large values that type double" cannot handle. Surely I can compute the function via Wolfram or Fortran, but I do need a common language to do the GPIB communication together with this calculation.
After asking this to some guys, I was told Python or C has a type called "big float". But none of them can tell me what is the precision nor the max value of it, not to mention the efficiency. So can anyone suggest a solution? Or maybe there's another language can handle this problem (compute large complex and GPIB session).
You could use Julia:
julia> a = BigFloat("1e10")
1.0e+10
julia> b = BigFloat("1e500")
1.000000000000000000000000000000000000000000000000000000000000000000000000000004e+500
julia> sinh(a + b * im)
1.43445592092543814302692567115470616209662482064997303227590320999972133381932e+4342944818 + 5.194323395284352151694584377260055302504830707661913916785283453158333278974902e+4342944818im
but as others have commented, and as you can see in this example, sinh grows quickly with the real part of your complex number, so you can't have too big a a. (With a = 1e10, the result already reached 10^4342944818 !)
FWIW, in Julia, written down as an integer, the biggest BigFloat would have about 1,388,255,822,130,839,282 digits:
julia> prevfloat(typemax(BigFloat))
5.875653789111587590936911998878442589938516392745498308333779606469323584389875e+1388255822130839282
Also, I don't know how accurate this result is.
I want to translate this MATLAB code into Python, I guess I did everything right, even though I didn't get the same results.
MATLAB script:
n=2 %Filter_Order
Wn=[0.4 0.6] %# Normalized cutoff frequencies
[b,a] = butter(n,Wn,'bandpass') % Transfer function coefficients of the filter
Python script:
import numpy as np
from scipy import signal
n=2 #Filter_Order
Wn=np.array([0.4,0.6]) # Normalized cutoff frequencies
b, a = signal.butter(n, Wn, btype='band') #Transfer function coefficients of the filter
a coefficients in MATLAB: 1, -5.55e-16, 1.14, -1.66e-16, 0.41
a coefficients in Python: 1, -2.77e-16, 1.14, -1.94e-16, 0.41
Could it just be a question of precision, since the two different values (the 2nd and 4th) are both on the order of 10^(-16)?!
The b coefficients are the same on the other hand.
You machine precision is about 1e-16 (in MATLAB this can be checked easily with eps(), I presume about the same in Python). The 'error' you are dealing with is thus on the order of machine precision, i.e. not actually calculable within fitting precision.
Also of note is that MATLAB ~= Python (or != in Python), thus the implementations of butter() on one hand and signal.butter() on the other will be slightly different, even if you use the exact same numbers, due to the way both languages are translated to machine code.
It rarely matters to have coefficients differing 16 orders of magnitude; the smaller ones would be essentially neglected. In case you do need exact values, consider using either symbolic math, or some kind of Variable Precision Arithmetic (vpa() in MATLAB), but I guess that in your case the difference is irrelevant.
I wrote a C++ wrapper class to some functions in LAPACK. In order to test the class, I use the Python C Extension, where I call numpy, and do the same operations, and compare the results by taking the difference
For example, for the inverse of a matrix, I generate a random matrix in C++, then pass it as a string (with many, many digits, like 30 digits) to Python's terminal using PyRun_SimpleString, and assign the matrix as numpy.matrix(...,dtype=numpy.double) (or numpy.complex128). Then I use numpy.linalg.inv() to calculate the inverse of the same matrix. Finally, I take the difference between numpy's result and my result, and use numpy.isclose with a specific relative tolerance to see whether the results are close enough.
The problem: The problem is that when I use C++ floats, the relative precision I need to be able to compare is about 1e-2!!! And yet with this relative precision I get some statistical failures (with low probability).
Doubles are fine... I can do 1e-10 and it's statistically safe.
While I know that floats have intrinsic bit precision of about 1e-6, I'm wondering why I have to go so low to 1e-2 to be able to compare the results, and it still fails some times!
So, going so low down to 1e-2 got me wondering whether I'm thinking about this whole thing the wrong way. Is there something wrong with my approach?
Please ask for more details if you need it.
Update 1: Eric requested example of Python calls. Here is an example:
//create my matrices
Matrix<T> mat_d = RandomMatrix<T>(...);
auto mat_d_i = mat_d.getInverse();
//I store everything in the dict 'data'
PyRun_SimpleString(std::string("data={}").c_str());
//original matrix
//mat_d.asString(...) will return in the format [[1,2],[3,4]], where 32 is 32 digits per number
PyRun_SimpleString(std::string("data['a']=np.matrix(" + mat_d.asString(32,'[',']',',') + ",dtype=np.complex128)").c_str());
//pass the inverted matrix to Python
PyRun_SimpleString(std::string("data['b_c']=np.matrix(" + mat_d_i.asString(32,'[',']',',') + ",dtype=np.complex128)").c_str());
//inverse in numpy
PyRun_SimpleString(std::string("data['b_p']=np.linalg.inv(data['a'])").c_str());
//flatten the matrices to make comparing them easier (make them 1-dimensional)
PyRun_SimpleString("data['fb_p']=((data['b_p']).flatten().tolist())[0]");
PyRun_SimpleString("data['fb_c']=((data['b_c']).flatten().tolist())[0]");
//make the comparison. The function compare_floats(f1,f2,t) calls numpy.isclose(f1,f2,rtol=t)
//prec is an integer that takes its value from a template function, where I choose the precision I want based on type
PyRun_SimpleString(std::string("res=list(set([compare_floats(data['fb_p'][i],data['fb_c'][i],1e-"+ std::to_string(prec) +") for i in range(len(data['fb_p']))]))[0]").c_str());
//the set above eliminates repeated True and False. If all results are True, we expect that res=[True], otherwise, the test failed somewhere
PyRun_SimpleString(std::string("res = ((len(res) == 1) and res[0])").c_str());
//Now if res is True, then success
Comments in the code describe the procedure step-by-step.
I'm attempting to simulate a certain physical system. In order to propagate solutions I need to be able to multiply matrices of determinant = 1 which describe each part of the system. In the code below T(variables) is a 2-dimensional matrix with det(T) = 1, i just indicates the region number and the rest is irrelevant.
When I run this code for systems with more than 30 regions, the final Msys no longer has determinant = 1. I checked the value of the determinant of Msys throughout the calculation and it's 1 for the first few iterations but then it starts diverging from that. I've tried putting dtype = float64 when creating the array T to see if this would improve the precision and stop it from breaking down but I saw no improvement.
Is there any way I could write the code to avoid the error accumulating or a way I can increase the amount of decimal places numpy stores so to make the error negligible for systems with 100+ regions.
for i in range(n):
if i == 0:
Msys = T(L[i],i,k)
else:
Msys = numpy.dot(T(L[i]-L[i-1],i,k), Msys)
return Msys
All Floating point operations have limited precision and errors do accumulate. You need to decide how much precision is "good enough" or how much error accumulation is "negligible". If float64 is not precise enough for you, try float128. You can find out the precision of the float types like this:
In [83]: np.finfo(np.float32).eps
Out[83]: 1.1920929e-07
In [84]: np.finfo(np.float64).eps
Out[84]: 2.2204460492503131e-16
In [85]: np.finfo(np.float128).eps
Out[85]: 1.084202172485504434e-19
Here is a lot more info about floating point arithmetic: What Every Computer Scientist Should Know About Floating-Point Arithmetic