Related
I’m in the need of an efficient method to inverse a 7000x7000 aerodynamic influence coefficient (dense) matrix in python. I had started before a FORTRAN routine to handle the problem using the LU decompostition routines from LAPACK, which I had seen being used in other related apps quite efficiently. I’ve read, though, that the NumPy and SciPy linear system solvers are mostly based in direct calls to the same LAPACK/BLAS functions in C, and was wondering wether switching to FORTRAN would really reduce computation time to a level which justifies giving up an easier, higher level language.
If there are python solvers which would guarantee similar performance for matrixes of that size (1000 to 10000, square), which are they?
I really need the matrix inverse, so switching to iterative Ax=b solutions is not an option.
Indeed, Numpy and Scipy efficiently call LAPACK routines to perform numpy.linalg.inv and scipy.linalg.inv.
To inverse a general matrix, numpy.linalg.inv solves A.x=np.eye((n,n)). The function inv() calls ainv = _umath_linalg.inv(a, signature=signature, extobj=extobj), which calls call_#lapack_func#(¶ms); where params.B is the identitity matrix and #lapack_func# is one of sgesv, dgesv, cgesv, zgesv, that are linear solvers for general matrices.
On the other hand, scipy.linalg.inv calls getri , defined as get_lapack_funcs(('getri'),(a1,)). It corresponds to the DGETRI() function of lapack, designed to computes the inverse of a matrix using the LU factorization computed by DGETRF(). Hence, if you are using DGETRI() in Fortran, making use of scipy.linalg.inv() in python likely enable similar performances and results.
Most of Lapack functions can be called using scipy.linalg.lapack. Here is an example making use of scipy.linalg.cython_lapack.dgetri() in a cython module: How to compile C extension for Python where C function uses LAPACK library? Here goes a sample code, comparing scipy.linalg.cython_lapack.dgetrf()+scipy.linalg.cython_lapack.dgetri() , numpy and scipy.linalg.inv() on a 1000x1000 matrix:
import numpy as np
from scipy import linalg
import time
import myinverse
n=1000
A=np.random.rand(n,n)
start= time.time()
Am,info,string=myinverse.invert(A.copy())
end= time.time()
print 'DGETRF+DGETRI, ', end-start, ' seconds'
if info==0:
print 'residual ',np.linalg.norm(A.dot(Am)-np.identity(n), np.inf)
else :
print "inversion failed, info=",info, string
start= time.time()
Am=np.linalg.inv(A.copy())
end= time.time()
print 'np.linalg.inv ', end-start, ' seconds'
print 'residual ', np.linalg.norm(A.dot(Am)-np.identity(n), np.inf)
start= time.time()
Am=linalg.inv(A.copy())
end= time.time()
print 'scipy.linalg.inv ', end-start, ' seconds'
print 'residual ',np.linalg.norm(A.dot(Am)-np.identity(n), np.inf)
And the output is:
DGETRF+DGETRI, 0.22541308403 seconds
residual 4.2155882951089296e-11
np.linalg.inv 0.29932808876 seconds
residual 4.371813154546711e-11
scipy.linalg.inv 0.298856973648 seconds
residual 9.110997546690758e-11
For 2000x2000 matrix:
DGETRF+DGETRI, 1.64830899239 seconds
residual 8.541625644634121e-10
np.linalg.inv 2.02795410156 seconds
residual 7.448244269611659e-10
scipy.linalg.inv 1.61937093735 seconds
residual 1.6453560233026243e-09
A Fortran code chaining DGETRF()+DGETRI() is provided in LAPACK inversion routine strangely mixes up all variables
After a few changes, let' run:
PROGRAM solvelinear
implicit none
REAL(8), dimension(1000,1000) :: A,Ainv,M,LU
REAL(8),allocatable :: work(:)
REAL(8) :: wwork
INTEGER :: info,lwork
INTEGER,dimension(1000) :: ipiv
INTEGER :: i,j
real :: start, finish
! put code to test here
info=0
!work=0
ipiv=0
call RANDOM_NUMBER(A)
call cpu_time(start)
!-- LU factorisation
LU = A
CALL DGETRF(1000,1000,LU,1000,ipiv,info)
!-- Inversion of matrix A using the LU
Ainv=LU
lwork=-1
CALL DGETRI(1000,Ainv,1000,Ipiv,wwork,lwork,info)
lwork =INT( wwork+0.1)
allocate(work(lwork))
CALL DGETRI(1000,Ainv,1000,Ipiv,work,lwork,info)
deallocate(work)
call cpu_time(finish)
print '("Time = ",f6.3," seconds.")',finish-start
!-- computation of A^-1 * A to check the inverse
M = matmul(Ainv,A)
print*,"M = "
do i=1,3
do j=1,3
print*,M(i,j)
enddo
end do
END PROGRAM solvelinear
Once compiled by using gfortran main2.f90 -o main2 -llapack -lblas -lm -Wall, it takes 0.42s for 1000x1000 matrix and 3s for a 2000x2000 matrix.
Finally, different performances may occur if the Fortran code and python code do not link to the same Blas/Lapack libraries. To investigate this, type commands like np.__config__.show() as shown in Link ATLAS/MKL to an installed Numpy or commands reported in How to check BLAS/LAPACK linkage in NumPy and SciPy? .
To got further and make use of distributed computations, petsc discourages inverting full matrices, as it is rarely required. It is also written that MatMatSolve(A,B,X), where B and X are dense matrices can be used to do so. Furthermore, this function is provided in the python interface petsc4py as method matSolve(self, Mat B, Mat X) for the object petsc4py.PETSc.Mat. The no-maintained-anymore Elemental library is listed as implementing a direct solver for dense matrices. While the Elemental library supported a python interface, its fork Hydrogen does not support it anymore.
Nevertheless, the Elemental page lists some related open source projects for Distributed dense linear algebra. ScaLapack provides the routine PDGETRI()/PZGETRI() to invert distributed dense matrices using LU decomposition. That might leave some room for a faster inversion.
I used interp1 in matlab with the method 'spline', is there a function in any python packages can stead of it?
I have tried interp1d in scipy with the kind='cubic', but it has different results with interp1 in matlab with the method 'spline', although not very big.
And I have found people saying that InterplotationUnivariateSpline with k=3 is same as interp1d with kind='cubic', is it true? And what about CubicSpline?In docs they say that the resulting spline will be the same as returned by splrep (with s=0) and InterpolatedUnivariateSpline, but they also say that it doesn't use B-spline basis like splrep (with s=0) and InterpolatedUnivariateSpline.
Matlab code:
ppx = interp1(samplez,samplex,'spline','pp');
interpx = ppval(ppx,interpz);
python code:
from scipy import interpolate
interplinex = interpolate.interp1d(samplez, samplex, kind='cubic')
interpx = interplinex(interpz)
The result named interpx in Matlab is different with the result named interpx in python
I am working with complex functions in sympy (python 3) and am having trouble getting sympy to simplify the equations. In particular I can't get sympy to use Euler's Identity to break up the complex exponential into real and imaginary parts. Here is my code:
import sympy as sym
from sympy import I, init_printing
# setup printing
init_printing()
# complex potential cylinder in uniform flow
U,z,R,theta=sym.symbols('U z R theta')
F=U*z+U/z
# complex velocity cylinder in uniform flow
compVel=sym.diff(F,z)
exp1=sym.sympify('R*exp(I*theta)')
compVel=compVel.subs(z,exp1)
print(compVel)
phi,psi=sym.symbols('phi psi')
phi=sym.re(compVel)
psi=sym.im(compVel)
print(phi)
print(psi)
When I run this code the output is:
U - U*exp(-2*I*theta)/R**2
re(U) - re(U*exp(-2*I*theta)/R**2)
im(U) - im(U*exp(-2*I*theta)/R**2)
Am I missing something or is sympy not powerful enough to do recognize this simplification? Thanks in advance!
I figured it out; #Stelios is correct but in addition when you use sympify you must pass a dictionary with the local variables that the expression contains.
exp1=sym.sympify('R*exp(I*theta)',locals={'R':R,'theta':theta})
Hello and thanks in advance. Fresh off the heels of this question I acquired some more RAM and now have enough memory to fit all the matrices I need to run a linear programming solver. Now the problem is none of the Linear Programming packages in R seem to support long vectors (ie large matrices).
I've tried functions Rsymphony_solve_LP, Rglpk_solve_LP and lp from packages Rsymphony, Rglpk, and lpSolve respectively. All report a similar error to the one below:
Error in rbind(const.mat, const.dir.num, const.rhs) :
long vectors not supported yet: bind.c:1544
I also have my code below in case that helps...the constraint matrix mat is my big matrix (7062 rows by 364520 columns) created using the package bigmemory. When I run this this line the matrix is pulled into memory and then after a while the errors show.
Rsymph <- Rsymphony_solve_LP(obj,mat[1:nrow(mat),1:ncol(mat)],dir,rhs,types=types,max=max, write_lp=T)
I'm guessing this is a hard-coded error in each of the three functions? Is there currently a linear programming solver in R or even Python that supports long vectors? Should I contact the package maintainers or just edit the code myself? Thanks!
The package lpSolveAPI can solve long-vector linear programming problems. You have to first start my declaring a Linear Programming object, then add the constraints:
library(lpSolveAPI)
#Generate Linear Programming Object
lprec <- make.lp(nrow = 0 # Number of Constraints
, ncol = ncol(mat) # Number of Decision Variables
)
#Set Objective Function to Minimize
set.objfn(lprec, obj)
#Add Constraints
#Note Direction and RHS is included along with Constraint Value
for(i in 1:nrow(mat) ){
add.constraint(lprec,mat[i,], dir[i], rhs[i])
print(i)
}
#Set Decision Variable Type
set.type(lprec, c(1:ncol(mat)), type = c("binary"))
#Solve Model
solve(lprec)
#Obtain Solutions
get.total.iter(lprec)
get.objective(lprec)
get.variables(lprec)
There's a good introduction to this package here.
I am porting code from Matlab to Python and am having trouble finding a replacement for the firls( ) routine. It is used for, least-squares linear-phase Finite Impulse Response (FIR) filter design.
I looked at scipy.signal and nothing there looked like it would do the trick. Of course I was able to replace my remez and freqz algorithsm, so that's good.
On one blog I found an algorithm that implemented this filter without weighting, but I need one with weights.
Thanks, David
The firls equivalent in python now appears to be implemented as part of the signal package:
https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.firls.html#scipy.signal.firls
Also I agree with everything that #pev hall stated above especially how firls is optimum in many situations (such as when overall signal to noise is being optimized for a given number of taps), and to not use the boxcar window as he stated, they are not equivalent at all! firls generally outperforms all window and frequency sampling approaches to filter design when designing traditional FIR filters.
To my current understanding, scipy.signal and Octave only support odd length (even order) least squared filters. In this case when I need an even length (Type II or Type IV filter), I resort to using the windowed design approach with a Kaiser Window specifically. I have found the Kaiser window solution to come quite close to the optimum least squares solution.
This blog post contains code detailing how to use scipy.signal to implement FIR filters.
Obviously, this post is somewhat dated, but maybe it is still interesting for some:
I think there are two near-equivalents to firls in Python:
You can try the firwin function with window='boxcar'. This is similar to Matlab where fir1 with a boxcar window delivers the same (? or at least very similar results) as firls.
You could also try the firwin2 method (frequency sampling method, similar to fir2 in Matlab), again using window='boxcar'
I did try one example from the Matlab firls reference and achieved near-identical results for:
Matlab:
F = [0 0.3 0.4 0.6 0.7 0.9];
A = [0 1 0 0 0.5 0.5];
b = firls(24,F,A,'hilbert');
Python:
F = [0, 0.3, 0.4, 0.6, 0.7, 0.9, 1]
A = [0, 1, 0, 0, 0.5, 0.5, 0]
bb = sig.firwin2( 25, F,A, window='boxcar', antisymmetric=True )
I had to increase the order to N = 25 and I also had to add another data point (F = 1, A = 0) which Python insisted upon; the option antisymmetric = True is only necessary for this special case (Hilbert filter)
This post is really in response to
You can try the firwin function with window='boxcar'...
Do don't use boxcar it means no window at all (it is ideal but only works "ideally" with an infinite number of multipliers - sinc in time). The whole perpose of using a window is the reduce the number of multipliers required to get good stop band attenuation. See Window function
When comparing filters please use dB/log scale.
Scipy not having firls (FIR least squares filter) function is a large limitation (as it generates the optimum filter in many situations).
REMEZ has it's place but the flat roll off is a real killer when your trying to get the best results (and not just meeting some managers spec). ( warning scipy remez implementation can give amplification in stop band - see plot at bottom)
If you are using python (or need to use some window) I recommend using the kasiar window which gets very good results and can easily be tweaked for your attenuation vs transition vs multipliers requirement(attenuation (in dB) = 2.285 * (multipliers - 1) * pi * width + 7.95). It performance is not quite as good as firls but it has the benefit of being fast and easy to calculate (great if you don't store the coefficients).
I found a firls() implementation attached here in SciPy ticket 648
Minor changes to get it working:
Swap the following two lines:
bands, desired, weight = array(bands), array(desired), array(weight)
if weight==None : weight = ones(len(bands)/2)
import roots from numpy instead of scipy.signal
Since version 0.18 in July, 2016 scipy includes an implementation of firls, as scipy.signal.firls.
It seems unlikely that you'll find exactly what you seek already written in Python, but perhaps the Matlab function's help page gives or references a description of the algorithm?