Is there a way to use together sympy's TensorProduct from the sympy.physics.quantum module and the Pauli matrices from sympy.physics.paulialgebra?
My problem is the following:
physically: I have an Hamiltonian written as product of (fermionic) ladder operators and I need to implement the Jordan-Wigner transformation to "translate" it to a spin Hamiltonian
with sympy: I have used sympy's AnnihilateFermion and CreateFermion functions from sympy.physics.secondquant, and I have implemented manually the Jordan-Wigner transformation. The substitution from the ladder operators to the spin operators works fine. Ideally I then would use tensor_product_simp and evaluate_pauli_product to simplify the result. But sympy does not like this mixing. Any suggestions?
Below is the full minimal example. Thanks a lot for any help.
import sympy as sym
import sympy.physics.secondquant as sec
from sympy.physics.quantum import TensorProduct, Dagger, tensor_product_simp, IdentityOperator
from sympy.physics.paulialgebra import Pauli, evaluate_pauli_product
def JW(n, N_sites):
factors = []
for l in range(N_sites):
if l < n-1:
factors.append(-sym.I*Pauli(3))
elif l == n-1:
factors.append((Pauli(1)-sym.I*Pauli(2))/2)
else:
# add identity below
factors.append(Pauli(1)*Pauli(1))
res = factors[0]
for l in range(1,n):
res = TensorProduct(res, factors[l])
return res
if __name__ == '__main__':
N_sites = 4
loci = sym.symbols('l1:{}'.format(N_sites + 1))
chi_s = [sec.AnnihilateFermion(loci[k]) for k in range(N_sites)]
chi_dag_s = [sec.CreateFermion(loci[k]) for k in range(N_sites)]
H = sum((chi_dag_s[n] * chi_s[n + 1] - chi_dag_s[n + 1] * chi_s[n]) for n in range(N_sites - 1))
list_of_subs = []
for n in range(N_sites):
list_of_subs.append((chi_s[n], JW(n, N_sites)))
list_of_subs.append((chi_dag_s[n], Dagger(JW(n, N_sites))))
H_new = H.subs(list_of_subs)
print(H_new)
print(tensor_product_simp(sym.expand(H_new)))
How can I change this part [AIF,j]=get_AIF_j(InterpFact) and [~,j_index] = min(InterpFact-AIF_vect) correctly? And what about the remaining code? Thanks in advance.
%Matlab code
InterpFact = (fs_h/2/2)/(fd_max);
[AIF,j]=get_AIF_j(InterpFact);
function [AIF,j] = get_AIF_j (InterpFact)
j_vect = 1:10;
AIF_vect = floor(j_vect*InterpFact)./j_vect;
[~,j_index] = min(InterpFact-AIF_vect);
j = j_vect(j_index);
AIF = AIF_vect(j_index);
end
#Python code
InterpFact = (fs_h/2/2)/(fd_max)
[AIF,j]=get_AIF_j(InterpFact)
def get_AIF_j (InterpFact):
j_vect =np.arange(1,11)
AIF_vect = np.floor(j_vect*InterpFact)/j_vect
[~,j_index] = min(InterpFact-AIF_vect)
j = j_vect[j_index]
AIF = AIF_vect[j_index];
return AIF,j
This MATLAB:
[~,j_index] = min(InterpFact-AIF_vect);
would be translated to Python as:
j_index = np.argmin(InterpFact-AIF_vect)
Also, …/(fd_max) can only be translated the way you did if fd_max is a scalar. A division with a matrix in MATLAB solves a system of linear equations.
I strongly recommend that you run the two pieces of code side by side with the same input, to verify that they do the same thing. You cannot go by guesses as to what a piece of code does.
Try this to see if it delivers what it should (I am not sure here as I am not fluent in matlab):
#Python code
import numpy as np
def get_AIF_j (InterpFact):
j_vect = np.arange(1,11)
AIF_vect = np.floor(j_vect*InterpFact)/j_vect
j_index = int( min(InterpFact-AIF_vect) )
print(j_index)
j = j_vect[j_index]
AIF = AIF_vect[j_index];
return AIF, j
fs_h = 24; fd_max = 1
InterpFact = (fs_h/2/2)/(fd_max)
AIF, j = get_AIF_j(InterpFact)
print(AIF,j)
gives:
0
6.0 1
I have an excel function that i'm trying to replicate into python but struggling to replicate it. Here's the function (VBA):
Function Z(Lambda, conf) As Integer
Application.Volatile
Lambda = Application.Round(Lambda, 3)
Select Case Lambda
Case Is < 400
For i = 0 To 500
' v = Application.Poisson(i, Lambda, True)
v = Application.ChiDist(2 * Lambda, 2 * i + 2)
If v >= conf Then
Z = i
Exit Function
End If
Next
Case Else
Z = Application.NormInv(conf, Lambda, Sqr(Lambda))
End Select
End Function
In Excel if i run =z(2,95%), i get z=5
I thought I could use:
from scipy import stats
stats.chi2.cdf
but not getting far with it.
Any help is highly appreciated!
According to the ChiDist docs, (see also CHIDIST), ChiDist returns the right tail of the Χ² distribution. The corresponding function in SciPy is the sf (survival function) method of scipy.stats.chi2. In your Python code, change stats.chi2.cdf to stats.chi2.sf.
Managed to get the function working in python - thanks #Warren Weckesser for the guidance on the chi2.sf.
from scipy.stats import norm, chi2
def z(lamda_calc, prob):
if lamda_calc < 400:
z_calc = [i for i in range (0,500) if chi2.sf(2 * lamda_calc, 2 * i + 2) >=
prob][0]
else:
z_calc = int(norm.ppf(prob,lamda_calc,sqrt(lamda_calc)))
return z_calc
print
print ("z:'", z(1.4, 0.98))
What I am trying to do
I am trying to create a very simple function which I want to optimise with numba (or at least verify if numba makes any difference).
I am running numpy 1.19.2 and numba 0.51.2 in an Anaconda installation on Windows.
The function takes 3 numeric inputs: a, b , c; the inputs can be scalars or numpy arrays; the output will of course be, respectively, a scalar or a numpy array
The function is fairly simple:
if a == 0 --> it returns np.nan
if b == 0 --> it returns a certain number
otherwise it performs some very simple algebra
The issue
I have come up with the toy example below (my actual formulas are more complex but I can show what I need to show with this easier example).
if the inputs are arrays, it works perfectly
if the inputs are scalar, numba doesn't work (Cannot unify array(int64, 0d, C) and float64 for '$phi12.0.2' )
if the inputs are arrays of size 1 (I make an array out of each scalar) numba works again
What I tried / similar questions
The closest question I found was this, but the mismatch there was between an int and a float.
Here it is between an array(int64, 0d, C) and a float64. I can convert my inputs to float but the mismatch remains.
Any ideas? I am not sure what the array and the float being compared are, to be honest.
The one solution I have found is to add a = np.array([a]) at the beginning of the function, but I don't understand why, plus this returns an array of size 1, whereas I'd like a scalar returned in these cases.
Toy example
#numba.jit
def my_fun(a,b,c):
return np.where(a == 0, np.nan,
np.where(b ==0 , 0 , c**2) )
a = np.arange(0,11)
b = np.arange(3,14)
b[1] = 0
c = np.arange(10,21)
out_array = my_fun(a,b,c)
out_scalar = my_fun(0,0,1)
The exact warning:
NumbaWarning:
Compilation is falling back to object mode WITH looplifting enabled because Function my_fun failed at nopython mode lowering due to: Failed in nopython mode pipeline (step: nopython frontend)
Cannot unify array(int64, 0d, C) and float64 for '$phi12.0.2', defined at C:\Users\USERNAME\anaconda3\lib\site-packages\numba\np\arraymath.py (3276)
File "C:\Users\USERNAME\anaconda3\lib\site-packages\numba\np\arraymath.py", line 3276:
def scalar_where_impl(cond, x, y):
<source elided>
"""
scal = x if cond else y
^
During: typing of assignment at C:\Users\USERNAME\anaconda3\lib\site-packages\numba\np\arraymath.py (3276)
File "C:\Users\USERNAME\anaconda3\lib\site-packages\numba\np\arraymath.py", line 3276:
def scalar_where_impl(cond, x, y):
<source elided>
"""
scal = x if cond else y
^
During: lowering "$36call_method.17 = call $4load_method.1($10compare_op.4, $14load_attr.6, $34call_method.16, func=$4load_method.1, args=[Var($10compare_op.4, refactor numba.py:8), Var($14load_attr.6, refactor numba.py:8), Var($34call_method.16, refactor numba.py:9)], kws=(), vararg=None)" at D:\MY DATA\USERNAME\Python\scratch scripts\refactor numba.py (8)
#numba.jit
C:\Users\USERNAME\anaconda3\lib\site-packages\numba\core\object_mode_passes.py:177: NumbaWarning: Function "my_fun" was compiled in object mode without forceobj=True.
File "refactor numba.py", line 6:
#numba.jit
def my_fun(a,b,c):
^
warnings.warn(errors.NumbaWarning(warn_msg,
C:\Users\USERNAME\anaconda3\lib\site-packages\numba\core\object_mode_passes.py:187: NumbaDeprecationWarning:
Fall-back from the nopython compilation path to the object mode compilation path has been detected, this is deprecated behaviour.
For more information visit https://numba.pydata.org/numba-doc/latest/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit
File "refactor numba.py", line 6:
#numba.jit
def my_fun(a,b,c):
^
warnings.warn(errors.NumbaDeprecationWarning(msg,
I have found a solution, but it's far from elegant, and I am hoping there is a better one.
To recap, I needed a function which:
works with numba
works with both scalars and arrays
returns scalar (not a one-sized array) when the inputs are scalars, and arrays when the inputs are arrays
I have tried the following, and found option 2 to be the fastest.
my_fun_optimised_1: a function which, without numba, determines whether the inputs are scalar or not, and then calls, accordingly, a sub-function for the scalar case and one for the arrays. Both sub-functions run with numba, but take forever. I guess this is because numba must be re-initialised at each call of the main function.
my_fun_optimised_2: similar to the above, except the scalar and array functions, both running with numba, are main functions and not subfunctions. Much much faster.
my_fun_non_opt_no_numba : a function which runs without numba.
The results are:
+-------------------------+----------------------------+-----------------------------+
| Function | Array: time vs the fastest | Scalar: time vs the fastest |
+-------------------------+----------------------------+-----------------------------+
| optimised numba 1 | 54,403 | 42,961 |
| optimised numba 2 | 1 | 1 |
| non-optimised, no numba | 3.409 | 4.53892 |
+-------------------------+----------------------------+-----------------------------+
What this means is that, on my PC, the non-optimised, no numba code takes 4.5 times longer than "optimsied numba 2" to run on scalars and 3.4 times longer for arrays.
The "optimised numba 1" is not optimsied at all and takes an insane amount of time.
I hope all of this can be of use to other people.
PS I am very well familiar with pitfalls of premature optimisation. I am only doing this because I have a specific case where 60% of the time is spent doing a similar (but not identical) calculation to the one shown here.
The code to time the functions is:
import numpy as np
import numba
import timeit
import pandas as pd
def my_fun_optimised_1(a,b,c):
#numba.jit
def my_fun_vectorised(a,b,c):
return np.where(a == 0, np.nan,
np.where(b ==0 , 0 , b*a**3 + a*b**3 + a*b*c**3)
)
#numba.jit
def my_fun_scalar(a,b,c):
if a ==0:
return np.nan
elif b == 0:
return np.nan
else:
return b*a**3 + a*b**3 + a*b*c**3
if np.isscalar(a) and np.isscalar(b) and np.isscalar(c):
return my_fun_scalar(a,b,c)
else:
return my_fun_vectorised(a,b,c)
def my_fun_optimised_2(a,b,c):
if np.isscalar(a) and np.isscalar(b) and np.isscalar(c):
return fun_2_scalar(a,b,c)
else:
return fun_2_vectorised(a,b,c)
#numba.jit
def fun_2_scalar(a,b,c):
if a ==0:
return np.nan
elif b == 0:
return np.nan
else:
return b*a**3 + a*b**3 + a*b*c**3
#numba.jit
def fun_2_vectorised(a,b,c):
return np.where(a == 0, np.nan,
np.where(b ==0 , 0 , b*a**3 + a*b**3 + a*b*c**3)
)
def my_fun_non_opt_no_numba(a,b,c):
# multipl by 1 converts from array to scalar
return 1 * np.where(a == 0, np.nan,
np.where(b ==0 , 0 , b*a**3 + a*b**3 + a*b*c**3)
)
# I couldn't get this to work with Numba
##numba.jit
def my_fun_non_opt_numba(a,b,c):
a = np.array([a])
b = np.array([b])
c = np.array([c])
out = np.where(a == 0, np.nan,
np.where(b ==0 , 0 , b*a**3 + a*b**3 + a*b*c**3)
)
return out
r = 4
n = int(100)
a = 3
b = 4
c = 5
x = my_fun_optimised_2(a,b,c)
t_scalar_opt_numba_1 = timeit.Timer("my_fun_optimised_1(a,b,c) " , globals = globals() ).repeat(repeat = r, number = n)
t_scalar_opt_numba_2 = timeit.Timer("my_fun_optimised_2(a,b,c) " , globals = globals() ).repeat(repeat = r, number = n)
t_scalar_non_opt_no_numba = timeit.Timer("my_fun_non_opt_no_numba(a,b,c) " , globals = globals() ).repeat(repeat = r, number = n)
resdf_scalar = pd.DataFrame(index = ['min time'])
resdf_scalar['optimised numba 1'] = [min(t_scalar_opt_numba_1)]
resdf_scalar['optimised numba 2'] = [min(t_scalar_opt_numba_2)]
resdf_scalar['non-optimised, no numba'] = [min(t_scalar_non_opt_no_numba)]
# the docs explain why we should take the min and not the avg
resdf_scalar = resdf_scalar.transpose()
resdf_scalar['diff vs fastest'] = (resdf_scalar / resdf_scalar.min() )
a = np.arange(3,13)
b = np.arange(0,10)
c = np.arange(20,30)
t_array_opt_numba_1 = timeit.Timer("my_fun_optimised_1(a,b,c) " , globals = globals() ).repeat(repeat = r, number = n)
t_array_opt_numba_2 = timeit.Timer("my_fun_optimised_2(a,b,c) " , globals = globals() ).repeat(repeat = r, number = n)
t_array_non_opt_no_numba = timeit.Timer("my_fun_non_opt_no_numba(a,b,c) " , globals = globals() ).repeat(repeat = r, number = n)
resdf_array = pd.DataFrame(index = ['min time'])
resdf_array['optimised numba 1'] = [min(t_array_opt_numba_1)]
resdf_array['optimised numba 2'] = [min(t_array_opt_numba_2)]
resdf_array['non-optimised, no numba'] = [min(t_array_non_opt_no_numba)]
# the docs explain why we should take the min and not the avg
resdf_array = resdf_array.transpose()
resdf_array['diff vs fastest'] = (resdf_array / resdf_array.min() )
I have been trying to run a code using numba and I have also added a print to see the progress of my code :
from numba import jit,njit,prange
import numpy as np
# for minimum reproducible example
a=1e5
ar = np.random.rand(a)
at = np.random.rand(a)
an = np.random.rand(a)
###############################3
tau = 1 # time lag
window = 6000
#njit(parallel=True)
def func_DB(ar,at,an):
DBtotal= np.zeros((len(an)-tau))
k = 0
for i in prange(0,len(an)-tau,1):
DBtotal[i] = np.sqrt((ar[i + tau]- ar[i])**2 +(at[i + tau]- at[i])**2 +(an[i + tau]- an[i])**2)
## To print the progress
if i%1e5==0:
k+=1
print(k*1e5/len(DBtotal))
return DBtotal
#njit(parallel=True)
def func_PVI(tau, window):
PVI = np.zeros((len(DBtotal)))
k = 0
for i in prange(int(window/2),len(DBtotal)-int(window/2)):
PVI[i] = DBtotal[i]/np.sqrt((np.mean(DBtotal[i-int(window/2):i+int(window/2)]**2)))
# To print the progress
if i%1e5==0:
k+=1
print(k*1e5/len(DBtotal))
return PVI
DBtotal = func_DB(ar,at,an)
PVI = func_PVI(DBtotal,tau, window)
However, while the code is running I dont get what I expected (i.e. values that go from 0 to 1 as the code is progressing) Instead, I get this:
Out[:] 0.009479390005044932
0.009479390005044932
0.009479390005044932
0.009479390005044932
0.009479390005044932
0.018958780010089864
Could someone suggest a way to see the progress of the code?
Also, any suggestions to make the code more efficient would be much appreciated!
I broke the function down into pieces and wrapped a tqdm around it.
Instead of
#jit(nopython=True)
def dothings(A, rows, cols):
for r in range(rows):
for c in range(cols):
stuff...
dothings(data, data.shape[0], data.shape[1])
I used
rows=data.shape[0]
#jit(nopython=True)
def dothings(A, cols, r):
# for r in range(rows):
for c in range(cols):
stuff...
for r in tqdm.tqdm(range(rows), total=rows):
dothings(data, data.shape[1], r)
try this:
from numba import njit,prange,objmode
#njit(parallel=True)
def harmonic_load_flow_func_time_inside():
with objmode(time1='f8'):
time1 = time.perf_counter()
calc = 0
for x in prange(1000000):
calc += x
with objmode():
print('time: {}'.format(time.perf_counter() - time1),end='\r')