I would like to create a 3D matrix of specific size by calculating a value for each combination of indexes. Each value in the matrix will be symbolic.
What I tried up to now:
import numpy as np
import sympy as sp
var1 = np.arange(1,10,2)
var2 = np.arange(1,10,2)
var3 = np.arange(20,50,5)
myMatrix = np.zeros(shape = (len(var1), len(var2), len(var3)))
t = sp.symbols('t')
for ii in range(len(var1)):
for jj in range(len(var2)):
for kk in range(len(var3)):
myMatrix[ii][jj][kk] = var1[ii] * var2[jj] * var3[kk] * t
This gives me the error:
TypeError: can't convert expression to float
which as far as I understand is due to combining numpy and sympy. Therefore, I tried:
myMatrix = sp.MatrixSymbol('temp', len(var1), len(var2), len(var3))
instead of:
myMatrix = np.zeros(shape = (len(var1), len(var2), len(var3)))
and got an error:
TypeError: new() takes exactly 4 arguments (5 given)
To sum up, my question is: how can I create a 3D matrix with any variables inside to be able to use it in the nested loop, which involves symbolic calculation?
(This is my first post in this community, so please let me know if I did anything wrong.)
The first error you get is, as you suggested, because you try to save a sympy type object into a numpy zeros array which is of type numbers. One option would be to use a numpy array of objects, which works as follows,
import numpy as np
import sympy as sp
var1 = np.arange(1,10,2)
var2 = np.arange(1,10,2)
var3 = np.arange(20,50,5)
myMatrix = np.empty((len(var1), len(var2), len(var3)), dtype=object)
t = sp.symbols('t')
for ii in range(len(var1)):
for jj in range(len(var2)):
for kk in range(len(var3)):
myMatrix[ii][jj][kk] = var1[ii] * var2[jj] * var3[kk] * t
Although for large sizes this isn't too efficient and not the way numpy should work. For sympy arrays this may be the only way to go however as it seems that, at least in my version of sympy (0.7.1.rc1), 3D arrays are not supported. For
myMatrix = sp.zeros((len(var1), len(var2), len(var3)))
I get the following error
ValueError: Matrix dimensions should be a two-element tuple of ints or a single int!
Related
A is a k dimensional numpy array of floats (k could be pretty big, e.g. up to 10)
I need to implement an update to A by incrementing each of the values (as described below). I'm wondering if there is a numpy-style way that would be fast.
Let L_i be the length of axis i
An update to this array is generated in two steps follows:
For each axis of A a corresponding vector G is generated.
For example, corresponding to axis i a vector G_i of length L_i is generated (from data).
Update A at all positions by calculating an increment from the G vectors for each position in A
To do this at any particular position, let p be an array of k indices, corresponding to a position in A. Then A at p is incremented by a value calculated as the product:
Product(G_i[p[i]], for i from 0 to k-1)
A full update to A involves doing this operation for all locations in A (i.e. all possible values of p)
This operation would be very slow doing positions one by one via loops.
Is there a numpy style way to do this that would be fast?
edit
## this for three dimensions, final matrix at pos i,j,k has the
## product of c[i]*b[j]*a[k]
## but for arbitrary # of dimensions it will have a loop in a loop
## and will be slow
import numpy as np
a = np.array([1,2])
b = np.array([3,4,5])
c = np.array([6,7,8,9])
ab = []
for bval in b:
ab.append(bval*a)
ab = np.stack(ab)
abc = []
for cval in c:
abc.append(cval*ab)
abc = np.stack(abc)
as a function
def loopfunc(arraylist):
ndim = len(arraylist)
m = arraylist[0]
for i in range(1,ndim):
ml = []
for val in arraylist[i]:
ml.append(val*m)
m = np.stack(ml)
return m
This is a wacky problem, but I like it.
If I understand what you need from your example, you can accomplish this with some reshaping trickery and NumPy's usual broadcasting rules. The idea is to reshape each array so it has the right number of dimensions, then just directly multiply.
Here's a function that implements this.
from functools import reduce
import operator
import numpy as np
import scipy.linalg
def wacky_outer_product(*arrays):
assert len(arrays) >= 2
assert all(arr.ndim == 1 for arr in arrays)
ndim = len(arrays)
shapes = scipy.linalg.toeplitz((-1,) + (1,) * (ndim - 1))
reshaped = (arr.reshape(new_shape) for arr, new_shape in zip(arrays, shapes))
return reduce(operator.mul, reshaped).T
Testing this on your example arrays, we have:
>>> foo = wacky_outer_product(a, b, c)
>>> np.all(foo, abc)
True
Edit
Ok, the above function is fun, but the below is probably much better. No transposing, clearer, and much smaller:
from functools import reduce
import operator
import numpy as np
def wacky_outer_product(*arrays):
return reduce(operator.mul, np.ix_(*reversed(arrays)))
My code is running fine for first iteration but after that it outputs the following error:
ValueError: matrix must be 2-dimensional
To the best of my knowledge (which is not much in python), my code is correct. but I don't know, why it is not running correctly for all given iterations. Could anyone help me in this problem.
from __future__ import division
import numpy as np
import math
import matplotlib.pylab as plt
import sympy as sp
from numpy.linalg import inv
#initial guesses
x = -2
y = -2.5
i1 = 0
while i1<5:
F= np.matrix([[(x**2)+(x*y**3)-9],[(3*y*x**2)-(y**3)-4]])
theta = np.sum(F)
J = np.matrix([[(2*x)+y**3, 3*x*y**2],[6*x*y, (3*x**2)-(3*y**2)]])
Jinv = inv(J)
xn = np.array([[x],[y]])
xn_1 = xn - (Jinv*F)
x = xn_1[0]
y = xn_1[1]
#~ print theta
print xn
i1 = i1+1
I believe xn_1 is a 2D matrix. Try printing it you and you will see [[something], [something]]
Therefore to get the x and y, you need to use multidimensional indexing. Here is what I did
x = xn_1[0,0]
y = xn_1[1,0]
This works because within the 2D matrix xn_1 are two single element arrays. Therefore we need to further index 0 to get that single element.
Edit: To clarify, xn_1[1,0] means to index 1 and then take that subarray and index 0 on that. And although according to Scipy it may seem that it should be functionally equivalent to xn_1[1][0], that only applies to the general np.array type and not the np.matrix type. Here is an excellent thread on SO that explains this.
So you should use the xn_1[1,0] way to get the element you want.
xn_1 is a numpy matrix, so it's elements are accessed with the item() method, not like an array. (with []s)
So just change
x = xn_1[0]
y = xn_1[1]
to
x = xn_1.item(0)
y = xn_1.item(1)
I am trying to compute the Earth Mover Distance between two histograms I have (x1,x2), using the following snippet I obtained from http://docs.opencv.org/2.4/modules/imgproc/doc/histograms.html:
cv2.cv.CalcEMD2(x1, x2,cv2.cv.CV_DIST_L1)
However I keep getting this error.
TypeError: CvArr argument 'signature1' must be IplImage, CvMat or CvMatND. Use fromarray() to convert numpy arrays to CvMat or cvMatND"
I tried using .fromarray() to convert what I had into a CvMat format with no success yet. Is there something I am doing wrong?
Here are the two types of array i am working with (shortened to three elements for clarity) I tried to convert using cv2.cv.fromarray():
np.histogramdd(data, bins = 80))
giving me:
(array([ 28., 5., 0.]), [array([-1.71194523, -1.66131523, -1.61068523])])
and a normal np array:
[28. 5. 0.]
Neither work for me and neither allow me to compute the Earth mover distance between arrays. Any help would be greatly appreciated!
Right, after much trial and error I think this is the solution:
(obtained from best answer of How to compute "EMD" for 2 numpy arrays i.e "histogram" using opencv?)
ref = reference np array
meas = np array to compare it to
creates histogram of numpy array [[data][weights]], converts it to [data[i],weight[i]] format. Then to a cv array which then converts it into a signal function for the EMD function.
This is not an ideal solution - any further improvements/advice would be greatly appreciated for education purposes!!!
#reference array:
j = np.histogramdd(ref, bins = 80)
a = np.zeros((len(j[0]), 2))
for i, x in enumerate(j[0]):
a[i][0] = x
a[i][1] = j[1][0][i]
cv_array_ref = cv2.cv.fromarray(a)
a32 = cv2.cv.CreateMat(cv_array_ref.rows, cv_array_ref.cols,cv2.cv.CV_32FC1)
cv2.cv.Convert(cv_array_ref, a32)
#measured array:
jj = np.histogramdd(amplitude_norm_data[0], bins = 80)
aa = np.zeros((len(jj[0]), 2))
for ii, xx in enumerate(jj[0]):
aa[ii][0] = xx
aa[i][1] = jj[1][0][ii]
cv_array_meas = cv2.cv.fromarray(aa)
a322 = cv2.cv.CreateMat(cv_array_meas.rows, cv_array_meas.cols, cv2.cv.CV_32FC1)
cv2.cv.Convert(cv_array_meas, a322)
cv2.cv.CalcEMD2(a32, a322,cv2.cv.CV_DIST_L1)
import numpy as np
beta= 0.9
A=[1+1j,2+2j]
real=np.zeros((1,2))
for i in range(1):
for l in range(2):
real[i,j] = real[i,j]-beta*A[i,j]
I am not familiar with the computation of different types of arrays in numpy. How can I make the code work?
The problem with your original code is that the result of
real[i, j] - beta * A[i, j]
will be complex, but you created real using np.zeros, which will give you a float64 array unless you explicitly specify a different dtype. Since there is no safe way to cast a complex value to a float, the assignment to real[i, j] will raise a TypeError.
One way to solve the problem would be to initialize real with a complex dtype:
real = np.zeros((1, 2), dtype=np.complex)
If you make A a numpy array, you can use broadcasting to do the multiplication in one go without pre-allocating real and without looping:
import numpy as np
beta = 0.9
A = np.array([1 + 1j, 2 + 2j])
real = -beta * A
print(repr(real))
# array([-0.9-0.9j, -1.8-1.8j])
It looks like you'd probably benefit from reading some of the examples here.
Matlab Code:
AP(queryIdx) = diff([0;recall]')*prec
My python code:
AP[queryIdx] = np.dot(np.diff(np.concatenate(([[0]], recall), axis=0).transpose()),prec)
Variables:(Checked and am quite sure they are equivalent in python and in Matlab)
Recall: 1000x1 np array*
prec: 1000x1 np array
* prints out as [[.],.....,[.]]
Results:
Matlab: .1011
Python: 0.05263158
Only cause I can think of outside of the code is that python uses more
precision, but I doubt that would make such a large difference)
*Edit There was a problem with my prec variable. The above code worked
That code looks a bit messy. Try replacing it with this:
AP[queryIdx] = np.dot(np.diff(np.hstack([0, recall.ravel()])), prec.ravel())
In your post, you mentioned that you have a 1000 x 1 array for both recall and prec. This to me is interpreted as a 2D array with a singleton dimension: the second dimension. As such, you'd need to convert this back to a 1D array using ravel.
Now, np.hstack horizontally stacks 1D arrays together and so this will append a 0 at the front, then apply the diff operator, and the perform the dot product with prec.
One common gotcha that MATLAB coders have with numpy is the representation of 1D arrays in numpy. There is no such thing as the transpose of a 1D array. All numpy 1D arrays are row vectors. If you explicitly want to make the 1D array a column vector, you need to include an additional dimension and make the second dimension 1, then transpose it. Something like this:
r = v[:][None].T
In any case, let's verify the results:
MATLAB
>> recall = (1:1000).';
>> prec = (1000:-1:1).';
>> diff([0; recall].')*prec
ans =
500500
Python (IPython)
In [1]: import numpy as np
In [2]: recall = np.arange(1,1001)
In [3]: prec = np.arange(1000,0,-1)
In [4]: np.dot(np.diff(np.hstack([0, recall.ravel()])), prec.ravel())
Out[4]: 500500