I am trying to rewrite the following snippet of Matlab code about outer product of matrices into python code,
function Y = matlab_outer_product(X,x)
A = reshape(X, [size(X) ones(1,ndims(x))]);
B = reshape(x, [ones(1,ndims(X)) size(x)]);
Y = squeeze(bsxfun(#times,A,B));
end
My one-to-one translation of this to python code is as following (considering how the shape of numpy array and matlab matrices are arranged),
def python_outer_product(X, x):
X_shape = list(X.shape)
x_shape = list(x.shape)
A = X.reshape(*list(np.ones(np.ndim(x),dtype=int)),*X_shape)
B = x.reshape(*x_shape,*list(np.ones(np.ndim(X),dtype=int)))
Y = A*B
return Y.squeeze()
Then trying the inputs, for instance,
matlab_outer_product([1,2],[[3,4];[5,6]])
python_out_product(np.array([[1,2]], np.array([[3,4],[5,6]])))
The outputs don't quite match. In matlab, it outputs
output(:,:,1) = [[3,5];[6,10]]
output(:,:,2) = [[4,6];[8,12]]
In python, it outputs
output = array([
[[ 3, 6],
[ 4, 8]],
[[ 5, 10],
[ 6, 12]]
])
They're almost identical, but not quite. I wonder what's wrong with code and how to change the python code to match with matlab output?
In full gory detail (since my MATLAB memory is old):
Octave
>> X = [1,2];
>> x = [[3,4];[5,6]];
>> A = reshape(X, [size(X) ones(1,ndims(x))]);
>> B = reshape(x, [ones(1,ndims(X)) size(x)]);
>> A
A =
1 2
>> B
B =
ans(:,:,1,1) = 3
ans(:,:,2,1) = 5
ans(:,:,1,2) = 4
ans(:,:,2,2) = 6
>> bsxfun(#times,A,B)
ans =
ans(:,:,1,1) =
3 6
ans(:,:,2,1) =
5 10
ans(:,:,1,2) =
4 8
ans(:,:,2,2) =
6 12
>> squeeze(bsxfun(#times,A,B))
ans =
ans(:,:,1) =
3 5
6 10
ans(:,:,2) =
4 6
8 12
You start with a (1,2) and (2,2), expand the second to (1,1,2,2). The bsxfun produces a (1,2,2,2) which is squeezed to (2,2,2).
A is X reshaped to [1 2 1 1], but the two outer size 1 dimensions are squeeze out, resulting in no change.
This MATLAB outter is a bit convoluted, using bsxfun to perform elementwise multiplication of (1,2,1,1) with (1,1,1,2). At least in Octave it's the same as
A.*B
In numpy
In [77]: X
Out[77]: array([[1, 2]]) # (1,2)
In [78]: x
Out[78]:
array([[3, 4], # (2,2)
[5, 6]])
Note that the MATLAB/Octave x when flattened has elements (3,5,4,6), while the numpy ravel is [3,4,5,6].
In numpy I can simply do:
In [79]: X[:,:,None,None]*x
Out[79]:
array([[[[ 3, 4], (1,2,2,2)
[ 5, 6]],
[[ 6, 8],
[10, 12]]]])
or without the extra size 1 dimension of X:
In [84]: (X[0,:,None,None]*x)
Out[84]:
array([[[ 3, 4],
[ 5, 6]],
[[ 6, 8],
[10, 12]]])
In [85]: (X[0,:,None,None]*x).ravel()
Out[85]: array([ 3, 4, 5, 6, 6, 8, 10, 12])
compare that with the Octave ravel
>> squeeze(bsxfun(#times,A,B))(:)'
ans =
3 6 5 10 4 8 6 12
We could add a transpose to the numpy
In [96]: (X[0,:,None,None]*x).transpose(2,1,0).ravel()
Out[96]: array([ 3, 6, 5, 10, 4, 8, 6, 12])
In [97]: (X[0,:,None,None]*x).transpose(2,1,0)
Out[97]:
array([[[ 3, 6],
[ 5, 10]],
[[ 4, 8],
[ 6, 12]]])
At least in numpy we can tweak the dimension order in lots of ways, so I won't try to suggest an optimal. I still think it's better to write code that's "natural" to numpy than to slavishly match the MATLAB order.
another try
I realized, above, that the MATLAB is just doing A*.B with
(1,2,1,1) arrays (1,1,1,2), where the extra 1's were added to "broadcast".
Using transpose to the same dimension outermost (leading in numpy)
In [5]: X = X.T; x = x.T
In [6]: X.shape
Out[6]: (2, 1)
In [7]: x.shape
Out[7]: (2, 2)
In [8]: x
Out[8]:
array([[3, 5],
[4, 6]])
In [9]: x.ravel()
Out[9]: array([3, 5, 4, 6]) # compare with MATLAB (:)'
Elementwise multiplication with the same dimension expansion:
In [10]: X[None,None,:,:]*x[:,:,None,None]
Out[10]:
array([[[[ 3],
[ 6]],
[[ 5],
[10]]],
[[[ 4],
[ 8]],
[[ 6],
[12]]]])
In [11]: _.shape
Out[11]: (2, 2, 2, 1) # compare with octave (1,2,2,2)
In [12]: __.squeeze()
Out[12]:
array([[[ 3, 6],
[ 5, 10]],
[[ 4, 8],
[ 6, 12]]])
the ravel is the same as Octave:
In [13]: ___.ravel()
Out[13]: array([ 3, 6, 5, 10, 4, 8, 6, 12])
expand_dims can be used instead of the indexing. Internally it uses reshape:
In [15]: np.expand_dims(X,(0,1)).shape
Out[15]: (1, 1, 2, 1)
In [16]: np.expand_dims(x,(2,3)).shape
Out[16]: (2, 2, 1, 1)
Related
I am trying to extract a sub-array using logical indexes as,
a = np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]])
a
Out[45]:
array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12],
[13, 14, 15, 16]])
b = np.array([False, True, False, True])
a[b, b]
Out[49]: array([ 6, 16])
python evaluates the logical indexes in b per element of a. However in matlab you can do something like
>> a = [1 2 3 4; 5 6 7 8; 9 10 11 12; 13 14 15 16]
a =
1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16
>> b = [2 4]
b =
2 4
>> a(b, b)
ans =
6 8
14 16
how can I achieve the same result in python without doing,
c = a[:, b]
c[b,:]
Out[51]:
array([[ 6, 8],
[14, 16]])
Numpy supports logical indexing, though it is a little different than what you are familiar in MATLAB. To get the results you want you can do the following:
a[b][:,b] # first brackets isolates the rows, second brackets isolate the columns
Out[27]:
array([[ 6, 8],
[14, 16]])
The more "numpy" method will be understood after you will understand what happend in your case.
b = np.array([False, True, False, True]) is similar to b=np.array([1,3]) and will be easier for me to explain. When writing a[[1,3],[1,3]] what happens is that numpy crates a (2,1) shape array, and places a[1,1] in the [0] location and a[3,3] in the second location. To create an output of shape (2,2), the indexing must have the same dimensionality. Therefore, the following will get your result:
a[[[1,1],[3,3]],[[1,3],[1,3]]]
Out[28]:
array([[ 6, 8],
[14, 16]])
Explanation:
The indexing arrays are:
temp_rows = np.array([[1,1],
[3,3]])
temp_cols = np.array([[1,3],
[1,3])
both arrays have dimensions of (2,2) and therefore, numpy will create an output of shape (2,2). Then, it places a[1,1] in location [0,0], a[1,3] in [0,1], a[3,1] in location [1,0] and a[3,3] in location [1,1]. This can be expanded to any shape but for your purposes, you wanted a shape of (2,2)
After figuring this out, you can make things even simpler by utilizing the fact you if you insert a (2,1) array in the 1st dimension and a (1,2) array in the 2nd dimension, numpy will perform the broadcasting, similar to the MATLAB operation. This means that by using:
temp_rows = np.array([[1],[3]])
temp_cols = np.array([1,3])
you can do:
a[[[1],[3]], [1,3])
Out[29]:
array([[ 6, 8],
[14, 16]])
You could use np.ix_ here.
a[np.ix_(b, b)]
# array([[ 6, 8],
# [14, 16]])
Output returned by np.ix_
>>> np.ix_(b, b)
(array([[1],
[3]]),
array([[1, 3]]))
You could make use of a outer product of the b vector. The new dimesion you can obtain from the number of True values using a sum.
import numpy as np
a = np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]])
b = np.array([False, True, False, True])
#
M = np.outer(b, b)
new_dim = b.sum()
new_shape = (new_dim, new_dim)
selection = a[M].reshape(new_shape)
The result looks like
[[ 6 8]
[14 16]]
I'm a bit new to python and wanted to check for some values in my arrays to see if they go above or below a certain value and adjust them afterwards.
For the case of a 2d array with numpy I found this in some part of its manual.
import numpy as np
arr = np.array([[1, 2, 3, 4], [5, 6, 7, 8]])
for x in np.nditer(arr[:, ::2]):
print(x)
What's the syntax in python to change that initial value so it doesnt iterate over every value starting from the first but from one I can define such iterate from every 2nd or 3rd as I need to check every 1st, 2nd, 3rd and so on value in my arrays against a different value or is there maybe a better way to do this?
I suspect you need to read some more basic numpy docs.
You created a 2d array:
In [5]: arr = np.array([[1, 2, 3, 4], [5, 6, 7, 8]])
In [6]: arr
Out[6]:
array([[1, 2, 3, 4],
[5, 6, 7, 8]])
You can view it as a 1d array (nditer iterates as flat)
In [7]: arr.ravel()
Out[7]: array([1, 2, 3, 4, 5, 6, 7, 8])
You can use standard slice notation to select everyother - on the flattened:
In [8]: arr.ravel()[::2]
Out[8]: array([1, 3, 5, 7])
or every other column of the original:
In [9]: arr[:,::2]
Out[9]:
array([[1, 3],
[5, 7]])
You can test every value, such as for being odd:
In [10]: arr % 2 == 1
Out[10]:
array([[ True, False, True, False],
[ True, False, True, False]])
and use that array to select those values:
In [11]: arr[arr % 2 == 1]
Out[11]: array([1, 3, 5, 7])
or modify them:
In [12]: arr[arr % 2 == 1] += 10
In [13]: arr
Out[13]:
array([[11, 2, 13, 4],
[15, 6, 17, 8]])
The documentation for nditer tends to overhype it. It's useful in compiled code, but rarely useful in python. If the above whole-array methods don't work, you can iterate directly, with more control and understanding:
In [14]: for row in arr:
...: print(row)
...: for x in row:
...: print(x)
[11 2 13 4]
11
2
13
4
[15 6 17 8]
15
6
17
8
Here's what I think axis is and want to know if my understanding is correct
We count opening bracket [ from left indexed from 0 and that's the axis
eg1) [[1,2],[3,4]]
for [ find all elements that has one [ left to it, are the elements for the axis 0. ([[ for axis 1 and so on)
0 axis: you see `[`: [x, y] where x = [1,2], y=[3,4]
1 axis: you see `[[`: [[x, y]] where x = [1,3], y = [2,4]
eg2) [[[1,2,3], [4,5,6]], [[7,8,9], [10,11,12]]]
0 axis: you see `[` [x, y] where x = [[1,2,3], [4,5,6]], y= [[7,8,9], [10,11,12]]
1 axis: you see `[[`, x = [1,2,3], [7,8,9] y = [4,5,6], [10,11,12]
2 axis: you see `[[[`, x = [1,4,7,10] y = [2,5,8,11] z = [3,6,9,12]
If there's a function that takes a value along the axis, I could verify if I'm right, but..
closest thing I found was np.take
For the (2,2) shape array:
In [13]: arr = np.array([[1,2],[3,4]])
In [14]: arr
Out[14]:
array([[1, 2],
[3, 4]])
Python unpacking just iterates on the array - that is on the first dimension, 'rows':
In [15]: x, y = arr
In [16]: x,y
Out[16]: (array([1, 2]), array([3, 4]))
To unpack columns, we could transpose the array, so the 2nd dimension is first. But I think a list comprehension is clearer:
In [17]: x, y = [arr[:,i] for i in range(2)]
In [18]: x,y
Out[18]: (array([1, 3]), array([2, 4]))
For the 3d array:
In [19]: arr = np.arange(1,13).reshape(2,2,3)
In [20]: arr
Out[20]:
array([[[ 1, 2, 3],
[ 4, 5, 6]],
[[ 7, 8, 9],
[10, 11, 12]]])
The results of iterating on the first dimension should be obvious - the 2 blocks or panes.
You got the 2nd dimension right. For the third, the results are 3 (2,2) arrays:
In [21]: x,y,z=[arr[:,:,i] for i in range(arr.shape[2])]
In [22]: x
Out[22]:
array([[ 1, 4],
[ 7, 10]])
In [23]: y
Out[23]:
array([[ 2, 5],
[ 8, 11]])
In [24]: z
Out[24]:
array([[ 3, 6],
[ 9, 12]])
x,y,z= from this transpose would also work:
In [25]: arr.transpose(2,0,1)
Out[25]:
array([[[ 1, 4],
[ 7, 10]],
[[ 2, 5],
[ 8, 11]],
[[ 3, 6],
[ 9, 12]]])
np.take can be used like my indexing:
[np.take(arr,i,2) for i in range(3)]
I have an array that is size (214, 144). I need it to be (214,144,1) is there a way to do this easily in Python? Basically the dimensions are supposed to be (Days, Times, Stations). Since I only have 1 station's data that dimension would be a 1. However if I could also make the code flexible enough work for say 2 stations that would be great (e.g. changing the dimension size from (428,288) to (214,144,2)) that would be great!
You could use reshape:
>>> a = numpy.array([[1,2,3,4,5,6],[7,8,9,10,11,12]])
>>> a.shape
(2, 6)
>>> a.reshape((2, 6, 1))
array([[[ 1],
[ 2],
[ 3],
[ 4],
[ 5],
[ 6]],
[[ 7],
[ 8],
[ 9],
[10],
[11],
[12]]])
>>> _.shape
(2, 6, 1)
Besides changing the shape from (x, y) to (x, y, 1), you could use (x, y/n, n) as well, but you may want to specify the column order depending on the input:
>>> a.reshape((2, 3, 2))
array([[[ 1, 2],
[ 3, 4],
[ 5, 6]],
[[ 7, 8],
[ 9, 10],
[11, 12]]])
>>> a.reshape((2, 3, 2), order='F')
array([[[ 1, 4],
[ 2, 5],
[ 3, 6]],
[[ 7, 10],
[ 8, 11],
[ 9, 12]]])
1) To add a dimension to an array a of arbitrary dimensionality:
b = numpy.reshape (a, list (numpy.shape (a)) + [1])
Explanation:
You get the shape of a, turn it into a list, concatenate 1 to that list, and use that list as the new shape in a reshape operation.
2) To specify subdivisions of the dimensions, and have the size of the last dimension calculated automatically, use -1 for the size of the last dimension. e.g.:
b = numpy.reshape(a, [numpy.size(a,0)/2, numpy.size(a,1)/2, -1])
The shape of b in this case will be [214,144,4].
(obviously you could combine the two approaches if necessary):
b = numpy.reshape (a, numpy.append (numpy.array (numpy.shape (a))/2, -1))
If I have an 2d array such as
A = np.arange(16).reshape(4,4)
How can I select row = [0, 2] and column = [0, 2] using parameters?
In MATLAB, I can simply do A[row, column] but in python this will select 2 elements corresponding to (0,0) and (2,2).
Is there anyway I can do this using some parameters as in MATLAB?
The output should be like
[0 2
8 10]
To select a block of elements - as MATLAB does, the 1st index has to be column vector. There are several ways of doing this:
In [19]: A = np.arange(16).reshape(4,4)
In [20]: row=[0,2];column=[0,2]
In [21]: A[np.ix_(row,column)]
Out[21]:
array([[ 0, 2],
[ 8, 10]])
In [22]: np.ix_(row,column)
Out[22]:
(array([[0],
[2]]), array([[0, 2]]))
In [23]: A[[[0],[2]],[0,2]]
Out[23]:
array([[ 0, 2],
[ 8, 10]])
The other answer uses meshgrid. We could probably list a half dozen variations.
Good documentation in this section:
http://docs.scipy.org/doc/numpy/reference/arrays.indexing.html#purely-integer-array-indexing
You can use the following
A = np.arange(16).reshape(4,4)
print np.ravel(A[row,:][:,column])
to get:
array([ 0, 2, 8, 10])
MATLAB creates a 2D mesh when indexed with vectors across dimensions. So, in MATLAB, you would have -
A =
0 1 2 3
4 5 6 7
8 9 10 11
12 13 14 15
>> row = [1, 3]; column = [1, 3];
>> A(row,column)
ans =
0 2
8 10
Now, in NumPy/Python, indexing with the vectors across dimensions selects the elements after making tuplets from each element in those vectors. To replicate the MATLAB behaviour, you need to create a mesh of such indices from the vectors. For the same, you can use np.meshgrid -
In [18]: A
Out[18]:
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15]])
In [19]: row = [0, 2]; column = [0, 2];
In [20]: C,R = np.meshgrid(row,column)
In [21]: A[R,C]
Out[21]:
array([[ 0, 2],
[ 8, 10]])