This question already has answers here:
How to copy a 2D array into a 3rd dimension, N times?
(7 answers)
Closed 4 years ago.
I have a 3x3 numpy array and I want to create a 3x3xC matrix where the new dimension consists of exact copies of the original 3x3 array. I am sure this is asked somewhere but I couldn't find the best way. I worked out how to do this for a simple 1 dimensional array x:
new_x = np.tile(np.array(x, (C, 1))
which repeats the array, then do:
np.transpose(np.expand_dims(new_x, axis=2),(2,1,0))
which expands the dimension and switches the axis so that the array is repeated in the 3rd dimension (although this works I'm not sure if this is the best way to do it either) - what is the most efficient way to do this for a general n x n numpy array?
For a readonly version, broadcast_to can be used:
In [370]: x = np.arange(9).reshape(3,3)
In [371]: x
Out[371]:
array([[0, 1, 2],
[3, 4, 5],
[6, 7, 8]])
In [372]: x = np.broadcast_to(x[..., None],(3,3,10))
In [373]: x
Out[373]:
array([[[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
[2, 2, 2, 2, 2, 2, 2, 2, 2, 2]],
[[3, 3, 3, 3, 3, 3, 3, 3, 3, 3],
[4, 4, 4, 4, 4, 4, 4, 4, 4, 4],
[5, 5, 5, 5, 5, 5, 5, 5, 5, 5]],
[[6, 6, 6, 6, 6, 6, 6, 6, 6, 6],
[7, 7, 7, 7, 7, 7, 7, 7, 7, 7],
[8, 8, 8, 8, 8, 8, 8, 8, 8, 8]]])
Or with repeat:
In [378]: x=np.repeat(x[...,None],10,2)
In [379]: x
Out[379]:
array([[[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
[2, 2, 2, 2, 2, 2, 2, 2, 2, 2]],
[[3, 3, 3, 3, 3, 3, 3, 3, 3, 3],
[4, 4, 4, 4, 4, 4, 4, 4, 4, 4],
[5, 5, 5, 5, 5, 5, 5, 5, 5, 5]],
[[6, 6, 6, 6, 6, 6, 6, 6, 6, 6],
[7, 7, 7, 7, 7, 7, 7, 7, 7, 7],
[8, 8, 8, 8, 8, 8, 8, 8, 8, 8]]])
This is a larger array, whose elements can be changed individually.
Related
Assuming i have the following array:
a = array([[[4, 8, 7, 3, 1, 2],
[3, 1, 8, 7, 1, 9],
[0, 0, 3, 0, 7, 6],
[1, 1, 5, 0, 5, 1],
[1, 6, 7, 0, 6, 2]],
[[8, 1, 1, 0, 0, 0],
[2, 8, 1, 6, 4, 9],
[1, 8, 7, 2, 2, 2],
[6, 6, 2, 6, 0, 5],
[3, 2, 2, 0, 6, 8]],
[[4, 6, 3, 2, 1, 4],
[0, 4, 3, 5, 9, 4],
[1, 4, 6, 7, 2, 4],
[6, 3, 5, 7, 7, 8],
[1, 0, 3, 9, 2, 5]],
[[7, 7, 3, 9, 7, 0],
[8, 5, 1, 4, 3, 9],
[9, 7, 9, 5, 4, 9],
[2, 0, 6, 0, 8, 5],
[4, 4, 4, 7, 5, 2]],
[[4, 0, 8, 2, 1, 0],
[2, 4, 0, 7, 3, 7],
[4, 6, 8, 7, 9, 6],
[3, 2, 7, 5, 2, 3],
[7, 6, 3, 0, 1, 5]]])
Is there an easy way to reduce the column values by summing to get the following array:
b = array([[[12, 10, 3],
[4, 15, 10],
[0, 3, 13],
[2, 5, 6],
[7, 7, 8]],
...])
The first row is achieved by:[4+8, 7+3, 1+2]. I know we can use np.sum to merge columns but I am lost on how to select the right columns to add together. Help is greatly appreciated!
You can do simply this:
a.reshape((5,5,3,2)).sum(axis=-1)
reshape((5,5,3,2)) will 'split' the last dimension into groups of 2, the sum(axis=-1) will sum over that last freshly created dimension.
In the general case (if the dimensions of a change) you can also use
a.reshape(a.shape[:-1]+(-1,2)).sum(axis=-1)
We can do this with simple numpy slicing.
b = a[:,:,::2] + a[:,:,1::2]
This tells us to select the whole array, and at the last dimension select the even columns, then the odd columns, and add them together elementwise.
Problem
For the following array:
import numpy as np
arr = np.array([[i for i in range(10)] for j in range(5)])
# arr example
array([[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]])
For each row in arr, I'm attempting to swap n (in this case 2) indices according to some 2d array, such as.
swap = np.random.choice(arr.shape[1], [arr.shape[0], 2], replace=False)
# swap example
array([[8, 1],
[5, 0],
[7, 2],
[9, 4],
[3, 6]])
Question
I tried arr[:, swap] = arr[:, swap[:, ::-1]], but this performs every swap for each row, rather than only swapping indices row by row. The behaviour I am trying to achieve is given below. Is this possible without iterating over swap?
for idx, s in enumerate(swap):
arr[idx, s] = arr[idx, s[::-1]]
# new arr with indices swapped
array([[0, 8, 2, 3, 4, 5, 6, 7, 1, 9],
[5, 1, 2, 3, 4, 0, 6, 7, 8, 9],
[0, 1, 7, 3, 4, 5, 6, 2, 8, 9],
[0, 1, 2, 3, 9, 5, 6, 7, 8, 4],
[0, 1, 2, 6, 4, 5, 3, 7, 8, 9]])
You can to use a "helper" array to index arr. The helper coerces arr into the correct shape.
import numpy as np
arr = np.array([[i for i in range(10)] for j in range(5)])
swap = np.array([[8, 1], [5, 0], [7, 2], [9, 4], [3, 6]])
helper = np.arange(arr.shape[0])[:, None]
# helper is
# array([[0],
# [1],
# [2],
# [3],
# [4]])
# arr[helper] is
# array([[[0, 8, 2, 3, 4, 5, 6, 7, 1, 9]],
# [[5, 1, 2, 3, 4, 0, 6, 7, 8, 9]],
# [[0, 1, 7, 3, 4, 5, 6, 2, 8, 9]],
# [[0, 1, 2, 3, 9, 5, 6, 7, 8, 4]],
# [[0, 1, 2, 6, 4, 5, 3, 7, 8, 9]]])
arr[helper, swap] = arr[helper, swap[:, ::-1]]
# arr is
# array([[0, 8, 2, 3, 4, 5, 6, 7, 1, 9],
# [5, 1, 2, 3, 4, 0, 6, 7, 8, 9],
# [0, 1, 7, 3, 4, 5, 6, 2, 8, 9],
# [0, 1, 2, 3, 9, 5, 6, 7, 8, 4],
# [0, 1, 2, 6, 4, 5, 3, 7, 8, 9]])
I can use numpy.mgrid as follows:
a = numpy.mgrid[x0:x1, y0:y1] # 2 dimensional
b = numpy.mgrid[x0:x1, y0:y1, z0:z1] # 3 dimensional
Now, I'd like to create the expression in brackets programmatically, because I do not know whether I have 1, 2, 3 or more dimensions. I'm looking for something like:
shape = np.array([[x0, x1], [y0, y1], ... maybe more dimensions ...])
idx = (s[0]:s[1] for s in shape)
a = numpy.mgrid[idx]
That gives at least a syntax error in the second line. How can I properly generate those indices/slices programmatically? (The mgrid here is rather an example/use case, the question is really about indexing in general.)
Use the slice object. For example:
shape = np.array([[0, 10], [0, 10]])
idx = tuple(slice(s[0],s[1], 1) for s in shape)
#yields the following
#(slice(0, 10, 1), slice(0, 10, 1))
np.mgrid[idx]
yields
array([[[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
[2, 2, 2, 2, 2, 2, 2, 2, 2, 2],
[3, 3, 3, 3, 3, 3, 3, 3, 3, 3],
[4, 4, 4, 4, 4, 4, 4, 4, 4, 4],
[5, 5, 5, 5, 5, 5, 5, 5, 5, 5],
[6, 6, 6, 6, 6, 6, 6, 6, 6, 6],
[7, 7, 7, 7, 7, 7, 7, 7, 7, 7],
[8, 8, 8, 8, 8, 8, 8, 8, 8, 8],
[9, 9, 9, 9, 9, 9, 9, 9, 9, 9]],
[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]]])
Alternatively, you could use the Numpy shorthand np.s_, e.g. np.s_[0:10:1], instead of slice(1, 10, 1), but they are equivalent objects.
For example, x = np.random.randint(low=0, high=10, shape=(6,6)) gives me a 6x6 numpy array:
array([[3, 1, 0, 1, 5, 4],
[2, 9, 9, 4, 8, 8],
[2, 3, 4, 3, 2, 9],
[5, 8, 4, 5, 7, 6],
[3, 0, 8, 1, 8, 0],
[6, 7, 1, 9, 0, 5]])
How can I get a list of, say, all 2x3 submatrices? What about non-overlapping ones?
I could code this in myself, but I'm sure this is a common enough operation that it already exists in numpy, I just can't find it.
Listed in this post is a generic approach to get a list of submatrices with given shape. Based on the order of submatrices being row (C-style) or column major (fortran-way), you would have two choices. Here's the implementation with np.reshape , np.transpose and np.array_split -
def split_submatrix(x,submat_shape,order='C'):
p,q = submat_shape # Store submatrix shape
m,n = x.shape
if np.any(np.mod(x.shape,np.array(submat_shape))!=0):
raise Exception('Input array shape is not divisible by submatrix shape!')
if order == 'C':
x4D = x.reshape(-1,p,n/q,q).transpose(0,2,1,3).reshape(-1,p,q)
return np.array_split(x4D,x.size/(p*q),axis=0)
elif order == 'F':
x2D = x.reshape(-1,n/q,q).transpose(1,0,2).reshape(-1,q)
return np.array_split(x2D,x.size/(p*q),axis=0)
else:
print "Invalid output order."
return x
Sample run with a modified sample input -
In [201]: x
Out[201]:
array([[5, 2, 5, 6, 5, 6, 1, 5],
[1, 1, 8, 4, 4, 5, 2, 5],
[4, 1, 6, 5, 6, 4, 6, 1],
[5, 3, 7, 0, 5, 8, 6, 5],
[7, 7, 0, 6, 5, 2, 5, 4],
[3, 4, 2, 5, 0, 7, 5, 0]])
In [202]: split_submatrix(x,(3,4))
Out[202]:
[array([[[5, 2, 5, 6],
[1, 1, 8, 4],
[4, 1, 6, 5]]]), array([[[5, 6, 1, 5],
[4, 5, 2, 5],
[6, 4, 6, 1]]]), array([[[5, 3, 7, 0],
[7, 7, 0, 6],
[3, 4, 2, 5]]]), array([[[5, 8, 6, 5],
[5, 2, 5, 4],
[0, 7, 5, 0]]])]
In [203]: split_submatrix(x,(3,4),order='F')
Out[203]:
[array([[5, 2, 5, 6],
[1, 1, 8, 4],
[4, 1, 6, 5]]), array([[5, 3, 7, 0],
[7, 7, 0, 6],
[3, 4, 2, 5]]), array([[5, 6, 1, 5],
[4, 5, 2, 5],
[6, 4, 6, 1]]), array([[5, 8, 6, 5],
[5, 2, 5, 4],
[0, 7, 5, 0]])]
For example, given a python numpy.ndarray a = array([[1, 2], [3, 4], [5, 6]]), I want to select the 0th and 2nd row of array a into a new array b, such that b becomes array([[1,2],[5,6]].
I need to solution to work on more general problems, where the original 2d array can have more rows and I should be able to select the rows based on some disjoint ranges. In general, I was looking for something like a[i:j] + a[k:p] that works for 1-d list, but it seems 2d-arrays won't add up this way.
update
It seems that I can use vstack((a[i:j], a[k:p])) to get this working, but is there any elegant way to do this?
You can use list indexing:
a[ [0,2], ]
More generally, to select rows i:j and k:p (I'm assuming in the python sense, meaning rows i to j but not including j):
a[ range(i,j) + range(k,p) , ]
Note that the range(i,j) + range(k,p) creates a flat list of [ i, i+1, ..., j-1, k, k+1, ..., p-1 ], which is then used to index the rows of a.
numpy is kind of clever when it comes to indexing. You can give it a list of indexes and it will return the sliced part.
In : a = numpy.array([[i]*10 for i in range(10)])
In : a
Out:
array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
[2, 2, 2, 2, 2, 2, 2, 2, 2, 2],
[3, 3, 3, 3, 3, 3, 3, 3, 3, 3],
[4, 4, 4, 4, 4, 4, 4, 4, 4, 4],
[5, 5, 5, 5, 5, 5, 5, 5, 5, 5],
[6, 6, 6, 6, 6, 6, 6, 6, 6, 6],
[7, 7, 7, 7, 7, 7, 7, 7, 7, 7],
[8, 8, 8, 8, 8, 8, 8, 8, 8, 8],
[9, 9, 9, 9, 9, 9, 9, 9, 9, 9]])
In : a[[0,5,9]]
Out:
array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[5, 5, 5, 5, 5, 5, 5, 5, 5, 5],
[9, 9, 9, 9, 9, 9, 9, 9, 9, 9]])
In : a[range(0,2)+range(5,8)]
Out:
array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
[5, 5, 5, 5, 5, 5, 5, 5, 5, 5],
[6, 6, 6, 6, 6, 6, 6, 6, 6, 6],
[7, 7, 7, 7, 7, 7, 7, 7, 7, 7]])