Related
I have a large number of 3d numpy arrays, which when assembled together, form a single contiguous 3d dataset*. However, the arrays were created by breaking the larger space into chunks. I need to assemble the chunk arrays back together. To simplify the problem, I've reduced it to the following example, with four chunks, each of which has 2x2x2 values.
So I have:
yellow_chunk = np.array([[[1,2], [5,6]], [[17,18], [21,22]]])
green_chunk = np.array([[[3,4], [7,8]], [[19,20], [23,24]]])
blue_chunk = np.array([[[9,10], [13,14]], [[25,26], [29,30]]])
red_chunk = np.array([[[11,12], [15,16]], [[27,28], [31,32]]])
And I want to end up with:
>>> output
array([[[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12],
[13, 14, 15, 16]],
[[17, 18, 19, 20],
[21, 22, 23, 24],
[25, 26, 27, 28],
[29, 30, 31, 32]]])
Illustration for this small example:
Things I've tried
concatenate
>>> np.concatenate([yellow_chunk,green_chunk,blue_chunk,red_chunk],-1)
array([[[ 1, 2, 3, 4, 9, 10, 11, 12],
[ 5, 6, 7, 8, 13, 14, 15, 16]],
[[17, 18, 19, 20, 25, 26, 27, 28],
[21, 22, 23, 24, 29, 30, 31, 32]]])
This was close, but the shape is wrong: 8x2x2 instead of the 4x2x4 I need.
hstack
>>> np.hstack([yellow_chunk,green_chunk,blue_chunk,red_chunk])
array([[[ 1, 2],
[ 5, 6],
[ 3, 4],
[ 7, 8],
[ 9, 10],
[13, 14],
[11, 12],
[15, 16]],
[[17, 18],
[21, 22],
[19, 20],
[23, 24],
[25, 26],
[29, 30],
[27, 28],
[31, 32]]])
Also the wrong shape.
vstack
>>> np.vstack([yellow_chunk,green_chunk,blue_chunk,red_chunk])
array([[[ 1, 2],
[ 5, 6]],
[[17, 18],
[21, 22]],
[[ 3, 4],
[ 7, 8]],
[[19, 20],
[23, 24]],
[[ 9, 10],
[13, 14]],
[[25, 26],
[29, 30]],
[[11, 12],
[15, 16]],
[[27, 28],
[31, 32]]])
Wrong shape and order.
dstack
>>> np.dstack([yellow_chunk,green_chunk,blue_chunk,red_chunk])
array([[[ 1, 2, 3, 4, 9, 10, 11, 12],
[ 5, 6, 7, 8, 13, 14, 15, 16]],
[[17, 18, 19, 20, 25, 26, 27, 28],
[21, 22, 23, 24, 29, 30, 31, 32]]])
Wrong shape and order.
* In reality, I have 16x16 chunks, each of which has a shape of 16x128x16. So I'm stitching together "rows" of 256 values rather than the 4-value rows that I have in my small example above.
np.block([[yellow_chunk, green_chunk], [blue_chunk, red_chunk]])
>>>
[[[ 1 2 3 4]
[ 5 6 7 8]
[ 9 10 11 12]
[13 14 15 16]]
[[17 18 19 20]
[21 22 23 24]
[25 26 27 28]
[29 30 31 32]]]
What you are doing here is assembling an nd-array from nested lists of blocks.
If you want more information about joining arrays, you can read this numpy.org doc on all the relevant methods and functions useable.
Simply this for example:
np.hstack((np.dstack((y,g)), np.dstack((b,r))))
(renaming yellow_chunk to y and so on)
I have a multidimensional array of shape (n,x,y). For this example can use this array
A = array([[[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11]],
[[12, 13, 14],
[15, 16, 17],
[18, 19, 20],
[21, 22, 23]],
[[24, 25, 26],
[27, 28, 29],
[30, 31, 32],
[33, 34, 35]]])
I then have another multidimensional array that has index values that I want to use on the original array, A. This has shape (z,2) and the values represent row values index’s
Row_values = array([[0,1],
[0,2],
[1,2],
[1,3]])
So I want to use all the index values in row_values to apply to each of the three arrays in A so I end up with a final array of shape (12,2,3)
Result = ([[[0,1,2],
[3,4,5]],
[[0,1,2],
[6,7,8]],
[[3,4,5],
[6,7,8]]
[[3,4,5],
[9,10,11],
[[12,13,14],
[15,16,17]],
[[12,13,14],
[18,19,20]],
[[15,16,17],
[18,19,20]],
[[15,16,17],
[21,22,23]],
[[24,25,26],
[27,28,29]],
[[24,25,26],
[30,31,32]],
[[27,28,29],
[30,31,32]],
[[27,28,29],
[33,34,35]]]
I have tried using np.take() but haven’t been able to make it work. Not sure if there’s another numpy function that is easier to use
We can advantage of NumPy's advanced indexing and using np.repeat and np.tile along with it.
cidx = np.tile(Row_values, (A.shape[0], 1))
ridx = np.repeat(np.arange(A.shape[0]), Row_values.shape[0])
out = A[ridx[:, None], cidx]
# out.shape -> (12, 2, 3)
Using np.take
np.take(A, Row_values, axis=1).reshape((-1, 2, 3))
# Or
A[:, Row_values].reshape((-1, 2, 3))
Output:
array([[[ 0, 1, 2],
[ 3, 4, 5]],
[[ 0, 1, 2],
[ 6, 7, 8]],
[[ 3, 4, 5],
[ 6, 7, 8]],
[[ 3, 4, 5],
[ 9, 10, 11]],
[[12, 13, 14],
[15, 16, 17]],
[[12, 13, 14],
[18, 19, 20]],
[[15, 16, 17],
[18, 19, 20]],
[[15, 16, 17],
[21, 22, 23]],
[[24, 25, 26],
[27, 28, 29]],
[[24, 25, 26],
[30, 31, 32]],
[[27, 28, 29],
[30, 31, 32]],
[[27, 28, 29],
[33, 34, 35]]])
I have an array like matrix using numpy like this.
import numpy as np
a = np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16], [17, 18, 19, 20]])
the desired array is like this:
array([[ 9, 10, 11, 12],
[13, 14, 15, 16],
[17, 18, 19, 20],
[ 1, 2, 3, 4],
[ 5, 6, 7, 8],])
description: first and second arrays move to the end of the matrix.
I tried something with changing a to a list and used append and del functions and then convert it to a numpy array but it could not be something good to write in python.
is there any function to replace an array position in a larger array-like matrix in numpy?
Function that takes the number of rotations
In [5]: a
Out[5]:
array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12],
[13, 14, 15, 16],
[17, 18, 19, 20]])
In [14]: def rotate(n):
...: n = n%len(a)
...: return np.concatenate([a[n:], a[:n]])
In [13]: rotate(2)
Out[13]:
array([[ 9, 10, 11, 12],
[13, 14, 15, 16],
[17, 18, 19, 20],
[ 1, 2, 3, 4],
[ 5, 6, 7, 8]])
What if you give n more than the length of the array? It's handled - n = n%len(a)
In [16]: rotate(9)
Out[16]:
array([[17, 18, 19, 20],
[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12],
[13, 14, 15, 16]])
Another solution given in comments is roll() method.
In [6]: a
Out[6]:
array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12],
[13, 14, 15, 16],
[17, 18, 19, 20]])
In [7]: def rotate(n):
...: n = n % len(a)
...: return np.roll(a,-n,axis=0)
...:
In [8]: rotate(8)
Out[8]:
array([[13, 14, 15, 16],
[17, 18, 19, 20],
[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12]])
In [9]: rotate(2)
Out[9]:
array([[ 9, 10, 11, 12],
[13, 14, 15, 16],
[17, 18, 19, 20],
[ 1, 2, 3, 4],
[ 5, 6, 7, 8]])
This is so easy if you use this simple line of code. no function and other things are needed.
simply use numpy.roll. see explanations here.
# Assume your matrix is named a.
>>> a
array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12],
[13, 14, 15, 16],
[17, 18, 19, 20]])
>>> np.roll(a,-(n % len(a)),axis=0)
array([[ 9, 10, 11, 12],
[13, 14, 15, 16],
[17, 18, 19, 20],
[ 1, 2, 3, 4],
[ 5, 6, 7, 8]])
i have a 1d np array "array1d" and a 3d np array "array3d", i want to sum them so the n'th value in "array1d" will be added to each of the elements of the n'th plane in array3d.
this can be done in the following loop
for i, value in enumerate(array1d):
array3d[i] += value
question is, how can this be done in a single numpy line?
example arrays:
arr1d = np.array(range(3))
>>>array([0, 1, 2])
arr3d = np.array(range(27)).reshape(3, 3, 3)
>>>array([[[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8]],
[[ 9, 10, 11],
[12, 13, 14],
[15, 16, 17]],
[[18, 19, 20],
[21, 22, 23],
[24, 25, 26]]])
wanted result:
array([[[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8]],
[[ 10, 11, 12],
[13, 14, 15],
[16, 17, 18]],
[[20, 21, 22],
[23, 24, 25],
[26, 27, 28]]])
Use Numpy's broadcasting features:
In [23]: arr1d[:, None, None] + arr3d
Out[23]:
array([[[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8]],
[[10, 11, 12],
[13, 14, 15],
[16, 17, 18]],
[[20, 21, 22],
[23, 24, 25],
[26, 27, 28]]])
This basically copies the content of arr1d across the other two dimensions (without actually copying, it just provides a view of the memory which looks like it). Instead of None, you can also use numpy.newaxis.
Alternatively, you can also use reshape:
In [32]: arr1d.reshape(3, 1, 1) + arr3d
Out[32]:
array([[[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8]],
[[10, 11, 12],
[13, 14, 15],
[16, 17, 18]],
[[20, 21, 22],
[23, 24, 25],
[26, 27, 28]]])
I have an array of numbers whose shape is 26*43264. I would like to reshape this into an array of shape 208*208 but in chunks of 26*26.
[[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[10,11,12,13,14,15,16,17,18,19]]
becomes something like:
[[0, 1, 2, 3, 4],
[10,11,12,13,14],
[ 5, 6, 7, 8, 9],
[15,16,17,18,19]]
This kind of reshaping question has come up before. But rather than search I'll quickly demonstate a numpy approach
make your sample array:
In [473]: x=np.arange(20).reshape(2,10)
In [474]: x
Out[474]:
array([[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14, 15, 16, 17, 18, 19]])
Use reshape to split it into blocks of 5
In [475]: x.reshape(2,2,5)
Out[475]:
array([[[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9]],
[[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19]]])
and use transpose to reorder dimensions, and in effect reorder those rows
In [476]: x.reshape(2,2,5).transpose(1,0,2)
Out[476]:
array([[[ 0, 1, 2, 3, 4],
[10, 11, 12, 13, 14]],
[[ 5, 6, 7, 8, 9],
[15, 16, 17, 18, 19]]])
and another shape to consolidate the 1st 2 dimensions
In [477]: x.reshape(2,2,5).transpose(1,0,2).reshape(4,5)
Out[477]:
array([[ 0, 1, 2, 3, 4],
[10, 11, 12, 13, 14],
[ 5, 6, 7, 8, 9],
[15, 16, 17, 18, 19]])
If x is already a numpy array, these transpose and reshape operations are cheap (time wise). If x was really nested lists, then the other solution with list operations will be faster, since making a numpy array has overhead.
A little ugly, but here's a one-liner for the small example that you should be able to modify for the full size one:
In [29]: from itertools import chain
In [30]: np.array(list(chain(*[np.arange(20).reshape(4,5)[i::2] for i in xrange(2)])))
Out[30]:
array([[ 0, 1, 2, 3, 4],
[10, 11, 12, 13, 14],
[ 5, 6, 7, 8, 9],
[15, 16, 17, 18, 19]])
EDIT: Here's a more generalized version in a function. Uglier code, but the function just takes an array and a number of segments you'd like to end up with.
In [57]: def break_arr(arr, chunks):
....: to_take = arr.shape[1]/chunks
....: return np.array(list(chain(*[arr.take(xrange(x*to_take, x*to_take+to_take), axis=1) for x in xrange(chunks)])))
....:
In [58]: arr = np.arange(40).reshape(4,10)
In [59]: break_arr(arr, 5)
Out[59]:
array([[ 0, 1],
[10, 11],
[20, 21],
[30, 31],
[ 2, 3],
[12, 13],
[22, 23],
[32, 33],
[ 4, 5],
[14, 15],
[24, 25],
[34, 35],
[ 6, 7],
[16, 17],
[26, 27],
[36, 37],
[ 8, 9],
[18, 19],
[28, 29],
[38, 39]])
In [60]: break_arr(arr, 2)
Out[60]:
array([[ 0, 1, 2, 3, 4],
[10, 11, 12, 13, 14],
[20, 21, 22, 23, 24],
[30, 31, 32, 33, 34],
[ 5, 6, 7, 8, 9],
[15, 16, 17, 18, 19],
[25, 26, 27, 28, 29],
[35, 36, 37, 38, 39]])