reduce even array in half by merging columns - python

Assuming i have the following array:
a = array([[[4, 8, 7, 3, 1, 2],
[3, 1, 8, 7, 1, 9],
[0, 0, 3, 0, 7, 6],
[1, 1, 5, 0, 5, 1],
[1, 6, 7, 0, 6, 2]],
[[8, 1, 1, 0, 0, 0],
[2, 8, 1, 6, 4, 9],
[1, 8, 7, 2, 2, 2],
[6, 6, 2, 6, 0, 5],
[3, 2, 2, 0, 6, 8]],
[[4, 6, 3, 2, 1, 4],
[0, 4, 3, 5, 9, 4],
[1, 4, 6, 7, 2, 4],
[6, 3, 5, 7, 7, 8],
[1, 0, 3, 9, 2, 5]],
[[7, 7, 3, 9, 7, 0],
[8, 5, 1, 4, 3, 9],
[9, 7, 9, 5, 4, 9],
[2, 0, 6, 0, 8, 5],
[4, 4, 4, 7, 5, 2]],
[[4, 0, 8, 2, 1, 0],
[2, 4, 0, 7, 3, 7],
[4, 6, 8, 7, 9, 6],
[3, 2, 7, 5, 2, 3],
[7, 6, 3, 0, 1, 5]]])
Is there an easy way to reduce the column values by summing to get the following array:
b = array([[[12, 10, 3],
[4, 15, 10],
[0, 3, 13],
[2, 5, 6],
[7, 7, 8]],
...])
The first row is achieved by:[4+8, 7+3, 1+2]. I know we can use np.sum to merge columns but I am lost on how to select the right columns to add together. Help is greatly appreciated!


You can do simply this:
a.reshape((5,5,3,2)).sum(axis=-1)
reshape((5,5,3,2)) will 'split' the last dimension into groups of 2, the sum(axis=-1) will sum over that last freshly created dimension.
In the general case (if the dimensions of a change) you can also use
a.reshape(a.shape[:-1]+(-1,2)).sum(axis=-1)


We can do this with simple numpy slicing.
b = a[:,:,::2] + a[:,:,1::2]
This tells us to select the whole array, and at the last dimension select the even columns, then the odd columns, and add them together elementwise.

Related

Swap multiple indices in 2d array

Problem
For the following array:
import numpy as np
arr = np.array([[i for i in range(10)] for j in range(5)])
# arr example
array([[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]])
For each row in arr, I'm attempting to swap n (in this case 2) indices according to some 2d array, such as.
swap = np.random.choice(arr.shape[1], [arr.shape[0], 2], replace=False)
# swap example
array([[8, 1],
[5, 0],
[7, 2],
[9, 4],
[3, 6]])
Question
I tried arr[:, swap] = arr[:, swap[:, ::-1]], but this performs every swap for each row, rather than only swapping indices row by row. The behaviour I am trying to achieve is given below. Is this possible without iterating over swap?
for idx, s in enumerate(swap):
arr[idx, s] = arr[idx, s[::-1]]
# new arr with indices swapped
array([[0, 8, 2, 3, 4, 5, 6, 7, 1, 9],
[5, 1, 2, 3, 4, 0, 6, 7, 8, 9],
[0, 1, 7, 3, 4, 5, 6, 2, 8, 9],
[0, 1, 2, 3, 9, 5, 6, 7, 8, 4],
[0, 1, 2, 6, 4, 5, 3, 7, 8, 9]])

You can to use a "helper" array to index arr. The helper coerces arr into the correct shape.
import numpy as np
arr = np.array([[i for i in range(10)] for j in range(5)])
swap = np.array([[8, 1], [5, 0], [7, 2], [9, 4], [3, 6]])
helper = np.arange(arr.shape[0])[:, None]
# helper is
# array([[0],
# [1],
# [2],
# [3],
# [4]])
# arr[helper] is
# array([[[0, 8, 2, 3, 4, 5, 6, 7, 1, 9]],
# [[5, 1, 2, 3, 4, 0, 6, 7, 8, 9]],
# [[0, 1, 7, 3, 4, 5, 6, 2, 8, 9]],
# [[0, 1, 2, 3, 9, 5, 6, 7, 8, 4]],
# [[0, 1, 2, 6, 4, 5, 3, 7, 8, 9]]])
arr[helper, swap] = arr[helper, swap[:, ::-1]]
# arr is
# array([[0, 8, 2, 3, 4, 5, 6, 7, 1, 9],
# [5, 1, 2, 3, 4, 0, 6, 7, 8, 9],
# [0, 1, 7, 3, 4, 5, 6, 2, 8, 9],
# [0, 1, 2, 3, 9, 5, 6, 7, 8, 4],
# [0, 1, 2, 6, 4, 5, 3, 7, 8, 9]])

Numpy: stack duplicates of matrix in new dimension [duplicate]

This question already has answers here:
How to copy a 2D array into a 3rd dimension, N times?
(7 answers)
Closed 4 years ago.
I have a 3x3 numpy array and I want to create a 3x3xC matrix where the new dimension consists of exact copies of the original 3x3 array. I am sure this is asked somewhere but I couldn't find the best way. I worked out how to do this for a simple 1 dimensional array x:
new_x = np.tile(np.array(x, (C, 1))
which repeats the array, then do:
np.transpose(np.expand_dims(new_x, axis=2),(2,1,0))
which expands the dimension and switches the axis so that the array is repeated in the 3rd dimension (although this works I'm not sure if this is the best way to do it either) - what is the most efficient way to do this for a general n x n numpy array?

For a readonly version, broadcast_to can be used:
In [370]: x = np.arange(9).reshape(3,3)
In [371]: x
Out[371]:
array([[0, 1, 2],
[3, 4, 5],
[6, 7, 8]])
In [372]: x = np.broadcast_to(x[..., None],(3,3,10))
In [373]: x
Out[373]:
array([[[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
[2, 2, 2, 2, 2, 2, 2, 2, 2, 2]],
[[3, 3, 3, 3, 3, 3, 3, 3, 3, 3],
[4, 4, 4, 4, 4, 4, 4, 4, 4, 4],
[5, 5, 5, 5, 5, 5, 5, 5, 5, 5]],
[[6, 6, 6, 6, 6, 6, 6, 6, 6, 6],
[7, 7, 7, 7, 7, 7, 7, 7, 7, 7],
[8, 8, 8, 8, 8, 8, 8, 8, 8, 8]]])
Or with repeat:
In [378]: x=np.repeat(x[...,None],10,2)
In [379]: x
Out[379]:
array([[[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
[2, 2, 2, 2, 2, 2, 2, 2, 2, 2]],
[[3, 3, 3, 3, 3, 3, 3, 3, 3, 3],
[4, 4, 4, 4, 4, 4, 4, 4, 4, 4],
[5, 5, 5, 5, 5, 5, 5, 5, 5, 5]],
[[6, 6, 6, 6, 6, 6, 6, 6, 6, 6],
[7, 7, 7, 7, 7, 7, 7, 7, 7, 7],
[8, 8, 8, 8, 8, 8, 8, 8, 8, 8]]])
This is a larger array, whose elements can be changed individually.

Copying 2D Array into 3D w/ one less row n times

I have a 2D array, and I need to make it into a 3D array - with the next layer starting with the second row of the first layer.
This is my best attempt to visually show what I want to do, with four 'layers':
# original array
dat = np.array([[0, 0, 0, 0, 9]
[1, 1, 1, 1, 9],
[2, 2, 2, 2, 9],
[3, 3, 3, 3, 9],
[4, 4, 4, 4, 9],
[5, 5, 5, 5, 9],
[6, 6, 6, 6, 9],
[7, 7, 7, 7, 9],
[8, 8, 8, 8, 9]], np.int32
)
#dat.shape
#(8, 5)
layers = 4
# new 3d array
# first 'layer'
[0, 0, 0, 0, 9],
[1, 1, 1, 1, 9],
[2, 2, 2, 2, 9],
[3, 3, 3, 3, 9],
[4, 4, 4, 4, 9],
[5, 5, 5, 5, 9]
# second 'layer'
[1, 1, 1, 1, 9],
[2, 2, 2, 2, 9],
[3, 3, 3, 3, 9],
[4, 4, 4, 4, 9],
[5, 5, 5, 5, 9],
[6, 6, 6, 6, 9]
# third 'layer'
[2, 2, 2, 2, 9],
[3, 3, 3, 3, 9],
[4, 4, 4, 4, 9],
[5, 5, 5, 5, 9],
[6, 6, 6, 6, 9],
[7, 7, 7, 7, 9]
# fourth 'layer'
[3, 3, 3, 3, 9],
[4, 4, 4, 4, 9],
[5, 5, 5, 5, 9],
[6, 6, 6, 6, 9],
[7, 7, 7, 7, 9],
[8, 8, 8, 8, 9]
# new shape: (rows, layers, columns)
#dat.shape
#(6, 4, 5)
I realize my visual representation of the layers might not be the way I say it is at the end, but that is the shape that I'm trying to get it in.
Solutions that I've tried include a variation of np.repeat(dat[:, :, np.newaxis], steps, axis=2) but for some reason I struggle once it's more than two dimensions.
Appreciate any help!

Approach #1: Here's one approach using broadcasting -
layers = 4
L = dat.shape[0]-layers+1
out = dat[np.arange(L) + np.arange(layers)[:,None]]
If you actually need a (6,4,5) shaped array, we would need slight modification :
out = dat[np.arange(L)[:,None] + np.arange(layers)]
Approach #2: Here's another with NumPy strides -
strided = np.lib.stride_tricks.as_strided
m,n = dat.strides
N = dat.shape[1]
out = strided(dat, shape = (layers,L,N), strides= (m,N*n,n))
For (6,4,5) shaped output array,
out = strided(dat, shape = (L,layers,N), strides= (N*n,m,n))
Note that this second method would create a view into input array dat and is very efficient to be created. If you need a copy instead, append .copy() at the end : out.copy().
Sample output for (6,4,5) output -
In [267]: out[:,0,:]
Out[267]:
array([[0, 0, 0, 0, 9],
[1, 1, 1, 1, 9],
[2, 2, 2, 2, 9],
[3, 3, 3, 3, 9],
[4, 4, 4, 4, 9],
[5, 5, 5, 5, 9]])
In [268]: out[:,1,:]
Out[268]:
array([[1, 1, 1, 1, 9],
[2, 2, 2, 2, 9],
[3, 3, 3, 3, 9],
[4, 4, 4, 4, 9],
[5, 5, 5, 5, 9],
[6, 6, 6, 6, 9]])
In [269]: out[:,2,:]
Out[269]:
array([[2, 2, 2, 2, 9],
[3, 3, 3, 3, 9],
[4, 4, 4, 4, 9],
[5, 5, 5, 5, 9],
[6, 6, 6, 6, 9],
[7, 7, 7, 7, 9]])
In [270]: out[:,3,:]
Out[270]:
array([[3, 3, 3, 3, 9],
[4, 4, 4, 4, 9],
[5, 5, 5, 5, 9],
[6, 6, 6, 6, 9],
[7, 7, 7, 7, 9],
[8, 8, 8, 8, 9]])

How to get all submatrices of given size in numpy?

For example, x = np.random.randint(low=0, high=10, shape=(6,6)) gives me a 6x6 numpy array:
array([[3, 1, 0, 1, 5, 4],
[2, 9, 9, 4, 8, 8],
[2, 3, 4, 3, 2, 9],
[5, 8, 4, 5, 7, 6],
[3, 0, 8, 1, 8, 0],
[6, 7, 1, 9, 0, 5]])
How can I get a list of, say, all 2x3 submatrices? What about non-overlapping ones?
I could code this in myself, but I'm sure this is a common enough operation that it already exists in numpy, I just can't find it.

Listed in this post is a generic approach to get a list of submatrices with given shape. Based on the order of submatrices being row (C-style) or column major (fortran-way), you would have two choices. Here's the implementation with np.reshape , np.transpose and np.array_split -
def split_submatrix(x,submat_shape,order='C'):
p,q = submat_shape # Store submatrix shape
m,n = x.shape
if np.any(np.mod(x.shape,np.array(submat_shape))!=0):
raise Exception('Input array shape is not divisible by submatrix shape!')
if order == 'C':
x4D = x.reshape(-1,p,n/q,q).transpose(0,2,1,3).reshape(-1,p,q)
return np.array_split(x4D,x.size/(p*q),axis=0)
elif order == 'F':
x2D = x.reshape(-1,n/q,q).transpose(1,0,2).reshape(-1,q)
return np.array_split(x2D,x.size/(p*q),axis=0)
else:
print "Invalid output order."
return x
Sample run with a modified sample input -
In [201]: x
Out[201]:
array([[5, 2, 5, 6, 5, 6, 1, 5],
[1, 1, 8, 4, 4, 5, 2, 5],
[4, 1, 6, 5, 6, 4, 6, 1],
[5, 3, 7, 0, 5, 8, 6, 5],
[7, 7, 0, 6, 5, 2, 5, 4],
[3, 4, 2, 5, 0, 7, 5, 0]])
In [202]: split_submatrix(x,(3,4))
Out[202]:
[array([[[5, 2, 5, 6],
[1, 1, 8, 4],
[4, 1, 6, 5]]]), array([[[5, 6, 1, 5],
[4, 5, 2, 5],
[6, 4, 6, 1]]]), array([[[5, 3, 7, 0],
[7, 7, 0, 6],
[3, 4, 2, 5]]]), array([[[5, 8, 6, 5],
[5, 2, 5, 4],
[0, 7, 5, 0]]])]
In [203]: split_submatrix(x,(3,4),order='F')
Out[203]:
[array([[5, 2, 5, 6],
[1, 1, 8, 4],
[4, 1, 6, 5]]), array([[5, 3, 7, 0],
[7, 7, 0, 6],
[3, 4, 2, 5]]), array([[5, 6, 1, 5],
[4, 5, 2, 5],
[6, 4, 6, 1]]), array([[5, 8, 6, 5],
[5, 2, 5, 4],
[0, 7, 5, 0]])]

Python: what's an elegant way to select specific rows of an 2-d array into a new array?

For example, given a python numpy.ndarray a = array([[1, 2], [3, 4], [5, 6]]), I want to select the 0th and 2nd row of array a into a new array b, such that b becomes array([[1,2],[5,6]].
I need to solution to work on more general problems, where the original 2d array can have more rows and I should be able to select the rows based on some disjoint ranges. In general, I was looking for something like a[i:j] + a[k:p] that works for 1-d list, but it seems 2d-arrays won't add up this way.
update
It seems that I can use vstack((a[i:j], a[k:p])) to get this working, but is there any elegant way to do this?

You can use list indexing:
a[ [0,2], ]
More generally, to select rows i:j and k:p (I'm assuming in the python sense, meaning rows i to j but not including j):
a[ range(i,j) + range(k,p) , ]
Note that the range(i,j) + range(k,p) creates a flat list of [ i, i+1, ..., j-1, k, k+1, ..., p-1 ], which is then used to index the rows of a.

numpy is kind of clever when it comes to indexing. You can give it a list of indexes and it will return the sliced part.
In : a = numpy.array([[i]*10 for i in range(10)])
In : a
Out:
array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
[2, 2, 2, 2, 2, 2, 2, 2, 2, 2],
[3, 3, 3, 3, 3, 3, 3, 3, 3, 3],
[4, 4, 4, 4, 4, 4, 4, 4, 4, 4],
[5, 5, 5, 5, 5, 5, 5, 5, 5, 5],
[6, 6, 6, 6, 6, 6, 6, 6, 6, 6],
[7, 7, 7, 7, 7, 7, 7, 7, 7, 7],
[8, 8, 8, 8, 8, 8, 8, 8, 8, 8],
[9, 9, 9, 9, 9, 9, 9, 9, 9, 9]])
In : a[[0,5,9]]
Out:
array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[5, 5, 5, 5, 5, 5, 5, 5, 5, 5],
[9, 9, 9, 9, 9, 9, 9, 9, 9, 9]])
In : a[range(0,2)+range(5,8)]
Out:
array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
[5, 5, 5, 5, 5, 5, 5, 5, 5, 5],
[6, 6, 6, 6, 6, 6, 6, 6, 6, 6],
[7, 7, 7, 7, 7, 7, 7, 7, 7, 7]])

Categories