Related
i have a doubt. There is an efficient way to sum all neighbors of a numpy matrix without using several conditions?
This is an example:
array([[5, 4, 8, 3, 1, 4, 3, 2, 2, 3],
[2, 7, 4, 5, 8, 5, 4, 7, 1, 1],
[5, 2, 6, 4, 5, 5, 6, 1, 7, 3],
[6, 1, 4, 1, 3, 3, 6, 1, 4, 6],
[6, 3, 5, 7, 3, 8, 5, 4, 7, 8],
[4, 1, 6, 7, 5, 2, 4, 6, 4, 5],
[2, 1, 7, 6, 8, 4, 1, 7, 2, 1],
[6, 8, 8, 2, 8, 8, 1, 1, 3, 4],
[4, 8, 4, 6, 8, 4, 8, 5, 5, 4],
[5, 2, 8, 3, 7, 5, 1, 5, 2, 6]])
When I run m[0][-1] it returns me 3 and not an error, so if I want to add 1 to all neighbors of a value I need to use a lot of conditions because I can't just use m[0][-1] because in this case and in the other cases of the corners it returns me just a " False neighbor"
IIUC, you want to add 1 to each neighbour of a cell with a given value.
For the example, let's add 1 to each cell in the neighborhood of a 7:
from scipy.signal import convolve2d
v = np.array([[1,1,1],[1,0,1],[1,1,1]])
a + convolve2d(a==7, v, mode='same')
output:
array([[6, 5, 9, 3, 1, 4, 4, 3, 3, 3],
[3, 7, 5, 5, 8, 5, 5, 8, 3, 2],
[6, 3, 7, 4, 5, 5, 7, 3, 8, 4],
[6, 1, 5, 2, 4, 3, 6, 3, 6, 8],
[6, 3, 7, 8, 5, 8, 5, 5, 7, 9],
[4, 2, 9, 9, 7, 2, 5, 8, 6, 6],
[2, 2, 8, 8, 9, 4, 2, 7, 3, 1],
[6, 9, 9, 3, 8, 8, 2, 2, 4, 4],
[4, 8, 4, 7, 9, 5, 8, 5, 5, 4],
[5, 2, 8, 4, 7, 6, 1, 5, 2, 6]])
In addition to the good #mozway solution, one very efficient solution is to use the Numba stencil decorator combined with a parallel execution. Here is an example:
import numba as nb
# parallel=True is only useful for quite-big arrays
#nb.njit(parallel=True)
def kernel(v):
cond = np.zeros((v.shape[0]+2, v.shape[1]+2), dtype=np.bool_)
cond[1:-1, 1:-1] = v == 7
res = nb.stencil(lambda c: c[-1,-1]+c[-1,0]+c[-1,1]+c[0,-1]+c[0,1]+c[1,-1]+c[1,0]+c[1,1])(cond)
return v + res[1:-1, 1:-1]
kernel(m)
An even faster solution consist in working in-place (using v += res instead of the return v + res). Here are the performance results for a 2000x2000 integer array on my 6-core machine:
scipy.signal.convolve2d: 124 ms
Numba out-of-place: 20 ms
Numba in-place: 15 ms
Note that the first call to kernel is slower due to the compilation time.
I also got a similar speed-up for smaller arrays (200x200).
Problem
For the following array:
import numpy as np
arr = np.array([[i for i in range(10)] for j in range(5)])
# arr example
array([[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]])
For each row in arr, I'm attempting to swap n (in this case 2) indices according to some 2d array, such as.
swap = np.random.choice(arr.shape[1], [arr.shape[0], 2], replace=False)
# swap example
array([[8, 1],
[5, 0],
[7, 2],
[9, 4],
[3, 6]])
Question
I tried arr[:, swap] = arr[:, swap[:, ::-1]], but this performs every swap for each row, rather than only swapping indices row by row. The behaviour I am trying to achieve is given below. Is this possible without iterating over swap?
for idx, s in enumerate(swap):
arr[idx, s] = arr[idx, s[::-1]]
# new arr with indices swapped
array([[0, 8, 2, 3, 4, 5, 6, 7, 1, 9],
[5, 1, 2, 3, 4, 0, 6, 7, 8, 9],
[0, 1, 7, 3, 4, 5, 6, 2, 8, 9],
[0, 1, 2, 3, 9, 5, 6, 7, 8, 4],
[0, 1, 2, 6, 4, 5, 3, 7, 8, 9]])
You can to use a "helper" array to index arr. The helper coerces arr into the correct shape.
import numpy as np
arr = np.array([[i for i in range(10)] for j in range(5)])
swap = np.array([[8, 1], [5, 0], [7, 2], [9, 4], [3, 6]])
helper = np.arange(arr.shape[0])[:, None]
# helper is
# array([[0],
# [1],
# [2],
# [3],
# [4]])
# arr[helper] is
# array([[[0, 8, 2, 3, 4, 5, 6, 7, 1, 9]],
# [[5, 1, 2, 3, 4, 0, 6, 7, 8, 9]],
# [[0, 1, 7, 3, 4, 5, 6, 2, 8, 9]],
# [[0, 1, 2, 3, 9, 5, 6, 7, 8, 4]],
# [[0, 1, 2, 6, 4, 5, 3, 7, 8, 9]]])
arr[helper, swap] = arr[helper, swap[:, ::-1]]
# arr is
# array([[0, 8, 2, 3, 4, 5, 6, 7, 1, 9],
# [5, 1, 2, 3, 4, 0, 6, 7, 8, 9],
# [0, 1, 7, 3, 4, 5, 6, 2, 8, 9],
# [0, 1, 2, 3, 9, 5, 6, 7, 8, 4],
# [0, 1, 2, 6, 4, 5, 3, 7, 8, 9]])
This question already has answers here:
How to copy a 2D array into a 3rd dimension, N times?
(7 answers)
Closed 4 years ago.
I have a 3x3 numpy array and I want to create a 3x3xC matrix where the new dimension consists of exact copies of the original 3x3 array. I am sure this is asked somewhere but I couldn't find the best way. I worked out how to do this for a simple 1 dimensional array x:
new_x = np.tile(np.array(x, (C, 1))
which repeats the array, then do:
np.transpose(np.expand_dims(new_x, axis=2),(2,1,0))
which expands the dimension and switches the axis so that the array is repeated in the 3rd dimension (although this works I'm not sure if this is the best way to do it either) - what is the most efficient way to do this for a general n x n numpy array?
For a readonly version, broadcast_to can be used:
In [370]: x = np.arange(9).reshape(3,3)
In [371]: x
Out[371]:
array([[0, 1, 2],
[3, 4, 5],
[6, 7, 8]])
In [372]: x = np.broadcast_to(x[..., None],(3,3,10))
In [373]: x
Out[373]:
array([[[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
[2, 2, 2, 2, 2, 2, 2, 2, 2, 2]],
[[3, 3, 3, 3, 3, 3, 3, 3, 3, 3],
[4, 4, 4, 4, 4, 4, 4, 4, 4, 4],
[5, 5, 5, 5, 5, 5, 5, 5, 5, 5]],
[[6, 6, 6, 6, 6, 6, 6, 6, 6, 6],
[7, 7, 7, 7, 7, 7, 7, 7, 7, 7],
[8, 8, 8, 8, 8, 8, 8, 8, 8, 8]]])
Or with repeat:
In [378]: x=np.repeat(x[...,None],10,2)
In [379]: x
Out[379]:
array([[[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
[2, 2, 2, 2, 2, 2, 2, 2, 2, 2]],
[[3, 3, 3, 3, 3, 3, 3, 3, 3, 3],
[4, 4, 4, 4, 4, 4, 4, 4, 4, 4],
[5, 5, 5, 5, 5, 5, 5, 5, 5, 5]],
[[6, 6, 6, 6, 6, 6, 6, 6, 6, 6],
[7, 7, 7, 7, 7, 7, 7, 7, 7, 7],
[8, 8, 8, 8, 8, 8, 8, 8, 8, 8]]])
This is a larger array, whose elements can be changed individually.
I have a 2D array, and I need to make it into a 3D array - with the next layer starting with the second row of the first layer.
This is my best attempt to visually show what I want to do, with four 'layers':
# original array
dat = np.array([[0, 0, 0, 0, 9]
[1, 1, 1, 1, 9],
[2, 2, 2, 2, 9],
[3, 3, 3, 3, 9],
[4, 4, 4, 4, 9],
[5, 5, 5, 5, 9],
[6, 6, 6, 6, 9],
[7, 7, 7, 7, 9],
[8, 8, 8, 8, 9]], np.int32
)
#dat.shape
#(8, 5)
layers = 4
# new 3d array
# first 'layer'
[0, 0, 0, 0, 9],
[1, 1, 1, 1, 9],
[2, 2, 2, 2, 9],
[3, 3, 3, 3, 9],
[4, 4, 4, 4, 9],
[5, 5, 5, 5, 9]
# second 'layer'
[1, 1, 1, 1, 9],
[2, 2, 2, 2, 9],
[3, 3, 3, 3, 9],
[4, 4, 4, 4, 9],
[5, 5, 5, 5, 9],
[6, 6, 6, 6, 9]
# third 'layer'
[2, 2, 2, 2, 9],
[3, 3, 3, 3, 9],
[4, 4, 4, 4, 9],
[5, 5, 5, 5, 9],
[6, 6, 6, 6, 9],
[7, 7, 7, 7, 9]
# fourth 'layer'
[3, 3, 3, 3, 9],
[4, 4, 4, 4, 9],
[5, 5, 5, 5, 9],
[6, 6, 6, 6, 9],
[7, 7, 7, 7, 9],
[8, 8, 8, 8, 9]
# new shape: (rows, layers, columns)
#dat.shape
#(6, 4, 5)
I realize my visual representation of the layers might not be the way I say it is at the end, but that is the shape that I'm trying to get it in.
Solutions that I've tried include a variation of np.repeat(dat[:, :, np.newaxis], steps, axis=2) but for some reason I struggle once it's more than two dimensions.
Appreciate any help!
Approach #1: Here's one approach using broadcasting -
layers = 4
L = dat.shape[0]-layers+1
out = dat[np.arange(L) + np.arange(layers)[:,None]]
If you actually need a (6,4,5) shaped array, we would need slight modification :
out = dat[np.arange(L)[:,None] + np.arange(layers)]
Approach #2: Here's another with NumPy strides -
strided = np.lib.stride_tricks.as_strided
m,n = dat.strides
N = dat.shape[1]
out = strided(dat, shape = (layers,L,N), strides= (m,N*n,n))
For (6,4,5) shaped output array,
out = strided(dat, shape = (L,layers,N), strides= (N*n,m,n))
Note that this second method would create a view into input array dat and is very efficient to be created. If you need a copy instead, append .copy() at the end : out.copy().
Sample output for (6,4,5) output -
In [267]: out[:,0,:]
Out[267]:
array([[0, 0, 0, 0, 9],
[1, 1, 1, 1, 9],
[2, 2, 2, 2, 9],
[3, 3, 3, 3, 9],
[4, 4, 4, 4, 9],
[5, 5, 5, 5, 9]])
In [268]: out[:,1,:]
Out[268]:
array([[1, 1, 1, 1, 9],
[2, 2, 2, 2, 9],
[3, 3, 3, 3, 9],
[4, 4, 4, 4, 9],
[5, 5, 5, 5, 9],
[6, 6, 6, 6, 9]])
In [269]: out[:,2,:]
Out[269]:
array([[2, 2, 2, 2, 9],
[3, 3, 3, 3, 9],
[4, 4, 4, 4, 9],
[5, 5, 5, 5, 9],
[6, 6, 6, 6, 9],
[7, 7, 7, 7, 9]])
In [270]: out[:,3,:]
Out[270]:
array([[3, 3, 3, 3, 9],
[4, 4, 4, 4, 9],
[5, 5, 5, 5, 9],
[6, 6, 6, 6, 9],
[7, 7, 7, 7, 9],
[8, 8, 8, 8, 9]])
For example, x = np.random.randint(low=0, high=10, shape=(6,6)) gives me a 6x6 numpy array:
array([[3, 1, 0, 1, 5, 4],
[2, 9, 9, 4, 8, 8],
[2, 3, 4, 3, 2, 9],
[5, 8, 4, 5, 7, 6],
[3, 0, 8, 1, 8, 0],
[6, 7, 1, 9, 0, 5]])
How can I get a list of, say, all 2x3 submatrices? What about non-overlapping ones?
I could code this in myself, but I'm sure this is a common enough operation that it already exists in numpy, I just can't find it.
Listed in this post is a generic approach to get a list of submatrices with given shape. Based on the order of submatrices being row (C-style) or column major (fortran-way), you would have two choices. Here's the implementation with np.reshape , np.transpose and np.array_split -
def split_submatrix(x,submat_shape,order='C'):
p,q = submat_shape # Store submatrix shape
m,n = x.shape
if np.any(np.mod(x.shape,np.array(submat_shape))!=0):
raise Exception('Input array shape is not divisible by submatrix shape!')
if order == 'C':
x4D = x.reshape(-1,p,n/q,q).transpose(0,2,1,3).reshape(-1,p,q)
return np.array_split(x4D,x.size/(p*q),axis=0)
elif order == 'F':
x2D = x.reshape(-1,n/q,q).transpose(1,0,2).reshape(-1,q)
return np.array_split(x2D,x.size/(p*q),axis=0)
else:
print "Invalid output order."
return x
Sample run with a modified sample input -
In [201]: x
Out[201]:
array([[5, 2, 5, 6, 5, 6, 1, 5],
[1, 1, 8, 4, 4, 5, 2, 5],
[4, 1, 6, 5, 6, 4, 6, 1],
[5, 3, 7, 0, 5, 8, 6, 5],
[7, 7, 0, 6, 5, 2, 5, 4],
[3, 4, 2, 5, 0, 7, 5, 0]])
In [202]: split_submatrix(x,(3,4))
Out[202]:
[array([[[5, 2, 5, 6],
[1, 1, 8, 4],
[4, 1, 6, 5]]]), array([[[5, 6, 1, 5],
[4, 5, 2, 5],
[6, 4, 6, 1]]]), array([[[5, 3, 7, 0],
[7, 7, 0, 6],
[3, 4, 2, 5]]]), array([[[5, 8, 6, 5],
[5, 2, 5, 4],
[0, 7, 5, 0]]])]
In [203]: split_submatrix(x,(3,4),order='F')
Out[203]:
[array([[5, 2, 5, 6],
[1, 1, 8, 4],
[4, 1, 6, 5]]), array([[5, 3, 7, 0],
[7, 7, 0, 6],
[3, 4, 2, 5]]), array([[5, 6, 1, 5],
[4, 5, 2, 5],
[6, 4, 6, 1]]), array([[5, 8, 6, 5],
[5, 2, 5, 4],
[0, 7, 5, 0]])]