I want to pad:
[[1, 2]
[3, 4]]
to
[[0, 0, 0]
[0, 1, 2]
[0, 3, 4]]
I have no problem in doing so when input is 2D by vstack & hsrack
However, when I add 1 dim to represent the dim of number, such as:
img = np.random.randint(-10, 10, size=(2, 2, 2))
I cannot do so if I would not like to use for loop.
Is there any way to avoid for but only numpy's stack to make it?
ps: for this example I should get (2,3,3) array by each 2 img instance is pad by zero at 1st column & 1st row.
Thanks
You can use np.pad, which allows you to specify the number of values padded to the edges of each axis.
So for the first example 2D array you could do:
a = np.array([[1, 2],[3, 4]])
np.pad(a,((1, 0), (1, 0)), mode = 'constant')
array([[0, 0, 0],
[0, 1, 2],
[0, 3, 4]])
So here each tuple is representing the side which to pad with zeros along each axis, i.e. ((before_1, after_1), … (before_N, after_N)).
And for a 3D array the same applies, but in this case we must specify that we only want to zero pad the two last dimensions:
img = np.random.randint(-10, 10, size=(2, 2, 2))
np.pad(img, ((0,0), (1,0), (1,0)), 'constant')
array([[[ 0, 0, 0],
[ 0, -3, -2],
[ 0, 9, -5]],
[[ 0, 0, 0],
[ 0, 1, -9],
[ 0, -1, -3]]])
You can use np.pad and remove the last row/column:
import numpy as np
a = np.array([[1, 2], [3, 4]])
result = np.pad(a, 1, mode='constant', constant_values=0)[:-1, :-1]
print(result)
Output:
[[0 0 0]
[0 1 2]
[0 3 4]]
If you want to pad only the column
import numpy as np
max_length =20
a = np.random.randint(1,size=(1, 10))
print(a.shape)
print(a.shape[1])
a =np.pad(a, [(0, 0), (0, max_length - a.shape[1])], mode='constant')
print(a.shape)
Output
(1, 10)
10
(1, 20)
Related
I would like to know the fastest way to extract the indices of the first n non zero values per column in a 2D array.
For example, with the following array:
arr = [
[4, 0, 0, 0],
[0, 0, 0, 0],
[0, 4, 0, 0],
[2, 0, 9, 0],
[6, 0, 0, 0],
[0, 7, 0, 0],
[3, 0, 0, 0],
[1, 2, 0, 0],
With n=2 I would have [0, 0, 1, 1, 2] as xs and [0, 3, 2, 5, 3] as ys. 2 values in the first and second columns and 1 in the third.
Here is how it is currently done:
x = []
y = []
n = 3
for i, c in enumerate(arr.T):
a = c.nonzero()[0][:n]
if len(a):
x.extend([i]*len(a))
y.extend(a)
In practice I have arrays of size (405, 256).
Is there a way to make it faster?
Here is a method, although quite confusing as it uses a lot of functions, that does not require sorting the array (only a linear scan is necessary to get non null values):
n = 2
# Get indices with non null values, columns indices first
nnull = np.stack(np.where(arr.T != 0))
# split indices by unique value of column
cols_ids= np.array_split(range(len(nnull[0])), np.where(np.diff(nnull[0]) > 0)[0] +1 )
# Take n in each (max) and concatenate the whole
np.concatenate([nnull[:, u[:n]] for u in cols_ids], axis = 1)
outputs:
array([[0, 0, 1, 1, 2],
[0, 3, 2, 5, 3]], dtype=int64)
Here is one approach using argsort, it gives a different order though:
n = 2
m = arr!=0
# non-zero values first
idx = np.argsort(~m, axis=0)
# get first 2 and ensure non-zero
m2 = np.take_along_axis(m, idx, axis=0)[:n]
y,x = np.where(m2)
# slice
x, idx[y,x]
# (array([0, 1, 2, 0, 1]), array([0, 2, 3, 3, 5]))
Use dislocation comparison for the row results of the transposed nonzero:
>>> n = 2
>>> i, j = arr.T.nonzero()
>>> mask = np.concatenate([[True] * n, i[n:] != i[:-n]])
>>> i[mask], j[mask]
(array([0, 0, 1, 1, 2], dtype=int64), array([0, 3, 2, 5, 3], dtype=int64))
I am trying to add coordinates to each pixel of an image for this I am doing the following
import cv2
import numpy as np
img = cv2.imread('images/0001.jpg')
grayscale = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
np_grayscale = np.array(grayscale)
# make the array 3d
processed_image = np_grayscale[:, :, np.newaxis]
x = 0
y = 0
for pixel_line in reversed(processed_image):
for pixel in pixel_line:
pixel = np.append(pixel, [x, y])
x += 1
y += 1
print(processed_image)
But this does not seem to work because I am still getting the original array that is in the form
[[[255]
[255]
[255]
...
[255]
[255]
[255]]
...
...
...
[255]
[255]
[255]]]
Moreover I don't think this is the most efficient way of doing this because I read that append creates a new copy of the array, can someone please help
You can create a mesh of indices using meshgrid() and stack() them with the original image:
import numpy as np
x = np.asarray([
[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 255, 0, 0],
[0, 0, 0, 0],
])
indices = np.meshgrid(
np.arange(x.shape[0]),
np.arange(x.shape[1]),
sparse=False
)
x = np.stack((x, *indices)).T
# array([[[ 0, 0, 0],
# [ 0, 0, 1],
# [ 0, 0, 2],
# [ 0, 0, 3]],
# [[ 0, 1, 0],
# [ 0, 1, 1],
# [255, 1, 2],
# [ 0, 1, 3]],
# [[ 0, 2, 0],
# [ 0, 2, 1],
# [ 0, 2, 2],
# [ 0, 2, 3]],
# [[ 0, 3, 0],
# [ 0, 3, 1],
# [ 0, 3, 2],
# [ 0, 3, 3]]])
x[0, 0, :] # 0, 0, 0
x[1, 2, :] # 255, 1, 2
x[-1, -1, :] # 0, 3, 3
Use xarray.
The easiest way, by far, to add coordinates to a NumPy array is with an xarray.DataArray. For example, here's how you can add coordinates that are just row and column index:
import xarray as xr
data = np.arange(25).reshape(5, 5)
rows, cols = data.shape
xs = np.arange(cols)
ys = np.arange(rows)
da = xr.DataArray(data, name="mydata", coords=[ys, xs], dims=['y', 'x'])
This da thing is a DataArray, which is essentially an indexed array ('indexed' in a pandas sense: the indices can be integers, floats, dates, etc). It has some nice features (try da.plot()). Otherwise, it basically behaves like a NumPy array. For example, you can slice it like a normal array:
>>> da[:, 3:]
<xarray.DataArray 'mydata' (y: 5, x: 2)>
array([[ 3, 4],
[ 8, 9],
[13, 14],
[18, 19],
[23, 24]])
Coordinates:
* y (y) int64 0 1 2 3 4
* x (x) int64 3 4
As you can see, this subarray 'knows' its own coordinates. What's more, you can have as many dimensions as you want, and each axis can have multiple coordinates. It's very useful.
As others pointed out, you can probably achieve what you want with careful indexing alone, but if you really want coordinates this is how I'd do it.
As you have a grayscale image, you can convert it to the 3-channeled image in which the first channel will contain the pixel values, and the other 2 channels will contain the coordinates. So, if you split your image into two or more, you will still have the coordinates of the original image in the other two channels, and also to visualize the image, you can simply use only the first channel. Here's how you can do this.
processed_image = np.zeros((grayscale.shape[0], grayscale.shape[1], 3), dtype=np.uint64)
processed_image[:, :, 0] = np.asarray(grayscale, dtype=np.uint64)
for i in range(processed_image.shape[0]):
for j in range(processed_image.shape[1]):
processed_image[i][j][1] = i
processed_image[i][j][2] = j
print(processed_image)
# Displaying the image
cv2.imshow("img", np.array(processed_image[:, :, [0]], dtype=np.uint8))
cv2.waitKey(0)
Note: Take care of the datatypes of the numpy arrays. You cannot store the coordinates of the complete image in the np.uint8 array as it will only contain values from 0-255. Also, while displaying, you'll have to convert the first channel back to the np.uint8 datatype as OpenCV only understands images in this format (for integer pixel values). That is why I have used an array of np.uint64 datatype to store pixel values.
I'm writing in Python 3.6, with Numpy 1.20.1. The problem is I have an np.ndarray called A with size (10, 3), and I have another np.ndarray called B with size (4, 3). For the 4 arrays of size 3, I would like to put them into 4 specific positions in the first array.
For example:
A = np.zeros((10, 3))
B = np.array([[1,2,3],[4,5,6],[7,8,9],[10,11,12]])
idx = [7,3,1,4]
And I would like to put each row in B to Aby the order in idx. So after the conversion, A should look like:
[0, 0, 0],
[7, 8, 9],
[0, 0, 0],
[4, 5, 6],
[10, 11, 12],
[0, 0, 0],
[0, 0 ,0],
[1, 2, 3],
[0, 0, 0],
[0, 0, 0],
[0, 0, 0].
I especailly wonder if it's possible to accomplish this in one line code.
I tried A[idx] = B, and it gives me error: IndexError: too many indices for array
Numpy version one liner.
A = np.zeros((10, 3))
B = np.array([[1,2,3],[4,5,6],[7,8,9],[10,11,12]])
idx = [7,3,1,4]
A[idx] = B[ np.arange(B.shape[0]) ] # Source from B.shape
OR
a[idx] = B[[0,1,2,3]] # Source as constants
Your code works perfectly fine for me.
As for the problem, you can try:
for i, row in zip(idx, B):
A[i] = row
import numpy as np
data = np.array([[0, 0, 1, 1, 2, 2],
[1, 0, 0, 1, 2, 2],
[1, 0, 1, 0, 0, 0],
[1, 1, 0, 0, 2, 0]])
How can I do the followings?
Within 2 by 2 patch:
if any element is 2: put 2
if any element is 1: put 1
if all elements are 0: put 0
The expected result is:
np.array([[1, 1, 2],
[1, 1, 2]])
Using extract_patches from scikit-learn you can write this as follows (copy and paste-able code):
import numpy as np
from sklearn.feature_extraction.image import extract_patches
data = np.array([[0, 0, 1, 1, 2, 2],
[1, 0, 0, 1, 2, 2],
[1, 0, 1, 0, 0, 0],
[1, 1, 0, 0, 2, 0]])
patches = extract_patches(data, patch_shape=(2, 2), extraction_step=(2, 2))
output = patches.max(axis=-1).max(axis=-1)
Explanation: extract_patches gives you a view on patches of your array, of size patch_shape and lying on a grid of extraction_step. The result is a 4D array where the first two axes index the patch and the last two axes index the pixels within the patch. We then evaluate the maximum over the last two axes to obtain the maximum per patch.
EDIT This is actually very much related to this question
I'm not sure where you get your input from or where you are supposed to leave the output, but you can adapt this.
import numpy as np
data = np.array([[0, 0, 1, 1, 2, 2],
[1, 0, 0, 1, 2, 2],
[1, 0, 1, 0, 0, 0],
[1, 1, 0, 0, 2, 0]])
def patchValue(i,j):
return max([data[i][j],
data[i][j+1],
data[i+1][j],
data[i+1][j+1]])
result = np.array([[0, 0, 0],
[0, 0, 0]])
for (v,i) in enumerate(range(0,4,2)):
for (w,j) in enumerate(range(0,6,2)):
result[v][w] = patchValue(i,j)
print(result)
Here's a rather lengthy one-liner that relies solely on reshaping, transposes, and taking maximums along different axes. It is fairly fast too.
data.reshape((-1,2)).max(axis=1).reshape((data.shape[0],-1)).T.reshape((-1,2)).max(axis=1).reshape((data.shape[1]/2,data.shape[0]/2)).T
Essentially what this does is reshape to take the maximum in pairs of two horizontally, then shuffle things around again and take the maximum in pairs of two vertically, ultimately giving the maximum of each block of 4, matching your desired output.
If the original array is large, and performance is an issue, the loops can be pushed down to numpy C code by manipulating the shape and strides of the original array to create the windows that you are acting on:
import numpy as np
from numpy.lib.stride_tricks import as_strided
data = np.array([[0, 0, 1, 1, 2, 2],
[1, 0, 0, 1, 2, 2],
[1, 0, 1, 0, 0, 0],
[1, 1, 0, 0, 2, 0]])
patch_shape = (2,2)
data_shape = np.array(data.shape)
# transform data to a 2x3 array of 2x2 patches/windows
# final shape of the computation on the windows can be calculated with:
# tuple(((data_shape-patch_shape) // patch_shape) + 1)
final_shape = (2,3)
# the shape of the windowed array can be calculated with:
# final_shape + patch_shape
newshape = (2, 3, 2, 2)
# the strides of the windowed array can be calculated with:
# tuple(np.array(data.strides) * patch_shape) + data.strides
newstrides = (48, 8, 24, 4)
# use as_strided to 'transform' the array
patch_array = as_strided(data, shape = newshape, strides = newstrides)
# flatten the windowed array for iteration - dim of 6x2x2
# the number of windows is the product of the 'first' dimensions of the array
# which can be calculated with:
# (np.product(newshape[:-len(patch_shape)])) + (newshape[-len(patch_array):])
dim = (6,2,2)
patch_array = patch_array.reshape(dim)
# perfom computations on the windows and reshape to final dimensions
result = [2 if np.any(patch == 2) else
1 if np.any(patch == 1) else
0 for patch in patch_array]
result = np.array(result).reshape(final_shape)
A generalized 1-d function for creating the windowed array can be found at Efficient rolling statistics with NumPy
A generalised multi-dimension function and a nice explanation can be found at Efficient Overlapping Windows with Numpy
Briefly: there is a similar question and the best answer suggests using numpy.bincount. I need the same thing, but for a matrix.
I've got two arrays:
array([1, 2, 1, 1, 2])
array([2, 1, 1, 1, 1])
together they make indices that should be incremented:
>>> np.array([a, b]).T
array([[1, 2],
[2, 1],
[1, 1],
[1, 1],
[2, 1]])
I want to get this matrix:
array([[0, 0, 0],
[0, 2, 1], # (1,1) twice, (1,2) once
[0, 2, 0]]) # (2,1) twice
The matrix will be small (like, 5×5), and the number of indices will be large (somewhere near 10^3 or 10^5).
So, is there anything better (faster) than a for-loop?
You can still use bincount(). The trick is to convert a and b into a single 1D array of flat indices.
If the matrix is nxm, you could apply bincount() to a * m + b, and construct the matrix from the result.
To take the example in your question:
In [15]: a = np.array([1, 2, 1, 1, 2])
In [16]: b = np.array([2, 1, 1, 1, 1])
In [17]: cnt = np.bincount(a * 3 + b)
In [18]: cnt.resize((3, 3))
In [19]: cnt
Out[19]:
array([[0, 0, 0],
[0, 2, 1],
[0, 2, 0]])
If the shape of the array is more complicated, it might be easier to use np.ravel_multi_index() instead of computing flat indices by hand:
In [20]: cnt = np.bincount(np.ravel_multi_index(np.vstack((a, b)), (3, 3)))
In [21]: np.resize(cnt, (3, 3))
Out[21]:
array([[0, 0, 0],
[0, 2, 1],
[0, 2, 0]])
(Hat tip #Jaime for pointing out ravel_multi_index.)
m1 = m.view(numpy.ndarray) # Create view
m1.shape = -1 # Make one-dimensional array
m1 += np.bincount(a+m.shape[1]*b, minlength=m1.size)