I would like to slice a numpy array with a constant offset back in the array. I.e. start in the k'th position and go back n elements. I want to move the slight one index ahead at every iteration.
E.g. I have the following array
x = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
and let's say k is 5 and n is 3. That would give me the following (with ordering preserved)
x_sliced = [4, 5, 6]
In the next iteration k += 1 and n is still 3. That gives me the following array
x_sliced = [5, 6, 7]
I can sort of get the result but I'll have to flip the array to get back to the original order. Isn't there a clever way that just uses a position and an offset back in the array?
If I understand correctly, this could help:
from skimage.util.shape import view_as_windows
k = 5
n = 3
view_as_windows(x[k-n+1:], n)
output:
array([[ 4, 5, 6],
[ 5, 6, 7],
[ 6, 7, 8],
[ 7, 8, 9],
[ 8, 9, 10]])
Then you can loop over the output rows and process each window. One thing to note is that the overlapping windows share the same memory. If you wish that changing the values in one window does not affect the next overlapping window, simply copy it (.copy())
You can use np.lib.stride_tricks.as_strided for that which requires no other dependencies than NumPy itself!
x = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
b = np.lib.stride_tricks.as_strided(x, (len(x)-3+1, 3), 2 * x.strides)
b
> array([[ 1, 2, 3],
[ 2, 3, 4],
[ 3, 4, 5],
[ 4, 5, 6],
[ 5, 6, 7],
[ 6, 7, 8],
[ 7, 8, 9],
[ 8, 9, 10]])
Moreover, with list(b) you can turn the 2D array into a list of 1D arrays.
Related
I have three arrays
a = np.array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) #Original large array
b = np.array([4, 8, 5]) #Smaller array
c = np.array([2, 7, 9]) #Arguments
The result should look like the following
np.array([4, 4, 4, 8, 8, 8, 8, 8, 5, 5])
This means that [0, 1, 2] are replaced by 4, [3, 4, 5, 6, 7] are replaced by 8 and [8, 9] are placed by 5. Is there any numpy function/code for that?
I am not aware if such a function exists but the function down below gets the job done:
for i, element in enumerate(a):
if a[i] <= c[x]:
a[i] = b[x]
else:
x+=1
a[i] = b[x]
I have a 2-d numpy array of shape NxM which represents M contiguous samples from N different sequences. I need to present patches of L samples (L << M) covering the entire dataset as a 2-d numpy array. There is too much data to construct a new dataset by simply copying of all the patches.
If there was a single sequence, it would be very straight forward to generate the overlapping patches without copying any data using the as_strided trick:
patches = np.lib.stride_tricks.as_strided(data, shape(N*M-L+1,L), strides=(8,8))
The problem with this approach for my data is that it produces patches that overlap separate sequences.
I can also see how to generate a 3-d array of shape N,M-L+1,L using something like:
patches = np.lib.stride_ticks.as_strided(data, shape(N,M-L+1,L), strides=(8*M,8,8))
This produces the correct patches, but I am not sure how to collapse the first two dimensions into one.
There are obviously several SO answers related to as_strided, but I could not find any that address these particular requirements.
Any ideas are appreciated.
Edit: Short example follows
Here is an example of using as_strided to make a 3-d array that almost accomplishes the task:
>>> a = np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]])
>>> a
array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12]])
>>> b = np.lib.stride_tricks.as_strided(a, shape=(3, 3, 2), strides=(32,8,8))
>>> b
array([[[ 1, 2],
[ 2, 3],
[ 3, 4]],
[[ 5, 6],
[ 6, 7],
[ 7, 8]],
[[ 9, 10],
[10, 11],
[11, 12]]])
>>>
The issue with trying to flatten this 3-d array into 2-d as suggested by #Divakar is that the reshaping produces the correct data but does so by making a copy which creates an unmanageable amount of data for the actual problem at hand:
>>> c = b.reshape(-1,b.shape[-1])
>>> c
array([[ 1, 2],
[ 2, 3],
[ 3, 4],
[ 5, 6],
[ 6, 7],
[ 7, 8],
[ 9, 10],
[10, 11],
[11, 12]])
>>> b[0][0][0] = 9999
>>> c
array([[ 1, 2],
[ 2, 3],
[ 3, 4],
[ 5, 6],
[ 6, 7],
[ 7, 8],
[ 9, 10],
[10, 11],
[11, 12]])
>>>
In Numpy, given a stack of large images A of size(N,hl,wl), and coordinates x of size(N) and y of size(N) I want to get smaller images of size (N,16,16)
In a for loop it would look like this:
B=numpy.zeros((N,16,16))
for i in range(0,N):
B[i,:,:]=A[i,y[i]:y[i]+16,x[i]:x[i]+16]
But can I do this just with indexing?
Bonus question: Will this indexing also work in pytorch? If not how can I implement this there?
In numpy slicing is very simple and the same logic works with a pytorch example. For example
imgs = np.random.normal(size=(16,24,24))
imgs[:,0:12,0:12].shape
imgs_tensor = torch.from_numpy(imgs)
imgs_tensor[:,0:12,0:12].size()
where the first : in the slicing indicates to select all the images in the batch. The 2nd and 3rd : indicates the slicing for height and width.
Pretty simple really with view_as_windows from scikit-image, to get those sliding windowed views as a 6D array with the fourth axis being singleton. Then, use advanced-indexing to select the ones we want based off the y and x indices for indexing into the second and third axes of the windowed array to get our B.
Hence, the implementation would be -
from skimage.util.shape import view_as_windows
BSZ = 16, 16 # Blocksize
A6D = view_as_windows(A,(1,BSZ[0],BSZ[1]))
B_out = A6D[np.arange(N),y,x,0]
Explanation
To explain to other readers on what's really going on with the problem, here's a sample run on a smaller dataset and with a blocksize of (2,2) -
1) Input array (3D) :
In [78]: A
Out[78]:
array([[[ 5, 5, 3, 5, 3, 8],
[ 5, *2, 6, 2, 2, 4],
[ 4, 3, 4, 9, 3, 8],
[ 6, 3, 3, 10, 4, 5],
[10, 2, 5, 7, 6, 7],
[ 5, 4, 2, 5, 2, 10]],
[[ 4, 9, 8, 4, 9, 8],
[ 7, 10, 8, 2, 10, 9],
[10, *9, 3, 2, 4, 7],
[ 5, 10, 8, 3, 5, 4],
[ 6, 8, 2, 4, 10, 4],
[ 2, 8, 6, 2, 7, 5]],
[[ *4, 8, 7, 2, 9, 9],
[ 2, 10, 2, 3, 8, 8],
[10, 7, 5, 8, 2, 10],
[ 7, 4, 10, 9, 6, 9],
[ 3, 4, 9, 9, 10, 3],
[ 6, 4, 10, 2, 6, 3]]])
2) y and x indices to index into the second and third axes :
In [79]: y
Out[79]: array([1, 2, 0])
In [80]: x
Out[80]: array([1, 1, 0])
3) Finally the desired output, which is a block each from each of the 2D slice along the first axis and whose starting point (top left corner point) is (y,x) on that 2D slice. Refer to the asterisks in A for those -
In [81]: B
Out[81]:
array([[[ 2, 6],
[ 3, 4]],
[[ 9, 3],
[10, 8]],
[[ 4, 8],
[ 2, 10]]])
This is an implementation of extract_glimpse similar with tf.image.extract_glimpse in PyTorch. It should be satisfied your need:
https://github.com/jimmysue/xvision/blob/main/xvision/ops/extract_glimpse.py#L14
I have a 2 d numpy array. I need to keep all the rows whose value at a specific column is greater than a certain number. Right now, I have:
f_left = np.where(f_sorted[:,attribute] >= split_point)
And it is failing with: "Index Error: too many indices for array"
How should I do this? I can't figure it out from the numpy website, here
You actually don't even need where.
yy = np.array(range(12)).reshape((4,3))
yy[yy[:,1] > 2]
Output
array([[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11]])
x = np.array([[2,3,4],[5,6,7],[1,2,3],[8,9,10]])
array([[ 2, 3, 4],
[ 5, 6, 7],
[ 1, 2, 3],
[ 8, 9, 10]])
Find the rows where the second element are >=4
x[np.where(x[:,1] >= 4)]
array([[ 5, 6, 7],
[ 8, 9, 10]])
How can I turn a list such as:
data_list = [0,1,2,3,4,5,6,7,8,9]
into a array (I'm using numpy) that looks like:
data_array = [ [0,1] , [2,3] , [4,5] , [6,7] , [8,9] ]
Can I slice segments off the beginning of the list and append them to an empty array?
Thanks
>>> import numpy as np
>>> np.array(data_list).reshape(-1, 2)
array([[0, 1],
[2, 3],
[4, 5],
[6, 7],
[8, 9]])
(The reshape method returns a new "view" on the array; it doesn't copy the data.)
def nest_list(list1,rows, columns):
result=[]
start = 0
end = columns
for i in range(rows):
result.append(list1[start:end])
start +=columns
end += columns
return result
for:
list1=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
nest_list(list1,4,4)
Output:
[[0, 1, 2, 3], [4, 5, 6, 7], [8, 9, 10, 11], [12, 13, 14, 15]]