I am stuck on a little issue in the project I am currently working on.
Getting straight to the point, let's assume I have a 2-dimensional numpy.array - I will call it arr.
I need to slice arr, but this slice must contain some padding depending on the selected interval.
Example:
arr = numpy.array([
[ 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10],
[ 11, 12, 13, 14, 15],
[ 16, 17, 18, 19, 20],
[ 21, 22, 23, 24, 25]
])
Actually, numpy's response for arr[3:7, 3:7] is:
array([[19, 20],
[24, 25]])
But I need it to be padded as if arr were bigger than it really is.
Here is what I need as response for arr[3:7, 3:7]:
array([[19, 20, 0, 0],
[24, 25, 0, 0],
[ 0, 0, 0, 0],
[ 0, 0, 0, 0]])
This padding should also occur in case of negative indices. If the requested slice is bigger than the whole image, padding must occur in all sides, if needed.
Another example, negative indices. This is the expected result for arr[-2:2, -1:3]:
array([[ 0, 0, 0, 0],
[ 0, 0, 1, 2],
[ 0, 0, 6, 7],
[ 0, 0, 11, 12]])
Is there any native numpy function for this? If not, any idea of how can I implement this?
About the first part of your question you can use a simple indexing, and you can create a zero_like of your array with numpy.zeros_like then assign the special part :
>>> new=numpy.zeros_like(arr)
>>> part=arr[3:7, 3:7]
>>> i,j=part.shape
>>> new[:i,:j]=part
>>> new
array([[19, 20, 0, 0, 0],
[24, 25, 0, 0, 0],
[ 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0]])
But for the second case you can not use a negative indexing for for numpy arrays like this.Negative indices are interpreted as counting from the end of the array so if you are counting from -2 actually in a 5x5 array there are not any row between -2 and 2 so the result would be an empty array :
>>> arr[-2:2]
array([], shape=(0, 5), dtype=int64)
You can do something like:
print np.lib.pad(arr[3:7,3:7], ((0, 2), (0, 2)), 'constant', constant_values=(0,0 ))
[[19 20 0 0]
[24 25 0 0]
[ 0 0 0 0]
[ 0 0 0 0]]
For the negative indexing:
print np.lib.pad(arr[ max(0,-1):3 , 0:2 ], ((1, 0), (2, 0)), 'constant', constant_values=(0,0 ))
[[ 0 0 0 0]
[ 0 0 1 2]
[ 0 0 6 7]
[ 0 0 11 12]]
Check here for reference
import numpy as np
def convert(inarr, x1, x2, y1, y2):
xd = x2 - x1
yd = y2 - y1
outarr = np.zeros(xd * yd).reshape(xd, yd)
x1fr = max(0, x1)
x2fr = min(x2, inarr.shape[0])
y1fr = max(0, y1)
y2fr = min(y2, inarr.shape[1])
x1to = max(0, xd - x2)
x2to = x1to + x2fr - x1fr
y1to = max(0, yd - y2)
y2to = y1to + y2fr - y1fr
outarr[x1to:x2to, y1to:y2to] = inarr[x1fr:x2fr, y1fr:y2fr]
return outarr
arr = np.array([[ 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25]])
print(convert(arr, -2, 2, -1, 3))
Well this works but returns
[[ 0. 0. 0. 0.]
[ 0. 0. 0. 0.]
[ 0. 1. 2. 3.]
[ 0. 6. 7. 8.]]
for your -ve index example. You can play around to get it to do what you expect
Related
I have a 2d array with shape(3,6), then i want to create a condition to check a value of each array.
my data arry is as follows :
array([[ 1, 2, 3, 4, 5, 6],
7, 8, 9, 10, 11, 12],
[13, 14, 15, 16, 17, 18]])
if in an array there are numbers < 10 then the value will be 0
the result I expected
array([[ 0, 0, 0, 0, 0, 0],
0, 0, 0, 10, 11, 12],
[13, 14, 15, 16, 17, 18]])
the code i created is like this, but why can't it work as i expected
FCDataNew = []
a = [ [1,2,3,4,5,6],
[7,8,9,10,11,12],
[13,14,15,16,17,18]
]
a = np.array(a)
c = 0
c = np.array(c)
for i in range(len(a)):
if a[i].all()<10:
FCDataNew.append(c)
else:
FCDataNew.append(a[i])
FCDataNew = np.array(FCDataNew)
FCDataNew
If you want to modify the array in place, use boolean indexing:
FCDataNew = np.array([[1,2,3,4,5,6],
[7,8,9,10,11,12],
[13,14,15,16,17,18],
])
FCDataNew[FCDataNew<10] = 0
For a copy:
out = np.where(FCDataNew<10, 0, FCDataNew)
Output:
array([[ 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 10, 11, 12],
[13, 14, 15, 16, 17, 18]])
You can just use arr[arr < 10] = 0
I am trying to pad two dimensions of an N-dimensional matrix with different paddings and override the values. Consider the Following example:
def determineShifts(layer):
u = range(0, 2*layer + 1)
b = range(0, 2*layer + 1)
shifts = []
mat = np.zeros((2 * layer + 1, 2 * layer + 1), dtype=object)
for x, i in enumerate(u):
for y, j in enumerate(b):
up = (j, 2*layer - j)
left = (i, 2*layer - i)
mat[x, y] = (left, up)
return mat
layer = 1
b = np.ones((3,3,3,3))
shifts = determineShifts(layer)
I want to pad the second last and final dimension of the array b such that the resulting shape is (3,3,5,5) and override the element of that matrix and repeat the process for all nodes, which in this case is (3,3). I would prefer to override the values (currently I receive a broadcasting error) rather than making a copy of the desired shape and iterating through the first and second dimension. A sample is included below:
c = np.ones((3,3,5,5))
for i in range(np.shape(c)[0]):
for j in range(np.shape(c)[1]):
c[i,j] = np.pad(b[i,j], shifts[i,j])
Is there some way to apply a function to the matrix to apply all the shifts to each of the elements (3,3, 3, 3) -> (3, 3, 5, 5) such that the code is computationally efficient?
np.pad() accepts different padding values for each axis, but not different ones within each axis, as per your example.
One general approach is to do a bit of arithmetic for the relocation of elements and then use fancy indexing. In your case, it looks like you are trying to stagger the 2D blocks of the last two dimensions in such a way that they move by 1: vertically for axis 0 and horizontally for axis 1.
You can do the same with the following arithmetic:
def stagger_ix(s):
r = np.arange(np.prod(s))
block = r // np.prod(s[-2:])
shift_i, shift_j = block // s[1], block % s[1]
i, j = r // s[-1] % s[-2], r % s[-1]
newshape = np.array(s)
newshape[-2:] += newshape[:2] - 1
ix = (
block * np.prod(newshape[-2:])
+ (i + shift_i) * newshape[-1]
+ (j + shift_j)
)
return newshape, ix
def stagger(b):
newshape, ix = stagger_ix(b.shape)
# now insert b in a zero(newshape), as per shift logic
c = np.zeros(np.prod(newshape), dtype=b.dtype)
c[ix] = b.ravel()
c = c.reshape(newshape)
return c
Your c array can be obtained as:
c = stagger(np.ones((3,3,3,3)))
Other examples -
# for example matrices
def rp1(s):
return (np.arange(np.prod(s)) + 1).reshape(s)
>>> stagger(rp1((2,2,2,2)))
array([[[[ 1, 2, 0],
[ 3, 4, 0],
[ 0, 0, 0]],
[[ 0, 5, 6],
[ 0, 7, 8],
[ 0, 0, 0]]],
[[[ 0, 0, 0],
[ 9, 10, 0],
[11, 12, 0]],
[[ 0, 0, 0],
[ 0, 13, 14],
[ 0, 15, 16]]]])
>>> stagger(rp1((2,3,2,5)))
array([[[[ 1, 2, 3, 4, 5, 0, 0],
[ 6, 7, 8, 9, 10, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0]],
[[ 0, 11, 12, 13, 14, 15, 0],
[ 0, 16, 17, 18, 19, 20, 0],
[ 0, 0, 0, 0, 0, 0, 0]],
[[ 0, 0, 21, 22, 23, 24, 25],
[ 0, 0, 26, 27, 28, 29, 30],
[ 0, 0, 0, 0, 0, 0, 0]]],
[[[ 0, 0, 0, 0, 0, 0, 0],
[31, 32, 33, 34, 35, 0, 0],
[36, 37, 38, 39, 40, 0, 0]],
[[ 0, 0, 0, 0, 0, 0, 0],
[ 0, 41, 42, 43, 44, 45, 0],
[ 0, 46, 47, 48, 49, 50, 0]],
[[ 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 51, 52, 53, 54, 55],
[ 0, 0, 56, 57, 58, 59, 60]]]])
I'm trying to reshape an array from its original shape, to make the elements of each row descend along a diagonal:
np.random.seed(0)
my_array = np.random.randint(1, 50, size=(5, 3))
array([[45, 48, 1],
[ 4, 4, 40],
[10, 20, 22],
[37, 24, 7],
[25, 25, 13]])
I would like the result to look like this:
my_array_2 = np.array([[45, 0, 0],
[ 4, 48, 0],
[10, 4, 1],
[37, 20, 40],
[25, 24, 22],
[ 0, 25, 7],
[ 0, 0, 13]])
This is the closest solution I've been able to get:
my_diag = []
for i in range(len(my_array)):
my_diag_ = np.diag(my_array[i], k=0)
my_diag.append(my_diag_)
my_array1 = np.vstack(my_diag)
array([[45, 0, 0],
[ 0, 48, 0],
[ 0, 0, 1],
[ 4, 0, 0],
[ 0, 4, 0],
[ 0, 0, 40],
[10, 0, 0],
[ 0, 20, 0],
[ 0, 0, 22],
[37, 0, 0],
[ 0, 24, 0],
[ 0, 0, 7],
[25, 0, 0],
[ 0, 25, 0],
[ 0, 0, 13]])
From here I think it might be possible to remove all zero diagonals, but I'm not sure how to do that.
One way using numpy.pad:
n = my_array.shape[1] - 1
np.dstack([np.pad(a, (i, n-i), "constant")
for i, a in enumerate(my_array.T)])
Output:
array([[[45, 0, 0],
[ 4, 48, 0],
[10, 4, 1],
[37, 20, 40],
[25, 24, 22],
[ 0, 25, 7],
[ 0, 0, 13]]])
In [134]: arr = np.array([[45, 48, 1],
...: [ 4, 4, 40],
...: [10, 20, 22],
...: [37, 24, 7],
...: [25, 25, 13]])
In [135]: res= np.zeros((arr.shape[0]+arr.shape[1]-1, arr.shape[1]), arr.dtype)
Taking a hint from how np.diag indexes a diagonal, iterate on the rows of arr:
In [136]: for i in range(arr.shape[0]):
...: n = i*arr.shape[1]
...: m = arr.shape[1]
...: res.flat[n:n+m**2:m+1] = arr[i,:]
...:
In [137]: res
Out[137]:
array([[45, 0, 0],
[ 4, 48, 0],
[10, 4, 1],
[37, 20, 40],
[25, 24, 22],
[ 0, 25, 7],
[ 0, 0, 13]])
There's probably a shift capability in numpy, but I'm not familiar w/it, so here's a solution using pandas. You concat np.zeros to the original array with the number of rows being equal to ncols - 1. Then iterate over each col and shift it down by the number equal to the column number.
import numpy as np
import pandas as pd
np.random.seed(0)
my_array = np.random.randint(1,50, size=(5,3))
df = pd.DataFrame(np.concatenate((my_array,np.zeros((my_array.shape[1]-1,
my_array.shape[1])))))
for col in df.columns:
df[col] = df[col].shift(int(col))
df.fillna(0).values
Output
array([[45., 0., 0.],
[ 4., 48., 0.],
[10., 4., 1.],
[37., 20., 40.],
[25., 24., 22.],
[ 0., 25., 7.],
[ 0., 0., 13.]])
You can create a fancy index for the output using simple broadcasting and padding. First pad the end of your data:
a = np.concatenate((a, np.zeros((a.shape[1] - 1, a.shape[1]), a.dtype)), axis=0)
Now make an index that gets the elements using their negative index. This will make it trivial to roll around the end:
cols = np.arange(a.shape[1])
rows = np.arange(a.shape[0]).reshape(-1, 1) - cols
Now just simply index:
result = a[rows, cols]
For large arrays, this may not be as efficient as running a small loop. At the same time, this avoids actual looping, and allows you to write a one-liner (but please don't):
result = np.concatenate((a, np.zeros((a.shape[1] - 1, a.shape[1]), a.dtype)), axis=0)[np.arange(a.shape[0] + a.shape[1] - 1).reshape(-1, 1) - np.arange(a.shape[1]), np.arange(a.shape[1])]
I have a 3D array a with shape (m, n, p) and a 2D array idx with shape (m, n). I want all elements in a where the last axis index is smaller than the corresponding element in idx to be set to 0.
The following code works. My question is : is there a more efficient approach?
a = np.array([[[1, 2, 3],
[4, 5, 6]],
[[7, 8, 9],
[10, 11, 12]],
[[21, 22, 23],
[25, 26, 27]]])
idx = np.array([[2, 1],
[0, 1],
[1, 1]])
for (i, j), val in np.ndenumerate(idx):
a[i, j, :val] = 0
The result is
array([[[ 0, 0, 3],
[ 0, 5, 6]],
[[ 7, 8, 9],
[ 0, 11, 12]],
[[ 0, 22, 23],
[ 0, 26, 27]]])
Use broadcasting to create the 3D mask and then assign zeros with boolean-indexing -
mask = idx[...,None] > np.arange(a.shape[2])
a[mask] = 0
Alternatively, we can also use NumPy builtin for outer-greater comparison to get that mask -
mask = np.greater.outer(idx, np.arange(a.shape[2]))
Run on given sample -
In [34]: mask = idx[...,None] > np.arange(a.shape[2])
In [35]: a[mask] = 0
In [36]: a
Out[36]:
array([[[ 0, 0, 3],
[ 0, 5, 6]],
[[ 7, 8, 9],
[ 0, 11, 12]],
[[ 0, 22, 23],
[ 0, 26, 27]]])
Let's say I have a 2d numpy array of the size (5,5). I can get the neighbors of the index (i,j) with the following statement:
a = range(25)
a = np.reshape(a, (5,5))
n = a[i-1:i+2, j-1:j+2]
That works great for 0 < i,j < 4. My problem is that I always want to get a 3x3 array but if one of the indices is 0 or 4 I do not get it (in case i=0 the range is (-1, 2) = (4, 2) and we get an empty range)
Do you have any ideas how I can always get a 3x3 matrix and fill the "failed indices" with zeros?
Use np.pad to extend your array first and index into the result instead. You'll have to shift your indices accordingly.
>>> b = np.pad(a, pad_width=1, mode='constant')
>>> b
array([[ 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 1, 2, 3, 4, 0],
[ 0, 5, 6, 7, 8, 9, 0],
[ 0, 10, 11, 12, 13, 14, 0],
[ 0, 15, 16, 17, 18, 19, 0],
[ 0, 20, 21, 22, 23, 24, 0],
[ 0, 0, 0, 0, 0, 0, 0]])
>>>