Related
Say for example I have these 3 arrays:
# Array 1:
array_1 = [[100, 0, 0, 0, 0, 100],
[0, 100, 0, 0, 0, 100],
[0, 0, 100, 100, 0, 0]]
# Array 2:
array_2 = [[0, 0, 0, 0, 100, 0],
[0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 100]]
# Array 3:
array_3 = [[0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 100, 0],
[0, 0, 0, 0, 0, 0]]
How will I be able to combine the 3 arrays into a one single array?
This will be the expected output:
[[100 0 0 0 100 100]
[0 100 0 0 100 100]
[0 0 100 100 0 100]]
As you can see, the 100s from array_1, array_2 and array_3 can be seen in the newly created array.
Combination of 100s must be with the same row as the other.
In this case, you can just add the arrays together
>>> a = np.arange(18).reshape((3,6))
>>> b = np.arange(18).reshape((3,6))
>>> c = np.arange(18).reshape((3,6))
>>> a
array([[ 0, 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10, 11],
[12, 13, 14, 15, 16, 17]])
>>> a + b + c
array([[ 0, 3, 6, 9, 12, 15],
[18, 21, 24, 27, 30, 33],
[36, 39, 42, 45, 48, 51]])
I am trying to pad two dimensions of an N-dimensional matrix with different paddings and override the values. Consider the Following example:
def determineShifts(layer):
u = range(0, 2*layer + 1)
b = range(0, 2*layer + 1)
shifts = []
mat = np.zeros((2 * layer + 1, 2 * layer + 1), dtype=object)
for x, i in enumerate(u):
for y, j in enumerate(b):
up = (j, 2*layer - j)
left = (i, 2*layer - i)
mat[x, y] = (left, up)
return mat
layer = 1
b = np.ones((3,3,3,3))
shifts = determineShifts(layer)
I want to pad the second last and final dimension of the array b such that the resulting shape is (3,3,5,5) and override the element of that matrix and repeat the process for all nodes, which in this case is (3,3). I would prefer to override the values (currently I receive a broadcasting error) rather than making a copy of the desired shape and iterating through the first and second dimension. A sample is included below:
c = np.ones((3,3,5,5))
for i in range(np.shape(c)[0]):
for j in range(np.shape(c)[1]):
c[i,j] = np.pad(b[i,j], shifts[i,j])
Is there some way to apply a function to the matrix to apply all the shifts to each of the elements (3,3, 3, 3) -> (3, 3, 5, 5) such that the code is computationally efficient?
np.pad() accepts different padding values for each axis, but not different ones within each axis, as per your example.
One general approach is to do a bit of arithmetic for the relocation of elements and then use fancy indexing. In your case, it looks like you are trying to stagger the 2D blocks of the last two dimensions in such a way that they move by 1: vertically for axis 0 and horizontally for axis 1.
You can do the same with the following arithmetic:
def stagger_ix(s):
r = np.arange(np.prod(s))
block = r // np.prod(s[-2:])
shift_i, shift_j = block // s[1], block % s[1]
i, j = r // s[-1] % s[-2], r % s[-1]
newshape = np.array(s)
newshape[-2:] += newshape[:2] - 1
ix = (
block * np.prod(newshape[-2:])
+ (i + shift_i) * newshape[-1]
+ (j + shift_j)
)
return newshape, ix
def stagger(b):
newshape, ix = stagger_ix(b.shape)
# now insert b in a zero(newshape), as per shift logic
c = np.zeros(np.prod(newshape), dtype=b.dtype)
c[ix] = b.ravel()
c = c.reshape(newshape)
return c
Your c array can be obtained as:
c = stagger(np.ones((3,3,3,3)))
Other examples -
# for example matrices
def rp1(s):
return (np.arange(np.prod(s)) + 1).reshape(s)
>>> stagger(rp1((2,2,2,2)))
array([[[[ 1, 2, 0],
[ 3, 4, 0],
[ 0, 0, 0]],
[[ 0, 5, 6],
[ 0, 7, 8],
[ 0, 0, 0]]],
[[[ 0, 0, 0],
[ 9, 10, 0],
[11, 12, 0]],
[[ 0, 0, 0],
[ 0, 13, 14],
[ 0, 15, 16]]]])
>>> stagger(rp1((2,3,2,5)))
array([[[[ 1, 2, 3, 4, 5, 0, 0],
[ 6, 7, 8, 9, 10, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0]],
[[ 0, 11, 12, 13, 14, 15, 0],
[ 0, 16, 17, 18, 19, 20, 0],
[ 0, 0, 0, 0, 0, 0, 0]],
[[ 0, 0, 21, 22, 23, 24, 25],
[ 0, 0, 26, 27, 28, 29, 30],
[ 0, 0, 0, 0, 0, 0, 0]]],
[[[ 0, 0, 0, 0, 0, 0, 0],
[31, 32, 33, 34, 35, 0, 0],
[36, 37, 38, 39, 40, 0, 0]],
[[ 0, 0, 0, 0, 0, 0, 0],
[ 0, 41, 42, 43, 44, 45, 0],
[ 0, 46, 47, 48, 49, 50, 0]],
[[ 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 51, 52, 53, 54, 55],
[ 0, 0, 56, 57, 58, 59, 60]]]])
In the documentation of numpy.take, it is stated that a is indexed according to indices and axis, then the result is optionally stored in the out parameter. Does exists a function that perform the indexing on out instead? Using fancy indexing it would be something like:
out[:, :, indices, :] = a
Here I assume that axis=2 but in my case I don't know the axis in advance.
A solution using 1d boolean masks instead of indices is acceptable as well.
You can use swapaxes like so:
>>> A = np.arange(24).reshape(2,3,4)
>>> out = np.empty_like(A)
>>> I = [2,0,1]
>>> axis = 1
>>> out.swapaxes(0, axis)[I] = A.swapaxes(0, axis)
>>> out
array([[[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[ 0, 1, 2, 3]],
[[16, 17, 18, 19],
[20, 21, 22, 23],
[12, 13, 14, 15]]])
Some of the numpy functions construct an indexing tuple when operating on a specified axis.
The code isn't particularly pretty, but is general and reasonably efficient.
In [700]: out = np.zeros((2,1,4,5),int)
In [701]: out
Out[701]:
array([[[[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0]]],
[[[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0]]]])
In [702]: indices = [3,0,1]
Make an indexing tuple. Start with a list or array for ease of construction, and then convert to tuple when indexing:
In [703]: idx = [slice(None)]*out.ndim
In [704]: idx[2] = indices
In [705]: idx
Out[705]:
[slice(None, None, None),
slice(None, None, None),
[3, 0, 1],
slice(None, None, None)]
In [706]: out[tuple(idx)] = 10
In [707]: out
Out[707]:
array([[[[10, 10, 10, 10, 10],
[10, 10, 10, 10, 10],
[ 0, 0, 0, 0, 0],
[10, 10, 10, 10, 10]]],
[[[10, 10, 10, 10, 10],
[10, 10, 10, 10, 10],
[ 0, 0, 0, 0, 0],
[10, 10, 10, 10, 10]]]])
It matches the take:
In [708]: np.take(out, indices, axis=2)
Out[708]:
array([[[[10, 10, 10, 10, 10],
[10, 10, 10, 10, 10],
[10, 10, 10, 10, 10]]],
[[[10, 10, 10, 10, 10],
[10, 10, 10, 10, 10],
[10, 10, 10, 10, 10]]]])
We can set more complex values, as long as we get the broadcasting right:
out[tuple(idx)] = np.array([10,11,12])[...,None]
I have also seen numpy functions that shift the axis of interest to a known location - either beginning or end. Depending on the action it may require swapping back.
There are functions like place, put, copyto that provide other ways of controlling assignment (besides the usual indexing). But none take an axis parameter like np.take.
having an array like this for example:
[1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1]
What's the fastest way in Python to get the non-zero elements organized in a list where each element contains the indexes of blocks of continuous non-zero values?
Here the result would be a list containing many arrays:
([0, 1, 2, 3], [9, 10, 11], [14, 15], [20, 21])
>>> L = [1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1]
>>> import itertools
>>> import operator
>>> [[i for i,value in it] for key,it in itertools.groupby(enumerate(L), key=operator.itemgetter(1)) if key != 0]
[[0, 1, 2, 3], [9, 10, 11], [14, 15], [20, 21]]
A trivial change to my answer at Finding the consecutive zeros in a numpy array gives the function find_runs:
def find_runs(value, a):
# Create an array that is 1 where a is `value`, and pad each end with an extra 0.
isvalue = np.concatenate(([0], np.equal(a, value).view(np.int8), [0]))
absdiff = np.abs(np.diff(isvalue))
# Runs start and end where absdiff is 1.
ranges = np.where(absdiff == 1)[0].reshape(-1, 2)
return ranges
For example,
In [43]: x
Out[43]: array([1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1])
In [44]: find_runs(1, x)
Out[44]:
array([[ 0, 4],
[ 9, 12],
[14, 16],
[20, 22]])
In [45]: [range(*run) for run in find_runs(1, x)]
Out[45]: [[0, 1, 2, 3], [9, 10, 11], [14, 15], [20, 21]]
If the value 1 in your example was not representative, and you really want runs of any non-zero values (as suggested by the text of the question), you can change np.equal(a, value) to (a != 0) and change the arguments and comments appropriately. E.g.
def find_nonzero_runs(a):
# Create an array that is 1 where a is nonzero, and pad each end with an extra 0.
isnonzero = np.concatenate(([0], (np.asarray(a) != 0).view(np.int8), [0]))
absdiff = np.abs(np.diff(isnonzero))
# Runs start and end where absdiff is 1.
ranges = np.where(absdiff == 1)[0].reshape(-1, 2)
return ranges
For example,
In [63]: y
Out[63]:
array([-1, 2, 99, 99, 0, 0, 0, 0, 0, 12, 13, 14, 0, 0, 1, 1, 0,
0, 0, 0, 42, 42])
In [64]: find_nonzero_runs(y)
Out[64]:
array([[ 0, 4],
[ 9, 12],
[14, 16],
[20, 22]])
Have a look at scipy.ndimage.measurements.label:
import numpy as np
from scipy.ndimage.measurements import label
x = np.asarray([1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1])
labelled, numfeats = label(x)
indices = [np.nonzero(labelled == k) for k in np.unique(labelled)[1:]]
indices contains exactly what you asked for. Note that, depending on your ultimate goal, labelled might also give you useful (extra) information.
You can use np.split, once you know the interval of non-zeros' lengths and the corresponding indices in A. Assuming A as the input array, the implementation would look something like this -
# Append A on either sides with zeros
A_ext = np.diff(np.hstack(([0],A,[0])))
# Find interval of non-zeros lengths
interval_lens = np.where(A_ext==-1)[0] - np.where(A_ext==1)[0]
# Indices of non-zeros places in A
idx = np.arange(A.size)[A!=0]
# Finally split indices based on the interval lengths
out = np.split(idx,interval_lens.cumsum())[:-1]
Sample input, output -
In [53]: A
Out[53]: array([1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1])
In [54]: out
Out[54]: [array([0, 1, 2, 3]), array([ 9, 10, 11]), array([14, 15]), array([20, 21])]
I am stuck on a little issue in the project I am currently working on.
Getting straight to the point, let's assume I have a 2-dimensional numpy.array - I will call it arr.
I need to slice arr, but this slice must contain some padding depending on the selected interval.
Example:
arr = numpy.array([
[ 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10],
[ 11, 12, 13, 14, 15],
[ 16, 17, 18, 19, 20],
[ 21, 22, 23, 24, 25]
])
Actually, numpy's response for arr[3:7, 3:7] is:
array([[19, 20],
[24, 25]])
But I need it to be padded as if arr were bigger than it really is.
Here is what I need as response for arr[3:7, 3:7]:
array([[19, 20, 0, 0],
[24, 25, 0, 0],
[ 0, 0, 0, 0],
[ 0, 0, 0, 0]])
This padding should also occur in case of negative indices. If the requested slice is bigger than the whole image, padding must occur in all sides, if needed.
Another example, negative indices. This is the expected result for arr[-2:2, -1:3]:
array([[ 0, 0, 0, 0],
[ 0, 0, 1, 2],
[ 0, 0, 6, 7],
[ 0, 0, 11, 12]])
Is there any native numpy function for this? If not, any idea of how can I implement this?
About the first part of your question you can use a simple indexing, and you can create a zero_like of your array with numpy.zeros_like then assign the special part :
>>> new=numpy.zeros_like(arr)
>>> part=arr[3:7, 3:7]
>>> i,j=part.shape
>>> new[:i,:j]=part
>>> new
array([[19, 20, 0, 0, 0],
[24, 25, 0, 0, 0],
[ 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0]])
But for the second case you can not use a negative indexing for for numpy arrays like this.Negative indices are interpreted as counting from the end of the array so if you are counting from -2 actually in a 5x5 array there are not any row between -2 and 2 so the result would be an empty array :
>>> arr[-2:2]
array([], shape=(0, 5), dtype=int64)
You can do something like:
print np.lib.pad(arr[3:7,3:7], ((0, 2), (0, 2)), 'constant', constant_values=(0,0 ))
[[19 20 0 0]
[24 25 0 0]
[ 0 0 0 0]
[ 0 0 0 0]]
For the negative indexing:
print np.lib.pad(arr[ max(0,-1):3 , 0:2 ], ((1, 0), (2, 0)), 'constant', constant_values=(0,0 ))
[[ 0 0 0 0]
[ 0 0 1 2]
[ 0 0 6 7]
[ 0 0 11 12]]
Check here for reference
import numpy as np
def convert(inarr, x1, x2, y1, y2):
xd = x2 - x1
yd = y2 - y1
outarr = np.zeros(xd * yd).reshape(xd, yd)
x1fr = max(0, x1)
x2fr = min(x2, inarr.shape[0])
y1fr = max(0, y1)
y2fr = min(y2, inarr.shape[1])
x1to = max(0, xd - x2)
x2to = x1to + x2fr - x1fr
y1to = max(0, yd - y2)
y2to = y1to + y2fr - y1fr
outarr[x1to:x2to, y1to:y2to] = inarr[x1fr:x2fr, y1fr:y2fr]
return outarr
arr = np.array([[ 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25]])
print(convert(arr, -2, 2, -1, 3))
Well this works but returns
[[ 0. 0. 0. 0.]
[ 0. 0. 0. 0.]
[ 0. 1. 2. 3.]
[ 0. 6. 7. 8.]]
for your -ve index example. You can play around to get it to do what you expect