In the documentation of numpy.take, it is stated that a is indexed according to indices and axis, then the result is optionally stored in the out parameter. Does exists a function that perform the indexing on out instead? Using fancy indexing it would be something like:
out[:, :, indices, :] = a
Here I assume that axis=2 but in my case I don't know the axis in advance.
A solution using 1d boolean masks instead of indices is acceptable as well.
You can use swapaxes like so:
>>> A = np.arange(24).reshape(2,3,4)
>>> out = np.empty_like(A)
>>> I = [2,0,1]
>>> axis = 1
>>> out.swapaxes(0, axis)[I] = A.swapaxes(0, axis)
>>> out
array([[[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[ 0, 1, 2, 3]],
[[16, 17, 18, 19],
[20, 21, 22, 23],
[12, 13, 14, 15]]])
Some of the numpy functions construct an indexing tuple when operating on a specified axis.
The code isn't particularly pretty, but is general and reasonably efficient.
In [700]: out = np.zeros((2,1,4,5),int)
In [701]: out
Out[701]:
array([[[[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0]]],
[[[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0]]]])
In [702]: indices = [3,0,1]
Make an indexing tuple. Start with a list or array for ease of construction, and then convert to tuple when indexing:
In [703]: idx = [slice(None)]*out.ndim
In [704]: idx[2] = indices
In [705]: idx
Out[705]:
[slice(None, None, None),
slice(None, None, None),
[3, 0, 1],
slice(None, None, None)]
In [706]: out[tuple(idx)] = 10
In [707]: out
Out[707]:
array([[[[10, 10, 10, 10, 10],
[10, 10, 10, 10, 10],
[ 0, 0, 0, 0, 0],
[10, 10, 10, 10, 10]]],
[[[10, 10, 10, 10, 10],
[10, 10, 10, 10, 10],
[ 0, 0, 0, 0, 0],
[10, 10, 10, 10, 10]]]])
It matches the take:
In [708]: np.take(out, indices, axis=2)
Out[708]:
array([[[[10, 10, 10, 10, 10],
[10, 10, 10, 10, 10],
[10, 10, 10, 10, 10]]],
[[[10, 10, 10, 10, 10],
[10, 10, 10, 10, 10],
[10, 10, 10, 10, 10]]]])
We can set more complex values, as long as we get the broadcasting right:
out[tuple(idx)] = np.array([10,11,12])[...,None]
I have also seen numpy functions that shift the axis of interest to a known location - either beginning or end. Depending on the action it may require swapping back.
There are functions like place, put, copyto that provide other ways of controlling assignment (besides the usual indexing). But none take an axis parameter like np.take.
Related
I have a 2d array with shape(3,6), then i want to create a condition to check a value of each array.
my data arry is as follows :
array([[ 1, 2, 3, 4, 5, 6],
7, 8, 9, 10, 11, 12],
[13, 14, 15, 16, 17, 18]])
if in an array there are numbers < 10 then the value will be 0
the result I expected
array([[ 0, 0, 0, 0, 0, 0],
0, 0, 0, 10, 11, 12],
[13, 14, 15, 16, 17, 18]])
the code i created is like this, but why can't it work as i expected
FCDataNew = []
a = [ [1,2,3,4,5,6],
[7,8,9,10,11,12],
[13,14,15,16,17,18]
]
a = np.array(a)
c = 0
c = np.array(c)
for i in range(len(a)):
if a[i].all()<10:
FCDataNew.append(c)
else:
FCDataNew.append(a[i])
FCDataNew = np.array(FCDataNew)
FCDataNew
If you want to modify the array in place, use boolean indexing:
FCDataNew = np.array([[1,2,3,4,5,6],
[7,8,9,10,11,12],
[13,14,15,16,17,18],
])
FCDataNew[FCDataNew<10] = 0
For a copy:
out = np.where(FCDataNew<10, 0, FCDataNew)
Output:
array([[ 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 10, 11, 12],
[13, 14, 15, 16, 17, 18]])
You can just use arr[arr < 10] = 0
I have a Numpy array that looks like
array([1, 2, 3, 4, 5, 6, 7, 8])
and I want to reshape it to an array
array([[5, 0, 0, 6],
[0, 1, 2, 0],
[0, 3, 4, 0],
[7, 0, 0, 8]])
More specifically, I'm trying to reshape a 2D numpy array to get a 3D Numpy array to go from
array([[ 1, 2, 3, 4, 5, 6, 7, 8],
[ 9, 10, 11, 12, 13, 14, 15, 16],
[17, 18, 19, 20, 21, 22, 23, 24],
...
[ 9, 10, 11, 12, 13, 14, 15, 16],
[89, 90, 91, 92, 93, 94, 95, 96]])
to a numpy array that looks like
array([[[ 5, 0, 0, 6],
[ 0, 1, 2, 0],
[ 0, 3, 4, 0],
[ 7, 0, 0, 8]],
[[13, 0, 0, 14],
[ 0, 9, 10, 0],
[ 0, 11, 12, 0],
[15, 0, 0, 16]],
...
[[93, 0, 0, 94],
[ 0, 89, 90, 0],
[ 0, 91, 92, 0],
[95, 0, 0, 96]]])
Is there an efficient way to do this using numpy functionality, particularly vectorized?
We can make use of slicing -
def expand(a): # a is 2D array
out = np.zeros((len(a),4,4),dtype=a.dtype)
out[:,1:3,1:3] = a[:,:4].reshape(-1,2,2)
out[:,::3,::3] = a[:,4:].reshape(-1,2,2)
return out
The benefit is memory and hence perf. efficiency, as only the output would occupy memory space. The steps involved work with views thanks to the slicing on the input and output.
Sample run -
2D input :
In [223]: a
Out[223]:
array([[ 1, 2, 3, 4, 5, 6, 7, 8],
[ 9, 10, 11, 12, 13, 14, 15, 16]])
In [224]: expand(a)
Out[224]:
array([[[ 5, 0, 0, 6],
[ 0, 1, 2, 0],
[ 0, 3, 4, 0],
[ 7, 0, 0, 8]],
[[13, 0, 0, 14],
[ 0, 9, 10, 0],
[ 0, 11, 12, 0],
[15, 0, 0, 16]]])
1D input (feed in 2D extended input with None) :
In [225]: a = np.array([1, 2, 3, 4, 5, 6, 7, 8])
In [226]: expand(a[None])
Out[226]:
array([[[5, 0, 0, 6],
[0, 1, 2, 0],
[0, 3, 4, 0],
[7, 0, 0, 8]]])
I have a vector A = [0,1,2,3,0,0,1,1,2,2,3,3]. I need to sort it in an increasing matter such that it is listed in an ordered fashion and from that extract the argsort. To better explain this I need to sort A to such that it returns B = [0,4,5,1,6,7,2,8,9,3,10,11]. However, when I use pyotrch's torch.argsort(A) it returns B = [4,5,0,1,6,7,2,8,9,3,10,11].
I'm assuming the algorithm that does so cannot be controlled on my end. Is there anyway to approach this without introducing for loops? Such operation are part of my NN model and will cause performance issues if not done efficiently. Thanks!
Here is a pure PyTorch based solution leveraging broadcasting, torch.unique(), and torch.nonzero(). This would be of great boost particularly for a GPU based implementation/run, which is not possible if we have to switch back to NumPy, argsort it and then transfer back to PyTorch (as suggested in other approaches).
# our input tensor
In [50]: A = torch.tensor([0,1,2,3,0,0,1,1,2,2,3,3])
# construct an intermediate boolean tensor
In [51]: boolean = A[:, None] == torch.unique(A)
In [52]: boolean
Out[52]:
tensor([[1, 0, 0, 0],
[0, 1, 0, 0],
[0, 0, 1, 0],
[0, 0, 0, 1],
[1, 0, 0, 0],
[1, 0, 0, 0],
[0, 1, 0, 0],
[0, 1, 0, 0],
[0, 0, 1, 0],
[0, 0, 1, 0],
[0, 0, 0, 1],
[0, 0, 0, 1]], dtype=torch.uint8)
Once we have this boolean tensor, we can find the desired indices by checking for positions where there is an 1 after transposing the boolean tensor.
That would give us both sorted input and the indices. Since we want only the indices, we can just grab those by indexing for the last column (1 or -1)
In [53]: torch.nonzero(boolean.t())[:, -1]
Out[53]: tensor([ 0, 4, 5, 1, 6, 7, 2, 8, 9, 3, 10, 11])
Here's the result for one more example provided by OP in the comments:
In [55]: A_large = torch.tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9])
In [56]: boolean_large = A_large[:, None] == torch.unique(A_large)
In [57]: torch.nonzero(boolean_large.t())[:, -1]
Out[57]:
tensor([ 0, 10, 11, 1, 12, 13, 2, 14, 15, 3, 16, 17, 4, 18, 19, 5, 20, 21,
6, 22, 23, 7, 24, 25, 8, 26, 27, 9, 28, 29])
Note: Unlike with NumPy-based solution proposed in other answers, here we don't have to worry about what kind of sorting algorithm we've to use because we are not using any sorting at all.
Here is one way:
sort the numpy array using numpy.argsort()
convert the result into tensor using torch.from_numpy()
import torch
import numpy as np
A = [0,1,2,3,0,0,1,1,2,2,3,3]
x = np.array(A)
y = torch.from_numpy(np.argsort(x, kind='mergesort'))
print(y)
I'd like to know how to make a simple data cube (matrix) with three 1D arrays or if there's a simpler way. I want to be able to call a specific value at the end from the cube such as cube[0,2,6].
x = arange(10)
y = arange(10,20,1)
z = arange(20,30,1)
cube = meshgrid(x,y,z)
But this doesn't give the desired result, as it gives mulitple arrays and can't call a specific number easily. I'd like to be able to use this for large data sets that would be laborious to do by hand, later on. Thanks
meshgrid as its name suggests creates an orthogonal mesh. If you call it with 3 arguments it will be a 3d mesh. Now the mesh is 3d arrangement of points but each point has 3 coordinates. Therefore meshgrid returns 3 arrays one for each coordinate.
The standard way of getting one 3d array out of that is to apply a vectorised function with three arguments. Here is a simple example:
>>> x = arange(7)
>>> y = arange(0,30,10)
>>> z = arange(0,200,100)
>>> ym, zm, xm = meshgrid(y, z, x)
>>> xm
array([[[0, 1, 2, 3, 4, 5, 6],
[0, 1, 2, 3, 4, 5, 6],
[0, 1, 2, 3, 4, 5, 6]],
[[0, 1, 2, 3, 4, 5, 6],
[0, 1, 2, 3, 4, 5, 6],
[0, 1, 2, 3, 4, 5, 6]]])
>>> ym
array([[[ 0, 0, 0, 0, 0, 0, 0],
[10, 10, 10, 10, 10, 10, 10],
[20, 20, 20, 20, 20, 20, 20]],
[[ 0, 0, 0, 0, 0, 0, 0],
[10, 10, 10, 10, 10, 10, 10],
[20, 20, 20, 20, 20, 20, 20]]])
>>> zm
array([[[ 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0]],
[[100, 100, 100, 100, 100, 100, 100],
[100, 100, 100, 100, 100, 100, 100],
[100, 100, 100, 100, 100, 100, 100]]])
>>> cube = xm + ym + zm
>>> cube
array([[[ 0, 1, 2, 3, 4, 5, 6],
[ 10, 11, 12, 13, 14, 15, 16],
[ 20, 21, 22, 23, 24, 25, 26]],
[[100, 101, 102, 103, 104, 105, 106],
[110, 111, 112, 113, 114, 115, 116],
[120, 121, 122, 123, 124, 125, 126]]])
>>> cube[0, 2, 6]
26
Let's say I have a 2d numpy array of the size (5,5). I can get the neighbors of the index (i,j) with the following statement:
a = range(25)
a = np.reshape(a, (5,5))
n = a[i-1:i+2, j-1:j+2]
That works great for 0 < i,j < 4. My problem is that I always want to get a 3x3 array but if one of the indices is 0 or 4 I do not get it (in case i=0 the range is (-1, 2) = (4, 2) and we get an empty range)
Do you have any ideas how I can always get a 3x3 matrix and fill the "failed indices" with zeros?
Use np.pad to extend your array first and index into the result instead. You'll have to shift your indices accordingly.
>>> b = np.pad(a, pad_width=1, mode='constant')
>>> b
array([[ 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 1, 2, 3, 4, 0],
[ 0, 5, 6, 7, 8, 9, 0],
[ 0, 10, 11, 12, 13, 14, 0],
[ 0, 15, 16, 17, 18, 19, 0],
[ 0, 20, 21, 22, 23, 24, 0],
[ 0, 0, 0, 0, 0, 0, 0]])
>>>