I have 0s and 1s store in a 3-dimensional numpy array:
g = np.array([[[0, 1], [0, 1], [1, 0]], [[0, 0], [1, 0], [1, 1]]])
# array([
# [[0, 1], [0, 1], [1, 0]],
# [[0, 0], [1, 0], [1, 1]]])
and I'd like to replace these values by those in another array using a row-wise replacement strategy. For example, replacing the vales of g by x:
x = np.array([[2, 3], [4, 5]])
array([[2, 3],
[4, 5]])
to obtain:
array([
[[2, 3], [2, 3], [3, 2]],
[[4, 4], [5, 4], [5, 5]]])
The idea here would be to have the first row of g replaced by the first elements of x (0 becomes 2 and 1 becomes 3) and the same for the other row (the first dimension - number of "rows" - will always be the same for g and x)
I can't seem to be able to use np.where because there's a ValueError: operands could not be broadcast together with shapes (2,3,2) (2,2) (2,2).
IIUC,
np.stack([x[i, g[i]] for i in range(x.shape[0])])
Output:
array([[[2, 3],
[2, 3],
[3, 2]],
[[4, 4],
[5, 4],
[5, 5]]])
Vectorized approach with np.take_along_axis to index into the last axis of x with g using axis=-1 -
In [20]: np.take_along_axis(x[:,None],g,axis=-1)
Out[20]:
array([[[2, 3],
[2, 3],
[3, 2]],
[[4, 4],
[5, 4],
[5, 5]]])
Or with manual integer-based indexing -
In [27]: x[np.arange(len(g))[:,None,None],g]
Out[27]:
array([[[2, 3],
[2, 3],
[3, 2]],
[[4, 4],
[5, 4],
[5, 5]]])
One solution, is to simply use comprehension directly here:
>>> np.array([[x[i][c] for c in r] for i, r in enumerate(g)])
array([[[2, 3],
[2, 3],
[3, 2]],
[[4, 4],
[5, 4],
[5, 5]]])
From what I understand, g is an array of indexes (indexes being 0 or 1) and x is the array to who's values you use.
Something like this should work (tested quickly)
import numpy as np
def swap_indexes(index_array, array):
out_array = []
for i, row in enumerate(index_array):
out_array.append([array[i,indexes] for indexes in row])
return np.array(out_array)
index_array = np.array([[[0, 1], [0, 1], [1, 0]], [[0, 0], [1, 0], [1, 1]]])
x = np.array([[2, 3], [4, 5]])
print(swap_indexes(index_array, x))
[EDIT: fixed typo that created duplicates]
Related
I'd like to ask how can I efficiently generate a numpy 3D array from a 2D array with each row filling the diagonal part of the new array?
For example, the input 2D array is
array([[1, 2],
[3, 4],
[5, 6],
[7, 8]])
and I want the output to be
array([[[1, 0],
[0, 2]],
[[3, 0],
[0, 4]],
[[5, 0],
[0, 6]],
[[7, 0],
[0, 8]]])
Typically, the size of the first dimensional is very large. Thanks in advance.
Assuming a the input and using indexing with unravel_index:
x, y = np.unravel_index(np.arange(a.size), a.shape)
out = np.zeros(a.shape+(a.shape[-1],), dtype=a.dtype)
out[x, y, y] = a.flat
Output:
array([[[1, 0],
[0, 2]],
[[3, 0],
[0, 4]],
[[5, 0],
[0, 6]],
[[7, 0],
[0, 8]]])
timings:
arr = np.array([[1, 2], [3, 4], [5, 6], [7, 8]])
res = np.apply_along_axis(np.diag, 1, arr)
Consider the following array:
x = np.array([[1, 1],[1, 1], [1, 2], [1, 2], [1, 2],
[2, 3], [2, 3], [2, 3], [2, 4], [2, 4]])
x
Out[12]:
array([[1, 1],
[1, 1],
[1, 2],
[1, 2],
[1, 2],
[2, 3],
[2, 3],
[2, 3],
[2, 4],
[2, 4],
[2, 5],
[2, 5],
[2, 5]])
How would I get the number of unique column 2 values for each column 1 value?
For example: if it can be done using a function V, then V(x) = [2, 3].
I have implemented this using a for loop. However, it seems more complicated than necessary and takes too much time (when applied to my actual dataset which is much larger than this example).
I am interested in performance and am willing to sacrifice code clarity for speed (although they usually are directly correlated!).
Use numpy.unique twice:
import numpy as np
x = np.array([[1, 1],[1, 1], [1, 2], [1, 2], [1, 2],
[2, 3], [2, 3], [2, 3], [2, 4], [2, 4]])
# drop duplicates
xx = np.unique(x, axis=0)
# count the first column
values, counts = np.unique(xx[:,0], return_counts=True)
print(values)
print(counts)
# [1, 2]
# [2, 2]
I have a large n x 2 numpy array that is formatted as (x, y) coordinates. I would like to filter this array so as to:
Identify coordinate pairs with duplicated x-values.
Keep only the coordinate pair of those duplicates with the highest y-value.
For example, in the following array:
arr = [[1, 4]
[1, 8]
[2, 3]
[4, 6]
[4, 2]
[5, 1]
[5, 2]
[5, 6]]
I would like the result to be:
arr = [[1, 8]
[2, 3]
[4, 6]
[5, 6]]
Ive explored np.unique and np.where but cannot figure out how to leverage them to solve this problem. Thanks so much!
Here's one way based on np.maximum.reduceat -
def grouby_maxY(a):
b = a[a[:,0].argsort()] # if first col is already sorted, skip this
grp_idx = np.flatnonzero(np.r_[True,(b[:-1,0] != b[1:,0])])
grp_maxY = np.maximum.reduceat(b[:,1], grp_idx)
return np.c_[b[grp_idx,0], grp_maxY]
Alternatively, if you want to bring np.unique, we can use it to find grp_idx with np.unique(b[:,0], return_index=1)[1].
Sample run -
In [453]: np.random.seed(0)
In [454]: arr = np.random.randint(0,5,(10,2))
In [455]: arr
Out[455]:
array([[4, 0],
[3, 3],
[3, 1],
[3, 2],
[4, 0],
[0, 4],
[2, 1],
[0, 1],
[1, 0],
[1, 4]])
In [456]: grouby_maxY(arr)
Out[456]:
array([[0, 4],
[1, 4],
[2, 1],
[3, 3],
[4, 0]])
>>> c= array([[[1, 2],
[3, 4]],
[[2, 1],
[4, 3]],
[[3, 2],
[1, 4]]])
>>> x
array([[0, 1, 2],
[3, 4, 5]])
return me a matrix such that each column is the product of each matrix in c multiply the each corresponding column of x in regular matrix multiplication. I'm trying to figure out a way to vectorized it or at least not using for loop to solve it.
array([[6, 6, 16]
12, 16, 22]])
to extends this operation further let's say that I have an array of matrices,say
>>> c
array([[[1, 2],
[3, 4]],
[[2, 1],
[4, 3]],
[[3, 2],
[1, 4]]])
>>> x
array([[[1, 2, 3],
[1, 2, 3]],
[[1, 0, 2],
[1, 0, 2]],
[[2, 3, 1],
[0, 1, 0]]])
def fun(c,x):
for i in range(len(x)):
np.einsum('ijk,ki->ji',c,x[i])
##something
So basically, I want to have each matrix in x multiply with all of c. return a structure similar to c without introducing this for loop
The reason I'm doing this because I've encounter a problem to solve a problem ,trying to vectorized
Xc (the operation follows the normal matrix column vector multiplication), c is 3D array; like the c from above-- a column vector that each element is a matrix (in numpy its the form in the above). X is the matrix with each elements is a 1D array. The output of the Xc should be 1D array.
You can use np.einsum -
np.einsum('ijk,ki->ji',c,x)
Sample run -
In [155]: c
Out[155]:
array([[[1, 2],
[3, 4]],
[[2, 1],
[4, 3]],
[[3, 2],
[1, 4]]])
In [156]: x
Out[156]:
array([[0, 1, 2],
[3, 4, 5]])
In [157]: np.einsum('ijk,ki->ji',c,x)
Out[157]:
array([[ 6, 6, 16],
[12, 16, 22]])
For the 3D case of x, simply append the new dimension at the start of the string notation for x and correspondingly at the output string notation too, like so -
np.einsum('ijk,lki->lji',c,x)
Sample run -
In [151]: c
Out[151]:
array([[[1, 2],
[3, 4]],
[[2, 1],
[4, 3]],
[[3, 2],
[1, 4]]])
In [152]: x
Out[152]:
array([[[1, 2, 3],
[1, 2, 3]],
[[1, 0, 2],
[1, 0, 2]],
[[2, 3, 1],
[0, 1, 0]]])
In [153]: np.einsum('ijk,lki->lji',c,x)
Out[153]:
array([[[ 3, 6, 15],
[ 7, 14, 15]],
[[ 3, 0, 10],
[ 7, 0, 10]],
[[ 2, 7, 3],
[ 6, 15, 1]]])
I have a 3D array as follow, 'b', which I want to represent an array of 2-D array. I want to remove the duplicates of my 2-D arrays and get the unique ones.
>>> a = [[[1, 2], [1, 2]], [[1, 2], [4, 5]], [[1, 2], [1, 2]]]
>>> b = numpy.array(a)
>>> b
array([[[1, 2],
[1, 2]],
[[1, 2],
[4, 5]],
[[1, 2],
[1, 2]]])
In this above example, I really want to return the following because there exist one duplicate which I want to remove.
unique = array([[[1, 2],
[1, 2]],
[[1, 2],
[4, 5]])
How should do this with numpy package? Thanks
See previous answer: Remove duplicate rows of a numpy array
convert to array of tuples and then apply np.unique()
Converting to tuple and back again is probably going to be quire expensive, instead you can do a generalized view:
def unique_by_first(a):
tmp = a.reshape(a.shape[0], -1)
b = np.ascontiguousarray(tmp).view(np.dtype((np.void, tmp.dtype.itemsize * tmp.shape[1])))
_, idx = np.unique(b, return_index=True)
return a[idx].reshape(-1, *a.shape[1:])
Usage:
print unique_by_first(a)
[[[1 2]
[1 2]]
[[1 2]
[4 5]]]
Effectively, a generalization of previous answers.
You can convert each such 2D slice off the last two axes into a scalar each by considering them as indices on a multi-dimensional grid. The intention is to map each such slice to a scalar based on their uniqueness. Then, using those scalars, we could use np.unique to keep one instance only.
Thus, an implementation would be -
idx = np.ravel_multi_index(a.reshape(a.shape[0],-1).T,a.max(0).ravel()+1)
out = a[np.sort(np.unique(idx, return_index=1)[1])]
Sample run -
In [43]: a
Out[43]:
array([[[8, 1],
[2, 8]],
[[3, 8],
[3, 4]],
[[2, 4],
[1, 0]],
[[3, 0],
[4, 8]],
[[2, 4],
[1, 0]],
[[8, 1],
[2, 8]]])
In [44]: idx = np.ravel_multi_index(a.reshape(a.shape[0],-1).T,a.max(0).ravel()+1)
In [45]: a[np.sort(np.unique(idx, return_index=1)[1])]
Out[45]:
array([[[8, 1],
[2, 8]],
[[3, 8],
[3, 4]],
[[2, 4],
[1, 0]],
[[3, 0],
[4, 8]]])
If you don't mind the order of such slices being maintained, skip the np.sort() at the last step.
Reshape, find the unique rows, then reshape again.
Finding unique tuples by converting to a set.
import numpy as np
a = [[[1, 2], [1, 2]], [[1, 2], [4, 5]], [[1, 2], [1, 2]]]
b = np.array(a)
new_array = [tuple(row) for row in b.reshape(3,4)]
uniques = list(set(new_array))
output = np.array(uniques).reshape(len(uniques), 2, 2)
output
Out[131]:
array([[[1, 2],
[1, 2]],
[[1, 2],
[4, 5]]])