I'm trying to access a numpy array A using another array B providing the indices at each position:
A = np.array([[1,2],[3,4]])
B = np.array([[[0,0],[0,0]],[[0,1],[0,1]]])
Desired output:
C = array([[1,1],[3,3]])
I haven't gotten it to work using np.take() or the advanced indexing.
I could do it iteratively but my arrays are on the order of 10**7 so I was hoping for a faster way.
I probably should have insisted on seeing the iterative solution first, but here's the array one:
In [45]: A[B[:,:,1], B[:,:,0]]
Out[45]:
array([[1, 1],
[3, 3]])
I first tried A[B[:,:,0], B[:,:,1]], the natural order of the inner dimension. Your own code could have saved me that trial.
The key with advanced indexing is that you have to create or define separate arrays (broadcastable) for each dimension. We can think of that index as a tuple:
idx = (B[:,:,0], B[:,:,1])
A[idx]
Adding on #hpaulj an alternative way is:
idx = tuple(B[:,:,[1,0]].transpose(2,0,1))
A[idx]
# array([[1, 1], [3, 3]])
Related
For a 2D numpy array A, the loop for a in A will loop through all the rows in A. This functionality is what I want for my code, but I'm having difficulty with the edge case where A only has one row (i.e., is essentially a 1-dimensional array). In this case, the for loop treats A as a 1D array and iterates through its elements. What I want to instead happen in this case is a natural extension of the 2D case, where the loop retrieves the (single) row in A. Is there a way to format the array A such that the for loop functions like this?
Depending on if you declare the array yourself you can do this:
A = np.array([[1, 2, 3]])
Else you can check the dim of your array before iterating over it
B = np.array([1, 2, 3])
if B.ndim == 1:
B = B[None, :]
Or you can use the function np.at_least2d
C = np.array([1, 2, 3])
C = np.atleast_2d(C)
If your array trully is a 2D array, even with one row, there is no edge case:
import numpy
a = numpy.array([[1, 2, 3]])
for line in a:
print(line)
>>> [1 2 3]
You seem to be confusing numpy.array([[1, 2, 3]]) which is a 2D array of one line and numpy.array([1, 2, 3]) which would be a 1D array.
I think you can use np.expand_dims to achieve your goal
X = np.expand_dims(X, axis=0)
Example of what I want to do:
import numpy as np
values = np.array([7, 7, 5, 2, 3, 9])
indices = np.array([
np.array([3,5]),
np.array([4]),
np.array([1,2,3])
])
>>> values[indices]
array([
array([2,9]),
array([3]),
array([7,5,2]),
])
Is it possible to achieve this using vectorization?
Right now I'm doing it with a for loop, but it can get slow.
Thanks!
We could concatenate the indices, index into values with those and finally split back -
idx = np.concatenate(indices)
all_out = values[idx]
lens = list(map(len,indices))
ssidx = np.r_[0,lens].cumsum()
out = [all_out[i:j] for (i,j) in zip(ssidx[:-1],ssidx[1:])]
For completeness, here's the straight-forward indexing based version -
[values[i] for i in indices]
So, with the proposed method we are making use of slicing and hence reducing per-iteration workload. As such, alongwith the step to get idx that needs concatenation of all indices in the proposed one, it makes sense for the case with small indexing arrays in indices.
I have a few simple questions I'm not able to find the answer to. They are both stated in the following example code. Thank you for any help!
import numpy as np
#here are two arrays to join together
a = np.array([1,2,3,4,5])
b = np.array([6,7,8,9,10])
#here comes the joining step I don't know how to do better
#QUESTION 1: How to form all permutations of two 1D arrays?
temp = np.array([]) #empty array to be filled with values
for aa in a:
for bb in b:
temp = np.append(temp,[aa,bb]) #fill the array
#QUESTION 2: Why do I have to reshape? How can I avoid this?
temp = temp.reshape((int(temp.size/2),2))
edit: made code more minimal
To answer your first question, you can use np.meshgrid to form those combinations between elements of the two input arrays and get to the final version of temp in a vectorized manner avoiding those loops, like so -
np.array(np.meshgrid(a,b)).transpose(2,1,0).reshape(-1,2)
As seen, we would still need a reshape if you intend to get a 2-column output array.
There are other ways we could construct the array with the meshed structure and thus avoid a reshape. One of those ways would be with np.column_stack, as shown below -
r,c = np.meshgrid(a,b)
temp = np.column_stack((r.ravel('F'), c.ravel('F')))
The proper way to build an array iteratively is with list append. np.append is poorly named, and often mis used.
In [274]: a = np.array([1,2,3,4,5])
...: b = np.array([6,7,8,9,10])
...:
In [275]: temp = []
In [276]: for aa in a:
...: for bb in b:
...: temp.append([aa,bb])
...:
In [277]: temp
Out[277]:
[[1, 6],
[1, 7],
[1, 8],
[1, 9],
[1, 10],
[2, 6],
....
[5, 9],
[5, 10]]
In [278]: np.array(temp).shape
Out[278]: (25, 2)
It's better to avoid loops at all, but if you must, use this list append approach.
I have a 2D numpy array and I want to change this array into a 1D which is sorted. For example:
A = [[1,0,2],
[0,3,0]]
I want this to be like:
B = [3,2,1,0,0,0]
Any idea how I can do this using python modules and not to write a sorting algorithm or anything like that ?
Thanks
Assuming you are looking to sort them in descending order -
In [127]: A
Out[127]: [[1, 0, 2], [0, 3, 0]]
In [128]: B = np.sort(np.array(A).ravel())
In [129]: B[::-1]
Out[129]: array([3, 2, 1, 0, 0, 0])
Basically, it involves three steps: Flatten the array with ravel(), sort it with np.sort and then reverse the indexing for an effect of descending order sorting.
This is my solution:
A = [[1,0,2], [0,3,0]]
B = []
for i in range(len(A)):
for j in range(len(A[i])):
B.append(A[i][j])
B.sort(cmp=None, key=None, reverse=True)
print B
You can see my code running here: http://ideone.com/P8xBPK
Say that I have 4 numpy arrays
[1,2,3]
[2,3,1]
[3,2,1]
[1,3,2]
In this case, I've determined [1,2,3] is the "minimum array" for my purposes, as it is one of two arrays with lowest value at index 0, and of those two arrays it has the the lowest index 1. If there were more arrays with similar values, I would need to compare the next index values, and so on.
How can I extract the array [1,2,3] in that same order from the pile?
How can I extend that to x arrays of size n?
Thanks
Using the python non-numpy .sort() or sorted() on a list of lists (not numpy arrays) automatically does this e.g.
a = [[1,2,3],[2,3,1],[3,2,1],[1,3,2]]
a.sort()
gives
[[1,2,3],[1,3,2],[2,3,1],[3,2,1]]
The numpy sort seems to only sort the subarrays recursively so it seems the best way would be to convert it to a python list first. Assuming you have an array of arrays you want to pick the minimum of you could get the minimum as
sorted(a.tolist())[0]
As someone pointed out you could also do min(a.tolist()) which uses the same type of comparisons as sort, and would be faster for large arrays (linear vs n log n asymptotic run time).
Here's an idea using numpy:
import numpy
a = numpy.array([[1,2,3],[2,3,1],[3,2,1],[1,3,2]])
col = 0
while a.shape[0] > 1:
b = numpy.argmin(a[:,col:], axis=1)
a = a[b == numpy.min(b)]
col += 1
print a
This checks column by column until only one row is left.
numpy's lexsort is close to what you want. It sorts on the last key first, but that's easy to get around:
>>> a = np.array([[1,2,3],[2,3,1],[3,2,1],[1,3,2]])
>>> order = np.lexsort(a[:, ::-1].T)
>>> order
array([0, 3, 1, 2])
>>> a[order]
array([[1, 2, 3],
[1, 3, 2],
[2, 3, 1],
[3, 2, 1]])