I am trying to implement a function, which can split a 3 dimensional numpy array in to 8 pieces, whilst keeping the order intact. Essentially I need the splits to be:
G[:21, :18,:25]
G[21:, :18,:25]
G[21:, 18:,:25]
G[:21, 18:,:25]
G[:21, :18,25:]
G[21:, :18,25:]
G[21:, 18:,25:]
G[:21, 18:,25:]
Where the original size of this particular matrix would have been 42, 36, 50. How is it possible to generalise these 8 "slices" so I do not have to hardcode all of them? essentially move the :in every possible position.
Thanks!
You could apply the 1d slice to successive (lists of) dimensions.
With a smaller 3d array
In [147]: X=np.arange(4**3).reshape(4,4,4)
A compound list comprehension produces a nested list. Here I'm using the simplest double split
In [148]: S=[np.split(z,2,0) for y in np.split(X,2,2) for z in np.split(y,2,1)]
In this case, all sublists have the same size, so I can convert it to an array for convenient viewing:
In [149]: SA=np.array(S)
In [150]: SA.shape
Out[150]: (4, 2, 2, 2, 2)
There are your 8 subarrays, but grouped (4,2).
In [153]: SAA = SA.reshape(8,2,2,2)
In [154]: SAA[0]
Out[154]:
array([[[ 0, 1],
[ 4, 5]],
[[16, 17],
[20, 21]]])
In [155]: SAA[1]
Out[155]:
array([[[32, 33],
[36, 37]],
[[48, 49],
[52, 53]]])
Is the order right? I can change it by changing the axis in the 3 split operations.
Another approach is write your indexing expressions as tuples
In [156]: x,y,z = 2,2,2 # define the split points
In [157]: ind = [(slice(None,x), slice(None,y), slice(None,z)),
(slice(x,None), slice(None,y), slice(None,z)),]
# and so on
In [158]: S1=[X[i] for i in ind]
In [159]: S1[0]
Out[159]:
array([[[ 0, 1],
[ 4, 5]],
[[16, 17],
[20, 21]]])
In [160]: S1[1]
Out[160]:
array([[[32, 33],
[36, 37]],
[[48, 49],
[52, 53]]])
Looks like the same order I got before.
That ind list of tuples can be produced with some sort of iteration and/or list comprehension. Maybe even using itertools.product or np.mgrid to generate the permutations.
An itertools.product version could look something like
In [220]: def foo(i):
return [(slice(None,x) if j else slice(x,None))
for j,x in zip(i,[2,2,2])]
In [221]: SAA = np.array([X[foo(i)] for i in
itertools.product(range(2),range(2),range(2))])
In [222]: SAA[-1]
Out[222]:
array([[[ 0, 1],
[ 4, 5]],
[[16, 17],
[20, 21]]])
product iterates the last value fastest, so the list is reversed (compared to your target).
To generate a particular order it may be easier to list the tuples explicitly, e.g.:
In [227]: [X[foo(i)] for i in [(1,1,1),(0,1,1),(0,0,1)]]
Out[227]:
[array([[[ 0, 1],
[ 4, 5]],
[[16, 17],
[20, 21]]]), array([[[32, 33],
[36, 37]],
[[48, 49],
[52, 53]]]), array([[[40, 41],
[44, 45]],
[[56, 57],
[60, 61]]])]
This highlights the fact that there are 2 distinct issues - generating the iteration pattern, and splitting the array based on this pattern.
Related
Here is the example:
import numpy as np
a = np.array([1,2],[3,4],[5,6])
b = np.array([7,8],[9,10],[11,12],[12,13])
what I want is to use each item in a to plus every item in b then plus them together. For example, [1,2] should plus every row in b 1+7=8,2+8=10, 8+10=18;1+9=10,2+10=12,10+12=22... The result would like to be that[[18,22,26,28...],[22,26,....],[26,30....]]
My question is how to fulfil that? I know use numpy can be more efficient than loop but how to use the matrix to calculate this?
I believe this does what you want:
>>> a = np.array([[1,2],[3,4],[5,6]])
>>> b = np.array([[7,8],[9,10],[11,12],[12,13]])
>>> np.sum(a, axis=1)[:,None] + np.sum(b, axis=1)[None,:]
array([[18, 22, 26, 28],
[22, 26, 30, 32],
[26, 30, 34, 36]])
You can use list comprehensions:
import numpy as np
a = np.array([[1, 2], [3, 4], [5, 6]])
b = np.array([[7, 8], [9, 10], [11, 12], [12, 13]])
[[sum(i) + sum(j) for j in b] for i in a]
Output:
[[18, 22, 26, 28], [22, 26, 30, 32], [26, 30, 34, 36]]
This can be done in the most precise and succinct way as follows:
np.einsum('ijk-> ij', a[:,None,:]+b)
Let me explain each step.It combines einsum and broadcasting concepts of numpy.
Step 1- a[:,None,:] reshapes matrix a to shape (3,1,2). This mid axis with value 1 is helpful for broadcasting.
Step 2- a[:,None,:] + b broadcasts a and adds matrix b to get a resultant matrix of shape (3,4,2).
Step 3- np.einsum('ijk-> ij', a[:,None,:]+b) does sum reduction along the last axis of matrix obtained from previous step.
I have the following matrix:
M = np.matrix([[1,2,3,4,5,6,7,8,9,10],
[11,12,13,14,15,16,17,18,19,20],
[21,22,23,24,25,26,27,28,29,30]])
And I receive a vector indexing the columns of the matrix:
index = np.array([1,1,2,2,2,2,3,4,4,4])
This vector has 4 different values, so my objective is to create a list containing four new matrices so that the first matrix is made by the first two columns of M, the second matrix is made by columns 3 to 6 and so on:
M1 = np.matrix([[1,2],[11,12],[21,22]])
M2 = np.matrix([[3,4,5,6],[13,14,15,16],[23,24,25,26]])
M3 = np.matrix([[7],[17],[27]])
M4 = np.matrix([[8,9,10],[18,19,20],[28,29,30]])
l = list(M1,M2,M3,M4)
I need to do this in a automated way, since the number of rows and columns of M as well as the indexing scheme are not fixed. How can I do this?
There are 3 points to note:
For a variable number of variables, as in this case, the recommended solution is to use a dictionary.
You can use simple numpy indexing for the individual case.
Unless you have a very specific reason, use numpy.array instead of numpy.matrix.
Combining these points, you can use a dictionary comprehension:
d = {k: np.array(M[:, np.where(index==k)[0]]) for k in np.unique(index)}
Result:
{1: array([[ 1, 2],
[11, 12],
[21, 22]]),
2: array([[ 3, 4, 5, 6],
[13, 14, 15, 16],
[23, 24, 25, 26]]),
3: array([[ 7],
[17],
[27]]),
4: array([[ 8, 9, 10],
[18, 19, 20],
[28, 29, 30]])}
import numpy as np
M = np.matrix([[1,2,3,4,5,6,7,8,9,10],
[11,12,13,14,15,16,17,18,19,20],
[21,22,23,24,25,26,27,28,29,30]])
index = np.array([1,1,2,2,2,2,3,4,4,4])
m = [[],[],[],[]]
for i,c in enumerate(index):
m[k-1].append(c)
for idx in m:
print M[:,idx]
this is a little hard coded, I assumed you will always want 4 matrixes and such.. you can change it for more generalisation
I am trying to obtain the top 2 values from each row in a matrix using argsort. The indexing is working, as in argsort is returning the correct values. However, when I put the argsort result as an index, it returns a 3 dimensional result.
For example:
test_mat = np.matrix([[0 for i in range(5)] for j in range(5)])
for i in range(5):
for j in range(5):
test_mat[i, j] = i * j
test_mat[range(2,3)] = test_mat[range(2,3)] * -1
last_two = range(-1, -3, -1)
index = np.argsort(test_mat, axis=1)
index = index[:, last_k]
This gives:
index.shape
Out[402]: (5L, 5L)
test_mat[index].shape
Out[403]: (5L, 5L, 5L)
Python is new to me and I find indexing to be very confusing in general even after reading the various array manuals. I spend more time trying to get the right values out of objects than actually solving problems. I'd welcome any tips on where to properly learn what is going on. Thanks.
You can use linear indexing to solve your case, like so -
# Say A is your 2D input array
# Get sort indices for the top 2 values in each row
idx = A.argsort(1)[:,::-1][:,:2]
# Get row offset numbers
row_offset = A.shape[1]*np.arange(A.shape[0])[:,None]
# Add row offsets with top2 sort indices giving us linear indices of
# top 2 elements in each row. Index into input array with those for output.
out = np.take( A, idx + row_offset )
Here's a step-by-step sample run -
In [88]: A
Out[88]:
array([[34, 45, 16, 20, 24],
[37, 13, 49, 37, 21],
[42, 36, 35, 24, 18],
[26, 28, 21, 13, 44]])
In [89]: idx = A.argsort(1)[:,::-1][:,:2]
In [90]: idx
Out[90]:
array([[1, 0],
[2, 3],
[0, 1],
[4, 1]])
In [91]: row_offset = A.shape[1]*np.arange(A.shape[0])[:,None]
In [92]: row_offset
Out[92]:
array([[ 0],
[ 5],
[10],
[15]])
In [93]: np.take( A, idx + row_offset )
Out[93]:
array([[45, 34],
[49, 37],
[42, 36],
[44, 28]])
You can directly get the top 2 values from each row with just sorting along the second axis and some slicing, like so -
out = np.sort(A,1)[:,:-3:-1]
I have a multidimensional array, in which I want to get 1D slices, something like mega_array[:, i, j, k, .....]
To do it, I try numpy.ndindex:
for idx in np.ndindex(mega_array.shape[1:]):
print mega_array[:, index]
But alas: this still gives me multidimensional slices, where only the dimension, other than first, are equal to one.
I want to use the slices as l-value, so, simple ravel() is not suitable here.
What should I use to get normal, 1D slices?
UPD: Here's a small example:
in_array = np.asarray([[7, 40], [777, 440]])
for index in np.ndindex(in_array.shape[1:]):
print "---"
print index
print in_array[:, index] # gives 2D array
UPD: Here's a 3D example:
in_array = np.asarray([[[7, 40, 5], [777, 440, 0]], [[8, 41, 6], [778, 441, 1]]])
print in_array
print in_array.shape
# print in_array[:, 0, 2]
for index in np.ndindex(in_array.shape[1:]):
print index
print in_array[:, index] # FAILS
# expected [7, 8], [40, 41], [5, 6], [778, 441] and so on.
You need to add a slice to the index.
In:
in_array = np.asarray([[7, 40], [777, 440]])
for index in np.ndindex(in_array.shape[1:]):
print "---"
print index
print in_array[:, index] # gives 2D array
index has values like (0,),(1,), i.e. tuples.
in_array[:,(1,)] is not the same as in_array[:,1]. To get the latter you need to use in_array[(slice(None),1)]. The slice must be part of the index tuple. We can do that by concatenating tuples.
in_array = np.asarray([[7, 40], [777, 440]])
for index in np.ndindex(in_array.shape[1:]):
print "---"
index = (slice(None),)+index
print index
print in_array[index]
printing:
---
(slice(None, None, None), 0)
[ 7 777]
---
(slice(None, None, None), 1)
[ 40 440]
Same adjustment should work with the nD array case
You can use np.dstack that stack arrays in sequence depth wise (along third axis). :
>>> a
array([[[ 7, 40, 5],
[777, 440, 0]],
[[ 8, 41, 6],
[778, 441, 1]]])
>>> np.dstack(a)
array([[[ 7, 8],
[ 40, 41],
[ 5, 6]],
[[777, 778],
[440, 441],
[ 0, 1]]])
Also based on your dimensions you can use other numpy joining functions :http://docs.scipy.org/doc/numpy/reference/routines.array-manipulation.html#joining-arrays
do you need to transpose the matrix to get the rows?
In [5]: in_array = numpy.asarray([[7, 40], [777, 440]])
In [6]: in_array.transpose()[0]
Out[6]: array([ 7, 777])
After you posted your 3D example, now I see what you wish to do. The following should work for arrays with more than 3 dimensions, too:
Save the number of items in the dimensions > 0,
nitems = np.product(in_array.shape[1:])
Reshape the array (similar to np.dstack pointed out by Kasra),
new_array = np.reshape(in_array, [in_array.shape[0], nitems])
Loop over new array:
for i in range(new_array.shape[1]):
print(new_array[:, i])
For me, that gives the expected output.
Given a numpy 2D array of points, aka 3D array with size of the 3rd dimension equals to 2, how do I get the minimum x and y coordinate over all points?
Examples:
First:
I edited my original example, since it was wrong.
data = np.array(
[[[ 0, 1],
[ 2, 3],
[ 4, 5]],
[[11, 12],
[13, 14],
[15, 16]]])
minx = 0 # data[0][0][0]
miny = 1 # data[0][0][1]
4 x 4 x 2:
Second:
array([[[ 0, 77],
[29, 12],
[28, 71],
[46, 17]],
[[45, 76],
[33, 82],
[14, 17],
[ 3, 18]],
[[99, 40],
[96, 3],
[74, 60],
[ 4, 57]],
[[67, 57],
[23, 81],
[12, 12],
[45, 98]]])
minx = 0 # data[0][0][0]
miny = 3 # data[2][1][1]
Is there an easy way to get now the minimum x and y coordinates of all points of the data? I played around with amin and different axis values, but nothing worked.
Clarification:
My array stores positions from different robots over time. First dimension is time, second is the index of an robot. The third dimension is then either x or y of a robots for a given time.
Since I want to draw their paths to pixels, I need to normalize my data, so that the points are as close as possible to the origin without getting negative. I thought that subtracting [minx,miny] from every point will do that for me.
alko's answer didn't work for me, so here's what I did:
import numpy as np
array = np.arange(15).reshape(5,3)
x,y = np.unravel_index(np.argmin(array),array.shape)
Seems you need consecutive min alongaxis. For your first example:
>>> np.min(np.min(data, axis=1), axis=0)
array([ 0, 1])
For the second:
>>> np.min(np.min(data, axis=1), axis=0)
array([0, 3])
The same expression can be stated (in numpy older than 1.7), as pointed out by #Jamie, s
>>> np.min(data, axis=(1, 0))
array([0, 3])
You should take the min of a slice of the array.
Assuming first coordinate is x and second is y
minx = min(a[:,0])
miny = min(a[:,1])
>>>a=np.array([[1,2],[3,4],[5,6]])
>>> a
array([[1, 2],
[3, 4],
[5, 6]])
>>> min(a[:,0])
1
>>> max(a[:,0])
5
>>> max(a[:,1])
6
minx = np.min(data[0])
miny = np.min(data[1])
You can use function numpy.amin to find the minima along the desired axis.