Related
I have a 40 x 40 x 40 array in Python and would like to extract all i values with k=10 index. I understand how to do this with a nxn array but not a nxnxn array.
Thanks
As per the Numpy Index documentation.
import numpy as np
arr = np.arange(3*4*5).reshape( 3,4,5 )
arr
# array([[[ 0, 1, 2, 3, 4],
# [ 5, 6, 7, 8, 9],
# [10, 11, 12, 13, 14],
# [15, 16, 17, 18, 19]],
# [[20, 21, 22, 23, 24],
# [25, 26, 27, 28, 29],
# [30, 31, 32, 33, 34],
# [35, 36, 37, 38, 39]],
# [[40, 41, 42, 43, 44],
# [45, 46, 47, 48, 49],
# [50, 51, 52, 53, 54],
# [55, 56, 57, 58, 59]]])
arr[:,:,1] # For k = 1
# or arr[ ...,1] as its the last index
This returns a 2d array.
array([[ 1, 6, 11, 16],
[21, 26, 31, 36],
[41, 46, 51, 56]])
Let's assume that I have input of 4x4 image with 3 channels with following pixel values:
And I want to make it to 12 x 9 matrix of image patches like this (using 2x2 kernel on a 4x4 image):
How can I achieve this using numpy?
Thank you for your help.
Assuming 4x4x3 as input and 12x9 as output
from scipy.signal import convolve
import numpy as np
# creating the 4x4x3 input image
a = np.arange( 1,16+1).reshape(4,4)
b = np.arange(17,32+1).reshape(4,4)
c = np.arange(33,48+1).reshape(4,4)
i_4x4x3 = np.dstack((a, b, c))
# creating four 2x2 kernels
mask_tl = np.array([0,0,0,1]).reshape(2,2)
mask_tr = np.array([0,0,1,0]).reshape(2,2)
mask_bl = np.array([0,1,0,0]).reshape(2,2)
mask_br = np.array([1,0,0,0]).reshape(2,2)
mask_tl = mask_tl[:,:,None]
mask_tr = mask_tr[:,:,None]
mask_bl = mask_bl[:,:,None]
mask_br = mask_br[:,:,None]
# convolving the input with all four kernels
tl = convolve(i_4x4x3, mask_tl, mode='valid')
tr = convolve(i_4x4x3, mask_tr, mode='valid')
bl = convolve(i_4x4x3, mask_bl, mode='valid')
br = convolve(i_4x4x3, mask_br, mode='valid')
i = np.dstack((
tl.reshape(-1,3),
tr.reshape(-1,3),
bl.reshape(-1,3),
br.reshape(-1,3)))
i=i.reshape(i.shape[0],-1).transpose()
display(a,b,c)
display(i)
Output:
array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12],
[13, 14, 15, 16]])
array([[17, 18, 19, 20],
[21, 22, 23, 24],
[25, 26, 27, 28],
[29, 30, 31, 32]])
array([[33, 34, 35, 36],
[37, 38, 39, 40],
[41, 42, 43, 44],
[45, 46, 47, 48]])
array([[ 1, 2, 3, 5, 6, 7, 9, 10, 11],
[ 2, 3, 4, 6, 7, 8, 10, 11, 12],
[ 5, 6, 7, 9, 10, 11, 13, 14, 15],
[ 6, 7, 8, 10, 11, 12, 14, 15, 16],
[17, 18, 19, 21, 22, 23, 25, 26, 27],
[18, 19, 20, 22, 23, 24, 26, 27, 28],
[21, 22, 23, 25, 26, 27, 29, 30, 31],
[22, 23, 24, 26, 27, 28, 30, 31, 32],
[33, 34, 35, 37, 38, 39, 41, 42, 43],
[34, 35, 36, 38, 39, 40, 42, 43, 44],
[37, 38, 39, 41, 42, 43, 45, 46, 47],
[38, 39, 40, 42, 43, 44, 46, 47, 48]])
How would one (efficiently) do the following:
x = np.arange(49)
x2 = np.reshape(x, (7,7))
x2
array([[ 0, 1, 2, 3, 4, 5, 6],
[ 7, 8, 9, 10, 11, 12, 13],
[14, 15, 16, 17, 18, 19, 20],
[21, 22, 23, 24, 25, 26, 27],
[28, 29, 30, 31, 32, 33, 34],
[35, 36, 37, 38, 39, 40, 41],
[42, 43, 44, 45, 46, 47, 48]])
From here I want to roll a couple of things.
I want to roll 0,7,14,21 etc so 14 comes to top.
Then the same with 4,11,18,25 etc so 39 comes to top.
Result should be:
x2
array([[14, 1, 2, 3, 39, 5, 6],
[21, 8, 9, 10, 46, 12, 13],
[28, 15, 16, 17, 4, 19, 20],
[35, 22, 23, 24, 11, 26, 27],
[42, 29, 30, 31, 18, 33, 34],
[ 0, 36, 37, 38, 25, 40, 41],
[ 7, 43, 44, 45, 32, 47, 48]])
I looked up numpy.roll, here and google but couldn't find how one would do this.
For horizontal rolls, I could do:
np.roll(x2[0], 3, axis=0)
x3
array([4, 5, 6, 0, 1, 2, 3])
But how do I return the full array with this roll change as a new copy?
Roll with a negative shift:
x2[:, 0] = np.roll(x2[:, 0], -2)
Roll with a positive shift:
x2[:, 4] = np.roll(x2[:, 4], 2)
gives:
>>>x2
array([[14, 1, 2, 3, 39, 5, 6],
[21, 8, 9, 10, 46, 12, 13],
[28, 15, 16, 17, 4, 19, 20],
[35, 22, 23, 24, 11, 26, 27],
[42, 29, 30, 31, 18, 33, 34],
[ 0, 36, 37, 38, 25, 40, 41],
[ 7, 43, 44, 45, 32, 47, 48]])
Here's a way to roll multiple columns in one go with advanced-indexing -
# Params
cols = [0,4] # Columns to be rolled
dirn = [2,-2] # Offset with direction as sign
n = x2.shape[0]
x2[:,cols] = x2[np.mod(np.arange(n)[:,None] + dirn,n),cols]
Sample run -
In [45]: x2
Out[45]:
array([[ 0, 1, 2, 3, 4, 5, 6],
[ 7, 8, 9, 10, 11, 12, 13],
[14, 15, 16, 17, 18, 19, 20],
[21, 22, 23, 24, 25, 26, 27],
[28, 29, 30, 31, 32, 33, 34],
[35, 36, 37, 38, 39, 40, 41],
[42, 43, 44, 45, 46, 47, 48]])
In [46]: cols = [0,4,5] # Columns to be rolled
...: dirn = [2,-2,4] # Offset with direction as sign
...: n = x2.shape[0]
...: x2[:,cols] = x2[np.mod(np.arange(n)[:,None] + dirn,n),cols]
...:
In [47]: x2 # Three columns rolled
Out[47]:
array([[14, 1, 2, 3, 39, 33, 6],
[21, 8, 9, 10, 46, 40, 13],
[28, 15, 16, 17, 4, 47, 20],
[35, 22, 23, 24, 11, 5, 27],
[42, 29, 30, 31, 18, 12, 34],
[ 0, 36, 37, 38, 25, 19, 41],
[ 7, 43, 44, 45, 32, 26, 48]])
You have to overwrite the column
e.g.:
x2[:,0] = np.roll(x2[:,0], 3)
See here a useful method for shifting a 2D array in all 4 directions (up, down, left, right):
def image_shift_roll(img, x_shift, y_roll):
img_roll = img.copy()
img_roll = np.roll(img_roll, -y_roll, axis = 0) # Positive y rolls up
img_roll = np.roll(img_roll, x_roll, axis = 1) # Positive x rolls right
return img_roll
I have data in a numpy array:
a = np.arange(100)
a = a.reshape((20,5))
When I type
a[:10]
it returns
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24],
[25, 26, 27, 28, 29],
[30, 31, 32, 33, 34],
[35, 36, 37, 38, 39],
[40, 41, 42, 43, 44],
[45, 46, 47, 48, 49]])
Now i decided to reshape the array into 3d array.
b = a.reshape((5,4,5))
array([[[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19]],
[[20, 21, 22, 23, 24],
[25, 26, 27, 28, 29],
[30, 31, 32, 33, 34],
[35, 36, 37, 38, 39]],
[[40, 41, 42, 43, 44],
[45, 46, 47, 48, 49],
[50, 51, 52, 53, 54],
[55, 56, 57, 58, 59]],
[[60, 61, 62, 63, 64],
[65, 66, 67, 68, 69],
[70, 71, 72, 73, 74],
[75, 76, 77, 78, 79]],
[[80, 81, 82, 83, 84],
[85, 86, 87, 88, 89],
[90, 91, 92, 93, 94],
[95, 96, 97, 98, 99]]])
How do I slice b to that I obtain the values like a[:10]?
I tried
b[:10,0,:5]
array([[ 0, 1, 2, 3, 4],
[10, 11, 12, 13, 14],
[20, 21, 22, 23, 24],
[30, 31, 32, 33, 34],
[40, 41, 42, 43, 44],
[50, 51, 52, 53, 54],
[60, 61, 62, 63, 64],
[70, 71, 72, 73, 74],
[80, 81, 82, 83, 84],
[90, 91, 92, 93, 94]])
But its not correct.
Thank you in advance!
When you use b = a.reshape((5,4,5)) you just create a different view on the same data used by the array a. (ie changes to the elements of a will appear in b). reshape() does not copy data in this case, so it is a very fast operation. Slicing b and slicing a accesses the same memory, so there shouldn't be any need for a different syntax for the b array (just use a[:10]). If you have created a copy of the data, perhaps with np.resize(), and discarded a, just reshape b: b.reshape((20,5))[:10].
By reshaping (20,5) to (5,4,5), there's no way you can pull out the 1st half of the values. You can't split those 5 rows into 2 even groups:
In [9]: b[:2]
Out[9]:
array([[[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19]],
[[20, 21, 22, 23, 24],
[25, 26, 27, 28, 29],
[30, 31, 32, 33, 34],
[35, 36, 37, 38, 39]]])
In [10]: b[:3]
Out[10]:
array([[[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19]],
[[20, 21, 22, 23, 24],
[25, 26, 27, 28, 29],
[30, 31, 32, 33, 34],
[35, 36, 37, 38, 39]],
[[40, 41, 42, 43, 44],
[45, 46, 47, 48, 49],
[50, 51, 52, 53, 54],
[55, 56, 57, 58, 59]]])
The last row of a[:10] is in the middle of b[3,:,:].
Note that b[:2] is (2,4,5), 8 rows of a, grouped into 2 sets of 4.
Now if you'd done c=a.reshape(4,5,5), then c[:2] would have those same 10 rows - in 2 sets of 5. And c[:2].reshape(10,-1) will look just like a[:10].
There could be a programmatic way to get what you want, but not a python slice.
It is important to understand what every component in the shape tells us about the arrangement. I like to think in terms of vectors.
Let's talk about the shape (20, 5) - this would mean, I have 20 vectors where every vector has 5 elements.
For the shape (5, 4, 5) - this would mean, I have 5 vectors, where each vector again has 4 vectors where every vector within has 5 elements.
This might sound complicated but with some deliberation, this could be understood.
Coming to your question, by a[:10] you want to retrieve the first 10 rows where each row should be a vector containing 5 elements but using a shape of (5, 4, 5).
This is only possible if you retrieve the first 4 vectors from 1st vector of the leftmost dimension (5), next 4 vectors from the next vector and next 2 from the 3rd.
Python slicing might not be the best tool to achieve this.
I am using nested arrays as a matrix representation. I created the following function for spliting quadratic matrices with size 2^k into four equal parts (used for Strassen algorithm):
import itertools
def splitmat(mat):
n = len(mat)
return map( \
lambda (x,y):map(lambda z:z[y[0]:y[1]],mat[x[0]:x[1]]), \
itertools.product([(0,n/2),(n/2,n)],repeat=2)
)
Now I'm trying to find an inverse function that joins the four parts back to a full matrix. I could use two nested loops, but may there be any pythonic way to achieve this? I would prefer to not use numpy but only builtin modules. Do you have any idea or hint how to achieve this?
Your inverse operation can be split into 2 simplier operation:
concatenate rows(numpy.vstack)
concatenate columns(numpy.hstack)
So, if you have matrix divided into 4 submatrix:
M = |m1|m2|
|m3|m4|
then M = hstack(vstack(m1, m2), vstack(m3, m4).
This operations can be code like this:
import itertools
import math
# iterators
def ihstack(*matrixes):
return map(lambda rows: itertools.chain(*rows), zip(*matrixes))
def ivstack(*matrixes):
return itertools.chain(*matrixes)
# main function
def squarejoin(*matrixes):
size = int(math.sqrt(len(matrixes)))
assert size ** 2 == len(matrixes), 'Incorrect number of matrices'
return _matrixjoin(matrixes, size, size)
def _matrixjoin(matrixes, hsize, vsize):
print(matrixes, hsize, vsize)
return ivstack(*(ihstack(*itertools.islice(matrixes, i*hsize, (i+1)*hsize)) for i in range(vsize)))
Here I have an example program where a 2 loops implementation works and is crystal clear in its intent, a 1 loop implementation works and is, imho, slightly less clear and eventually a 0 (explicit, btw) loops implementation that, alas, is buggy.
My vote goes to the two loops... further, I'd like to be shown what's wrong with my 0 loops attempt
Code
import itertools
def pm(m):
for row in m: print row
mat = []
n = 8
for i in range(n):
mat.append(range(i*n, i*n+n))
# this is shorthand for your splitmat function
res = map(lambda (x,y):
map(lambda z:z[y[0]:y[1]],mat[x[0]:x[1]]),
itertools.product([(0,n/2),(n/2,n)],repeat=2))
pm(res)
print "\n2 cycles"
mat = []
for i, j in ((0,1),(2,3)):
for a, b in zip(res[i],res[j]):
mat.append(a+b)
pm(mat)
print "\n1 cycle"
mat = []
for i, j in ((0,1),(2,3)):
map(lambda x: mat.append(x[0]+x[1]), zip(res[i],res[j]))
pm(mat)
print "\n0 cycles"
mat = map(lambda i_j:
map(lambda x: x[0]+x[1], zip(res[i_j[0]],res[i_j[1]])), ((0,1),(2,3)))
pm(mat)
Output
[[0, 1, 2, 3], [8, 9, 10, 11], [16, 17, 18, 19], [24, 25, 26, 27]]
[[4, 5, 6, 7], [12, 13, 14, 15], [20, 21, 22, 23], [28, 29, 30, 31]]
[[32, 33, 34, 35], [40, 41, 42, 43], [48, 49, 50, 51], [56, 57, 58, 59]]
[[36, 37, 38, 39], [44, 45, 46, 47], [52, 53, 54, 55], [60, 61, 62, 63]]
2 cicli
[0, 1, 2, 3, 4, 5, 6, 7]
[8, 9, 10, 11, 12, 13, 14, 15]
[16, 17, 18, 19, 20, 21, 22, 23]
[24, 25, 26, 27, 28, 29, 30, 31]
[32, 33, 34, 35, 36, 37, 38, 39]
[40, 41, 42, 43, 44, 45, 46, 47]
[48, 49, 50, 51, 52, 53, 54, 55]
[56, 57, 58, 59, 60, 61, 62, 63]
1 ciclo
[0, 1, 2, 3, 4, 5, 6, 7]
[8, 9, 10, 11, 12, 13, 14, 15]
[16, 17, 18, 19, 20, 21, 22, 23]
[24, 25, 26, 27, 28, 29, 30, 31]
[32, 33, 34, 35, 36, 37, 38, 39]
[40, 41, 42, 43, 44, 45, 46, 47]
[48, 49, 50, 51, 52, 53, 54, 55]
[56, 57, 58, 59, 60, 61, 62, 63]
0 cicli
[[0, 1, 2, 3, 4, 5, 6, 7], [8, 9, 10, 11, 12, 13, 14, 15], [16, 17, 18, 19, 20, 21, 22, 23], [24, 25, 26, 27, 28, 29, 30, 31]]
[[32, 33, 34, 35, 36, 37, 38, 39], [40, 41, 42, 43, 44, 45, 46, 47], [48, 49, 50, 51, 52, 53, 54, 55], [56, 57, 58, 59, 60, 61, 62, 63]]