I have a multiband raster having 23 bands for example. I am reading the raster using GDAL and converting it to numpy array. In numpy when I am checking the array shape it is showing as 23,4,5 i.e. 23 = bands, 4 = row and 5 = col. I want to loop through each pixel in a manner that I am able to get the vector/array of all 23 values related to one pixel.
I am explaining it further in detail by example...
Band 1
[[1, 2, 3, 4, 5],
[6, 7, 8, 9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20]]
Band 2
[[21, 22, 23, 24, 25],
[26, 27, 28, 29, 30],
[31, 32, 33, 34, 35],
[36, 37, 38, 39, 40]]
Band 3
[[31, 32, 33, 34, 35],
[36, 37, 38, 39, 40],
[41, 42, 43, 44, 45],
[46, 47, 48, 49, 50]]
Band 4
[[41, 42, 43, 44, 45],
[46, 47, 48, 49, 50],
[51, 52, 53, 54, 55],
[56, 57, 58, 59, 60]]
Now I want to loop through all bands in such a way that I should get array of values [1,21,31,41] in first iteration and [2,22,32,42] in second iteration and so on. I can do that in R using the pointer math and extract function easily but in python I am not able to do that because I have very less experience with looping and array indexing in python.
I have tried some codes given on the StackOverflow
for cell in arr:
print cell
for cell in arr.flat:
print cell
for row in arr:
for cell in row:
print cell
for (i,row) in enumerate(arr):
for (j,value) in enumerate(row):
print value
The output I am getting is not as required.
As i see it what you need is to have pixel values of same index grouped together. For grouping together same index e use zip here since the bandx itself is not flat we may first flatten then band and then do zip on the flattend bands
Here is example with 3 bands
band1 = np.array( [[1, 2, 3, 4, 5],
[6, 7, 8, 9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20]])
band2 = np.array([[21, 22, 23, 24, 25],
[26, 27, 28, 29, 30],
[31, 32, 33, 34, 35],
[36, 37, 38, 39, 40]])
band3 = np.array([[31, 32, 33, 34, 35],
[36, 37, 38, 39, 40],
[41, 42, 43, 44, 45],
[46, 47, 48, 49, 50]])
arr = [band1,band2,band3]
Following will give you array of tuples as you expected
zip(*map(lambda x:x.flatten(),arr))
If the bands are not numpy arrays but normal python arrays ,you may do the following
zip(*map(lambda x:x.flatten(),np.array(arr)))
A simple way is using np.nditer:
>>> for i in np.nditer((Band1, Band2, Band3, Band4)):
... print(i)
This prints
(array(1), array(21), array(31), array(41))
(array(2), array(22), array(32), array(42))
(array(3), array(23), array(33), array(43))
(array(4), array(24), array(34), array(44))
(array(5), array(25), array(35), array(45))
(array(6), array(26), array(36), array(46))
(array(7), array(27), array(37), array(47))
(array(8), array(28), array(38), array(48))
(array(9), array(29), array(39), array(49))
(array(10), array(30), array(40), array(50))
(array(11), array(31), array(41), array(51))
(array(12), array(32), array(42), array(52))
(array(13), array(33), array(43), array(53))
(array(14), array(34), array(44), array(54))
(array(15), array(35), array(45), array(55))
(array(16), array(36), array(46), array(56))
(array(17), array(37), array(47), array(57))
(array(18), array(38), array(48), array(58))
(array(19), array(39), array(49), array(59))
(array(20), array(40), array(50), array(60))
Even though it seems strange that this returns arrays actually makes it a really cool feature because these are views of the original array so you don't have the overhead of unpacking the value as python type and these zero-dimensional arrays behave (almost) like native python types.
Related
I have a 40 x 40 x 40 array in Python and would like to extract all i values with k=10 index. I understand how to do this with a nxn array but not a nxnxn array.
Thanks
As per the Numpy Index documentation.
import numpy as np
arr = np.arange(3*4*5).reshape( 3,4,5 )
arr
# array([[[ 0, 1, 2, 3, 4],
# [ 5, 6, 7, 8, 9],
# [10, 11, 12, 13, 14],
# [15, 16, 17, 18, 19]],
# [[20, 21, 22, 23, 24],
# [25, 26, 27, 28, 29],
# [30, 31, 32, 33, 34],
# [35, 36, 37, 38, 39]],
# [[40, 41, 42, 43, 44],
# [45, 46, 47, 48, 49],
# [50, 51, 52, 53, 54],
# [55, 56, 57, 58, 59]]])
arr[:,:,1] # For k = 1
# or arr[ ...,1] as its the last index
This returns a 2d array.
array([[ 1, 6, 11, 16],
[21, 26, 31, 36],
[41, 46, 51, 56]])
This question already has answers here:
Most efficient way to reverse a numpy array
(8 answers)
Closed 1 year ago.
I try to rotate the matrix so that the first line is the last and the last is the first. And proceed in this way with the other lines in array
import numpy as np
m = np.array([[11, 12, 13, 14, 15],
[21, 22, 23, 24, 25],
[31, 32, 33, 34, 35],
[41, 42, 43, 44, 45]])
My attempt:
p = np.rot90(m, 2)
array([[45, 44, 43, 42, 41],
[35, 34, 33, 32, 31],
[25, 24, 23, 22, 21],
[15, 14, 13, 12, 11]])
I need
array([[41, 42, 43, 44, 45],
[31, 32, 33, 34, 35],
[21, 22, 23, 24, 25],
[11, 12, 13, 14, 15]])
Do you have any advice?
np.flip(m,axis=0)
output
array([[41, 42, 43, 44, 45],
[31, 32, 33, 34, 35],
[21, 22, 23, 24, 25],
[11, 12, 13, 14, 15]])
You're essentially trying to flip the matrix vertically and the function is literally flipud:
np.flipud(m)
Aside from flipud(m) and flip(m, axis=0), you can use simple indexing to reverse the first dimension:
m[::-1, :]
or even just
m[::-1]
If you really wanted arcane overkill, you could do something with np.lib.stride_tricks.as_strided:
np.lib.stride_tricks.as_strided(m[-1], shape=m.shape, strides=(-m.strides[0], m.strides[1]))
I have array A = np.ones((4,4,4)) and another array that represent the coordinates of point in array A called B, lets assume that B = [[2,2,2], [3,2,1]].
I tried to access A by array indexing like A[B], but it didn't works.
How i can do it in elegant way, that also work for B that it's have a higher dimensions like B of shape (10, 20, 3) ?
You can pass a list of coordinates, but you should transpose the list. Such that the items of the i-th dimension are passed as the i-th element in the indexing, for example with:
A[tuple(np.transpose(B))]
For a 4×4×4 matrix:
>>> A = np.arange(64).reshape(4,4,4)
>>> A
array([[[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15]],
[[16, 17, 18, 19],
[20, 21, 22, 23],
[24, 25, 26, 27],
[28, 29, 30, 31]],
[[32, 33, 34, 35],
[36, 37, 38, 39],
[40, 41, 42, 43],
[44, 45, 46, 47]],
[[48, 49, 50, 51],
[52, 53, 54, 55],
[56, 57, 58, 59],
[60, 61, 62, 63]]])
we get for the given coordinates:
>>> A[tuple(np.transpose(B))]
array([42, 57])
and if we calculate these manually, we get:
>>> A[2,2,2]
42
>>> A[3,2,1]
57
Background:
A[1,2,3] is short for A[(1,2,3)] (so in a tuple). You can fetch multiple items with A[([2,3], [2,2], [2,1])] but then you thus first need to transpose the data.
Since the data is represented as [[2,2,2], [3,2,1]], we thus first need to transpose it to [[2,3], [2,2], [2,1]]. Next we wrap it in a tuple, and can use this to subscript A.
I have data in a numpy array:
a = np.arange(100)
a = a.reshape((20,5))
When I type
a[:10]
it returns
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24],
[25, 26, 27, 28, 29],
[30, 31, 32, 33, 34],
[35, 36, 37, 38, 39],
[40, 41, 42, 43, 44],
[45, 46, 47, 48, 49]])
Now i decided to reshape the array into 3d array.
b = a.reshape((5,4,5))
array([[[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19]],
[[20, 21, 22, 23, 24],
[25, 26, 27, 28, 29],
[30, 31, 32, 33, 34],
[35, 36, 37, 38, 39]],
[[40, 41, 42, 43, 44],
[45, 46, 47, 48, 49],
[50, 51, 52, 53, 54],
[55, 56, 57, 58, 59]],
[[60, 61, 62, 63, 64],
[65, 66, 67, 68, 69],
[70, 71, 72, 73, 74],
[75, 76, 77, 78, 79]],
[[80, 81, 82, 83, 84],
[85, 86, 87, 88, 89],
[90, 91, 92, 93, 94],
[95, 96, 97, 98, 99]]])
How do I slice b to that I obtain the values like a[:10]?
I tried
b[:10,0,:5]
array([[ 0, 1, 2, 3, 4],
[10, 11, 12, 13, 14],
[20, 21, 22, 23, 24],
[30, 31, 32, 33, 34],
[40, 41, 42, 43, 44],
[50, 51, 52, 53, 54],
[60, 61, 62, 63, 64],
[70, 71, 72, 73, 74],
[80, 81, 82, 83, 84],
[90, 91, 92, 93, 94]])
But its not correct.
Thank you in advance!
When you use b = a.reshape((5,4,5)) you just create a different view on the same data used by the array a. (ie changes to the elements of a will appear in b). reshape() does not copy data in this case, so it is a very fast operation. Slicing b and slicing a accesses the same memory, so there shouldn't be any need for a different syntax for the b array (just use a[:10]). If you have created a copy of the data, perhaps with np.resize(), and discarded a, just reshape b: b.reshape((20,5))[:10].
By reshaping (20,5) to (5,4,5), there's no way you can pull out the 1st half of the values. You can't split those 5 rows into 2 even groups:
In [9]: b[:2]
Out[9]:
array([[[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19]],
[[20, 21, 22, 23, 24],
[25, 26, 27, 28, 29],
[30, 31, 32, 33, 34],
[35, 36, 37, 38, 39]]])
In [10]: b[:3]
Out[10]:
array([[[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19]],
[[20, 21, 22, 23, 24],
[25, 26, 27, 28, 29],
[30, 31, 32, 33, 34],
[35, 36, 37, 38, 39]],
[[40, 41, 42, 43, 44],
[45, 46, 47, 48, 49],
[50, 51, 52, 53, 54],
[55, 56, 57, 58, 59]]])
The last row of a[:10] is in the middle of b[3,:,:].
Note that b[:2] is (2,4,5), 8 rows of a, grouped into 2 sets of 4.
Now if you'd done c=a.reshape(4,5,5), then c[:2] would have those same 10 rows - in 2 sets of 5. And c[:2].reshape(10,-1) will look just like a[:10].
There could be a programmatic way to get what you want, but not a python slice.
It is important to understand what every component in the shape tells us about the arrangement. I like to think in terms of vectors.
Let's talk about the shape (20, 5) - this would mean, I have 20 vectors where every vector has 5 elements.
For the shape (5, 4, 5) - this would mean, I have 5 vectors, where each vector again has 4 vectors where every vector within has 5 elements.
This might sound complicated but with some deliberation, this could be understood.
Coming to your question, by a[:10] you want to retrieve the first 10 rows where each row should be a vector containing 5 elements but using a shape of (5, 4, 5).
This is only possible if you retrieve the first 4 vectors from 1st vector of the leftmost dimension (5), next 4 vectors from the next vector and next 2 from the 3rd.
Python slicing might not be the best tool to achieve this.
I am using nested arrays as a matrix representation. I created the following function for spliting quadratic matrices with size 2^k into four equal parts (used for Strassen algorithm):
import itertools
def splitmat(mat):
n = len(mat)
return map( \
lambda (x,y):map(lambda z:z[y[0]:y[1]],mat[x[0]:x[1]]), \
itertools.product([(0,n/2),(n/2,n)],repeat=2)
)
Now I'm trying to find an inverse function that joins the four parts back to a full matrix. I could use two nested loops, but may there be any pythonic way to achieve this? I would prefer to not use numpy but only builtin modules. Do you have any idea or hint how to achieve this?
Your inverse operation can be split into 2 simplier operation:
concatenate rows(numpy.vstack)
concatenate columns(numpy.hstack)
So, if you have matrix divided into 4 submatrix:
M = |m1|m2|
|m3|m4|
then M = hstack(vstack(m1, m2), vstack(m3, m4).
This operations can be code like this:
import itertools
import math
# iterators
def ihstack(*matrixes):
return map(lambda rows: itertools.chain(*rows), zip(*matrixes))
def ivstack(*matrixes):
return itertools.chain(*matrixes)
# main function
def squarejoin(*matrixes):
size = int(math.sqrt(len(matrixes)))
assert size ** 2 == len(matrixes), 'Incorrect number of matrices'
return _matrixjoin(matrixes, size, size)
def _matrixjoin(matrixes, hsize, vsize):
print(matrixes, hsize, vsize)
return ivstack(*(ihstack(*itertools.islice(matrixes, i*hsize, (i+1)*hsize)) for i in range(vsize)))
Here I have an example program where a 2 loops implementation works and is crystal clear in its intent, a 1 loop implementation works and is, imho, slightly less clear and eventually a 0 (explicit, btw) loops implementation that, alas, is buggy.
My vote goes to the two loops... further, I'd like to be shown what's wrong with my 0 loops attempt
Code
import itertools
def pm(m):
for row in m: print row
mat = []
n = 8
for i in range(n):
mat.append(range(i*n, i*n+n))
# this is shorthand for your splitmat function
res = map(lambda (x,y):
map(lambda z:z[y[0]:y[1]],mat[x[0]:x[1]]),
itertools.product([(0,n/2),(n/2,n)],repeat=2))
pm(res)
print "\n2 cycles"
mat = []
for i, j in ((0,1),(2,3)):
for a, b in zip(res[i],res[j]):
mat.append(a+b)
pm(mat)
print "\n1 cycle"
mat = []
for i, j in ((0,1),(2,3)):
map(lambda x: mat.append(x[0]+x[1]), zip(res[i],res[j]))
pm(mat)
print "\n0 cycles"
mat = map(lambda i_j:
map(lambda x: x[0]+x[1], zip(res[i_j[0]],res[i_j[1]])), ((0,1),(2,3)))
pm(mat)
Output
[[0, 1, 2, 3], [8, 9, 10, 11], [16, 17, 18, 19], [24, 25, 26, 27]]
[[4, 5, 6, 7], [12, 13, 14, 15], [20, 21, 22, 23], [28, 29, 30, 31]]
[[32, 33, 34, 35], [40, 41, 42, 43], [48, 49, 50, 51], [56, 57, 58, 59]]
[[36, 37, 38, 39], [44, 45, 46, 47], [52, 53, 54, 55], [60, 61, 62, 63]]
2 cicli
[0, 1, 2, 3, 4, 5, 6, 7]
[8, 9, 10, 11, 12, 13, 14, 15]
[16, 17, 18, 19, 20, 21, 22, 23]
[24, 25, 26, 27, 28, 29, 30, 31]
[32, 33, 34, 35, 36, 37, 38, 39]
[40, 41, 42, 43, 44, 45, 46, 47]
[48, 49, 50, 51, 52, 53, 54, 55]
[56, 57, 58, 59, 60, 61, 62, 63]
1 ciclo
[0, 1, 2, 3, 4, 5, 6, 7]
[8, 9, 10, 11, 12, 13, 14, 15]
[16, 17, 18, 19, 20, 21, 22, 23]
[24, 25, 26, 27, 28, 29, 30, 31]
[32, 33, 34, 35, 36, 37, 38, 39]
[40, 41, 42, 43, 44, 45, 46, 47]
[48, 49, 50, 51, 52, 53, 54, 55]
[56, 57, 58, 59, 60, 61, 62, 63]
0 cicli
[[0, 1, 2, 3, 4, 5, 6, 7], [8, 9, 10, 11, 12, 13, 14, 15], [16, 17, 18, 19, 20, 21, 22, 23], [24, 25, 26, 27, 28, 29, 30, 31]]
[[32, 33, 34, 35, 36, 37, 38, 39], [40, 41, 42, 43, 44, 45, 46, 47], [48, 49, 50, 51, 52, 53, 54, 55], [56, 57, 58, 59, 60, 61, 62, 63]]