Given a numpy 2D array of points, aka 3D array with size of the 3rd dimension equals to 2, how do I get the minimum x and y coordinate over all points?
Examples:
First:
I edited my original example, since it was wrong.
data = np.array(
[[[ 0, 1],
[ 2, 3],
[ 4, 5]],
[[11, 12],
[13, 14],
[15, 16]]])
minx = 0 # data[0][0][0]
miny = 1 # data[0][0][1]
4 x 4 x 2:
Second:
array([[[ 0, 77],
[29, 12],
[28, 71],
[46, 17]],
[[45, 76],
[33, 82],
[14, 17],
[ 3, 18]],
[[99, 40],
[96, 3],
[74, 60],
[ 4, 57]],
[[67, 57],
[23, 81],
[12, 12],
[45, 98]]])
minx = 0 # data[0][0][0]
miny = 3 # data[2][1][1]
Is there an easy way to get now the minimum x and y coordinates of all points of the data? I played around with amin and different axis values, but nothing worked.
Clarification:
My array stores positions from different robots over time. First dimension is time, second is the index of an robot. The third dimension is then either x or y of a robots for a given time.
Since I want to draw their paths to pixels, I need to normalize my data, so that the points are as close as possible to the origin without getting negative. I thought that subtracting [minx,miny] from every point will do that for me.
alko's answer didn't work for me, so here's what I did:
import numpy as np
array = np.arange(15).reshape(5,3)
x,y = np.unravel_index(np.argmin(array),array.shape)
Seems you need consecutive min alongaxis. For your first example:
>>> np.min(np.min(data, axis=1), axis=0)
array([ 0, 1])
For the second:
>>> np.min(np.min(data, axis=1), axis=0)
array([0, 3])
The same expression can be stated (in numpy older than 1.7), as pointed out by #Jamie, s
>>> np.min(data, axis=(1, 0))
array([0, 3])
You should take the min of a slice of the array.
Assuming first coordinate is x and second is y
minx = min(a[:,0])
miny = min(a[:,1])
>>>a=np.array([[1,2],[3,4],[5,6]])
>>> a
array([[1, 2],
[3, 4],
[5, 6]])
>>> min(a[:,0])
1
>>> max(a[:,0])
5
>>> max(a[:,1])
6
minx = np.min(data[0])
miny = np.min(data[1])
You can use function numpy.amin to find the minima along the desired axis.
Related
Here is the example:
import numpy as np
a = np.array([1,2],[3,4],[5,6])
b = np.array([7,8],[9,10],[11,12],[12,13])
what I want is to use each item in a to plus every item in b then plus them together. For example, [1,2] should plus every row in b 1+7=8,2+8=10, 8+10=18;1+9=10,2+10=12,10+12=22... The result would like to be that[[18,22,26,28...],[22,26,....],[26,30....]]
My question is how to fulfil that? I know use numpy can be more efficient than loop but how to use the matrix to calculate this?
I believe this does what you want:
>>> a = np.array([[1,2],[3,4],[5,6]])
>>> b = np.array([[7,8],[9,10],[11,12],[12,13]])
>>> np.sum(a, axis=1)[:,None] + np.sum(b, axis=1)[None,:]
array([[18, 22, 26, 28],
[22, 26, 30, 32],
[26, 30, 34, 36]])
You can use list comprehensions:
import numpy as np
a = np.array([[1, 2], [3, 4], [5, 6]])
b = np.array([[7, 8], [9, 10], [11, 12], [12, 13]])
[[sum(i) + sum(j) for j in b] for i in a]
Output:
[[18, 22, 26, 28], [22, 26, 30, 32], [26, 30, 34, 36]]
This can be done in the most precise and succinct way as follows:
np.einsum('ijk-> ij', a[:,None,:]+b)
Let me explain each step.It combines einsum and broadcasting concepts of numpy.
Step 1- a[:,None,:] reshapes matrix a to shape (3,1,2). This mid axis with value 1 is helpful for broadcasting.
Step 2- a[:,None,:] + b broadcasts a and adds matrix b to get a resultant matrix of shape (3,4,2).
Step 3- np.einsum('ijk-> ij', a[:,None,:]+b) does sum reduction along the last axis of matrix obtained from previous step.
Suppose I have a numpy array img, with img.shape == (468,832,3). What does img[::2, ::2] do? It reduces the shape to (234,416,3) Can you please explain the logic?
Let's read documentation together (Source).
(Just read the bold part first)
The basic slice syntax is i:j:k where i is the starting index, j is the stopping index, and k is the step (k \neq 0). This selects the m elements (in the corresponding dimension) with index values i, i + k, ..., i + (m - 1) k where m = q + (r\neq0) and q and r are the quotient and remainder obtained by dividing j - i by k: j - i = q k + r, so that i + (m - 1) k < j.
...
Assume n is the number of elements in the dimension being sliced.
Then, if i is not given it defaults to 0 for k > 0 and n - 1 for k < 0
. If j is not given it defaults to n for k > 0 and -n-1 for k < 0 . If
k is not given it defaults to 1. Note that :: is the same as : and
means select all indices along this axis.
Now looking at your part.
[::2, ::2] will be translated to [0:468:2, 0:832:2] because you do not specify the first two or i and j in the documentation. (You only specify k here. Recall the i:j:k notation above.) You select elements on these axes at the step size 2 which means you select every other elements along the axes specified.
Because you did not specify for the 3rd dimension, all will be selected.
It slices every alternate row, and then every alternate column, from an array, returning an array of size (n // 2, n // 2, ...).
Here's an example of slicing with a 2D array -
>>> a = np.arange(16).reshape(4, 4)
>>> a
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15]])
>>> a[::2, ::2]
array([[ 0, 2],
[ 8, 10]])
And, here's another example with a 3D array -
>>> a = np.arange(27).reshape(3, 3, 3)
>>> a
array([[[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8]],
[[ 9, 10, 11],
[12, 13, 14],
[15, 16, 17]],
[[18, 19, 20],
[21, 22, 23],
[24, 25, 26]]])
>>> a[::2, ::2] # same as a[::2, ::2, :]
array([[[ 0, 1, 2],
[ 6, 7, 8]],
[[18, 19, 20],
[24, 25, 26]]])
Well, we have the RGB image as a 3D array of shape:
img.shape=(468,832,3)
Now, what does img[::2, ::2] do?
we're just downsampling the image (i.e. we're shrinking the image size by half by taking only every other pixel from the original image and we do this by using a step size of 2, which means to skip one pixel). This should be clear from the example below.
Let's take a simple grayscale image for easier understanding.
In [13]: arr
Out[13]:
array([[10, 11, 12, 13, 14, 15],
[20, 21, 22, 23, 24, 25],
[30, 31, 32, 33, 34, 35],
[40, 41, 42, 43, 44, 45],
[50, 51, 52, 53, 54, 55],
[60, 61, 62, 63, 64, 65]])
In [14]: arr.shape
Out[14]: (6, 6)
In [15]: arr[::2, ::2]
Out[15]:
array([[10, 12, 14],
[30, 32, 34],
[50, 52, 54]])
In [16]: arr[::2, ::2].shape
Out[16]: (3, 3)
Notice which pixels are in the sliced version. Also, observe how the array shape changes after slicing (i.e. it is reduced by half).
Now, this downsampling happens for all three channels in the image since there's no slicing happening in the third axis. Thus, you will get the shape reduced only for the first two axis in your example.
(468, 832, 3)
. . |
. . |
(234, 416, 3)
I'm having difficulty conceptualizing a 3 dimensional matrix, which I need for a truth table of sorts. I am trying to go through a 3D matrix, and compare each element to all other elements adjacent to it (but not on diagonals). I have a function (truth_table) which is supposed to take a set of 3D coordinates and return a matrix which tells me which surrounding coordinates are safe to call upon without fear of an INDEX_OUT_OF_BOUNDS exception. I am new to N-dimensional matrices and am having trouble making this truth table, which was easy enough in 2D.
import numpy as np
array_rows=10
array_cols=10
array_height=10
laser_power=2e-3
T_MATRIX=np.full((array_rows,array_cols,array_height),0.0)
H_MATRIX=np.full((array_rows,array_cols,array_height),0.0)
H_MATRIX[5,5,5]=laser_power
constants=0.5
def truth_table(row,col,height):
skeleton=np.array([
[[False,False,False],[False,height<array_height-1,False],[False,False,False]],[[False,row>0,False],[col>0,False,col<array_cols-1],[False,row<array_rows-1,False]],[[False,False,False],[False,height>0,False],[False,False,False]]])
def update_T():
global T_MATRIX
for row in range(array_rows):
for col in range(array_cols):
for height in range(array_height):
T_MATRIX[row][col][height]+=H_MATRIX[row][col][height]*constants
def reset_H():
global H_MATRIX
for row in range(array_rows):
for col in range(array_cols):
for height in range(array_height):
H_MATRIX[row][col][height]=0.0
H_MATRIX[5,5,5]=laser_power
def update_H():
global H_MATRIX
for row in range(array_rows):
for col in range(array_cols):
for height in range(array_height):
my_table=truth_table(row,col,height)
count=np.sum(my_table)
T_DIFF=0.0
AVG_T=0.0
for r in range(3):
for c in range(3):
for h in range(3):
if(my_table[r][c][h]):
T_DIFF+=T_MATRIX[row][col][height]-T_MATRIX[row+(r-1)][col+(c-1)][height+(h-1)]
if(count>0):
AVG_T=T_DIFF/count
H_MATRIX[row][col][height]+=T_DIFF*constants
def update_stuff():
##UPDATE THE TEMP
update_T()
reset_H()
update_H()
update_stuff()
print T_MATRIX[5][5][5]
#print constants
Because this question is likely going to be abandoned or closed, here is something that might help your thinking about this problem. It looks like you are doing things where you are interested in calculating differences/averages between a cell's values and the value's of its nearest neighbors.
On a 3D lattice, a cell's nearest neighbors are located at the following positions (centered on the cell of interest):
[[[0, 0, 0],
[0, 1, 0],
[0, 0, 0]],
[[0, 1, 0],
[1, 0, 1],
[0, 1, 0]],
[[0, 0, 0],
[0, 1, 0],
[0, 0, 0]]]
which can be constructed with numpy using
f = np.zeros((3,3,3))
f[[0,1,1,1,1,2],[1,0,1,1,2,1],[1,1,0,2,1,1]] = 1
The convolution of this nearest-neighbor filter (or kernel) with any 3D array A1 of shape (N1,N2,N3) will give you a new array A2 of shape (N1,N2,N3) where each value in A2 is the sum of the neighbors' values in A1. For example:
>>> A1 = np.arange(27).reshape(3,3,3)
>>> A1
array([[[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8]],
[[ 9, 10, 11],
[12, 13, 14],
[15, 16, 17]],
[[18, 19, 20],
[21, 22, 23],
[24, 25, 26]]])
>>> A2 = scipy.ndimage.convolve(A1, f)
>>> A2
array([[[ 13, 18, 23],
[ 28, 33, 38],
[ 43, 48, 53]],
[[ 58, 63, 68],
[ 73, 78, 83],
[ 88, 93, 98]],
[[103, 108, 113],
[118, 123, 128],
[133, 138, 143]]])
The first entry of A2 is 13 = 1 + 3 + 9. The middle entry of A2 is 78 = 4 + 10 + 12 + 14 + 16 + 22.
You can use these convolutions to easily get some of the quantities you are looking for. It looks like H_MATRIX could be constructed simply by
H_MATRIX = constants * (T_MATRIX - convolve(T_MAXTRIX, f))
using the f defined above. Alternatively, you could define a different f that has a 1 in the center and -1 at the other 6 positions which might give you just H_MATRIX = constants * convolve(T_MATRIX, f).
How do I sum over the columns of a tensor?
torch.Size([10, 100]) ---> torch.Size([10])
The simplest and best solution is to use torch.sum().
To sum all elements of a tensor:
torch.sum(x) # gives back a scalar
To sum over all rows (i.e. for each column):
torch.sum(x, dim=0) # size = [ncol]
To sum over all columns (i.e. for each row):
torch.sum(x, dim=1) # size = [nrow]
It should be noted that the dimension summed over is eliminated from the resulting tensor.
Alternatively, you can use tensor.sum(axis) where axis indicates 0 and 1 for summing over rows and columns respectively, for a 2D tensor.
In [210]: X
Out[210]:
tensor([[ 1, -3, 0, 10],
[ 9, 3, 2, 10],
[ 0, 3, -12, 32]])
In [211]: X.sum(1)
Out[211]: tensor([ 8, 24, 23])
In [212]: X.sum(0)
Out[212]: tensor([ 10, 3, -10, 52])
As, we can see from the above outputs, in both cases, the output is a 1D tensor. If you, on the other hand, wish to retain the dimension of the original tensor in the output as well, then you've set the boolean kwarg keepdim to True as in:
In [217]: X.sum(0, keepdim=True)
Out[217]: tensor([[ 10, 3, -10, 52]])
In [218]: X.sum(1, keepdim=True)
Out[218]:
tensor([[ 8],
[24],
[23]])
If you have tensor my_tensor, and you wish to sum across the second array dimension (that is, the one with index 1, which is the column-dimension, if the tensor is 2-dimensional, as yours is), use torch.sum(my_tensor,1) or equivalently my_tensor.sum(1) see documentation here.
One thing that is not mentioned explicitly in the documentation is: you can sum across the last array-dimension by using -1 (or the second-to last dimension, with -2, etc.)
So, in your example, you could use: outputs.sum(1) or torch.sum(outputs,1), or, equivalently, outputs.sum(-1) or torch.sum(outputs,-1). All of these would give the same result, an output tensor of size torch.Size([10]), with each entry being the sum over the all rows in a given column of the tensor outputs.
To illustrate with a 3-dimensional tensor:
In [1]: my_tensor = torch.arange(24).view(2, 3, 4)
Out[1]:
tensor([[[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]],
[[12, 13, 14, 15],
[16, 17, 18, 19],
[20, 21, 22, 23]]])
In [2]: my_tensor.sum(2)
Out[2]:
tensor([[ 6, 22, 38],
[54, 70, 86]])
In [3]: my_tensor.sum(-1)
Out[3]:
tensor([[ 6, 22, 38],
[54, 70, 86]])
Based on doc https://pytorch.org/docs/stable/generated/torch.sum.html
it should be
dim (int or tuple of python:ints) – the dimension or dimensions to reduce.
dim=0 means reduce row dimensions: condense all rows = sum by col
dim=1 means reduce col dimensions: condense cols= sum by row
Torch sum along multiple axis or dimensions
Just for the sake of completeness (I could not find it easily) I include how to sum along multiple dimensions with torch.sum which is heavily used in computer vision tasks where you have to reduce along H and W dimensions.
If you have an image x with shape C x H x W and want to compute the average pixel intensity value per channel you could do:
avg = torch.sum(x, dim=(1,2)) / (H*W) # Sum along (H,W) and norm
I am trying to implement a function, which can split a 3 dimensional numpy array in to 8 pieces, whilst keeping the order intact. Essentially I need the splits to be:
G[:21, :18,:25]
G[21:, :18,:25]
G[21:, 18:,:25]
G[:21, 18:,:25]
G[:21, :18,25:]
G[21:, :18,25:]
G[21:, 18:,25:]
G[:21, 18:,25:]
Where the original size of this particular matrix would have been 42, 36, 50. How is it possible to generalise these 8 "slices" so I do not have to hardcode all of them? essentially move the :in every possible position.
Thanks!
You could apply the 1d slice to successive (lists of) dimensions.
With a smaller 3d array
In [147]: X=np.arange(4**3).reshape(4,4,4)
A compound list comprehension produces a nested list. Here I'm using the simplest double split
In [148]: S=[np.split(z,2,0) for y in np.split(X,2,2) for z in np.split(y,2,1)]
In this case, all sublists have the same size, so I can convert it to an array for convenient viewing:
In [149]: SA=np.array(S)
In [150]: SA.shape
Out[150]: (4, 2, 2, 2, 2)
There are your 8 subarrays, but grouped (4,2).
In [153]: SAA = SA.reshape(8,2,2,2)
In [154]: SAA[0]
Out[154]:
array([[[ 0, 1],
[ 4, 5]],
[[16, 17],
[20, 21]]])
In [155]: SAA[1]
Out[155]:
array([[[32, 33],
[36, 37]],
[[48, 49],
[52, 53]]])
Is the order right? I can change it by changing the axis in the 3 split operations.
Another approach is write your indexing expressions as tuples
In [156]: x,y,z = 2,2,2 # define the split points
In [157]: ind = [(slice(None,x), slice(None,y), slice(None,z)),
(slice(x,None), slice(None,y), slice(None,z)),]
# and so on
In [158]: S1=[X[i] for i in ind]
In [159]: S1[0]
Out[159]:
array([[[ 0, 1],
[ 4, 5]],
[[16, 17],
[20, 21]]])
In [160]: S1[1]
Out[160]:
array([[[32, 33],
[36, 37]],
[[48, 49],
[52, 53]]])
Looks like the same order I got before.
That ind list of tuples can be produced with some sort of iteration and/or list comprehension. Maybe even using itertools.product or np.mgrid to generate the permutations.
An itertools.product version could look something like
In [220]: def foo(i):
return [(slice(None,x) if j else slice(x,None))
for j,x in zip(i,[2,2,2])]
In [221]: SAA = np.array([X[foo(i)] for i in
itertools.product(range(2),range(2),range(2))])
In [222]: SAA[-1]
Out[222]:
array([[[ 0, 1],
[ 4, 5]],
[[16, 17],
[20, 21]]])
product iterates the last value fastest, so the list is reversed (compared to your target).
To generate a particular order it may be easier to list the tuples explicitly, e.g.:
In [227]: [X[foo(i)] for i in [(1,1,1),(0,1,1),(0,0,1)]]
Out[227]:
[array([[[ 0, 1],
[ 4, 5]],
[[16, 17],
[20, 21]]]), array([[[32, 33],
[36, 37]],
[[48, 49],
[52, 53]]]), array([[[40, 41],
[44, 45]],
[[56, 57],
[60, 61]]])]
This highlights the fact that there are 2 distinct issues - generating the iteration pattern, and splitting the array based on this pattern.