We Keep Coding

Searching within Nested lists in Python || How do I match a 1-index nested list, with a 2-index nested list

here is the problem I am trying to solve:
coord = [[0, 0], [1, 0], [1, 1], [2, 0], [2, 1], [2, 1], [2, 2] ..]
new_arr = [[[0, 0], 1], [[1, 0], 1], [[1, 1], 1], [[2, 0], 1], [[2, 1], 2], [[2, 2], 1] ..]
This is the target I am trying to map to
[0, 0][0, 1][0, 2]
[1, 0][1, 1][1, 2]
[2, 0][2, 1][2, 2]
the ultimate output would be the counts against each of the coordinates
1 0 0
1 1 0
1 2 1
------ clarifications --------
the goal is to generate this square of numbers (counts) which is the second element in new_arr. E.g. [[0, 0], 1], [[1, 0], 1], can be interpreted as the value 1 for the coordinate [0,0] and value 1 for coordinate [1,0] 
the first list (coord) is simply a map of the coordinates. The goal is to get the corresponding value (from new_arr) and display it in the form of a square. Hope this clarified. The output will be a grid of the format
1 0 0
1 1 0
1 2 1
to the question of N (I just took a sample value of 3). The actual use case is when the user enters an integer, say 6 and the result is in a 6 X 6 square. The counts are chess move computations on the ways to reach a specific cell (two movements only (i+1, j) & (i+1, j+1) ....... starting from (0,0)

The logic is not fully clear, but is looks like you want to map the values of new_arr on the Cartesian product of coordinates:
N = 3 # how this is determined is unclear
d = {tuple(l):x for l, x in new_arr}
# {(0, 0): 1, (1, 0): 1, (1, 1): 1, (2, 0): 1, (2, 1): 2, (2, 2): 1}
out = [d.get((i,j), 0) for i in range(N) for j in range(N)]
# [1, 0, 0, 1, 1, 0, 1, 2, 1]
# 2D variant
out2 = [[d.get((i,j), 0) for j in range(N)] for i in range(N)]
# [[1, 0, 0],
# [1, 1, 0],
# [1, 2, 1]]
alternative with numpy
import numpy as np
N = 3
a = np.zeros((N,N), dtype=int)
# get indices and values
idx, val = zip(*new_arr)
# assign values (option 1)
a[tuple(zip(*idx))] = val
# assign values (option 2)
a[tuple(np.array(idx).T.tolist())] = val
print(a)
output:
array([[1, 0, 0],
[1, 1, 0],
[1, 2, 1]])

Use numpy:
import numpy as np
i = []
coord = [[0, 0], [1, 0], [1, 1], [2, 0], [2, 1], [2, 1], [2, 2]]
new_arr = [[[0, 0], 1], [[1, 0], 1], [[1, 1], 1], [[2, 0], 1], [[2, 1], 2], [[2, 2], 1]]
result = np.zeros([coord[-1][0] + 1, coord[-1][1] + 1])
for i in new_arr:
for j in coord:
if i[0] == j:
result[j[0],j[1]]= i[1]
print(result)
Output:
[[1. 0. 0.]
[1. 1. 0.]
[1. 2. 1.]]

Is there a vectorized way to apply a transformation matrix to a group of vectors?

Imagine you have a group of vectors, e.g. in the form of a trajectory. Is there a vectorized way of applying the transformation matrix to all data points at once, or are you stuck with a for-loop? Here is some sample code:
import numpy as np
angle = np.deg2rad(90)
rotM = np.array(
[
[np.cos(angle), -np.sin(angle), 0],
[np.sin(angle), np.cos(angle), 0],
[ 0, 0, 1],
]
)
# trajectory with columns t, x, y, z
trajectory = np.array(
[
[1, 1, 0, 0],
[2, 2, 1, 0],
[3, 3, 2, 0],
[4, 4, 3, 1],
[5, 6, 4, 2],
[6, 9, 5, 3],
]
)
# transform coordinates
for i in range(len(trajectory)):
trajectory[i][1:] = np.dot(rotM, trajectory[i][1:])
All I found so far is numpy.linalg.multi_dot, and these two posts (one, two), none of which seem to apply to my case.

For this case, use broadcasting along with np.matmul/#. You can multiply a 3x3 martrix by an Nx3x1 array of vectors:
trajectory[:, 1:] = rotM # trajectory[:, 1:, None]
A cleaner and more flexible solution might be to use scipy.spatial.transform.Rotation objects instead of hand-crafting the matrix yourself:
rotM = Rotation.from_euler('z', angle)
trajectory[:, 1:] = rotM.apply(trajectory[:, 1:])
No need to add shim dimensions in this case.

Zero pad ndarray along axis

I want to pad:
[[1, 2]
[3, 4]]
to
[[0, 0, 0]
[0, 1, 2]
[0, 3, 4]]
I have no problem in doing so when input is 2D by vstack & hsrack
However, when I add 1 dim to represent the dim of number, such as:
img = np.random.randint(-10, 10, size=(2, 2, 2))
I cannot do so if I would not like to use for loop.
Is there any way to avoid for but only numpy's stack to make it?
ps: for this example I should get (2,3,3) array by each 2 img instance is pad by zero at 1st column & 1st row.
Thanks

You can use np.pad, which allows you to specify the number of values padded to the edges of each axis.
So for the first example 2D array you could do:
a = np.array([[1, 2],[3, 4]])
np.pad(a,((1, 0), (1, 0)), mode = 'constant')
array([[0, 0, 0],
[0, 1, 2],
[0, 3, 4]])
So here each tuple is representing the side which to pad with zeros along each axis, i.e. ((before_1, after_1), … (before_N, after_N)).
And for a 3D array the same applies, but in this case we must specify that we only want to zero pad the two last dimensions:
img = np.random.randint(-10, 10, size=(2, 2, 2))
np.pad(img, ((0,0), (1,0), (1,0)), 'constant')
array([[[ 0, 0, 0],
[ 0, -3, -2],
[ 0, 9, -5]],
[[ 0, 0, 0],
[ 0, 1, -9],
[ 0, -1, -3]]])

You can use np.pad and remove the last row/column:
import numpy as np
a = np.array([[1, 2], [3, 4]])
result = np.pad(a, 1, mode='constant', constant_values=0)[:-1, :-1]
print(result)
Output:
[[0 0 0]
[0 1 2]
[0 3 4]]

If you want to pad only the column
import numpy as np
max_length =20
a = np.random.randint(1,size=(1, 10))
print(a.shape)
print(a.shape[1])
a =np.pad(a, [(0, 0), (0, max_length - a.shape[1])], mode='constant')
print(a.shape)
Output
(1, 10)
10
(1, 20)

How do I calculate xi^j in a matrix in Numpy

I am trying to calculate a matrix from an array that is inputted.
I would like to be able to input
a = [0,1,2]
in python and would like to reshape it with Numpy such that the result is that the array is in the form of x_i^j at row i and column j,
so for example
the input is:
a = [0,1,2]
and the output should be
[[1,0,0],
[1,1,1],
[1,2,4]]
and I have used the following code
xij = np.matrix([np.power(xi,j) for j in x for xi in x]).reshape(3,3)
[[ 1, 2, 3],
[ 1, 4, 9],
[ 1, 8, 27]]
I assume I'm using the wrong formula for Numpy,
please could you assist me in this to solve the problem.
Thanks in advance

You need to use a range(len(a)) to get the exponents and the correct order of for loops
a = [0,1,2]
xij = np.matrix([np.power(xi,j) for xi in a for j in range(len(a))]).reshape(3,3)
# matrix([[1, 0, 0],
# [1, 1, 1],
# [1, 2, 4]])

With array broadcasting:
In [823]: np.array([0,1,2])**np.arange(3)[:,None]
Out[823]:
array([[1, 1, 1],
[0, 1, 2],
[0, 1, 4]])
In [825]: np.array([1,2,3])**np.arange(1,4)[:,None]
Out[825]:
array([[ 1, 2, 3],
[ 1, 4, 9],
[ 1, 8, 27]])

Getting positions of specific values of a 2D NumPy array with mask

I need some help to detect all values (coordinates) of 2D array which verify a specific conditional.
I have already asked a similar question, but now I masked specific values which don't interest me...
Last time, a person suggested to use zip(*np.where(test2D < 5000.))
For example:
import numpy as np
test2D = np.array([[ 3051.11, 2984.85, 3059.17],
[ 3510.78, 3442.43, 3520.7 ],
[ 4045.91, 3975.03, 4058.15],
[ 4646.37, 4575.01, 4662.29],
[ 5322.75, 5249.33, 5342.1 ],
[ 6102.73, 6025.72, 6127.86],
[ 6985.96, 6906.81, 7018.22],
[ 7979.81, 7901.04, 8021. ],
[ 9107.18, 9021.98, 9156.44],
[ 10364.26, 10277.02, 10423.1 ],
[ 11776.65, 11682.76, 11843.18]])
So I can get all positions which verify < 5000 :
positions=zip(*np.where(test2D < 5000.))
Now I want to reject some values which are useless for me (it s an array with coordinates):
rejectedvalues = np.array([[0, 0], [2, 2], [3, 1], [10, 2]])
i, j = rejectedvalues.T
mask = np.zeros(test2D.shape, bool)
mask[i,j] = True
m = np.ma.array(test2D, mask=mask)
positions2=zip(*np.where(m < 5000.))
But positions2 gives me the same as positions...

np.ma.where respects the mask -- it does not return indices in the condition (e.g. m < 5000.) that are masked.
In [58]: np.asarray(np.column_stack(np.ma.where(m < 5000.)))
Out[58]:
array([[0, 1],
[0, 2],
[1, 0],
[1, 1],
[1, 2],
[2, 0],
[2, 1],
[3, 0],
[3, 2]])
Compare that with the analogous expression using np.where:
In [57]: np.asarray(np.column_stack(np.where(m < 5000.)))
Out[57]:
array([[0, 0],
[0, 1],
[0, 2],
[1, 0],
[1, 1],
[1, 2],
[2, 0],
[2, 1],
[2, 2],
[3, 0],
[3, 1],
[3, 2]])