We Keep Coding

How to extrapolate points of a regular grid in python

I have a numpy array of points and want to extend my point (x, y and z) in a defined direction. This is my array:
coordinates=np.array([[1., 19., 4.], [2., 19., 4.5], [1., 20., 4.],[2., 20., 4.5], [3., 20., 4.]])
These points are on two y grids: 19 and 20. I want to extend my points in these grid lines. First grid has two point. x and y coordinates of extrapolated points are fixed. y equals the grid value (19 or 20) and x equals one unit more that the last point of the grid. For first grid the new x and y of two points are (3.,19.) and (4.,19.) and z should be calculated as:
z of last point + (z of last point - z of one point before the last)/2
In case of having just one point in the grid, I copy the value of that point. In the first grid, z value of first extrapolated point is :
4.5 + (4.5-4)/2 = 4.75
and for the second point is
4.75 + (4.75-4.5)/2 = 4.875
I want to do the same for the next grid (y=20) to finally add all four point to the existing array:
all_points=np.array([[1.,19.,4.], [1.,20.,4.], [2.,19.,4.5], [2.,20.,4.5], [3.,20.,4.],\
[3.,19.,4.75], [4.,19.,4.875], [4.,20,3.75], [5.,20, 3.625]])
I tried the following but firstly I have no idea how to generate z value dynamically. At the moment it is calculating the same z values for the new two generated points of each grid. Another issue is that my code is not effiecient and I believe there are much faster ways to do it:
nums, counts=np.unique(coordinates[:,1],return_counts=True) # gives the y grids
new_y=nums
cum=np.cumsum(counts)-1
new_x=[]
for i in cum:
new_x.append (coordinates[i,0])
new_x=np.array(new_x)
new_x_y=[]
for m, n in zip (new_x, new_y):
new_x_y.append([m,n])
new_x_y.append([m+1,n])
new_x_y=np.array(new_x_y)
z_difference=[]
for i in cum:
z_difference.append((coordinates[i,-1]-coordinates[i-1,-1])/2)# to find the difference of last two points of each grid
new_z=[]
for i in range (len(cum)-1):
new_z.append (coordinates[cum[i],-1]+z_difference)#to find the new z of each extrapolated point
new_z=np.array(new_z)
all_z=np.repeat (new_z, repeats=new_z.shape[1], axis=1)
final_arr=np.hstack ([new_x_y, all_z.T])
In advance, I do appreciate any help to solve my problem in python.

cv2 triangulatePoints always returns same Z value

I am trying to get 3D points using cv2.triangulatePoints but it always returns almost same Z value. My output looks like this: As it seen, all points are in almost same Z value. There is no depth.
Here is my triangulation:
def triangulate(self, proj_mat1, pts0, pts1):
proj_mat0 = np.zeros((3,4))
proj_mat0[:, :3] = np.eye(3)
pts0, pts1 = self.normalize(pts0), self.normalize(pts1)
pts4d = cv2.triangulatePoints(proj_mat0, proj_mat1, pts0.T, pts1.T).T
pts4d /= pts4d[:, 3:]
out = np.delete(pts4d, 3, 1)
print(out)
return out
Here is my projection matrix calculation:
def getP(self, rmat, tvec):
P = np.concatenate([rmat, tvec.reshape(3, 1)], axis = 1)
return P
Here is the part that I get rmat, tvec and call triangulation:
E, mask = cv2.findEssentialMat(np.array(aa), np.array(bb), self.K)
_, R, t, mask = cv2.recoverPose(E, np.array(aa), np.array(bb), self.K)
proj_mat1 = self.getP(R, t)
out = self.triangulate(proj_mat1, np.array(aa, dtype = np.float32), np.array(bb, dtype = np.float32))
My camera matrix:
array([[787.8113353 , 0. , 318.49905794],
[ 0. , 786.9638204 , 245.98673477],
[ 0. , 0. , 1. ]])
My projection matrix 1:
array([[1., 0., 0., 0.],
[0., 1., 0., 0.],
[0., 0., 1., 0.]])
Explanations:
aa and bb are matched points from 2 frames.
self.K is my camera matrix
rotation and translation matrices are extracted from Essential matrix
Essential matrix calculated from matched keypoints. It changes every frame.
Projection matrix 2 changes every frame.
Output after changing first projection matrix (I switched from matplotlib to pangolin as 3D visualization tool):
Output after using P1 and P2 that I mentioned in comments:
Where is my mistake? Please let me know if any further information needed. I will update my question.

Unfortunately I don't have the possibility to double-check directly but my gut feeling is that the issues you are facing are essentially due to the choice of your first projection matrix
I did some research and I found this great paper with both theory and practice. Despite differing a little bit from your approach, there is a thing that is worth saying
If you check carefully, the first projection matrix is exactly the camera matrix with an additional last column equal to zero. In fact, the rotation matrix for the first camera reduces to the identity matrix and the corresponding translation vector is a null vector, so using this general formula:
P = KT
where P is the projection matrix, K the camera matrix and T the matrix obtained by the rotation matrix R flanked by the translation vector t according to:
T = [R|t]
then you will get:
Coming back to your case, first of all I would suggest to change your first projection matrix as just said
Also, I understand that you are planned to work with something different at every frame but if after the suggested change the things still don't match then in your shoes I'd start working with just 2 images [I think you implicitly did already to create the correspondence between aa and bb], calculating first the matrices with your algorithm and then checking with the ones obtained following the article above
In this way you would be able to understand/debug which matrices are creating you troubles

Thank you so much for all the effort #Antonino. My webcams were pretty bad. After changing every part of my code and making many trials I decided to change my webcams and bought good webcams. It worked :D Here is the result:

Is there a quick way to ignore certain values of an array when performing numpy operations?

Sorry if the title is confusing, but it is very hard to put what I would like to do in a single sentence. Image you have an image stack stack in the form of N m x n matrices as a numpy array in the shape of (m, n, N). Now, if I want to perform the numpy.median for example along the stack axis N it is very easy: numpy.median(stack, 0). The problem is that for each image of the stack, I also have a mask of pixels that I would not like to include in the operation, in this case numpy.median. Is there any efficient way to do that?
So far, all I could think of is this, but it is increadibly slow and absolutely not feasible:
median = [[]]*images[0].flatten().shape
for i in range(len(images)):
image = images[i].flatten()
mask = mask[i].flatten()
for j in range(len(median)):
if mask[j] == 0:
median[j].append(image[j])
for i in range(len(median)):
median[j] = np.median(median[j]) if median[j] else 0
median = np.array(median).reshape(images[0].shape)
There has to be a better way.

What you can do is build a an array with NaNs in the non-masked values and compute np.nanmedian (which ignores NaNs). You can build such an array "on the fly" using np.where:
x = np.arange(4*3*4).reshape((4,3,4))
m = x%2 == 0
np.nanmedian(np.where(m, x, np.NaN), axis=2)
>>array([[ 1., 5., 9.],
[13., 17., 21.],
[25., 29., 33.],
[37., 41., 45.]])

I have a hard time understanding what you are trying to say, but hopefully this will help:
You can use np.whereto find and replace - or ignore/remove - values that you want to exclude.
Or you can use bin_mask = stack != value_you_want_to_ignore to get a boolean array that you can use to ignore your critical values.

How to append all subdivided polygon coordinates?

let A be set of coordinate (e.g. array([[ 10., 110.],[200., 90.],[210., 0.], [-10., 10.],[ 10., 110.]]).
If subdivision(coordinate) is a division code, giving two outcomes - subdivisionR, subdivisionL, then subdivision(A) gives array([[ 10., 110.],[110.0, 99.47],[110.0, 4.54],[-10., 10.],[ 10., 110.]]), array([[110.0, 99.47],[200.,90.],[210., 0.],[110.0, 4.54],[110.0, 99.47]]).
Since I wanted to divide the polygon until all subdivided polygons reach the area closest to 200 and get all subdivided coordinates, I used following code;
n=1
t=0
r=0
k=A
while area(subdivision(k)[n-1])>200:
for i in range(t,n):
k.append(subdivision(k[i])[0])
k.append(subdivision(k[i])[1])
t=2**r
r=r+1
n=2**r
However, I get warning - 'numpy.ndarray' object has no attribute 'append'.
How can I fix this?

The syntax for appending to a numpy array is different to say that of a python list. You want to use:
numpy.append(array, value, axis)
So something like:
numpy.append(k, subdivision(k[i]), axis = 0)
Note axis is optional and would default to 1. 'value' is a numpy array, not a discrete value.

Multiple coefficient sets for least squares fitting in numpy/scipy

Is there a way to perform multiple simultaneous (but unrelated) least-squares fits with different coefficient matrices in either numpy.linalg.lstsq or scipy.linalg.lstsq? For example, here is a trivial linear fit that I would like to be able to do with different x-values but the same y-values. Currently, I have to write a loop:
x = np.arange(12.0).reshape(4, 3)
y = np.arange(12.0, step=3.0)
m = np.stack((x, np.broadcast_to(1, x.shape)), axis=0)
fit = np.stack(tuple(np.linalg.lstsq(w, y, rcond=-1)[0] for w in m), axis=-1)
This results in a set of fits with the same slope and different intercepts, such that fit[n] corresponds to coefficients m[n].
Linear least squares is not a great example since it is invertible, and both functions have an option for multiple y-values. However, it serves to illustrate my point.
Ideally, I would like to extend this to any "broadcastable" combination of a and b, where a.shape[-2] == b.shape[0] exactly, and the last dimensions have to either match or be one (or missing). I am not really hung up on which dimension of a is the one representing the different matrices: it was just convenient to make it the first one to shorten the loop.
Is there a built in method in numpy or scipy to avoid the Python loop? I am very much interested in using lstsq rather than manually transposing, multiplying and inverting the matrices.

You could use scipy.sparse.linalg.lsqr together with scipy.sparse.block_diag. I'm just not sure it will be any faster.
Example:
>>> import numpy as np
>>> from scipy.sparse import block_diag
>>> from scipy.sparse import linalg as sprsla
>>>
>>> x = np.random.random((3,5,4))
>>> y = np.random.random((3,5))
>>>
>>> for A, b in zip(x, y):
... print(np.linalg.lstsq(A, b))
...
(array([-0.11536962, 0.22575441, 0.03597646, 0.52014899]), array([0.22232195]), 4, array([2.27188101, 0.69355384, 0.63567141, 0.21700743]))
(array([-2.36307163, 2.27693405, -1.85653264, 3.63307554]), array([0.04810252]), 4, array([2.61853881, 0.74251282, 0.38701194, 0.06751288]))
(array([-0.6817038 , -0.02537582, 0.75882223, 0.03190649]), array([0.09892803]), 4, array([2.5094637 , 0.55673403, 0.39252624, 0.18598489]))
>>>
>>> sprsla.lsqr(block_diag(x), y.ravel())
(array([-0.11536962, 0.22575441, 0.03597646, 0.52014899, -2.36307163,
2.27693405, -1.85653264, 3.63307554, -0.6817038 , -0.02537582,
0.75882223, 0.03190649]), 2, 15, 0.6077437777160813, 0.6077437777160813, 6.226368324510392, 106.63227777368986, 1.3277892240815807e-14, 5.36589277249043, array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]))