We Keep Coding

python: DELETE points out of a very big 2D array and elements are float, like discarding unwanted points in KNN

I have a 2D array and I want to delete a point out of it but suppose it's so big meaning I can't specify an index and just grab it and the values of the array are float
How can I delete this point? With a LOOP and WITHOUT LOOP?? the following is 2D array and I want to delete [ 32.9, 23.]
[[ 1. , -1.4],
[ -2.9, -1.5],
[ -3.6, -2. ],
[ 1.5, 1. ],
[ 24. , 11. ],
[ -1. , 1.4],
[ 2.9, 1.5],
[ 3.6, 2. ],
[ -1.5, -1. ],
[ -24. , -11. ],
[ 32.9, 23. ],
[-440. , 310. ]]
I tried this but doesn't work:
this_point = np.asarray([ 32.9, 23.])
[x for x in y if x == point]
del datapoints[this_point]
np.delete(datapoints,len(datapoints), axis=0)
for this_point in datapoints:
del this_point
when I do this, the this_point stays in after printing all points, what should I do?

Python can remove a list element by content, but numpy does only by index. So, use "where" to find the coordinates of the matching row:
import numpy as np
a = np.array([[ 1. , -1.4],
[ -2.9, -1.5],
[ -3.6, -2. ],
[ 1.5, 1. ],
[ 24. , 11. ],
[ -1. , 1.4],
[ 2.9, 1.5],
[ 3.6, 2. ],
[ -1.5, -1. ],
[ -24. , -11. ],
[ 32.9, 23. ],
[-440. , 310. ]])
find = np.array([32.9,23.])
row = np.where( (a == find).all(axis=1))
print( row )
print(np.delete( a, row, axis=0 ) )
Output:
(array([10], dtype=int64),)
[[ 1. -1.4]
[ -2.9 -1.5]
[ -3.6 -2. ]
[ 1.5 1. ]
[ 24. 11. ]
[ -1. 1.4]
[ 2.9 1.5]
[ 3.6 2. ]
[ -1.5 -1. ]
[ -24. -11. ]
[-440. 310. ]]
C:\tmp>

How to access the x axis index of a scipy denogram?

I have data, for which I create a linkage model, like this:
model = sc.linkage(data, 'ward')
Where model is as follows:
Z = np.array([
[ 2. , 9. , 20.12172148, 2. ],
[ 0. , 1. , 26.16772232, 2. ],
[ 11. , 12. , 29.40258214, 2. ],
[ 14. , 16. , 30.89332011, 3. ],
[ 3. , 7. , 33.70695832, 2. ],
[ 5. , 13. , 34.22180543, 2. ],
[ 4. , 15. , 35.52080322, 3. ],
[ 17. , 21. , 45.3919152 , 5. ],
[ 6. , 20. , 45.56339627, 3. ],
[ 8. , 23. , 66.42828305, 4. ],
[ 10. , 22. , 87.52531145, 6. ],
[ 18. , 24. , 93.78070161, 7. ],
[ 19. , 26. , 124.09967826, 9. ],
[ 25. , 27. , 160.11685636, 15. ]])
Z == model # returns true
I can then plot this linkage model using matplotlib:
# calculate full dendrogram
plt.figure(figsize=(25, 10))
plt.title('Hierarchical Clustering Dendrogram for signature data')
plt.xlabel('sample index')
plt.ylabel('distance')
sc.dendrogram(
model,
leaf_rotation=90., # rotates the x axis labels
leaf_font_size=8., # font size for the x axis labels
)
plt.show()
Now, this plots the dendogram and sets the xticks to the index. I would want to replace this with actual labels, which are
labels = ['wood', 'stone', 'flora', 'liquid', 'food', 'metal', 'ceramic',
'sky', 'glass', 'paper', 'animal', 'skin', 'fabrics', 'gem', 'ground']
as in, the first tick on the x-axis reads 10. Which would be labels[10]. However, I can't find out how to access this index.

There is no need to access the index. scipy.cluster.hierarchy.dendrogram provides a labels argument which you should use to supply your labels.
scipy.cluster.hierarchy.dendrogram(Z, labels=labels, ....)
Complete code:
import numpy as np
import scipy.cluster.hierarchy as sc
import matplotlib.pyplot as plt
Z = np.array([
[ 2. , 9. , 20.12172148, 2. ],
[ 0. , 1. , 26.16772232, 2. ],
[ 11. , 12. , 29.40258214, 2. ],
[ 14. , 16. , 30.89332011, 3. ],
[ 3. , 7. , 33.70695832, 2. ],
[ 5. , 13. , 34.22180543, 2. ],
[ 4. , 15. , 35.52080322, 3. ],
[ 17. , 21. , 45.3919152 , 5. ],
[ 6. , 20. , 45.56339627, 3. ],
[ 8. , 23. , 66.42828305, 4. ],
[ 10. , 22. , 87.52531145, 6. ],
[ 18. , 24. , 93.78070161, 7. ],
[ 19. , 26. , 124.09967826, 9. ],
[ 25. , 27. , 160.11685636, 15. ]])
labels = ['wood', 'stone', 'flora', 'liquid', 'food', 'metal', 'ceramic',
'sky', 'glass', 'paper', 'animal', 'skin', 'fabrics', 'gem', 'ground']
# calculate full dendrogram
plt.figure()
plt.title('Hierarchical Clustering Dendrogram for signature data')
plt.xlabel('sample index')
plt.ylabel('distance')
sc.dendrogram(
Z,
labels=labels,
leaf_rotation=90., # rotates the x axis labels
leaf_font_size=8., # font size for the x axis labels
)
plt.tight_layout()
plt.show()

I do not have dendogram module but the following should work for you. The idea is:
Create an axis instance ax and pass it to dendogram plot as axis argument
Get the existing x-ticklabels, and convert them to integers. Use these integers as indices to access the values from labels. This is just to have the labels in the order as you want them to be displayed on the x-axis
Set these new labels using set_xticklabels
Following is the relevant piece of code you can use
fig, ax = plt.subplots(figsize=(25, 10))
sc.dendrogram(
model,
leaf_rotation=90.,
leaf_font_size=8., ax=ax)
fig.canvas.draw()
new_labels = [labels[int(i.get_text())] for i in ax.get_xticklabels()]
ax.set_xticklabels(new_labels)

Recover data from matplotlib scatter plot [duplicate]

This question already has an answer here:
Extracting data from a scatter plot in Matplotlib
(1 answer)
Closed 2 years ago.
From a matplotlib scatter plot, I'm trying the recover the point data. Consider
from matplotlib import pyplot as plt
import numpy as np
fig = plt.figure()
x = np.linspace(0.0, 1.0, 5)
y = np.linspace(0.0, 1.0, 5)
plt.scatter(x, y)
ax = fig.get_children()[1]
pc = ax.get_children()[2]
for path in pc.get_paths():
print
print('path:')
print(path)
print
print('segments:')
for vert, code in path.iter_segments():
print(code, vert)
plt.show()
This yields
path:
Path(array([[ 0. , -0.5 ],
[ 0.13260155, -0.5 ],
[ 0.25978994, -0.44731685],
[ 0.35355339, -0.35355339],
[ 0.44731685, -0.25978994],
[ 0.5 , -0.13260155],
[ 0.5 , 0. ],
[ 0.5 , 0.13260155],
[ 0.44731685, 0.25978994],
[ 0.35355339, 0.35355339],
[ 0.25978994, 0.44731685],
[ 0.13260155, 0.5 ],
[ 0. , 0.5 ],
[-0.13260155, 0.5 ],
[-0.25978994, 0.44731685],
[-0.35355339, 0.35355339],
[-0.44731685, 0.25978994],
[-0.5 , 0.13260155],
[-0.5 , 0. ],
[-0.5 , -0.13260155],
[-0.44731685, -0.25978994],
[-0.35355339, -0.35355339],
[-0.25978994, -0.44731685],
[-0.13260155, -0.5 ],
[ 0. , -0.5 ],
[ 0. , -0.5 ]]), array([ 1, 4, 4, 4, 4, 4, 4, 4, 4,
4, 4, 4, 4, 4, 4, 4, 4,
4, 4, 4, 4, 4, 4, 4, 4, 79], dtype=uint8))
segments:
(1, array([ 0. , -0.5]))
(4, array([ 0.13260155, -0.5 , 0.25978994, -0.44731685, 0.35355339,
-0.35355339]))
(4, array([ 0.44731685, -0.25978994, 0.5 , -0.13260155, 0.5 , 0.
]))
(4, array([ 0.5 , 0.13260155, 0.44731685, 0.25978994, 0.35355339,
0.35355339]))
(4, array([ 0.25978994, 0.44731685, 0.13260155, 0.5 , 0. ,
0.5 ]))
(4, array([-0.13260155, 0.5 , -0.25978994, 0.44731685, -0.35355339,
0.35355339]))
(4, array([-0.44731685, 0.25978994, -0.5 , 0.13260155, -0.5 , 0.
]))
(4, array([-0.5 , -0.13260155, -0.44731685, -0.25978994, -0.35355339,
-0.35355339]))
(4, array([-0.25978994, -0.44731685, -0.13260155, -0.5 , 0. ,
-0.5 ]))
(79, array([ 0. , -0.5]))
/usr/local/lib/python2.7/dist-packages/matplotlib/collections.py:590:
FutureWarning: elementwise comparison failed; returning scalar instead, but in
the future will perform elementwise comparison
if self._edgecolors == str('face'):
but I don't see any of that data correlate with the actual scatter input data. Perhaps it's not the ax.get_children()[2] path collection I need to look at?

Given the PathCollection returned by plt.scatter, you could call its get_offsets method:
from matplotlib import pyplot as plt
import numpy as np
fig = plt.figure()
x = np.linspace(0.0, 1.0, 5)
y = np.linspace(0.0, 1.0, 5)
s = plt.scatter(x, y)
print(s.get_offsets())
# [[ 0. 0. ]
# [ 0.25 0.25]
# [ 0.5 0.5 ]
# [ 0.75 0.75]
# [ 1. 1. ]]
Or, given the axes object, ax, you could access the PathCollection via ax.collections, and then call get_offsets:
In [110]: ax = fig.get_axes()[0]
In [129]: ax.collections[0].get_offsets()
Out[131]:
array([[ 0. , 0. ],
[ 0.25, 0.25],
[ 0.5 , 0.5 ],
[ 0.75, 0.75],
[ 1. , 1. ]])

You could also get the z coordinate. In case you used 3d data:
from matplotlib import pyplot as plt
import numpy as np
fig = plt.figure()
x = np.linspace(0.0, 1.0, 5)
y = np.linspace(0.0, 1.0, 5)
z = np.linspace(0.0, 10, 5)
s = plt.scatter(x, y, c=z)
cbar=plt.colorbar(s)
To retrieve information of x,y,z:
ax=fig.get_axes()[0]
x_r=ax.collections[0].get_offsets()[:,0]
y_r=ax.collections[0].get_offsets()[:,1]
z_r=ax.collections[0].get_array()

Curve curvature in numpy

I am measuring x,y coordinates (in cm) of an object with a special camera in fixed time intervals of 1s. I have the data in a numpy array:
a = np.array([ [ 0. , 0. ],[ 0.3 , 0. ],[ 1.25, -0.1 ],[ 2.1 , -0.9 ],[ 2.85, -2.3 ],[ 3.8 , -3.95],[ 5. , -5.75],[ 6.4 , -7.8 ],[ 8.05, -9.9 ],[ 9.9 , -11.6 ],[ 12.05, -12.85],[ 14.25, -13.7 ],[ 16.5 , -13.8 ],[ 19.25, -13.35],[ 21.3 , -12.2 ],[ 22.8 , -10.5 ],[ 23.55, -8.15],[ 22.95, -6.1 ],[ 21.35, -3.95],[ 19.1 , -1.9 ]])
And the curve looks like this:
plt.scatter(a[:,0], a[:,1])
Question:
How can I calculate the tangential and the radial aceleration vectors at each point? I found some formulas that might be relevant:
I am able to easily calculate the vx and the vy projections with np.diff(a, axis=0) but I am a numpy/python noob and it is way over my head to continue. If I could calculate the curvature at each point, also my problem would be solved. Can somebody help?

EDIT: I put together this answer off and on over a couple of hours, so I missed your latest edits indicating that you only needed curvature. Hopefully, this answer will be helpful regardless.
Other than doing some curve-fitting, our method of approximating derivatives is via finite differences. Thankfully, numpy has a gradient method that does these difference calculations for us, taking care of the details of averaging previous and next slopes for each interior point and leaving each endpoint alone, etc.
import numpy as np
a = np.array([ [ 0. , 0. ],[ 0.3 , 0. ],[ 1.25, -0.1 ],
[ 2.1 , -0.9 ],[ 2.85, -2.3 ],[ 3.8 , -3.95],
[ 5. , -5.75],[ 6.4 , -7.8 ],[ 8.05, -9.9 ],
[ 9.9 , -11.6 ],[ 12.05, -12.85],[ 14.25, -13.7 ],
[ 16.5 , -13.8 ],[ 19.25, -13.35],[ 21.3 , -12.2 ],
[ 22.8 , -10.5 ],[ 23.55, -8.15],[ 22.95, -6.1 ],
[ 21.35, -3.95],[ 19.1 , -1.9 ]])
Now, we compute the derivatives of each variable and put them together (for some reason, if we just call np.gradient(a), we get a list of arrays...not sure off the top of my head what's going on there, but I'll just work around it for now):
dx_dt = np.gradient(a[:, 0])
dy_dt = np.gradient(a[:, 1])
velocity = np.array([ [dx_dt[i], dy_dt[i]] for i in range(dx_dt.size)])
This gives us the following vector for velocity:
array([[ 0.3 , 0. ],
[ 0.625, -0.05 ],
[ 0.9 , -0.45 ],
[ 0.8 , -1.1 ],
[ 0.85 , -1.525],
[ 1.075, -1.725],
[ 1.3 , -1.925],
[ 1.525, -2.075],
[ 1.75 , -1.9 ],
[ 2. , -1.475],
[ 2.175, -1.05 ],
[ 2.225, -0.475],
[ 2.5 , 0.175],
[ 2.4 , 0.8 ],
[ 1.775, 1.425],
[ 1.125, 2.025],
[ 0.075, 2.2 ],
[-1.1 , 2.1 ],
[-1.925, 2.1 ],
[-2.25 , 2.05 ]])
which makes sense when glancing at the scatterplot of a.
Now, for speed, we take the length of the velocity vector. However, there's one thing that we haven't really kept in mind here: everything is a function of t. Thus, ds/dt is really a scalar function of t (as opposed to a vector function of t), just like dx/dt and dy/dt. Thus, we will represent ds_dt as a numpy array of values at each of the one second time intervals, each value corresponding to an approximation of the speed at each second:
ds_dt = np.sqrt(dx_dt * dx_dt + dy_dt * dy_dt)
This yields the following array:
array([ 0.3 , 0.62699681, 1.00623059, 1.36014705, 1.74588803,
2.03254766, 2.32284847, 2.57512136, 2.58311827, 2.48508048,
2.41518633, 2.27513736, 2.50611752, 2.52982213, 2.27623593,
2.31651678, 2.20127804, 2.37065392, 2.8487936 , 3.04384625])
which, again, makes some sense as you look at the gaps between the dots on the scatterplot of a: the object picks up speed, slowing down a bit as it takes the corner, and then speeds back up even more.
Now, in order to find the unit tangent vector, we need to make a small transformation to ds_dt so that its size is the same as that of velocity (this effectively allows us to divide the vector-valued function velocity by the (representation of) the scalar function ds_dt):
tangent = np.array([1/ds_dt] * 2).transpose() * velocity
This yields the following numpy array:
array([[ 1. , 0. ],
[ 0.99681528, -0.07974522],
[ 0.89442719, -0.4472136 ],
[ 0.5881717 , -0.80873608],
[ 0.48685826, -0.87348099],
[ 0.52889289, -0.84868859],
[ 0.55965769, -0.82872388],
[ 0.5922051 , -0.80578727],
[ 0.67747575, -0.73554511],
[ 0.80480291, -0.59354215],
[ 0.90055164, -0.43474907],
[ 0.97796293, -0.2087786 ],
[ 0.99755897, 0.06982913],
[ 0.9486833 , 0.31622777],
[ 0.77979614, 0.62603352],
[ 0.48564293, 0.87415728],
[ 0.03407112, 0.99941941],
[-0.46400699, 0.88583154],
[-0.67572463, 0.73715414],
[-0.73919634, 0.67349 ]])
Note two things: 1. At each value of t, tangent is pointing in the same direction as velocity, and 2. At each value of t, tangent is a unit vector. Indeed:
In [12]:
In [12]: np.sqrt(tangent[:,0] * tangent[:,0] + tangent[:,1] * tangent[:,1])
Out[12]:
array([ 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1.])
Now, since we take the derivative of the tangent vector and divide by its length to get the unit normal vector, we do the same trick (isolating the components of tangent for convenience):
tangent_x = tangent[:, 0]
tangent_y = tangent[:, 1]
deriv_tangent_x = np.gradient(tangent_x)
deriv_tangent_y = np.gradient(tangent_y)
dT_dt = np.array([ [deriv_tangent_x[i], deriv_tangent_y[i]] for i in range(deriv_tangent_x.size)])
length_dT_dt = np.sqrt(deriv_tangent_x * deriv_tangent_x + deriv_tangent_y * deriv_tangent_y)
normal = np.array([1/length_dT_dt] * 2).transpose() * dT_dt
This gives us the following vector for normal:
array([[-0.03990439, -0.9992035 ],
[-0.22975292, -0.97324899],
[-0.48897562, -0.87229745],
[-0.69107645, -0.72278167],
[-0.8292422 , -0.55888941],
[ 0.85188045, 0.52373629],
[ 0.8278434 , 0.56095927],
[ 0.78434982, 0.62031876],
[ 0.70769355, 0.70651953],
[ 0.59568265, 0.80321988],
[ 0.41039706, 0.91190693],
[ 0.18879684, 0.98201617],
[-0.05568352, 0.99844847],
[-0.36457012, 0.93117594],
[-0.63863584, 0.76950911],
[-0.89417603, 0.44771557],
[-0.99992445, 0.0122923 ],
[-0.93801622, -0.34659137],
[-0.79170904, -0.61089835],
[-0.70603568, -0.70817626]])
Note that the normal vector represents the direction in which the curve is turning. The vector above then makes sense when viewed in conjunction with the scatterplot for a. In particular, we go from turning down to turning up after the fifth point, and we start turning to the left (with respect to the x axis) after the 12th point.
Finally, to get the tangential and normal components of acceleration, we need the second derivatives of s, x, and y with respect to t, and then we can get the curvature and the rest of our components (keeping in mind that they are all scalar functions of t):
d2s_dt2 = np.gradient(ds_dt)
d2x_dt2 = np.gradient(dx_dt)
d2y_dt2 = np.gradient(dy_dt)
curvature = np.abs(d2x_dt2 * dy_dt - dx_dt * d2y_dt2) / (dx_dt * dx_dt + dy_dt * dy_dt)**1.5
t_component = np.array([d2s_dt2] * 2).transpose()
n_component = np.array([curvature * ds_dt * ds_dt] * 2).transpose()
acceleration = t_component * tangent + n_component * normal

getting a list of coordinates from a 2D matrix

Let's say I have a 10 x 20 matrix of values (so 200 data points)
values = np.random.rand(10,20)
with a known regular spacing between coordinates so that the x and y coordinates are defined by
coord_x = np.arange(0,5,0.5) --> gives [0.0,0.5,1.0,1.5...4.5]
coord_y = np.arange(0,5,0.25) --> gives [0.0,0.25,0.50,0.75...4.5]
I'd like to get an array representing each coordinates points so that
the shape of the array is (200,2), 200 being the total number of points and the extra dimension simply representing x and y such as
coord[0][0]=0.0, coord[0][1]=0.0
coord[1][0]=0.0, coord[1][1]=0.25
coord[2][0]=0.0, coord[2][1]=0.50
...
coord[19][0]=0.0, coord[19][1]=5.0
coord[20][0]=0.5, coord[20][1]=0.0
coord[21][0]=0.5, coord[21][1]=0.25
coord[22][0]=0.5, coord[22][1]=0.50
...
coord[199][0]=4.5, coord[199][1]=4.5
That would a fairly easy thing to do with a double for loop, but I wonder if there is more elegant solution using built-in numpy (or else) functions.
?

I think meshgrid may be what you're looking for.
Here's an example, with smaller number of datapoints:
>>> from numpy import fliplr, dstack, meshgrid, linspace
>>> x, y, nx, ny = 4.5, 4.5, 3, 10
>>> Xs = linspace(0, x, nx)
>>> Ys = linspace(0, y, ny)
>>> fliplr(dstack(meshgrid(Xs, Ys)).reshape(nx * ny, 2))
array([[ 0. , 0. ],
[ 0. , 2.25],
[ 0. , 4.5 ],
[ 0.5 , 0. ],
[ 0.5 , 2.25],
[ 0.5 , 4.5 ],
[ 1. , 0. ],
[ 1. , 2.25],
[ 1. , 4.5 ],
[ 1.5 , 0. ],
[ 1.5 , 2.25],
[ 1.5 , 4.5 ],
[ 2. , 0. ],
[ 2. , 2.25],
[ 2. , 4.5 ],
[ 2.5 , 0. ],
[ 2.5 , 2.25],
[ 2.5 , 4.5 ],
[ 3. , 0. ],
[ 3. , 2.25],
[ 3. , 4.5 ],
[ 3.5 , 0. ],
[ 3.5 , 2.25],
[ 3.5 , 4.5 ],
[ 4. , 0. ],
[ 4. , 2.25],
[ 4. , 4.5 ],
[ 4.5 , 0. ],
[ 4.5 , 2.25],
[ 4.5 , 4.5 ]])

I think you meant coord_y = np.arange(0,5,0.25) in your question. You can do
from numpy import meshgrid,column_stack
x,y=meshgrid(coord_x,coord_y)
coord = column_stack((x.T.flatten(),y.T.flatten()))