I have data points of time and voltage that create the curve shown below.
The time data is
array([ 0.10810811, 0.75675676, 1.62162162, 2.59459459,
3.56756757, 4.21621622, 4.97297297, 4.97297297,
4.97297297, 4.97297297, 4.97297297, 4.97297297,
4.97297297, 4.97297297, 5.08108108, 5.18918919,
5.2972973 , 5.51351351, 5.72972973, 5.94594595,
6.27027027, 6.59459459, 7.13513514, 7.67567568,
8.32432432, 9.18918919, 10.05405405, 10.91891892,
11.78378378, 12.64864865, 13.51351351, 14.37837838,
15.35135135, 16.32432432, 17.08108108, 18.16216216,
19.02702703, 20. , 20. , 20. ,
20. , 20. , 20. , 20. ,
20.10810811, 20.21621622, 20.43243243, 20.64864865,
20.97297297, 21.40540541, 22.05405405, 22.91891892,
23.78378378, 24.86486486, 25.83783784, 26.7027027 ,
27.56756757, 28.54054054, 29.51351351, 30.48648649,
31.56756757, 32.64864865, 33.62162162, 34.59459459,
35.67567568, 36.64864865, 37.62162162, 38.59459459,
39.67567568, 40.75675676, 41.83783784, 42.81081081,
43.89189189, 44.97297297, 46.05405405, 47.02702703,
48.10810811, 49.18918919, 50.27027027, 51.35135135,
52.43243243, 53.51351351, 54.48648649, 55.56756757,
56.75675676, 57.72972973, 58.81081081, 59.89189189])
and the volts data is
array([ 4.11041056, 4.11041056, 4.11041056, 4.11041056, 4.11041056,
4.11041056, 4.11041056, 4.10454545, 4.09794721, 4.09208211,
4.08621701, 4.07961877, 4.07228739, 4.06568915, 4.05909091,
4.05175953, 4.04516129, 4.03782991, 4.03123167, 4.02463343,
4.01803519, 4.01217009, 4.00557185, 3.99970674, 3.99384164,
3.98797654, 3.98284457, 3.97771261, 3.97331378, 3.96891496,
3.96451613, 3.96085044, 3.95645161, 3.95205279, 3.9483871 ,
3.94398827, 3.94032258, 3.93665689, 3.94325513, 3.94985337,
3.95645161, 3.96378299, 3.97038123, 3.97624633, 3.98284457,
3.98944282, 3.99604106, 4.0026393 , 4.00923754, 4.01510264,
4.02096774, 4.02609971, 4.02903226, 4.03196481, 4.03416422,
4.0356305 , 4.03709677, 4.03856305, 4.03929619, 4.04002933,
4.04076246, 4.04222874, 4.04296188, 4.04296188, 4.04369501,
4.04442815, 4.04516129, 4.04516129, 4.04589443, 4.04589443,
4.04662757, 4.04662757, 4.0473607 , 4.0473607 , 4.04809384,
4.04809384, 4.04809384, 4.04882698, 4.04882698, 4.04882698,
4.04956012, 4.04956012, 4.04956012, 4.04956012, 4.05029326,
4.05029326, 4.05029326, 4.05029326])
I would like to determine the location of the points labeled A, B, C, D, and E. Point A is the first location where the slope goes from zero to undefined. Point B is the location where the line is no longer vertical. Point C is the minimum of the curve. Point D is where the curve is no longer vertical. Point E is where the slope is close to zero again. The Python code below determines the locations for points A and C.
tdiff = np.diff(time)
vdiff = np.diff(volts)
# point A
idxA = np.where(vdiff < 0)[0][0]
timeA = time[idxA]
voltA = volts[idxA]
# point C
idxC = volts.idxmin()
timeC = time[idxC]
voltC = volts[idxC]
How can I determine the other locations on the curve represented by points B, D, and E?
You are looking for the points that mark any location where the slope changes to or from zero or infinity. We do not not actually need to compute slopes anywhere: either yn - yn-1 == 0 and yn+1 - yn != 0, or vice versa, or the same for x.
We can take the diff of x. If one of two successive elements is zero, then the diff of the diff will be the diff or the negative diff at that point. So we just want to find and label all points where diff(x) == diff(diff(x)) and diff(x) != 0, properly adjusted for differences in size between the arrays of course. We also want all the points where the same is true for y.
In numpy terms, this is can be written as follows
def masks(vec):
d = np.diff(vec)
dd = np.diff(d)
# Mask of locations where graph goes to vertical or horizontal, depending on vec
to_mask = ((d[:-1] != 0) & (d[:-1] == -dd))
# Mask of locations where graph comes from vertical or horizontal, depending on vec
from_mask = ((d[1:] != 0) & (d[1:] == dd))
return to_mask, from_mask
to_vert_mask, from_vert_mask = masks(time)
to_horiz_mask, from_horiz_mask = masks(volts)
Keep in mind that the masks are computed on second order differences, so they are two elements shorter than the inputs. Elements in the masks correspond to elements in the input arrays with a one-element border on the leading and trailing edge (hence the index [1:-1] below). You can convert the mask to indices using np.nonzero or you can get the x- and y-values directly using the masks as indices:
def apply_mask(mask, x, y):
return x[1:-1][mask], y[1:-1][mask]
to_vert_t, to_vert_v = apply_mask(to_vert_mask, time, volts)
from_vert_t, from_vert_v = apply_mask(from_vert_mask, time, volts)
to_horiz_t, to_horiz_v = apply_mask(to_horiz_mask, time, volts)
from_horiz_t, from_horiz_v = apply_mask(from_horiz_mask, time, volts)
plt.plot(time, volts, 'b-')
plt.plot(to_vert_t, to_vert_v, 'r^', label='Plot goes vertical')
plt.plot(from_vert_t, from_vert_v, 'kv', label='Plot stops being vertical')
plt.plot(to_horiz_t, to_horiz_v, 'r>', label='Plot goes horizontal')
plt.plot(from_horiz_t, from_horiz_v, 'k<', label='Plot stops being horizontal')
plt.legend()
plt.show()
Here is the resulting plot:
Notice that because the classification is done separately, "Point A" is correctly identified as being both a spot where verticalness starts and horizontalness ends. The problem is that "Point E" does not appear to be resolvable as such according to these criteria. Zooming in shows that all of the proliferated points correctly identify horizontal line segments:
You could choose a "correct" version of "Point E" by discarding from_horiz completely, and only the last value from to_horiz:
to_horiz_t, to_horiz_v = apply_mask(to_horiz_mask, time, volts)
to_horiz_t, to_horiz_v = to_horiz_t[-1], to_horiz_v[-1]
plt.plot(time, volts, 'b-')
plt.plot(*apply_mask(to_vert_mask, time, volts), 'r^', label='Plot goes vertical')
plt.plot(*apply_mask(from_vert_mask, time, volts), 'kv', label='Plot stops being vertical')
plt.plot(to_horiz_t, to_horiz_v, 'r>', label='Plot goes horizontal')
plt.legend()
plt.show()
I am using this as a showcase for the star expansion of the results of apply_mask. The resulting plot is:
This is pretty much exactly the plot you were looking for. Discarding from_horiz also makes "Point A" be identified only as a drop to vertical, which is nice.
As multiple values in to_horiz show, this method is very sensitive to noise within the data. Your data is quite smooth, but this approach is unlikely to ever work with raw unfiltered measurements.
Related
I have been struggling for apparently no reason trying to fit a sin function to a small dataset that resembles a sinusoid. I've looked at many other questions and tried different libraries and can't seem to find any glaring mistake in my code. Also in many answers people are fitting a function onto data where y = f(x); but I'm retrieving both of my lists independently from stellar spectra.
These are the lists for reference:
time = np.array([2454294.5084288 , 2454298.37039515, 2454298.6022165 ,
2454299.34790096, 2454299.60750029, 2454300.35176022,
2454300.61361622, 2454301.36130122, 2454301.57111912,
2454301.57540159, 2454301.57978822, 2454301.5842906 ,
2454301.58873511, 2454302.38635047, 2454302.59553152,
2454303.41548415, 2454303.56765036, 2454303.61479213,
2454304.38528718, 2454305.54043812, 2454306.36761011,
2454306.58025083, 2454306.60772791, 2454307.36686591,
2454307.49460991, 2454307.58258509, 2454308.3698358 ,
2454308.59468672, 2454309.40004997, 2454309.51208756,
2454310.43078368, 2454310.6091061 , 2454311.40121502,
2454311.5702085 , 2454312.39758274, 2454312.54580053,
2454313.52984047, 2454313.61734047, 2454314.37609003,
2454315.56721061, 2454316.39218499, 2454316.5672538 ,
2454317.49410168, 2454317.6280825 , 2454318.32944441,
2454318.56913047])
velocities = np.array([-2.08468951, -2.26117398, -2.44703149, -2.10149768, -2.09835213,
-2.20540079, -2.4221183 , -2.1394637 , -2.0841663 , -2.2458154 ,
-2.06177386, -2.47993416, -2.13462117, -2.26602791, -2.47359571,
-2.19834895, -2.17976339, -2.37745005, -2.48849617, -2.15875901,
-2.27674409, -2.39054554, -2.34029665, -2.09267843, -2.20338104,
-2.49483926, -2.08860222, -2.26816951, -2.08516229, -2.34925637,
-2.09381667, -2.21849357, -2.43438148, -2.28439031, -2.43506056,
-2.16953358, -2.24405359, -2.10093237, -2.33155007, -2.37739938,
-2.42468714, -2.19635302, -2.368558 , -2.45959665, -2.13392004,
-2.25268181]
These are radial velocities of a star observed at different times. When plotted they look like this:
Plotted Data
This is then the code I'm using to fit a test sine on the data:
x = time
y = velocities
def sin_fit(x, A, w):
return A * np.sin(w * x)
popt, pcov = curve_fit(sin_fit,x,y) #try to calculate exoplanet parameters with these data
xfit = np.arange(min(x),max(x),0.1)
fit = sin_fit(xfit,*popt)
mod = plt.figure()
plt.xlabel("Time (G. Days)")
plt.ylabel("Radial Velocity")
plt.scatter(x,[i for i in y],color="b",label="Data")
plt.plot(x,[i for i in y],color="b",alpha=0.2)
plt.plot(xfit,fit,color="r",label="Model Fit")
plt.legend()
mod.savefig("Data with sin fit.png")
plt.show()
I thought this was right, and it seems right by looking at other answers, but then this is what I get:
Data with model sine
What am I doing wrong?
Thank you in advanceee
I guess it's due the sin_fit function is not able to fit the data at all. The sin function per default whirls around y=0 while your data whirls somewhere around y=-2.3.
I tried your code and extended the sin_fit with an offset, yielding way better results (althought looking not too perfect):
def sin_fit(x, A, w, offset):
return A * np.sin(w * x) + offset
with this the function has at least a chance to fit
So I have one problem for motion. The problem is finding for which angle, a object would fall off a sphere. Suppose the sphere is frictionless and all that, so it just slides off. Logically this would say that when the normal force acting on the object is 0, the object leaves the surface. So when the normal force is 0, the object has travelled an angle from the start. At what point does it leave? Well analytically it was simple to find the answer, it is arccos(2/3*cos(startingposition)). Right now I'm trying to find this numerically. It requires a system of equations that solve themselves basically, because all i have is just inital conditions, while the rest is just differential equations, so I'm implementing a form for Eulers method in a for-loop. So for what angle value is N equal to zero is what im trying to find, but all my numbers are just weird. My code below:
g=9.8
m=0.2 #mass of object
R=0.5 #radius of the circle that its sliding down from
r=0.07 #radius of the object sliding down
grad = 180/np.pi #turn radiens to degrees
theta=np.zeros(91) #thetas
N = np.zeros(91) #normal force
w = np.zeros(91) #omega
a = np.zeros(91) #tangential acceleration
v = np.zeros(91) #speed
def eulersMethod():
dt=0.01 #small timestep
thetai=[np.radians(0)] #startangle theta (where on the circle do i let it slide from start angle 0 degrees)
a[0]=g*np.sin(thetai[0]) #initial condition for acceleration
v[0]=0 #initial condition for velocity (starts at rest)
w[0]=(v[0]/(R+r)) #initial condition for angular velocity
N[0]=m*g*np.cos(thetai[0]) #initial condition for normal force
theta[0]=np.radians(0) #initial angle which is 0 degrees
for i in range(0,90):
v[i+1]=v[i]+a[i]*dt #v=v+at (Eulers method)
w[i+1]=v[i+1]/(R+r) #omega=v/r
theta[i+1]=theta[i]+w[i]*dt #0(t+dt)=0(t)+(d0/dt)*dt (Eulers method) finding theta iteratively
N[i+1]=(m*g*np.cos(theta[i+1]))-(m/(R+r)*(v[i+1])**2) #N=mgcos(theta)-mv^2/r
a[i+1]=g*np.sin(np.radians(theta[i+1])) #a=gsin(theta)
print(v[i])
theta_falloff=[] #where will the object fall off?
for i in N:
if i==0: #for the i-th element that is equal to zero (N=0), then append that theta value to a list
theta_falloff.append(theta[i])
return theta_falloff
For the velocity I'm getting weird numbers like:
[0. 0. 0.00268639 0.00537278 0.00805925 0.01074589
0.01343279 0.01612002 0.01880768 0.02149584 0.0241846 0.02687403
0.02956422 0.03225525 0.03494721 0.03764019 0.04033427 0.04302953
0.04572606 0.04842394 0.05112325 0.05382409 0.05652654 0.05923068
0.0619366 0.06464437 0.0673541 0.07006586 0.07277973 0.07549581
0.07821417 0.08093491 0.08365811 0.08638385 0.08911222 0.0918433
0.09457719 0.09731397 0.10005371 0.10279652 0.10554247 0.10829166
0.11104416 0.11380006 0.11655946 0.11932243 0.12208907 0.12485945
0.12763368 0.13041183 0.13319399 0.13598026 0.13877071 0.14156543
0.14436452 0.14716805 0.14997613 0.15278883 0.15560624 0.15842846
0.16125557 0.16408765 0.16692481 0.16976713 0.17261469 0.17546759
0.17832592 0.18118976 0.1840592 0.18693434 0.18981527 0.19270208
0.19559485 0.19849367 0.20139865 0.20430986 0.20722741 0.21015138
0.21308186 0.21601895 0.21896273 0.22191331 0.22487077 0.2278352
0.2308067 0.23378537 0.23677128 0.23976455 0.24276526 0.24577351
0. ]
[0. 0. 0.00268639 0.00537278 0.00805925 0.01074589
0.01343279 0.01612002 0.01880768 0.02149584 0.0241846 0.02687403
0.02956422 0.03225525 0.03494721 0.03764019 0.04033427 0.04302953
0.04572606 0.04842394 0.05112325 0.05382409 0.05652654 0.05923068
0.0619366 0.06464437 0.0673541 0.07006586 0.07277973 0.07549581
0.07821417 0.08093491 0.08365811 0.08638385 0.08911222 0.0918433
0.09457719 0.09731397 0.10005371 0.10279652 0.10554247 0.10829166
0.11104416 0.11380006 0.11655946 0.11932243 0.12208907 0.12485945
0.12763368 0.13041183 0.13319399 0.13598026 0.13877071 0.14156543
0.14436452 0.14716805 0.14997613 0.15278883 0.15560624 0.15842846
0.16125557 0.16408765 0.16692481 0.16976713 0.17261469 0.17546759
0.17832592 0.18118976 0.1840592 0.18693434 0.18981527 0.19270208
0.19559485 0.19849367 0.20139865 0.20430986 0.20722741 0.21015138
0.21308186 0.21601895 0.21896273 0.22191331 0.22487077 0.2278352
0.2308067 0.23378537 0.23677128 0.23976455 0.24276526 0.24577351
0.24878938]
I'm trying to create a Voronoi diagram given a set of scatterplot points. However, several "extra unintended lines" appear to get calculated in the process. Some of these "extra" lines appear to be the infinite edges getting incorrectly calculated. But others are appearing randomly in the middle of the plot as well. How can I only create an extra edge when it's needed/required to connect a polygon to the edge of the plot (e.g. plot boundaries)?
My graph outer boundaries are:
boundaries = np.array([[0, -2], [0, 69], [105, 69], [105, -2], [0, -2]])
Here's the section dealing with the voronoi diagram creation:
def voronoi_polygons(voronoi, diameter):
centroid = voronoi.points.mean(axis=0)
ridge_direction = defaultdict(list)
for (p, q), rv in zip(voronoi.ridge_points, voronoi.ridge_vertices):
u, v = sorted(rv)
if u == -1:
t = voronoi.points[q] - voronoi.points[p] # tangent
n = np.array([-t[1], t[0]]) / np.linalg.norm(t) # normal
midpoint = voronoi.points[[p, q]].mean(axis=0)
direction = np.sign(np.dot(midpoint - centroid, n)) * n
ridge_direction[p, v].append(direction)
ridge_direction[q, v].append(direction)
for i, r in enumerate(voronoi.point_region):
region = voronoi.regions[r]
if -1 not in region:
# Finite region.
yield Polygon(voronoi.vertices[region])
continue
# Infinite region.
inf = region.index(-1) # Index of vertex at infinity.
j = region[(inf - 1) % len(region)] # Index of previous vertex.
k = region[(inf + 1) % len(region)] # Index of next vertex.
if j == k:
# Region has one Voronoi vertex with two ridges.
dir_j, dir_k = ridge_direction[i, j]
else:
# Region has two Voronoi vertices, each with one ridge.
dir_j, = ridge_direction[i, j]
dir_k, = ridge_direction[i, k]
# Length of ridges needed for the extra edge to lie at least
# 'diameter' away from all Voronoi vertices.
length = 2 * diameter / np.linalg.norm(dir_j + dir_k)
# Polygon consists of finite part plus an extra edge.
finite_part = voronoi.vertices[region[inf + 1:] + region[:inf]]
extra_edge = [voronoi.vertices[j] + dir_j * length,
voronoi.vertices[k] + dir_k * length]
combined_finite_edge = np.concatenate((finite_part, extra_edge))
poly = Polygon(combined_finite_edge)
yield poly
Here are the points being used:
['52.629' '24.28099822998047']
['68.425' '46.077999114990234']
['60.409' '36.7140007019043']
['72.442' '28.762001037597656']
['52.993' '43.51799964904785']
['59.924' '16.972000122070312']
['61.101' '55.74899959564209']
['68.9' '13.248001098632812']
['61.323' '29.0260009765625']
['45.283' '36.97500038146973']
['52.425' '19.132999420166016']
['37.739' '28.042999267578125']
['48.972' '2.3539962768554688']
['33.865' '30.240001678466797']
['52.34' '64.94799995422363']
['52.394' '45.391000747680664']
['52.458' '34.79800033569336']
['31.353' '43.14500045776367']
['38.194' '39.24399948120117']
['98.745' '32.15999984741211']
['6.197' '32.606998443603516']
Most likely this is due to the errors associated with floating point arithmetic while computing the voronoi traingulation from your data (esp. the second column).
Assuming that, there is no single solution for such kinds of problems. I urge you to go through this page* of the Qhull manual and try iterating through those parameters in qhull_options before generating the voronoi object that you are inputting in the function. An example would be qhull_options='Qbb Qc Qz QJ'.
Other than that I doubt there is anything that could be modified in the function to avoid such a problem.
*This will take some time though. Just be patient.
Figured out what was wrong: after each polygon I needed to add a null x and y value or else it would attempt to 'stitch' one polygon to another, drawing an additional unintended line in order to do so. So the data should really look more like this:
GameTime,Half,ObjectType,JerseyNumber,X,Y,PlayerIDEvent,PlayerIDTracking,MatchIDEvent,Position,teamId,i_order,v_vor_x,v_vor_y
0.0,1,1,22,None,None,578478,794888,2257663,3,35179.0,0,22.79645297,6.20866756
0.0,1,1,22,None,None,578478,794888,2257663,3,35179.0,1,17.63464264,3.41230187
0.0,1,1,22,None,None,578478,794888,2257663,3,35179.0,2,20.27639318,34.29191902
0.0,1,1,22,None,None,578478,794888,2257663,3,35179.0,3,32.15600546,36.60432421
0.0,1,1,22,None,None,578478,794888,2257663,3,35179.0,4,38.34639812,33.62806739
0.0,1,1,22,None,None,578478,794888,2257663,3,35179.0,5,22.79645297,6.20866756
0.0,1,1,22,None,None,578478,794888,2257663,3,35179.0,5,nan,nan
0.0,1,1,22,33.865,30.240001678466797,578478,794888,2257663,3,35179.0,,,
0.0,1,0,92,None,None,369351,561593,2257663,1,32446.0,0,46.91696938,29.44801535
0.0,1,0,92,None,None,369351,561593,2257663,1,32446.0,1,55.37574848,29.5855499
0.0,1,0,92,None,None,369351,561593,2257663,1,32446.0,2,58.85876401,23.20381766
0.0,1,0,92,None,None,369351,561593,2257663,1,32446.0,3,57.17455086,21.5228301
0.0,1,0,92,None,None,369351,561593,2257663,1,32446.0,4,44.14237744,22.03925667
0.0,1,0,92,None,None,369351,561593,2257663,1,32446.0,5,45.85962774,28.83613332
0.0,1,0,92,None,None,369351,561593,2257663,1,32446.0,5,nan,nan
0.0,1,0,92,52.629,24.28099822998047,369351,561593,2257663,1,32446.0,,,
0.0,1,0,27,None,None,704169,704169,2257663,2,32446.0,0,65.56965667,33.4292025
0.0,1,0,27,None,None,704169,704169,2257663,2,32446.0,1,57.23303682,32.43809027
0.0,1,0,27,None,None,704169,704169,2257663,2,32446.0,2,55.65704152,38.97814049
0.0,1,0,27,None,None,704169,704169,2257663,2,32446.0,3,60.75304149,44.53251169
0.0,1,0,27,None,None,704169,704169,2257663,2,32446.0,4,65.14170295,40.77562188
0.0,1,0,27,None,None,704169,704169,2257663,2,32446.0,5,65.56965667,33.4292025
0.0,1,0,27,None,None,704169,704169,2257663,2,32446.0,5,nan,nan
I have a 1d signal that is given as following
x = np.array([34.69936612, 34.70083619, 37.38802174, 39.67141565, 49.05662135,
63.87593075, 67.70815746, 72.06562117, 79.31063707, 85.13125285,
83.34185985, 72.74589905, 57.34778159, 58.63283664, 64.92526896,
65.89153823, 66.07273386, 59.68722257, 59.6801125 , 59.41456929,
58.19250575, 59.92192524, 58.42078866, 55.45131784, 55.09849914,
54.95270916, 49.60804717, 43.05198366, 36.10104167, 26.88848229,
25.38550393, 28.71305461, 30.03802157, 31.3520023 , 32.59509437,
32.67600055, 32.68801666, 32.61500098, 32.65303828, 32.72752018,
32.84099458, 31.46154937, 32.70809456, 27.67842221, 25.65302641,
30.08500957, 31.41003082, 32.91935844, 32.92452782, 35.56587345,
30.09272452, 35.60898454, 49.12005244, 85.79396522, 71.81950127,
63.91915245, 69.14879246, 70.43600086, 71.71703424, 71.74830965,
70.51400086, 70.50201501, 70.50202228, 67.91157904, 66.62396413,
67.90736076, 66.5410636 , 67.96748026, 67.94177515, 65.30929726,
65.29901863, 66.60282538, 66.60666811, 66.55100589, 65.33825435,
66.55222626, 65.29656691, 66.56003543, 65.30964145, 64.07556963,
63.99339626, 62.86668124, 60.43549001, 61.68116229, 61.61140279,
62.65181523, 62.70844205, 62.77783077, 64.03882299, 65.39701193,
65.40123835, 65.41845477, 65.42941287, 65.38851043, 65.36201151,
73.33102635, 73.84443755, 70.94806114, 68.18793023, 69.20003749,
66.61045573, 65.38106858, 64.05484531, 63.88684974, 62.64420529,
62.69196131, 62.74418993, 62.72175294, 64.01210311, 64.1590297 ,
63.0284751 , 64.27265024, 64.24984689, 62.90213438, 62.68704697,
62.65233151, 63.09040365, 63.10330994, 62.72787413, 63.95427977,
63.89707325, 61.38203635, 57.48587612, 60.05764178, 62.70293674,
61.38484666, 60.07995823, 61.34569129, 62.66307354, 61.38549663,
61.34835356, 61.3888718 , 61.48381576, 62.74226583, 62.83945058,
41.78731982, 38.06452548, 40.57553545, 43.10410628, 43.17965777,
44.41576623, 45.67422069, 44.44681128, 44.52855717, 45.69118569,
45.7559632 , 49.9019806 , 50.90898633, 52.2603325 , 36.83061979,
48.36714502, 53.60110239, 53.58750501, 51.03745637, 52.15201941,
50.94600264, 48.50758345, 51.03154956, 51.32249134, 51.49705585,
53.46467209, 51.708078 , 48.1404585 , 46.32157084, 53.20416229,
60.52216104, 67.14976382, 66.6844348 , 63.99400013, 63.89292312,
63.94972283, 65.33551293, 66.54723199, 65.29004129, 67.87224117,
69.3810433 , 69.28915977, 65.32064534, 64.07644938, 64.59988251,
65.55365125, 64.3440046 , 64.4526091 , 63.38977665, 64.61810574,
63.52989024, 63.55126155, 64.4263114 , 64.43874937, 64.78594756,
66.03974204, 67.34958445, 70.07248445, 67.40968741, 66.56554542,
67.59965865, 67.85658168, 67.62022101, 67.87089721, 61.22552792,
54.07823817, 47.96332512, 53.22944931, 54.77573267, 59.55033053,
62.24247612, 62.24529416, 63.9429676 , 63.13145527, 63.29764489,
63.2723988 , 62.96359318, 63.3025575 , 63.47790181, 63.29642863,
63.50702402, 63.71413853, 63.71470992, 62.25079434, 63.46787461,
63.73497156, 63.77631175, 63.69024723, 63.55254533, 63.97794376,
64.05815662, 63.57687055, 66.80917018, 66.82863683, 66.27964922,
65.04852024, 65.29135318, 65.57783886, 65.52090561, 65.29656225,
65.32543578, 66.52825603, 67.1314033 , 50.03567181, 53.53803024,
53.56862071, 55.10515723, 55.14010716, 63.30760687, 62.7114906 ,
62.95237442, 62.75869066, 64.19585539, 62.70371169, 62.65204241,
62.69394807, 62.94844878, 58.36397143, 59.68285611, 60.89452752,
60.97356663, 60.72068974, 59.62036073, 60.52789377, 59.27245489,
58.82200393, 60.10430588, 60.90874661, 61.51060014, 61.74838059,
63.28503148, 61.12237542, 60.87046418, 61.23634728, 60.99214796,
60.18921274, 60.07774571, 61.20623845, 61.65825197, 60.11025633,
60.52832382, 61.18188688, 61.31380433, 58.80528487, 57.84584698,
58.73805752, 54.85645345, 58.79988199, 60.07737149, 56.20096342,
60.3929374 , 36.77761826, 49.22568866, 55.10930206, 65.24736292,
57.08641006, 54.08806036, 53.89556268, 53.5613321 , 53.51515767,
52.30442805, 52.24562597, 53.50311397, 53.49561038, 53.53878528,
49.66610081, 52.35633014, 55.17584864, 53.945292 , 53.79353353,
54.8626422 , 54.87102507, 56.14098197, 57.38968051, 55.1146169 ,
54.92290752, 54.87858275, 54.86639486, 56.34316676, 56.16200014,
69.90905494, 68.20948497, 68.51263756, 65.64670149, 65.53992678,
67.07185321, 67.0542345 , 66.79344433, 66.75400526, 66.76640135,
66.76742739, 65.53052634, 67.01174217, 67.98329773, 69.18915578,
66.69019707, 69.61506484, 67.94096632, 67.91401491, 66.84415179,
67.88935229, 67.89356226, 69.1984958 , 52.24244378, 52.37211419,
50.95591909, 51.07641848, 50.91919022, 52.13500015, 52.26717303,
60.03109894, 65.23341727, 72.11099746, 75.02859632, 81.93540828,
81.20708335, 80.86208705, 81.04817673, 71.74669785, 73.05200134,
74.34519255, 75.72326992, 78.55812705, 76.95800509, 77.08696036,
79.61302675, 79.68123466, 78.31207499, 77.08036041, 77.18815309,
77.11523959, 75.74423094, 75.73143868, 74.48319908, 73.17138546,
66.80804931, 53.88772644, 53.87714358, 53.6088119 , 53.65411471,
54.86536613, 53.49300076, 53.52447811, 53.52000034, 56.83649529,
57.43503283, 82.38440921, 83.83190983, 83.9128805 , 83.94305425,
83.06892508, 82.91998964, 82.29555463, 82.30635577, 82.23464297,
82.20709065, 80.98821075, 83.93336979, 81.32873456, 82.46698736,
82.70592498, 83.93335761, 83.80821766, 83.84313602, 82.59867874,
82.62361191, 83.94865746, 83.83137976, 83.46075784, 82.14902814,
82.18902896, 83.83722778, 83.60064452, 83.63187976, 85.04806926,
84.87213079, 84.92473511, 84.90790341, 83.55500539, 83.59501005,
84.81195299, 84.86952928, 84.85600059, 84.81955391, 82.33120262,
78.56908599, 73.14783901, 64.99883861, 66.78701764, 64.5916058 ,
64.77055337, 64.56918786, 65.02605783, 65.01019955, 64.78145201,
64.77581828, 64.55221044, 64.34285288, 62.8764752 , 64.57949744,
63.17957281, 61.89857751, 63.48365778, 55.62801456, 43.17986365])
I want to find all the slopes for this signal. I have tried first order difference and second order difference (np.diff and taking the difference of the difference). But the point on the slope will have every small difference, in contrast to the point in the beginning or the end, where the difference is bigger.
Here is what I have tried
def detect_slope(signal, window_size = 3, threshold = 5):
list_ = []
for i in range(window_size, len(signal)-window_size):
diff_ = np.mean(signal[i-window_size:i]) - np.mean(signal[i:window_size+i])
list_.append(diff_)
first_order_diff = np.array(list_)
d = np.where(np.abs(first_order_diff) < threshold, 0 , first_order_diff)
idx = np.where(np.abs(d) != 0)
# might need some offset because we are doing some smoothing, but just use raw idx for now
# second order different
diff_list = np.array(list_).copy()
dd = np.diff(diff_list)
print(dd.shape)
dd_idx = np.where(np.abs(dd) > 0.5)
return diff_list, dd, idx, dd_idx
I have played around the 1st-2nd order difference but nothing seems to work. I'm trying to find all the peaks and troughs and exclude all of them or neighbors with close enough values too.
Attached is my desired output. Sorry for the crappy pic.
Not clear what you want here, but if your issue is that the diff is picking up local changes and you want to focus your attention on global changes, smooth the signal first.
import numpy as np
import matplotlib.pyplot as plt
from scipy.signal import savgol_filter
plt.plot(x)
x = savgol_filter(x, 21, 3)
plt.plot(x)
diff_list, dd, idx, dd_idx = detect_slope(x)
plt.plot(diff_list)
plt.show()
This gives -
Blue is your original signal, orange is your smoothed signal and green is your new diff. You can set it to pick up changes at various levels by playing around with the two parameters of savgol_filter. The more aggressively you smooth your function, the more global changes(and less local changes) the derivative picks up.
You can try find_peaks function from scipy. As you guess it gets the peaks defining a parameter to be more or less sensitive. The best one in your case is prominence ("How much you have to go down before finding another peak"). I use with your function in positive for max peaks and negative for min peaks.
import numpy as np
from matplotlib import pyplot as plt
from scipy.ndimage import median_filter
from scipy.signal import find_peaks
#Find peaks (maximum)
yhat = x
max_peaks,_ = find_peaks(yhat, prominence=10 )
min_peaks,_ = find_peaks(-yhat, prominence=10 )
#Plot data and max peak
fig, ax2 = plt.subplots(figsize=(20, 10))
ax2.plot(max_peaks, yhat[max_peaks], "xr",markersize=20)
ax2.plot(min_peaks, yhat[min_peaks], "xr",markersize=20)
ax2.plot(yhat,'-')
ax2.plot(yhat,'o',markersize=4)
I have a project where I'm sampling analog data and attempting to analyze with matplotlib. Currently, my analog data source is a potentiometer hooked up to a microcontroller, but that's not really relevant to the issue. Here's my code
arrayFront = RunningMean(array(dataFront), 15)
arrayRear = RunningMean(array(dataRear), 15)
x = linspace(0, len(arrayFront), len(arrayFront)) # Generate x axis
y = linspace(0, len(arrayRear), len(arrayRear)) # Generate x axis
min_vals_front = scipy.signal.argrelmin(arrayFront, order=2)[0] # Min
min_vals_rear = scipy.signal.argrelmin(arrayRear, order=2)[0] # Min
max_vals_front = scipy.signal.argrelmax(arrayFront, order=2)[0] # Max
max_vals_rear = scipy.signal.argrelmax(arrayRear, order=2)[0] # Max
maxvalfront = max(arrayFront[max_vals_front])
maxvalrear = max(arrayRear[max_vals_rear])
minvalfront = min(arrayFront[min_vals_front])
minvalrear = min(arrayRear[min_vals_rear])
plot(x, arrayFront, label="Front Pressures")
plot(y, arrayRear, label="Rear Pressures")
plot(x[min_vals_front], arrayFront[min_vals_front], "x")
plot(x[max_vals_front], arrayFront[max_vals_front], "o")
plot(y[min_vals_rear], arrayRear[min_vals_rear], "x")
plot(y[max_vals_rear], arrayRear[max_vals_rear], "o")
xlim(-25, len(arrayFront) + 25)
ylim(-1000, 7000)
legend(loc='upper left')
show()
dataFront and dataRear are python lists that hold the sampled data from 2 potentiometers. RunningMean is a function that calls:
convolve(x, ones((N,)) / N, mode='valid')
The problem is that the argrelmax (and min) functions don't always find all the maxes and mins. Sometimes it doesn't find ANY max or mins, and that causes me problems in this block of code
maxvalfront = max(arrayFront[max_vals_front])
maxvalrear = max(arrayRear[max_vals_rear])
minvalfront = min(arrayFront[min_vals_front])
minvalrear = min(arrayRear[min_vals_rear])
because the [min_vals_(blank)] variables are empty. Does anyone have any idea what is happening here, and what I can do to fix the problem? Thanks in advance.
Here's one of graphs of data where not all the maxes and mins are found:
signal.argrelmin is a thin wrapper around signal.argrelextrema with comparator=np.less. np.less(a, b) returns the truth value of a < b element-wise. Notice that np.less requires a to be strictly less than b for it to be True.
Your data has the same minimum value at a lot of neighboring locations. At the local minima, the inequality between local minimum and its neighbors does not satisfy a strictly less than relationship; instead it only satisfies a strictly less than or equal to relationship.
Therefore, to find these extrema use signal.argrelmin with comparator=np.less_equal. For example, using a snippet from your data:
import numpy as np
from scipy import signal
arrayRear = np.array([-624.59309896, -624.59309896, -624.59309896,
-625., -625., -625.,])
print(signal.argrelmin(arrayRear, order=2)[0])
# []
print(signal.argrelextrema(arrayRear, np.less_equal)[0])
# [0 1 3 4 5]
print(signal.argrelextrema(arrayRear, np.less_equal, order=2)[0])
# [0 3 4 5]