I'm trying to implement some algorithms in Python from different papers and some require a peak/through detection where local minima and maxima are alternating. That should always be the case of course in an analog signal.
However when working with discrete Signals this is not always the case as you can see with this simple example:
from scipy.signal import argrelmax,argrelmin,argrelextrema
import numpy as np
import matplotlib.pyplot as plt
fig = plt.figure("local extrema")
ax = fig.subplots()
x = range(0,10)
y = np.array([0,1,0,0,1,2,1,0,0,0])
local_minima, = list(argrelmin(np.array(y)))
local_maxima, = list(argrelmax(np.array(y)))
ax.plot(local_maxima, y[local_maxima],"x",color="orange")
ax.plot(local_minima, y[local_minima],"x",color="orange")
ax.plot(x,y)
plt.show()
I know that I can adjust the argrelmax and argrelmin functions but no matter what I do, its always somehow flawed.
Here only two maxima are found. So there are no alternating extrema.
Is there a way to automatically detect extremas and assure they are alternating, or do I have to implement some kind of workaround? (Ideas for that are welcome too)
I found a solution I previously disregarded unfortunately. The Problem with argrelmax,argrelmin and argrelextrema was that they had difficulties detecting flat peaks/throughs because sometimes they found two maxima using np.greater_equal as comparator (beginning and end of flat peak), which lead to non alternating maxima/minima.
After Flows answer I revisited find_peaks from scipy.signal, which I previously disregarded, because in the first few sentences it said: "finds all local maxima by simple comparison of neighboring values" so I thought it had the same flaws as the others.
But in reality it detects flat peaks with only one index at the middle (rounded down) of the peak, no matter how wide apparently.
Thus this solved my Problem for this simple case:
from scipy.signal import argrelmax,argrelmin, argrelextrema, find_peaks
fig = plt.figure("local extrema")
ax = fig.subplots()
x = range(0,10)
y = np.array([0,1,0,0,1,2,1,0,0,0])
peaks,_ = find_peaks(y)
throughs,_ = find_peaks(np.negative(y))
local_maxima = list(peaks)
local_minima = list(throughs)
ax.plot(x,y)
ax.plot(local_maxima, y[local_maxima],"x",color="orange")
ax.plot(local_minima, y[local_minima],"x",color="orange")
plt.show()
As you can see now the minima is detected.
I will have to see if it works on my real world data though, but I think it should. Thx!
Related
Maybe this will be duplicate question but I couldn't find any solution for this.
Normally what I coded should show me a curved line in python. But with this code I cant see it. Is there a problem with my code or pycharm ? This code only shows me an empty graphic with the correct axes.
And I did adding "ro" in plt.plot(at[i], st, "ro"). This showed me the spots on the graph but what I want to see the complete line.
at = [0,1,2,3,4,5,6]
for i in range(len(at)):
st = at[i]**2
plt.plot(at[i], st)
plt.show()
This is how you would normally do this:
import numpy as np
import matplotlib.pyplot as plt
at = np.array([0,1,2,3,4,5,6])
at2 = at ** 2
plt.plot(at,at2)
plt.show()
you can use something like plt.plot(at,at2, c='red', marker='o') to see the spots.
for detailed explanation please read the documentation.
Maybe rather calculate the to be plotted values entirely before plotting.
at = [0,1,2,3,4,5,6]
y = [xi**2 for xi in at]
plt.plot(at, y)
Or do it alternatively with a function
from math import pow
at = [0,1,2,3,4,5,6]
def parabolic(x):
return [pow(xi,2) for xi in x]
plt.plot(at, parabolic(at))
both return the following plot:
the other answers give fixes for your question, but don't tell you why your code is not working.
the reason for not "seeing anything" is that plt.plot(at[i], st) was trying to draw lines between the points you give it. but because you were only ever giving it single values it didn't have anything to draw lines between. as a result, nothing appeared on the plot
when you changed to call plt.plot(at[i], st, 'ro') you're telling it to draw single circles at points and these don't go between points so would appear
the other answers showed you how to pass multiple values to plot and hence matplotlib could draw lines between these values.
one of your comments says "its not parabolic still" and this is because matplotlib isn't a symbolic plotting library. you just give it numeric values and it draws these onto the output device. sympy is a library for doing symbolic computation and supports plotting, e.g:
from sympy import symbols, plot
x = symbols('x')
plot(x**2, (x, 0, 6))
does the right thing for me. the current release (1.4) doesn't handle discontinuities, but this will be fixed in the next release
I receive this error in scipy interp1d function. Normally, this error would be generated if the x was not monotonically increasing.
import scipy.interpolate as spi
def refine(coarsex,coarsey,step):
finex = np.arange(min(coarsex),max(coarsex)+step,step)
intfunc = spi.interp1d(coarsex, coarsey,axis=0)
finey = intfunc(finex)
return finex, finey
for num, tfile in enumerate(files):
tfile = tfile.dropna(how='any')
x = np.array(tfile['col1'])
y = np.array(tfile['col2'])
finex, finey = refine(x,y,0.01)
The code is correct, because it successfully worked on 6 data files and threw the error for the 7th. So there must be something wrong with the data. But as far as I can tell, the data increase all the way down.
I am sorry for not providing an example, because I am not able to reproduce the error on an example.
There are two things that could help me:
Some brainstorming - if the data are indeed monotonically
increasing, what else could produce this error? Another hint,
regarding the decimals, could be in this question, but I think
my solution (the min and max of x) is robust enough to avoid it. Or
isn't it?
Is it possible (how?) to return the value of x_new and
it's index when throwing the ValueError: A value in x_new is above the interpolation range. so that I could actually see where in the
file is the problem?
UPDATE
So the problem is that, for some reason, max(finex) is larger than max(coarsex) (one is .x39 and the other is .x4). I hoped rounding the original values to 2 significant digits would solve the problem, but it didn't, it displays fewer digits but still counts with the undisplayed. What can I do about it?
If you are running Scipy v. 0.17.0 or newer, then you can pass fill_value='extrapolate' to spi.interp1d, and it will extrapolate to accomadate these values of your's that lie outside the interpolation range. So define your interpolation function like so:
intfunc = spi.interp1d(coarsex, coarsey,axis=0, fill_value="extrapolate")
Be forewarned, however!
Depending on what your data looks like and the type on interpolation you are performing, the extrapolated values can be erroneous. This is especially true if you have noisy or non-monotonic data. In your case you might be ok because your x_new value is only slighly beyond your interpolation range.
Here's simple demonstration of how this feature can work nicely but also give erroneous results.
import scipy.interpolate as spi
import numpy as np
x = np.linspace(0,1,100)
y = x + np.random.randint(-1,1,100)/100
x_new = np.linspace(0,1.1,100)
intfunc = spi.interp1d(x,y,fill_value="extrapolate")
y_interp = intfunc(x_new)
import matplotlib.pyplot as plt
plt.plot(x_new,y_interp,'r', label='interp/extrap')
plt.plot(x,y, 'b--', label='data')
plt.legend()
plt.show()
So the interpolated portion (in red) worked well, but the extrapolated portion clearly fails to follow the otherwise linear trend in this data because of the noise. So have some understanding of your data and proceed with caution.
A quick test of your finex calc shows that it can (always?) gets into the extrapolation region.
In [124]: coarsex=np.random.rand(100)
In [125]: max(coarsex)
Out[125]: 0.97393109991816473
In [126]: step=.01;finex=np.arange(min(coarsex), max(coarsex)+step, step);(max(
...: finex),max(coarsex))
Out[126]: (0.98273730602114795, 0.97393109991816473)
In [127]: step=.001;finex=np.arange(min(coarsex), max(coarsex)+step, step);(max
...: (finex),max(coarsex))
Out[127]: (0.97473730602114794, 0.97393109991816473)
Again it is a quick test, and may be missing some critical step or value.
When looking closely at contour plots made with matplotlib, I noticed that smaller contours have inaccurate endpoints which do not close perfectly in PDF figures. Consider the minimal example:
plt.gca().set_aspect('equal')
x,y = np.meshgrid(np.linspace(-1,1,100), np.linspace(-1,1,100))
r = x*x + y*y
plt.contour(np.log(r))
plt.savefig("test.pdf")
The central part of the resulting test.pdf file, shown below, clearly shows the problem. Is there a way to solve this or is it a bug/intrinsic inaccuracy of matploplib?
Disclaimer: this is more an explanation + hack than a real answer.
I believe that there is a fundamental problem the way matplotlib makes contour plots. Essentially, all contours are collections of lines (LineCollection), while they should be collection of possibly closed lines (PolyCollection). There might be good reasons why things are done this way, but in the simple example I made this choice clearly produces artifacts. A not-very-nice solution is to convert a posteriori all LineCollection's into PolyCollection's. This is what is done in the following code
from matplotlib.collections import PolyCollection
eps = 1e-5
plt.gca().set_aspect('equal')
x,y = np.meshgrid(np.linspace(-1,1,100), np.linspace(-1,1,100))
r = x*x + y*y
plt.contour(np.log(r/1.2))
ca = plt.gca()
N = len(ca.collections)
for n in range(N):
c = ca.collections.pop(0)
for s in c.get_segments():
closed = (abs(s[0,0] - s[-1,0]) < eps) and (abs(s[0,1] - s[-1,1]) < eps)
p = PolyCollection([s], edgecolors=c.get_edgecolors(),
linewidths=c.get_linewidths(), linestyles=c.get_linestyles(),
facecolors=c.get_facecolors(), closed=closed)
ca.add_collection(p)
plt.savefig("test.pdf")
A zoom of the result obtained shows that everything is OK now:
Some care is taken to check if a contour is closed: in the present code, this is done with an approximate equality check for the first and last point: I am wandering if there is a better way to do this (perhaps matplotlib returns some data to check closed contours?). In any case, again, this is hack: I would be happy to hear if anyone has a better solution (or has a way to fix this within matplotlib).
So I just did the same thing, but got closed contours (see images). Did you check for any updates on the package?
import matplotlib.pyplot as plt
import numpy as np
plt.gca().set_aspect('equal')
x,y = np.meshgrid(np.linspace(-1,1,100), np.linspace(-1,1,100))
r = x*x + y*y
plt.contour(np.log(r))
plt.show()
Zoomed Out
Zoomed In
I have to just read Using adaptive step sizes with scipy.integrate.ode and the accepted solution to that problem, and have even reproduced the results by copy-and-paste in my Python interpreter.
My problem is that when I try and adapt the solution code to my own code I only get flat lines.
My code is as follows:
from scipy.integrate import ode
from matplotlib.pyplot import plot, show
initials = [1,1,1,1,1]
integration_range = (0, 100)
f = lambda t,y: [1.0*y[0]*y[1], -1.0*y[0]*y[1], 1.0*y[2]*y[3] - 1.0*y[2], -1.0*y[2]*y[3], 1.0*y[2], ]
y_solutions = []
t_solutions = []
def solution_getter(t,y):
t_solutions.append(t)
y_solutions.append(y)
backend = "dopri5"
ode_solver = ode(f).set_integrator(backend)
ode_solver.set_solout(solution_getter)
ode_solver.set_initial_value(y=initials, t=0)
ode_solver.integrate(integration_range[1])
plot(t_solutions,y_solutions)
show()
And the plot it yields:
In the line
y_solutions.append(y)
you think that you are appending the current vector. What actally happens is that you are appending the object reference to y. Since apparently the integrator reuses the vector y during the integration loop, you are always appending the same object reference. Thus at the end, each position of the list is filled by the same reference pointing to the vector of the last state of y.
Long story short: replace with
y_solutions.append(y.copy())
and everything is fine.
I am working in image processing right now in python using numpy and scipy all the time. I have one piece of code that can enlarge an image, but not sure how this works.
So please some expert in scipy/numpy in python can explain to me line by line. I am always eager to learn.
import numpy as N
import os.path
import scipy.signal
import scipy.interpolate
import matplotlib.pyplot as plt
import matplotlib.cm as cm
def enlarge(img, rowscale, colscale, method='linear'):
x, y = N.meshgrid(N.arange(img.shape[1]), N.arange(img.shape[0]))
pts = N.column_stack((x.ravel(), y.ravel()))
xx, yy = N.mgrid[0.:float(img.shape[1]):1/float(colscale),
0.:float(img.shape[0]):1/float(rowscale)]
large = scipy.interpolate.griddata(pts, img.flatten(), (xx, yy), method).T
large[-1,:] = large[-2,:]
large[:,-1] = large[:,-2]
return large
Thanks a lot.
First, a grid of empty points is created with point per pixel.
x, y = N.meshgrid(N.arange(img.shape[1]), N.arange(img.shape[0]))
The actual image pixels are placed into the variable pts which will be needed later.
pts = N.column_stack((x.ravel(), y.ravel()))
After that, it creates a mesh grid with one point per pixel for the enlarged image; if the original image was 200x400, the colscale set to 4 and rowscale set to 2, the mesh grid would have (200*4)x(400*2) or 800x800 points.
xx, yy = N.mgrid[0.:float(img.shape[1]):1/float(colscale),
0.:float(img.shape[0]):1/float(rowscale)]
Using scipy, the points in pts variable are interpolated into the larger grid. Interpolation is the manner in which missing points are filled or estimated usually when going from a smaller set of points to a larger set of points.
large = scipy.interpolate.griddata(pts, img.flatten(), (xx, yy), method).T
I am not 100% certain what the last two lines do without going back and looking at what the griddata method returns. It appears to be throwing out some additional data that isn't needed for the image or performing a translation.
large[-1,:] = large[-2,:]
large[:,-1] = large[:,-2]
return large