I'm trying to use Python to fit a curve to a set of points. Essentially the points look like this.
The blue curve indicates the data entered (in this case 4 points) with the green being a curve fit using np.polyfit and polyfit1d. What I essentially want is a curve fit that looks very similar to the blue line but with a smoother change in gradient at points 1 and 2 (meaning I don't require the line to pass through these points).
What would be the best way to do this? The line looks like an arctangent, is there any way to specify an arctangent fit?
I realise this is a bit of a rubbish question but I want to get away without specifying more points. Any help would be greatly appreciated.
It seems that you might be after interpolation between points rather than fitting a polynomial References: Spline Interpolation with Python and Fitting polynomials to data
However, in either case here is a code snippet that should get you started:
import numpy as np
import scipy as sp
from scipy.interpolate import interp1d
x = np.array([0,5,10,15,20,30,40,50])
y = np.array([0,0,0,12,40,40,40,40])
coeffs = np.polyfit(x, y, deg=4)#you can change degree as you see fit
poly = np.poly1d(coeffs)
yp = np.polyval(poly, x)
interpLength = 10
new_x = np.linspace(x.min(), x.max(), new_length)
new_y = sp.interpolate.interp1d(x, y, kind='cubic')(new_x)
plt.plot(x, y, '.', x, yp, '-', new_x,new_y, '--')
plt.show()
Related
One of the assumptions behind the Natural Cubic Spline is that at the endpoints of the interval of interpolation, the second derivative of the spline polynomials is set to be equal to 0. I tried to show that using the Natural Cubic Spline via from scipy.interpolate import CubicSplines in the example (code below).
from scipy.interpolate import CubicSpline
from numpy import linspace
import matplotlib.pyplot as plt
runge_f = lambda x: 1 / (1 + 25*x**2)
x = linspace(-2, 2, 11)
y = runge_f(x)
cs = CubicSpline(x, y, bc_type = "natural")
t = linspace(-5, 5, 1000)
plt.plot(x, y, "p", color="red")
plt.plot(t, runge_f(t), color="black")
plt.plot(t, cs(t), color="lightblue")
plt.show()
In the presented example, the extrapolated points' curvature is not equal to zero - shouldn't the extrapolation outside the interval be linear in the Natural Cubic Spline?
The curvature (second derivative) of the spline at the end points is indeed 0, as you can check by running this
print(cs(x[0],2), cs(x[-1], 2))
which calculates second derivatives at both ends of your x interpolation interval. However that does not mean the spline is flat beyond the limits -- it continues on as a cubic polynomial. If you want to extrapolate linearly outside the range, you have to do it yourself. It is easy enough to extrapolate flat: replace your cs=.. line with
from scipy.interpolate import interp1d
cs = interp1d(x,y,fill_value = (y[0],y[-1]), bounds_error = False)
to get something like this:
but a bit more work to extrapolate linearly. I am not sure there is a Python function for that (or rather I am sure there is one I just don't know what it is)
I have what seems to be a relatively simplistic geometric optimization question, but one in which I have absolutely no experience.
I have a bunch of data that I am fitting a spline through to find its equation, such as the below,
And this is the equation I fit to the dots,
with the code,
import matplotlib.pyplot as plt
from scipy.interpolate import UnivariateSpline
plt.figure(figsize=(6.5, 4))
plt.axis([0.017,.045,.0045,.014])
plt.scatter(x,y,color='orange')
spl = UnivariateSpline(x, y)
plt.plot(xs, spl(xs), 'b', lw=3)
plt.show()
And what I want to do is find the equation of a line, y = mx + b, with a certain y-intercept a that hits the blue curve at some tangent point. So, for example, if my line has to pass through y=.006, what is the slope b of its line that makes it hit the blue line's tangent?
I though about doing a regular optimization over the derivatives of the curve at a large amount of points, but I am unsure how to set this up in a rigorous way.
I have a set of data that when plotted most points congregate to the left of the x axis:
plt.plot(x, y, marker='o')
plt.title('Original')
plt.show()
ORIGINAL GRAPH
I want to use scipy to interpolate the data and later try to fit a quadratic line to the data. I am avoiding to simply fit a quadratic curve without interpolation since this will make the obtained curve biased towards the mass of data at one extreme end of the x axis. I tried this by using
f = interp1d(x, y, kind='quadratic')
# Array with points in between min(x) and max(x) for interpolation
x_interp = np.linspace(min(x), max(x), num=np.size(x))
# Plot graph with interpolation
plt.plot(x_interp, f(x_interp), marker='o')
plt.title('Interpolated')
plt.show()
and got INTERPOLATED GRAPH.
However, what I intend to get is something like this:
EXPECTED GRAPH
What am I doing wrong?
My values for x can be found here and values for y here.
Thank you!
Solution 1
I'm pretty sure this does what you want. It fits a second degree (quadratic) polynomial to your data, then plots that function on an evenly spaced array of x values ranging from the minimum to the maximum of your original x data.
new_x = np.linspace(min(x), max(x), num=np.size(x))
coefs = np.polyfit(x,y,2)
new_line = np.polyval(coefs, new_x)
Plotting it returns:
plt.scatter(x,y)
plt.scatter(new_x,new_line,c='g', marker='^', s=5)
plt.xlim(min(x)-0.00001,max(x)+0.00001)
plt.xticks(rotation=90)
plt.tight_layout()
plt.show()
if that wasn't what you meant...
However, from your question, it seems like you might be trying to force all your original y-values onto evenly spaced x-values (if that's not your intention, let me know, and I'll just delete this part).
This is also possible, there are lots of ways to do this, but I've done it here in pandas:
import pandas as pd
xy_df=pd.DataFrame({'x_orig': x, 'y_orig': y})
sorted_x_y=xy_df.sort_values('x_orig')
sorted_x_y['new_x'] = np.linspace(min(x), max(x), np.size(x))
plt.figure(figsize=[5,5])
plt.scatter(sorted_x_y['new_x'], sorted_x_y['y_orig'])
plt.xlim(min(x)-0.00001,max(x)+0.00001)
plt.xticks(rotation=90)
plt.tight_layout()
Which looks pretty different from your original data... which is why I think it might not be exactly what you're looking for.
I'm new to Python and having some trouble with matplotlib. I currently have data that is contained in two numpy arrays, call them x and y, that I am plotting on a scatter plot with coordinates for each point (x, y) (i.e I have points x[0], y[0] and x1, y1 and so on on my plot). I have been using the following code segment to color the points in my scatter plot based on the spatial density of nearby points (found this on another stackoverflow post):
http://prntscr.com/abqowk
import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import gaussian_kde
x = np.random.normal(size=1000)
y = x*3 + np.random.normal(size=1000)
xy = np.vstack([x,y])
z = gaussian_kde(xy)(xy)
idx = z.argsort()
fig,ax = plt.subplots()
ax.scatter(x,y,c=z,s=50,edgecolor='')
plt.show()
Output:
I've been using it without being sure exactly how it works (namely the point density calculation - if someone could explain how exactly that works, would also be much appreciated).
However, now I'd like to color code by the ratio of the spatial density of points in x,y to that of the spatial density of points in another set of numpy arrays, call them x2, y2. That is, I would like to make a plot such that I can identify how the density of points in x,y compares to the points in x2,y2 on the same scatter plot. Could someone please explain how I could go about doing this?
Thanks in advance for your help!
I've been trying to do the same thing based on that same earlier post, and I think I just figured it out! The trick is to use matplotlib.colors.Normalize() to define a scale and then weight it according to some data set (xnorm,ynorm):
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.colors as mplc
import matplotlib.cm as cm
from scipy.stats import gaussian_kde
def kdeplot(x,y,xnorm,ynorm):
xy = np.vstack([x,y])
z = gaussian_kde(xy)(xy)
wt = 1.0*len(x)/(len(xnorm)*1.0)
norm = mplc.Normalize(vmin=0, vmax=8/wt)
cmap = cm.gnuplot
idx = z.argsort()
x, y, z = x[idx], y[idx], z[idx]
args = (x,y)
kwargs = {'c':z,'s':10,'edgecolor':'','cmap':cmap,'norm':norm}
return args, kwargs
# (x1,y1) is some data set whose density map coloring you
# want to scale to (xnorm,ynorm)
args,kwargs = kdeplot(x1,y1,xnorm,ynorm)
plt.scatter(*args,**kwargs)
I used trial and error to optimize my normalization for my particular data and choice of colormap. Here's what my data looks like scaled to itself; here's my data scaled to some comparison data (which is on the bottom of that image).
I'm not sure this method is entirely general, but it works in my case: I know that my data and the comparison data are in similar regions of parameter space, and they both have gaussian scatter, so I can use a naive linear scaling determined by the number of data points and it results in something that gives the right idea visually.
Given some data of shape 20x45, where each row is a separate data set, say 20 different sine curves with 45 data points each, how would I go about getting the same data, but with shape 20x100?
In other words, I have some data A of shape 20x45, and some data B of length 20x100, and I would like to have A be of shape 20x100 so I can compare them better.
This is for Python and Numpy/Scipy.
I assume it can be done with splines, so I am looking for a simple example, maybe just 2x10 to 2x20 or something, where each row is just a line, to demonstrate the solution.
Thanks!
Ubuntu beat me to it while I was typing this example, but his example just uses linear interpolation, which can be more easily done with numpy.interpolate... (The difference is only a keyword argument in scipy.interpolate.interp1d, however).
I figured I'd include my example, as it shows using scipy.interpolate.interp1d with a cubic spline...
import numpy as np
import scipy as sp
import scipy.interpolate
import matplotlib.pyplot as plt
# Generate some random data
y = (np.random.random(10) - 0.5).cumsum()
x = np.arange(y.size)
# Interpolate the data using a cubic spline to "new_length" samples
new_length = 50
new_x = np.linspace(x.min(), x.max(), new_length)
new_y = sp.interpolate.interp1d(x, y, kind='cubic')(new_x)
# Plot the results
plt.figure()
plt.subplot(2,1,1)
plt.plot(x, y, 'bo-')
plt.title('Using 1D Cubic Spline Interpolation')
plt.subplot(2,1,2)
plt.plot(new_x, new_y, 'ro-')
plt.show()
One way would be to use scipy.interpolate.interp1d:
import scipy as sp
import scipy.interpolate
import numpy as np
x=np.linspace(0,2*np.pi,45)
y=np.zeros((2,45))
y[0,:]=sp.sin(x)
y[1,:]=sp.sin(2*x)
f=sp.interpolate.interp1d(x,y)
y2=f(np.linspace(0,2*np.pi,100))
If your data is fairly dense, it may not be necessary to use higher order interpolation.
If your application is not sensitive to precision or you just want a quick overview, you could just fill the unknown data points with averages from neighbouring known data points (in other words, do naive linear interpolation).