We Keep Coding

np.polyval to np.polynomial.polynomial.polyval

I don't understand how I can migrate np.polyval to np.polynomial.polynomial.polyval. Because of the reversal of coefficient order, I account for this with [::-1] and there still is an issue plotting.
1- Creating a function to plot later
def f(x):
return np.sin(x) + 0.5 * x
x = np.linspace(-2 * np.pi, 2 * np.pi, 50)
Using old method, it works:
old_polyfit =np.polyfit(x, f(x), 1)
old_polyval = np.polyval(old_polyfit , x)
plt.plot(x, f(x), 'b', label='f(x)')
plt.plot(x, old_polyval, 'r.', label='regression')
My problem: I use the new method and there are two issues,
1-
new_polyfit =np.polynomial.polynomial.polyfit(x, f(x), 1)
new_polyval = np.polynomial.polynomial.polyval(new_polyfit[::-1], x)
I see that New_polfit does not math the inverse of old_polyfit, which it should from coef order change..
2- Plotting with the new_polyval = np.polynomial.polynomial.polyval(new_polyfit[::-1], x)
error --> 1) plt.plot(x, new_polyval, 'r.', label='regression')
2) x and y must have same first dimension, but have shapes (50,) and (2,)
plt.plot(x, f(x), 'b', label='f(x)')
plt.plot(x, new_polyval, 'r.', label='regression')
I don't understand why the old_polyval would size (50,) while newpolyval is (2,).

The new poly packages not only reverses the order of the coefficients, it also reverses the order of the arguments. Once you started to get errors, did you go back and read the docs?
your function:
In [305]: def f(x):
...: return np.sin(x) + 0.5 * x
...:
...: x = np.linspace(-2 * np.pi, 2 * np.pi, 50)
the old fit:
In [308]: old_polyfit =np.polyfit(x, f(x), 1)
...: old_polyval = np.polyval(old_polyfit , x)
In [309]: old_polyfit
Out[309]: array([ 4.28841952e-01, -7.45793918e-17])
In [310]: old_polyval
Out[310]:
array([-2.69449345, -2.58451412, -2.4745348 , -2.36455548, -2.25457615,
-2.14459683, -2.0346175 , -1.92463818, -1.81465885, -1.70467953,
-1.5947002 , ..., 2.14459683,
2.25457615, 2.36455548, 2.4745348 , 2.58451412, 2.69449345])
Your new try:
In [311]: new_polyfit =np.polynomial.polynomial.polyfit(x, f(x), 1)
...: new_polyval = np.polynomial.polynomial.polyval(new_polyfit[::-1], x)
In [312]: new_polyfit
Out[312]: array([-1.88411095e-16, 4.28841952e-01])
In [313]: new_polyval
Out[313]: array([-10.66365141, -6.28318531])
The switch in the coefficients order is clear; why the switch? That's for the developers to explain.
Reading the docs I see the new polyval expects the (x,c) order:
In [317]:
...: new_polyval = np.polynomial.polynomial.polyval(x, new_polyfit)
In [318]: new_polyval
Out[318]:
array([-2.69449345, -2.58451412, -2.4745348 , -2.36455548, -2.25457615,
-2.14459683, -2.0346175 , -1.92463818, -1.81465885, -1.70467953,
-1.5947002 , ... 2.14459683,
2.25457615, 2.36455548, 2.4745348 , 2.58451412, 2.69449345])
these match the old.
We could also use the newfit values with the old val, by just reversing the coeff.
In [320]: old_polyval = np.polyval(new_polyfit[::-1] , x)
In [321]: old_polyval
Out[321]:
array([-2.69449345, -2.58451412, -2.4745348 , -2.36455548, -2.25457615,
-2.14459683, -2.0346175 , -1.92463818, -1.81465885, -1.70467953,
-1.5947002 , ... 2.14459683,
2.25457615, 2.36455548, 2.4745348 , 2.58451412, 2.69449345])

Constrain specific values in Scipy curve fitting

I have what may be quite a basic question, but a quick googling was not able to solve it.
So I have some experimental data that I need to fit with an equation like
a * exp^{-x/t}
in the case of needing more components the expression is
a * exp^{-x/t1} + b * exp^{-x/t2} ... + n * exp^{-x/tn}
for n elements
Right now I have the following code
x = np.array([0.0001, 0.0004, 0.0006, 0.0008, 0.001, 0.0015, 0.002, 0.004, 0.006, 0.008, 0.01, 0.05, 0.1, 0.2, 0.5, 0.6, 0.8, 1, 1.5, 2, 4, 6, 8])
y1= np.array([5176350.00, 5144208.69, 4998297.04, 4787100.79, 4555731.93, 4030741.17, 3637802.79, 2949911.45, 2816472.26, 2831962.09, 2833262.53, 2815205.34, 2610685.14, 3581566.94, 1820610.74, 2100882.80, 1762737.50, 1558251.40, 997259.21, 977892.00, 518709.91, 309594.88, 186184.52])
y2 = np.array([441983.26, 423371.31, 399370.82, 390603.58, 378351.08, 356511.93, 349582.29, 346425.39, 351191.31, 329363.40, 325154.86, 352906.21, 333150.81, 301613.81, 94043.05, 100885.77, 86193.40, 75548.26, 27958.11, 20262.68, 27945.10])
def fitcurve (x, a, b, t1, t2):
return a * np.exp(- x / t1) + b * np.exp(- x / t2)
popt, pcov = curve_fit(fitcurve, x, y)
print('a = ', popt[0], 'b = ', popt[1], 't1 = ', popt[2], 't2 = ', popt[3])
plt.plot(x,y, 'bo')
plt.plot(x,fitcurve(x, *popt))
Something important is that a+b+...n = is equal to 1. Basically the percentage of each component. Ideally, I want to plot 1, 2, 3 and 4 components and see which ones provide a better fitting

I am afraid that your data cannot be fitted with a simple sum of exponential functions. Did you draw the points on a graph in order to see what is the shape of the curve ?
This looks more like a function of logistic kind (but not exactly logistic) than a sum of exponentials.
I could provide some advises to fit a sum of exponential (even with condition about the sum of coefficients). But this would be of no use with your data. Of course if you have other data convenient to fit a sum of exponentials, I would be pleased to show how to proceed.

I am not going into the model-fitting procedure but what you can do is argparse variable number of paramters and then try to fit for various numbers of exponentials. You can make use of the broadcasting feature of numpy to achieve this.
EDIT: you have to take care of the number of elements in argparse. Only even numbers works now. I leave it up to you to edit that part in (trivial).
Target
We want to fit $$\sum_i^N a_i \exp(-b_i x)$$ for variable $n$
Output:
Implementation:
from scipy import optimize, ndimage, interpolate
x = np.array([0.0001, 0.0004, 0.0006, 0.0008, 0.0010, 0.0015, 0.0020, 0.0040, 0.0060, 0.0080, 0.0100, 0.0500, 0.1000, 0.2000, 0.5000, 0.6000, 0.8000, 1.0000, 1.5000, 2.0000, 4.0000, 6.0000, 8.0000, 10.0000])
y = np.array([416312.6500, 387276.6400, 364153.7600, 350981.7000, 336813.8800, 314992.6100, 310430.4600, 318255.1700, 318487.1700, 291768.9700, 276617.3000, 305250.2100, 272001.3500, 260540.5600, 173677.1900, 155821.5500, 151502.9700, 83559.9000, 256097.3600, 20761.8400, 1.0000, 1.0000, 1.0000, 1.0000])
# variable args fit
def fitcurve (x, *args):
args = np.array(args)
half = len(args)//2
y = args[:half] * np.exp(-x[:, None] * args[half:])
return y.sum(-1)
# data seems to contain outlier?
# y = ndimage.median_filter(y, 5)
popt, pcov = optimize.curve_fit(fitcurve, x, y,
bounds = (0, np.inf),
p0 = np.ones(6), # set variable size
maxfev = 1e3,
)
fig, ax = plt.subplots()
ax.plot(x,y, 'bo')
# ax.set_yscale('log')
ax.set_xscale('symlog')
ax.plot(x,fitcurve(x, *popt))
fig.show()

Keep receiving Too many indices for array for interpolation

interp - Program to interpolate data using Lagrange
I am not able to complete the for-loop in the coding sequence below. I don't see anything wrong with it, since I choose np.empty(nplot) to create the 1D array for xi, and for some reason the loop won't fill those values.
def intrpf(xi,x,y):
"""Function to interpolate between data points
using Lagrange polynomial (quadratic)
Inputs
x Vector of x coordinates of data points (3 values)
y Vector of y coordinates of data points (3 values)
xi The x value where interpolation is computed
Output
yi The interpolation polynomial evaluated at xi
"""
#* Calculate yi = p(xi) using Lagrange polynomial
yi = ( (xi-x[1])*(xi-x[2])/((x[0]-x[1])*(x[0]-x[2])) * y[0]
+ (xi-x[0])*(xi-x[2])/((x[1]-x[0])*(x[1]-x[2])) * y[1]
+ (xi-x[0])*(xi-x[1])/((x[2]-x[0])*(x[2]-x[1])) * y[2] )
return yi
#* Initialize the data points to be fit by quadratic
x = np.empty(3)
y = np.empty(3)
print ('Enter data points as x,y pairs (e.g., [1, 2]')
for i in range(3):
temp = np.array(input('Enter data point: '))
x[i] = temp[0]
y[i] = temp[1]
#* Establish the range of interpolation (from x_min to x_max)
xr = np.array(input('Enter range of x values as [x_min, x_max]: '))
I'm getting stuck on this part, where it seems properly set up, but "Too many indices for array" appears on xi[i] within the for loop.
#* Find yi for the desired interpolation values xi using
# the function intrpf
nplot = 100 # Number of points for interpolation curve
xi = np.empty(nplot)
yi = np.empty(nplot)
for i in range(nplot) :
xi[i] = xr[0] + (xr[1]-xr[0])* i/float(nplot)
yi[i] = intrpf(xi[i], x, y) # Use intrpf function to interpolate

From the docs of np.array:
Parameters:
object: _array_like_
An array, any object exposing the array interface, an object whose array method returns an array, or any (nested) sequence.
This means array should receive something like a list, in order to make the casting, while input returns a string. What python is trying to do here at the end of the day is something like
np.array('[1, 2]')
While it might be tempting to do something like
np.array(eval(input()))
you should never do this because it is unsafe as it allows the user to execute any kind of code in your program. If you really need that kind of input I would suggest something like
np.array(list(map(int, input('Enter data point: ')
.replace('[','')
.replace(']','')
.split(','))))

The error occurs with your data input lines:
Enter data points as x,y pairs (e.g., [1, 2]
Enter data point: [1,2]
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
<ipython-input-6-8d648ad8c9e4> in <module>
22 for i in range(3):
23 temp = np.array(input('Enter data point: '))
---> 24 x[i] = temp[0]
25 y[i] = temp[1]
26
IndexError: too many indices for array: array is 0-dimensional, but 1 were indexed
The code doesn't even get to " I choose np.empty(nplot) to create the 1D array for xi, and for some reason the loop won't fill those values." part.
When asking for help, give full and accurate information about the error.
If I change the input lines to:
...: x = np.empty(3)
...: y = np.empty(3)
...: print ('Enter data points as x,y pairs')
...: for i in range(3):
...: temp = input('Enter data point: ').split()
...: x[i] = temp[0]
...: y[i] = temp[1]
...:
...: #* Establish the range of interpolation (from x_min to x_max)
...: xr = np.array(input('Enter range of x values as x_min, x_max: ').split(),float)
Enter data points as x,y pairs
Enter data point: 1 2
Enter data point: 3 4
Enter data point: 5 6
Enter range of x values as x_min, x_max: 0 4
In [9]: x
Out[9]: array([1., 3., 5.])
In [10]: y
Out[10]: array([2., 4., 6.])
In [11]: xr
Out[11]: array([0., 4.])
Getting array values via user input is not ideal, but this at least works. input (in Py3) does not evaluate the inputs; it just returns a string. I split it (on default space), and then assign the values to an array. x is defined as a float array, so the x[i]=temp[0] takes care of converting the string to float. Similarly the xr line makes a float array from the string inputs. This input style is not very robust; I could easily raise an error with wrong input.
===
The rest of the code runs with this input:
In [12]: nplot = 100 # Number of points for interpolation curve
...: xi = np.empty(nplot)
...: yi = np.empty(nplot)
...: for i in range(nplot) :
...: xi[i] = xr[0] + (xr[1]-xr[0])* i/float(nplot)
...: yi[i] = intrpf(xi[i], x, y) # Use intrpf function to interpolate
...:
In [13]: xi
Out[13]:
array([0. , 0.04, 0.08, 0.12, 0.16, 0.2 , 0.24, 0.28, 0.32, 0.36, 0.4 ,
0.44, 0.48, 0.52, 0.56, 0.6 , 0.64, 0.68, 0.72, 0.76, 0.8 , 0.84,
...
3.52, 3.56, 3.6 , 3.64, 3.68, 3.72, 3.76, 3.8 , 3.84, 3.88, 3.92,
3.96])
In [14]: yi
Out[14]:
array([1. , 1.04, 1.08, 1.12, 1.16, 1.2 , 1.24, 1.28, 1.32, 1.36, 1.4 ,
1.44, 1.48, 1.52, 1.56, 1.6 , 1.64, 1.68, 1.72, 1.76, 1.8 , 1.84,
....
4.52, 4.56, 4.6 , 4.64, 4.68, 4.72, 4.76, 4.8 , 4.84, 4.88, 4.92,
4.96])

Vectorizing A Function With Array Parameter

I'm currently trying to apply Chi-Squared analysis to some data.
I want to plot a colourmap of varying values depending on the two coefficients of a model
def f(x, coeff):
return coeff[0] + numpy.exp(coeff[1] * x)
def chi_squared(coeff, x, y, y_err):
return numpy.sum(((y - f(x, coeff) / y_err)**2)
us = numpy.linspace(u0, u1, n)
vs = numpy.linspace(v0, v1, n)
rs = numpy.meshgrid(us, vs)
chi = numpy.vectorize(chi_squared)
chi(rs, x, y, y_error)
I tried vectorizing the function to be able to pass a meshgrid of the varying coefficents to produce the colormap.
The values of x, y, y_err are all 1D arrays of length n.
And u, v are the various changing coefficients.
However this doesn't work, resulting in
IndexError: invalid index to scalar variable.
This is because coeff is passed as a scalar rather than a vector, however I don't know how to correct this.
Update
My aim is to take an array of coordinates
rs = [[[u0, v0], [u1, v0],..,[un, v0]],...,[[u0, vm],..,[un,vm]]
Where each coordinate is the coefficient parameters to be passed to the chi-squared method.
This should return a 2D array populated with Chi-Squared values for the appropriate coordinate
chi = [[c00, c10, ..., cn0], ..., [c0m, c1m, ..., cnm]]
I can then use this data to plot a colormap using imshow

Here's my first attempt to run your code:
In [44]: def f(x, coeff):
...: return coeff[0] + numpy.exp(coeff[1] * x)
...:
...: def chi_squared(coeff, x, y, y_err):
...: return numpy.sum((y - f(x, coeff) / y_err)**2)
(I had to remove the ( in that last line)
First guess at possible array values:
In [45]: x = np.arange(3)
In [46]: y = x
In [47]: y_err = x
In [48]: us = np.linspace(0,1,3)
In [49]: rs = np.meshgrid(us,us)
In [50]: rs
Out[50]:
[array([[ 0. , 0.5, 1. ],
[ 0. , 0.5, 1. ],
[ 0. , 0.5, 1. ]]),
array([[ 0. , 0. , 0. ],
[ 0.5, 0.5, 0.5],
[ 1. , 1. , 1. ]])]
In [51]: chi_squared(rs, x, y, y_err)
/usr/local/bin/ipython3:5: RuntimeWarning: divide by zero encountered in true_divide
import sys
Out[51]: inf
oops, y_err shouldn't have a 0. Try again:
In [52]: y_err = np.array([1,1,1])
In [53]: chi_squared(rs, x, y, y_err)
Out[53]: 53.262865105526018
It also works if I turn the rs list into an array:
In [55]: np.array(rs).shape
Out[55]: (2, 3, 3)
In [56]: chi_squared(np.array(rs), x, y, y_err)
Out[56]: 53.262865105526018
Now, what was the purpose of vectorize?
The f function returns a (n,n) array:
In [57]: f(x, rs)
Out[57]:
array([[ 1. , 1.5 , 2. ],
[ 1. , 2.14872127, 3.71828183],
[ 1. , 3.21828183, 8.3890561 ]])
Lets modify the chi_squared to give sum an axis
In [61]: def chi_squared(coeff, x, y, y_err, axis=None):
...: return numpy.sum((y - f(x, coeff) / y_err)**2, axis=axis)
In [62]: chi_squared(np.array(rs), x, y, y_err)
Out[62]: 53.262865105526018
In [63]: chi_squared(np.array(rs), x, y, y_err, axis=0)
Out[63]: array([ 3. , 6.49033483, 43.77253028])
In [64]: chi_squared(np.array(rs), x, y, y_err, axis=1)
Out[64]: array([ 1.25 , 5.272053 , 46.74081211])
I'm tempted to change the coeff to coeff0, coeff1, to give more control from the start on how this parameter is passed, but it probably doesn't make a difference.
update
Now that you've been more specific about how the coeff values relate to x, y etc, I see that this can be solved with simple broadcasting. No need to use np.vectorize.
First, define a grid that has a different size; that way we, and the code, won't think that each dimension of the coeff grid has anything to do with the x,y values.
In [134]: rs = np.meshgrid(np.linspace(0,1,4), np.linspace(0,1,5), indexing='ij')
In [135]: coeff=np.array(rs)
In [136]: coeff.shape
Out[136]: (2, 4, 5)
Now look at what f looks like when given this coeff and x.
In [137]: f(x, coeff[...,None]).shape
Out[137]: (4, 5, 3)
coeff is effectively (4,5,1), while x is (1,1,3), resulting in a (4,5,3) (by broadcasting rules)
The same thing happens inside chi_squared, with the final step of sum on the last axis (size 3):
In [138]: chi_squared(coeff[...,None], x, y, y_err, axis=-1)
Out[138]:
array([[ 2. , 1.20406718, 1.93676807, 8.40646968,
32.99441808],
[ 2.33333333, 2.15923164, 3.84810347, 11.80559574,
38.73264336],
[ 3.33333333, 3.78106277, 6.42610554, 15.87138846,
45.13753532],
[ 5. , 6.06956056, 9.67077427, 20.60384785,
52.20909393]])
In [139]: _.shape
Out[139]: (4, 5)
One value for each coeff pair of values, the (4,5) grid.

Getting coefficients of a cubic spline from scipy.interpolate.splrep

I am doing a cubic spline interpolation using scipy.interpolate.splrep as following:
import numpy as np
import scipy.interpolate
x = np.linspace(0, 10, 10)
y = np.sin(x)
tck = scipy.interpolate.splrep(x, y, task=0, s=0)
F = scipy.interpolate.PPoly.from_spline(tck)
I print t and c:
print F.x
array([ 0. , 0. , 0. , 0. ,
2.22222222, 3.33333333, 4.44444444, 5.55555556,
6.66666667, 7.77777778, 10. , 10. ,
10. , 10. ])
print F.c
array([[ -1.82100357e-02, -1.82100357e-02, -1.82100357e-02,
-1.82100357e-02, 1.72952212e-01, 1.26008293e-01,
-4.93704109e-02, -1.71230879e-01, -1.08680287e-01,
1.00658224e-01, 1.00658224e-01, 1.00658224e-01,
1.00658224e-01],
[ -3.43151441e-01, -3.43151441e-01, -3.43151441e-01,
-3.43151441e-01, -4.64551679e-01, 1.11955696e-01,
5.31983340e-01, 3.67415303e-01, -2.03354294e-01,
-5.65621916e-01, 1.05432909e-01, 1.05432909e-01,
1.05432909e-01],
[ 1.21033389e+00, 1.21033389e+00, 1.21033389e+00,
1.21033389e+00, -5.84561936e-01, -9.76335250e-01,
-2.60847433e-01, 7.38484392e-01, 9.20774403e-01,
6.63563923e-02, -9.56285846e-01, -9.56285846e-01,
-9.56285846e-01],
[ -4.94881722e-18, -4.94881722e-18, -4.94881722e-18,
-4.94881722e-18, 7.95220057e-01, -1.90567963e-01,
-9.64317117e-01, -6.65101515e-01, 3.74151231e-01,
9.97097891e-01, -5.44021111e-01, -5.44021111e-01,
-5.44021111e-01]])
So I had supplied the x array as :
array([ 0. , 1.11111111, 2.22222222, 3.33333333,
4.44444444, 5.55555556, 6.66666667, 7.77777778,
8.88888889, 10. ])
Q.1: The F.x (knots) are not the same as original x array and has duplicate values (possibly to force first derivative to zero?). Also some values in x (1.11111111, 8.88888889) are missing in F.x. Any ideas?
Q.2 The shape of F.c is (4, 13). I understand that 4 comes from the fact that it is cubic spline fit. But I do not know how do I select coefficients for each of the 9 sections that I want (from x = 0 to x=1.11111, x = 1.111111 to x = 2.222222 and so on). Any help in extraction of the coefficients for different segments would be appreciated.

If you want to have the knots in specific locations along the curves you need to use the argument task=-1 of splrep and give an array of interior knots as the t argument.
The knots in t must satisfy the following condition:
If provided, knots t must satisfy the Schoenberg-Whitney conditions, i.e., there must be a subset of data points x[j] such that t[j] < x[j] < t[j+k+1], for j=0, 1,...,n-k-2.
See the documentation here.
Then you should get F.c of the following size (4, <length of t> + 2*(k+1)-1) corresponding to the consecutive intervals along the curve (k+1 knots are added at either end of the curve by splrep).
Try the following:
import numpy as np
import scipy.interpolate
x = np.linspace(0, 10, 20)
y = np.sin(x)
t = np.linspace(0, 10, 10)
tck = scipy.interpolate.splrep(x, y, t=t[1:-1])
F = scipy.interpolate.PPoly.from_spline(tck)
print(F.x)
print(F.c)
# Accessing coeffs of nth segment: index = k + n - 1
# Eg. for second segment:
print(F.c[:,4])