I'm having trouble getting a fit to converge, as it's either not converging or giving a NaN error, depending on my start parameters. I'm using quad to integrate and fitting using lmfit. Any help is appreciated.
I'm fitting my data to a Langevin function, weighted by a log-normal distribution. Stackoverflow won't let me post an image of the function because of my reputation score, but it's in the code below.
I'm plugging in H (field) and fitting for Ms, Dm, and sigma, while mu_0, Msb, kb, and T are all constants.
Here's what I'm working with, using some example data:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import scipy
from numpy import vectorize, sqrt, log, inf, exp, pi, tanh
from scipy.constants import k, mu_0
from lmfit import Parameters
from scipy.integrate import quad
x_data = [-7.0, -6.5, -6.0, -5.5, -5.0, -4.5, -4.0, -3.5, -3.0, -2.5, -2.0, -1.5, -1.0,
-0.95, -0.9, -0.85, -0.8, -0.75, -0.7, -0.65, -0.6, -0.55, -0.5, -0.45, -0.4,
-0.35, -0.3, -0.25, -0.2, -0.1,-0.05, 3e-6, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3,
0.35, 0.4, 0.45, 0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9, 0.95, 1.0,
1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0, 5.5, 6.0, 6.5, 7.0]
y_data = [-61.6, -61.6, -61.6, -61.5, -61.5, -61.4, -61.3, -61.2, -61.1, -61.0, -60.8,
-60.4, -59.8, -59.8, -59.7, -59.5, -59.4, -59.3, -59.1, -58.9, -58.7, -58.4,
-58.1, -57.7, -57.2, -56.5, -55.6, -54.3, -52.2, -48.7, -41.8, -27.3, 2.6,
30.1, 43.1, 49.3, 52.6, 54.5, 55.8, 56.6, 57.3, 57.8, 58.2, 58.5, 58.7, 59.0,
59.1, 59.3, 59.5, 59.6, 59.7, 59.8, 59.9, 60.5, 60.8, 61.0, 61.2, 61.3, 61.4,
61.4, 61.5, 61.6, 61.6, 61.7, 61.7]
params = Parameters()
params.add('Dm' , value = 8e-9 , vary = True, min = 0, max = 1) # magnetic diameter (m)
params.add('s' , value = 0.4 , vary = True, min = 0.0, max = 10.0) # sigma, unitless
params.add('Ms' , value = 61.0 , vary = True) #, min = 30.0 , max = 100.0) # saturation magnetization (emu/g)
params.add('Msb', value = 446000 * 1e-16, vary = False) # Bulk magnetite saturation magnetization (A/m)
params.add('T' , value = 300 , vary = False) # Temperature (K)
def Mag(x_data, params):
v = params.valuesdict() # put parameters into a dictionary
def numerator(D, x_data, params):
# langevin
a_numerator = pi * v['Msb'] * x_data * D**3
a_denominator = 6*k*v['T']
a = a_numerator / a_denominator
langevin = (1/tanh(a)) - (1/a)
# PDF
exp_num = (log(D/v['Dm']))**2
exp_denom = 2 * v['s']
exponential = exp(-exp_num/exp_denom)
pdf = exponential/(sqrt(2*pi) * v['s'] * D)
return D**3 * langevin * pdf
def denominator(D, params):
# PDF
exp_num = (log(D/v['Dm']))**2
exp_denom = 2 * v['s']
exponential = exp(-exp_num/exp_denom)
pdf = exponential/(sqrt(2*pi) * v['s'] * D)
return D**3 * pdf
# return integrals
return v['Ms'] * quad(numerator, 0, inf, args=(x_data, params))[0] / quad(denominator, 0, inf,args=(params))[0]
# vectorize
vcurve = np.vectorize(Mag, excluded=set([1]))
plt.plot(x_data, vcurve(x_data, params))
plt.scatter(x_data, y_data)
This plots the data and the fit equation with start parameters. I have an issue somewhere with units in the Langevin and have to multiply the numerator by 1e-16 to get the curve looking correct...
from lmfit import minimize, Minimizer, Parameters, Parameter, report_fit
def fit_function(params, x_data, y_data):
model1 = vcurve(x_data, params)
resid1 = y_data - model1
return resid1
minner = Minimizer(fit_function, params, fcn_args=(x_data, y_data))
result = minner.minimize()
report_fit(result)
result.params.pretty_print()
Depending on the sigma (s) value I choose, which should be able to range from 0 to infinity, the integral won't converge, giving the following error:
/var/folders/pz/tbd_dths0_512bm6l43vpg680000gp/T/ipykernel_68003/1413445460.py:39: IntegrationWarning: The algorithm does not converge. Roundoff error is detected
in the extrapolation table. It is assumed that the requested tolerance
cannot be achieved, and that the returned result (if full_output = 1) is
the best which can be obtained.
return v['Ms'] * quad(numerator, 0, inf, args=(x_data, params))[0] / quad(denominator, 0, inf,args=(params))[0]
I'm stuck on why the fit isn't converging. Is this an issue because I'm using very small numbers or is this an issue with quad/lmfit? Thank you!
Having parameters that are closer to order 1 (say, between 1e-7 and 1e7) is a good idea. If you expect a parameter is in the 1.e-9 (or 1.e-16!) range, you could definitely scale it (in the fitting function) so that the value passed back and forth by the fitting algorithm is closer to order 1. But, I sort of doubt that is the main problem you are having.
It looks to me like your Mag function is not very sensitive to the values of your variable parameters Dm and s. I am not 100% sure why that is. Have you verified that calculations using your "Mag" or "vcurve" do what you expect them to do?
Related
the fitted curve doesn't fit the datapoints (xH_data, nH_data) as expected. Does someone know what might be the issue here?
from scipy.optimize import curve_fit
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
xH_data = np.array([1., 1.03, 1.06, 1.1, 1.2, 1.3, 1.5, 1.7, 2., 2.6, 3., 4., 5., 6.])
nH_data = np.array([403., 316., 235., 160., 70.8, 37.6, 14.8, 7.11, 2.81, 0.665, 0.313, 0.090, 0.044, 0.029])*1.0e6
plt.plot(xH_data, nH_data)
plt.yscale("log")
plt.xscale("log")
def eTemp(x, A, a, B):
n = B*(A+x)**a
return n
parameters, covariance = curve_fit(eTemp, xH_data, nH_data, maxfev=200000)
fit_A = parameters[0]
fit_a = parameters[1]
fit_B = parameters[2]
print(fit_A)
print(fit_a)
print(fit_B)
r = np.logspace(0, 0.7, 1000)
ne = fit_B *(fit_A + r)**(fit_a)
plt.plot(r, ne)
plt.yscale("log")
plt.xscale("log")
Thanks in advance for the help.
Ok, here is a different approach. As usual, the main problem are initial guesses for the non linear fit (For details, check this). Here, those are evaluated by using an integro relation of the fit function y( x ) = a ( x - c )^p, namely int( y ) = ( x - c ) / ( p + 1 ) y + d = x y / ( p + 1 ) - c y / ( p + 1 ) + d This means we can get c and p via a linear fit of int y against x y and y. Once those are known, a is a simple linear fit. It will turn out that these guesses are already quite good. Nevertheless, those will go as initial values into a non-linear fit providing the final result. In detail this goes like this:
import matplotlib.pyplot as plt
import numpy as np
from scipy.integrate import cumtrapz
from scipy.optimize import curve_fit
xHdata = np.array(
[
1.0, 1.03, 1.06, 1.1, 1.2, 1.3, 1.5,
1.7, 2.0, 2.6, 3.0, 4.0, 5.0, 6.0
]
)
nHdata = np.array(
[
403.0, 316.0, 235.0, 160.0, 70.8, 37.6,
14.8, 7.11, 2.81, 0.665, 0.313, 0.090, 0.044, 0.029
]
) * 1.0e6
def fit_func( x, a, c, p ):
out = a * ( x - c )**p
return out
### fitting the non-linear parameters as part of an integro-equation
### this is the standard matrix formulation of a linear fit
Sy = cumtrapz( nHdata, x=xHdata, initial=0 ) ## int( y )
VMXT = np.array( [ xHdata * nHdata , nHdata, np.ones( len( nHdata ) ) ] ) ## ( x y, y, d )
VMX = VMXT.transpose()
A = np.dot( VMXT, VMX )
SV = np.dot( VMXT, Sy )
sol = np.linalg.solve( A , SV )
print ( sol )
pF = 1 / sol[0] - 1
print( pF )
cF = -sol[1] * ( pF + 1 )
print( cF )
### making a linear fit on the scale
### the short version of the matrix form if only one factor is calculated
fk = fit_func( xHdata, 1, cF, pF )
aF = np.dot( nHdata, fk ) / np.dot( fk, fk )
print( aF )
#### using these guesses as input for a final non-linear fit
sol, cov = curve_fit(fit_func, xHdata, nHdata, p0=( aF, cF, pF ) )
print( sol )
print( cov )
### plotting
xth = np.linspace( 1, 6, 125 )
fig = plt.figure()
ax = fig.add_subplot( 1, 1, 1 )
ax.scatter( xHdata, nHdata )
ax.plot( xth, fit_func( xth, aF, cF, pF ), ls=':' )
ax.plot( xth, fit_func( xth, *sol ) )
plt.show()
Providing:
[-3.82334284e-01 2.51613126e-01 5.41522867e+07]
-3.6155122388787175
0.6580972107001803
8504146.59883185
[ 5.32486242e+07 2.44780953e-01 -7.24897172e+00]
[[ 1.03198712e+16 -2.71798924e+07 -2.37545914e+08]
[-2.71798924e+07 7.16072922e-02 6.26461373e-01]
[-2.37545914e+08 6.26461373e-01 5.49910325e+00]]
(note the high correlation from a to c and p)
and
I know of two things that might help you
Provide the p0 input parameter to curve_fit with a set of appropriate starting parameters to the function. That can keep the algorithm from running wild.
Change the function you are fitting so that it returns np.log(n) and then make the fit to np.log(nH_data). As it is now, there is a far larger penalty for not fitting the first data points than for not fitting the last data points, as the values are about 10^2 larger for the first ones. Thus, the first data points become "more important" to fit for the algorithm. Taking the logarithm puts them more on the same scale, so that points are weighed equally.
Go ahead and play around with it. I managed a pretty fine fit with these parameters
[-7.21450545e-01 -3.36131028e+00 5.97293632e+06]
I think you're nearly there, just need to fit on a log scale and throw in a decent guess. To make the guess you just need to throw in a plot like
plt.figure()
plt.plot(np.log(xH_data), np.log(nH_data))
and you'll see it's nearly linear. So your B will be the exponentiated intercept (i.e. exp(20ish)) and the a is the approximate slope (-5ish). A is weird one, does it have some physical meaning or you just threw it in there? If there's no physical meaning, I'd say get rid of it.
from scipy.optimize import curve_fit
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
import numpy as np
xH_data = np.array([1., 1.03, 1.06, 1.1, 1.2, 1.3, 1.5, 1.7, 2., 2.6, 3., 4., 5., 6.])
nH_data = np.array([403., 316., 235., 160., 70.8, 37.6, 14.8, 7.11, 2.81, 0.665, 0.313, 0.090, 0.044, 0.029])*1.0e6
def eTemp(x, A, a, B):
logn = np.log(B*(x + A)**a)
return logn
parameters, covariance = curve_fit(eTemp, xH_data, np.log(nH_data), p0=[np.exp(0.1), -5, np.exp(20)], maxfev=200000)
fit_A = parameters[0]
fit_a = parameters[1]
fit_B = parameters[2]
print(fit_A)
print(fit_a)
print(fit_B)
r = np.logspace(0, 0.7, 1000)
ne = np.exp(eTemp(r, fit_A, fit_a, fit_B))
plt.plot(xH_data, nH_data)
plt.plot(r, ne)
plt.yscale("log")
plt.xscale("log")
There is a problem with your fit equation. If A is less than -1 and your a parameter is negative then you get an imaginary value for your function within your fit range. For this reason you need to add constraints and an initial set of parameters to your curve_fit function for example:
parameters, covariance = curve_fit(eTemp, xH_data, nH_data, method='dogbox', p0 = [100, -3.3, 10E8], bounds=((-0.9, -10, 0), (200, -1, 10e9)), maxfev=200000)
You need to change the method to 'dogbox' in order to perform this fit with the constraints.
I'm trying to plot a function but for some reason it returns me a type error that I can't seem to find. These are the reproducible steps:
import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit
mass = [0.02, 0.05, 0.1, 0.15, 0.2, 0.25]
y = [0.755, 0.725, 0.668, 0.61, 0.55, 0.491]
def y_for_ideal_spring(m, k, y_0):
a_g = 9.8 #m/s^2
return np.divide(np.multiply(a_g,m) + np.multiply(k,y_0), k)
def y_for_real_spring_1(m, k, y_0, alpha):
a_g = 9.8
return y_0 + np.divide(- k - np.sqrt(k**2 + np.prod([4, a_g, m, alpha])), np.multiply(2, alpha))
popt_0, pcov_0 = curve_fit(y_for_ideal_spring, mass, y)
popt_1, pcov_1 = curve_fit(y_for_real_spring_1, mass, y, p0 = [popt_0[0], popt_0[1],1])
but when I try to plot I get the following error:
TypeError: can't multiply sequence by non-int of type 'float'
This is how I'm plotting it:
plt.plot(mass, y, 'go')
plt.plot(mass, y_for_ideal_spring(mass, *popt_0), '--r')
plt.plot(mass, y_for_real_spring_1(mass, *popt_1), '--b')
This is because mass is of type list. It can be solved by casting it as a numpy array.
mass = np.array([0.02, 0.05, 0.1, 0.15, 0.2, 0.25])
y = np.array([0.755, 0.725, 0.668, 0.61, 0.55, 0.491])
The output albeit with a few warnings from curve_fit:
1
I have this exercise where I get to build a simple neural network with one input layer and one hidden layer... I made the code below to perform a simple matrix multiplication, but it's not doing it properly as when I do the multiplication by hand. What am I doing wrong in my code?
#toes %win #fans
ih_wgt = ([0.1, 0.2, -0.1], #hid[0]
[-0.1, 0.1, 0.9], #hid[1]
[0.1, 0.4, 0.1]) #hid[2]
#hid[0] hid[1] #hid[2]
ho_wgt = ([0.3, 1.1, -0.3], #hurt?
[0.1, 0.2, 0.0], #win?
[0.0, 1.3, 0.1]) #sad?
weights = [ih_wgt, ho_wgt]
def w_sum(a,b):
assert(len(a) == len(b))
output = 0
for i in range(len(a)):
output += (a[i] * b[i])
return output
def vect_mat_mul(vec, mat):
assert(len(vec) == len(mat))
output = [0, 0, 0]
for i in range(len(vec)):
output[i]= w_sum(vec, mat[i])
return output
def neural_network(input, weights):
hid = vect_mat_mul(input, weights[0])
pred = vect_mat_mul(hid, weights[1])
return pred
toes = [8.5, 9.5, 9.9, 9.0]
wlrec = [0.65, 0.8, 0.8, 0.9]
nfans = [1.2, 1.3, 0.5, 1.0]
input = [toes[0],wlrec[0],nfans[0]]
pred = neural_network(input, weights)
print(pred)
the output of my code is:
[0.258, 0, 0]
The way I attempted to solve it by hand is as follows:
I multiplied the input vector [8.5, 0.65, 1.2] with the input weight matrix
ih_wgt = ([0.1, 0.2, -0.1], #hid[0]
[-0.1, 0.1, 0.9], #hid[1]
[0.1, 0.4, 0.1]) #hid[2]
[0.86, 0.295, 1.23]
the output vector is then fed into the network as an input vector which is then multiplied by the hidden weight matrix
ho_wgt = ([0.3, 1.1, -0.3], #hurt?
[0.1, 0.2, 0.0], #win?
[0.0, 1.3, 0.1]) #sad?
the correct output prediction:
[0.2135, 0.145, 0.5065]
Your help would be much appreciated!
You're almost there! Only a simple indentation thing is the reason:
def vect_mat_mul(vec, mat):
assert(len(vec) == len(mat))
output = [0, 0, 0]
for i in range(len(vec)):
output[i]= w_sum(vec, mat[i])
return output # <-- This one was inside the for loop
I have four 1-D arrays of dependent variables. They contain hundreds of data points but I have cropped them to 20 in this example. Each point represents a grid cell on a map.
import numpy as np
A=np.asarray([0.195, 0.154, 0.208, 0.22, 0.204, 0.175, 0.184, 0.187, 0.171, 0.2, 0.222, 0.235, 0.206, 0.215, 0.222, 0.252, 0.269, 0.251, 0.285, 0.28])
B=np.asarray([0.119, 0.134, 0.132, 0.121, 0.11, 0.097, 0.13, 0.106, 0.103, 0.139, 0.124, 0.147, 0.152, 0.123, 0.177, 0.172, 0.18, 0.182, 0.197, 0.193])
C=np.asarray([0.11, 0.1, 0.103, 0.111, 0.105, 0.098, 0.099, 0.093, 0.105, 0.099, 0.113, 0.093, 0.104, 0.095, 0.099, 0.105, 0.108, 0.128, 0.125, 0.118])
D=np.asarray([-0.015, -0.015, -0.007, -0.02, 0.002, 0.009, 0.019, 0.0, -0.02, -0.001, -0.006, -0.015, -0.03, -0.036, -0.051, -0.058, -0.065, -0.081, -0.082, -0.055])
The error on each variable is contained within the arrays:
A_err=np.asarray([ 0.016, 0.015, 0.017, 0.016, 0.015, 0.016, 0.016, 0.018, 0.015, 0.014, 0.015, 0.016, 0.017, 0.016, 0.017, 0.017, 0.017, 0.017, 0.017, 0.017])
B_err=np.asarray([ 0.045, 0.049, 0.039, 0.044, 0.036, 0.027, 0.032, 0.033, 0.029, 0.036, 0.032, 0.027, 0.04 , 0.022, 0.034, 0.026, 0.021, 0.028, 0.035, 0.028])
C_err=np.zeros(20)+0.7
D_err=np.zeros(20)+0.9
Y is the sum of A, B, C and D:
Y=A+B+C+D
Clearly Y will have a different value in each of the 20 grid cells. What I am trying to calculate is the error on each Y measurement.
Initially I was calculating this simply as:
Y_err=np.sqrt(A_err**2 + B_err**2 + C_err**2 + D_err**2)
But this includes no covariance terms. Since I do not have expressions for the variables A, B, C and D in terms of one another, the only way I can see to calculate the covariances is to use the correlation-covariance matrix:
X = np.vstack([A,B,C,D])
C = np.cov(X)
print(C)
[[ 1.31819737e-03 9.52921053e-04 2.17881579e-04 -8.58197368e-04]
[ 9.52921053e-04 9.87252632e-04 1.69478947e-04 -7.97089474e-04]
[ 2.17881579e-04 1.69478947e-04 9.47868421e-05 -1.88007895e-04]
[ -8.58197368e-04 -7.97089474e-04 -1.88007895e-04 8.85081579e-04]]
I am unclear as to whether I can then simply add each of the six non-diagonal matrix terms:
σ_AB=9.52921053e-04
σ_AC=2.17881579e-04
σ_AD=-8.58197368e-04
σ_BC=1.69478947e-04
σ_BD=-7.97089474e-04
σ_CD=-1.88007895e-04
into the equation for the propagation of error, such that:
Y_err=np.sqrt(A_err**2 + B_err**2 + C_err**2 + D_err**2 + 2*σ_AB + 2*σ_AC + 2*σ_AD + 2*σ_BC + 2*σ_BD + 2*σ_CD)
Or this totally invalid?
This is what my understanding is of your measurements: if you had a multiple measurements for a single grid. Then you would have calculated std error in measurement from those observations including cross terms.
Here you have a single measurement for each grid and error associated to that measurement.
Since, you measured for all 20 grids using the same procedure, what you can do is to calculate std_err for your measurement procedure and add the same uncertainty to all of your 20 observations.
For modelling the uncertainties in error measurement (error terms only) i.e. standard error in Y_err. Covariance/ cross terms gives you an idea of how uncertainties/ error in measurement are related to each other (i.e. σ_A_err_B_err) and not how actual measurement variables relates to each other(i.e. σ_AB).
Instead of: X = np.vstack([A,B,C,D])
You need: X = np.vstack([A_err,B_err,C_err,D_err]), to calculate covariance in error terms.
Your formula should be: All of them contained in your covariance matrix (including diagonal terms): np.cov(np.vstack([A_err,B_err,C_err,D_err]))
Y_std_err= np.sqrt(σ_A_err**2 + σ_B_err**2 + σ_C_err**2 + σ_D_err**2 +
2*σ_A_err_B_error + 2*σ_A_err_C_err + 2*σ_A_err_D_err +
2*σ_B_err_C_err + 2*σ_B_err_D_err + 2*σ_C_err_D_err)
Also, you can directly multiply matrices to get the sample variance for y, instead of the summation formula.
Where S, here is the sample variance/ covariance matrix for error terms and c, coefficients of linear combination of measurement model.
X = np.vstack([A_err,B_err,C_err,D_err]) # matrix of error terms
cov = np.cov(X)
c = np.asarray([1, 1, 1, 1]) # coefficients: linear combination
var_y_err = np.matmul(c.T, np.matmul(cov,c))
np.sqrt(var_y_err) # standard error in measurement of Y: stdy_y_err
# 0.007379024325749306
Y values should simply be a sum of A,B,C,D and each array point having above (+ or -) Average Y_error (depending on direction of error), having its own standard error (+/-).
Y = A + B + C + D (+ or -) Y_err_avg # Y_err will be an interval for every grid:
Y_err_avg = sum(A_err + B_err + C_err + D_err)/ 20 + (+/-) std_y_err
PS: I am not an expert on propagation of error. I would recommend you to post your query on cross validated to get an expert opinion.
import numpy as np
from scipy.optimize import curve_fit
x1 = [0.25, 0.33, 0.40, 0.50, 0.60, 0.75, 1.00]
y1 = [1.02, 1.39, 1.67, 1.89, 2.08, 2.44, 2.50]
def mmfunc(x1, d, e):
return d*x1/(e + x1)
y2 = mmfunc(x1,6.0,1.0)
popt, pcov = curve_fit(mmfunc, x1, y1)
I get this error
TypeError: can't multiply sequence by non-int of type 'float'
(x1 is an array (floats), d, e are floats)
(I tried reading values from a file, printed the values (they are floats) ...
I tried a simpler function - nothing seems to work!)
The problem is that you're not converting your lists to numpy arrays, so you can't add to or multiply by scalars. This seems to work for me:
import numpy as np
x1 = np.array([0.25, 0.33, 0.40, 0.50, 0.60, 0.75, 1.00], dtype="float")
y1 = np.array([1.02, 1.39, 1.67, 1.89, 2.08, 2.44, 2.50], dtype="float")
def mmfunc(x1, d, e):
return d*x1/(e + x1)
y2 = mmfunc(x1,6.0,1.0)
(Note: I didn't have scipy installed so I wasn't able to check that the curve_fit function works, but the conversion to np.array fixed the exception related to arithmetic on lists.)