I'm currently working on an algorithm that determines the cost of a wind turbine support structure. The algorithm I'm writing needs to optimize the weight of an initial input support structure so that the stress levels will not exceed but be near to the failure criteria of the properties of the materials used. Another requirement is that the natural frequency of the structure needs to be bounded between 2 values. To optimize the structure 4 variables can be altered.
Can I use a function from the Scipy.Optimize library that optimizes the weight of this structure using several design parameters, but keeps into account the natural frequency and the maximum stress value in the support structure?
The function I'm optimizing looks like this:
def func(self, x):
self.properties.D_mp = x[0] # Set a new diameter for the monopile
self.properties.Dtrat_tower = x[1] # Set a new thickness ratio for the tower
self.properties.Dtrat_tp = x[2] # Set a new thickness ratio for the transition piece
self.properties.Dtrat_mud = x[3] # Set a new thickness ratio for the mudline region of the monopile
self.UpdateAll() # Update the support structure based on the changes in variables above
eig = self.GetEigenFrequency() # Get the natural frequency
maxUtil = self.GetMaximumUtilisationFactor() # Get the maximum utilisation ratio on the structure (more than 1 means stress is higher than maximum allowed)
# Natural frequency of 0.25 and utilisation ratio of 1 are ideal
# Create some penalty...
penalty = (100000 * abs((eig - 0.25)))
penalty += (100000 * abs(maxUtil - 1))
return self.GetTotalMass() + penalty
Thanks in advance!
It is probably easiest to make this a single-value optimization problem by folding in the frequency and stress constraints as penalties to the overall fitness, something like
LOW_COST = 10.
MID_COST = 150.
HIGH_COST = 400.
def weight(a, b, c, d):
return "calculated weight of structure"
def frequency(a, b, c, d):
return "calculated resonant frequency"
def freq_penalty(freq):
# Example linear piecewise penalty function -
# increasing cost for frequencies below 205 or above 395
if freq < 205:
return MID_COST * (205 - freq)
elif freq < 395:
return 0.
else:
return MID_COST * (freq - 395)
def stress_fraction(a, b, c, d):
return "calculated stress / failure criteria"
def stress_penalty(stress_frac):
# Example linear piecewise penalty function -
# low extra cost for stress fraction below 0.85,
# high extra cost for stress fraction over 0.98
if stress_frac < 0.85:
return LOW_COST * (0.85 - stress_frac)
elif stress_frac < 0.98:
return 0.
else:
return HIGH_COST * (stress_frac - 0.98)
def overall_fitness(parameter_vector):
a, b, c, d = parameter_vector
return (
# D'oh! it took me a while to get this right -
# we want _minimum_ weight and _minimum_ penalty
# to get _maximum_ fitness.
-weight(a, b, c, d)
- freq_penalty(frequency(a, b, c, d))
- stress_penalty(stress_fraction(a, b, c, d)
)
... you will of course want to find more suitable penalty functions and play with the relative weighting, but this should give you the general idea. Then you can maximize it like
from scipy.optimize import fmin
initial_guess = [29., 45., 8., 0.06]
result = fmin(lambda x: -overall_fitness(x), initial_guess, maxfun=100000, full_output=True)
(using the lambda lets fmin (a minimizer) find a maximum value for overall_fitness).
Alternatively, fmin allows a callback function which is applied after each iteration; you could use this to put hard limits on frequency IF you know how to tweak a,b,c,d appropriately - something like
def callback(x):
a, b, c, d = x # unpack parameter vector
freq = frequency(a, b, c, d)
if freq < 205:
# apply appropriate correction to put frequency back in bounds
return [a, b, c + 0.2, d]
elif freq < 395:
return x
else:
return [a, b, c - 0.2, d]
You can use the leastsq function of spicy.optimize.
In my case it was to fit an exponential function with two variables :
def func_exp(p, x, z):
# exponential function with multiple parameters
a, b, c, d, t, t2 = p[0], p[1], p[2], p[3], p[4], p[5]
return a*np.exp(b + pow(x,c)*t + pow(z,d)*t2)
But to use the leastsq function you need to create an error function, this this one you'll optimize.
def err(p, x,z, y):
# error function compare the previous to the estimate to
return func_exp(p, x,z) - y
# minimise the residuals
To use it :
p0=[1e4,-1e-3,1,1,-1e-2, -1e-6]
# First parameters
pfit_fin, pcov, infodict, errmsg, success = leastsq(err, p0, args=(X,Y,Z), full_output=1, epsfcn=0.000001)
So that's return the best parameters, to generate the results :
Y_2= func_exp(pfit_fin, X,Y)
I hope that will help you,
Chris.
Related
A set of points with coordinates x and y look like this. I want to construct a curve in the region below y = 0 of the form - a - np.exp(-(x - b)/c), where parameters a, b and c are found by the condition that 90% of the points below y = 0 are enclosed by this line and the function in question.
I've written the following code to do this, but the minimize function gives the initial guess as a result and I don't know what I'm missing.
from scipy.optimize import minimize
import numpy as np
def enclosed_points(params):
a, b, c = params
den = (y < 0).sum() # Calculate the number of points with y coordinate below y0
func = - a - np.exp(-(x - b)/c) # Calculate the value of the function for each x
num = ((y < 0) & (y > func)).sum() # Calculate the number of points with y coordinate
# below y0 and above the function
return np.abs(num/den - 0.9) # Return the absolute value of the difference between
# the ratio of num and den and the target number (0.9)
initial_guess = [0.1, 0.2, 1] # Dummy initial guess
result = minimize(enclosed_points, initial_guess)
Edit. Here I have uploaded a random sample of the whole data in npy format.
Well, I tried some different methods and change some parts of your code:
def func(x, a, b, c):
return - a - np.exp(-(x - b)/c)
def enclosed_points(params):
a, b, c = params
loss1 = y[np.argwhere( y > 0)]
loss2 = loss1[np.argwhere( loss1 < func(x[np.argwhere( y > 0)], a, b, c) )]
loss = ((loss2.sum() / len(y)) - 0.9)**2
return loss
initial_guess = [-0.1, 0.2, 1]
result = minimize(enclosed_points, initial_guess, method='SLSQP', options={'eps': 1e-2})
The loss1 and loss2 do the same work as your loss function, but I change abs to the power of 2 (because it's more common in my opinion) (also added "method='SLSQP', options={'eps': 1e-2}" to your minimizer based on another post on StackOverflow; try to read them carefully and get familiar with their issues about this minimizer too). However, I think the main issue is that your problem is non-convex, and minimize function tries to find a local minimum. Please see this post for a comprehensive description.
In the end, I would say that with [-0.1, 0.2, 1] initial point I could find a solution (try different negative value for a, and may you can find another solutions :) )
Good luck
A distribution is beta-binomial if p, the probability of success, in a binomial distribution has a beta distribution with shape parameters α > 0 and β > 0. The shape parameters define the probability of success.
I want to find the values for α and β that best describe my data from the perspective of a beta-binomial distribution. My dataset players consist of data about the number of hits (H), the number of at-bats (AB) and the conversion (H / AB) of a lot of baseball players. I estimate the PDF with the help of the answer of JulienD in Beta Binomial Function in Python
from scipy.special import beta
from scipy.misc import comb
pdf = comb(n, k) * beta(k + a, n - k + b) / beta(a, b)
Next, I write a loglikelihood function that we will minimize.
def loglike_betabinom(params, *args):
"""
Negative log likelihood function for betabinomial distribution
:param params: list for parameters to be fitted.
:param args: 2-element array containing the sample data.
:return: negative log-likelihood to be minimized.
"""
a, b = params[0], params[1]
k = args[0] # the conversion rate
n = args[1] # the number of at-bats (AE)
pdf = comb(n, k) * beta(k + a, n - k + b) / beta(a, b)
return -1 * np.log(pdf).sum()
Now, I want to write a function that minimizes loglike_betabinom
from scipy.optimize import minimize
init_params = [1, 10]
res = minimize(loglike_betabinom, x0=init_params,
args=(players['H'] / players['AB'], players['AB']),
bounds=bounds,
method='L-BFGS-B',
options={'disp': True, 'maxiter': 250})
print(res.x)
The result is [-6.04544138 2.03984464], which implies that α is negative which is not possible. I based my script on the following R-snippet. They get [101.359, 287.318]..
ll <- function(alpha, beta) {
x <- career_filtered$H
total <- career_filtered$AB
-sum(VGAM::dbetabinom.ab(x, total, alpha, beta, log=True))
}
m <- mle(ll, start = list(alpha = 1, beta = 10),
method = "L-BFGS-B", lower = c(0.0001, 0.1))
ab <- coef(m)
Can someone tell me what I am doing wrong? Help is much appreciated!!
One thing to pay attention to is that comb(n, k) in your log-likelihood might not be well-behaved numerically for the values of n and k in your dataset. You can verify this by applying comb to your data and see if infs appear.
One way to amend things could be to rewrite the negative log-likelihood as suggested in https://stackoverflow.com/a/32355701/4240413, i.e. as a function of logarithms of Gamma functions as in
from scipy.special import gammaln
import numpy as np
def loglike_betabinom(params, *args):
a, b = params[0], params[1]
k = args[0] # the OVERALL conversions
n = args[1] # the number of at-bats (AE)
logpdf = gammaln(n+1) + gammaln(k+a) + gammaln(n-k+b) + gammaln(a+b) - \
(gammaln(k+1) + gammaln(n-k+1) + gammaln(a) + gammaln(b) + gammaln(n+a+b))
return -np.sum(logpdf)
You can then minimize the log-likelihood with
from scipy.optimize import minimize
init_params = [1, 10]
# note that I am putting 'H' in the args
res = minimize(loglike_betabinom, x0=init_params,
args=(players['H'], players['AB']),
method='L-BFGS-B', options={'disp': True, 'maxiter': 250})
print(res)
and that should give reasonable results.
You could check How to properly fit a beta distribution in python? for inspiration if you want to rework further your code.
I'm tring to approximate an empirical cumulative distribution function (ECDF I want to approximate) with a smooth function (with less than 5 parameter) such as the generalized logistic function.
However, using scipy.optimize.curve_fit, the fitting operation gives really bad approximations or it doesn't work at all (depending on the initial values). The variable series represents my data stored as pandas.Series.
from scipy.optimize import curve_fit
def fit_ecdf(x):
x = np.sort(x)
def result(v):
return np.searchsorted(x, v, side='right') / x.size
return result
ecdf = fit_ecdf(series)
def genlogistic(x, B, M, Q, v):
return 1 / (1 + Q * np.exp(-B * (x - M))) ** (1 / v)
params = curve_fit(genlogistic, xdata = series, ydata = ecdf(series), p0 = (0.1, 10.0, 0.1, 0.1))[0]
Should I use another type of function for the fit?
Are there any code mistakes?
UPDATE - 1
As asked, I link to a csv containing the data.
UPDATE - 2
After a lot of search and trial and error I find out this function
f(x; a, b, c) = 1 - 1 / (1 + (x / b) ** a) ** c
with a = 4.61320000, b = 2.94570952, c = 0.5886922
which fits a lot better than the other one. The only problem is the little step that the ECDF shows near x=1. How can I modify f to improve the quality of the fit? I was thinking of adding some sort of function that is "relevant" only in those kind of points. Here are the graphical results of the fit where the solid blue line is the ECDF and the dotted line represents the (x, f(x)) points.
I find out how to deal with that little step near x=1. As expressed in the question, adding some sort of function that is significant only in that interval was the game changer.
The "step" ends at about (1.7, 0.04) so I needed a sort of function that flattens for x > 1.7 and has y = 0.04 as asymptote. The natural choice (just to stay on point) was to take a function like f(x) = 1/exp(x).
Thanks to JamesPhillips, I also picked up the proper data for the regression (no double values = no overweighted points).
Python Code
from scipy.optimize import curve_fit
def fit_ecdf(x):
x = np.sort(x)
def result(v):
return np.searchsorted(x, v, side = 'right') / x.size
return result
ecdf = fit_ecdf(series)
unique_series = series.unique().tolist()
def cdf_interpolation(x, a, b, c, d):
f_1 = 0.95 + (0 - 0.95) / (1 + (x / b) ** a) ** c + 0.05
f_2 = (0 - 0.05)/(np.exp(d * x))
return f_1 + f_2
params = curve_fit(cdf_interpolation,
xdata = unique_series ,
ydata = ecdf(unique_series),
p0 = (6.0, 3.0, 0.4, 1.0))[0]
Parameters
a = 6.03256462
b = 2.89418871
c = 0.42997956
d = 1.06864006
Graphical results
I got an OK fit for a 5-parameter logistic equation (see image and code) using unique values, not sure if the low end curve is sufficient for your needs, please check.
import numpy as np
def Sigmoidal_FiveParameterLogistic_model(x_in): # from zunzun.com
# coefficients
a = 9.9220221252324947E-01
b = -3.1572339989462903E+00
c = 2.2303376075685142E+00
d = 2.6271495036080207E-02
f = 3.4399008905318986E+00
return d + (a - d) / np.power(1.0 + np.power(x_in / c, b), f)
I currently have a system of odes with a time-dependent constant. E.g.
def fun(u, t, a, b, c):
x = u[0]
y = u[1]
z = u[2]
dx_dt = a * x + y * z
dy_dt = b * (y-z)
dz_dt = -x*y+c*y-z
return [dx_dt, dy_dt, dz_dt]
The constants are "a", "b" and "c". I currently have a list of "a"s for every time-step which I would like to insert at every time-step, when using the scipy ode solver...is this possible?
Thanks!
Yes, this is possible. In the case where a is constant, I guess you called scipy.integrate.odeint(fun, u0, t, args) where fun is defined as in your question, u0 = [x0, y0, z0] is the initial condition, t is a sequence of time points for which to solve for the ODE and args = (a, b, c) are the extra arguments to pass to fun.
In the case where a depends on time, you simply have to reconsider a as a function, for example (given a constant a0):
def a(t):
return a0 * t
Then you will have to modify fun which computes the derivative at each time step to take the previous change into account:
def fun(u, t, a, b, c):
x = u[0]
y = u[1]
z = u[2]
dx_dt = a(t) * x + y * z # A change on this line: a -> a(t)
dy_dt = b * (y - z)
dz_dt = - x * y + c * y - z
return [dx_dt, dy_dt, dz_dt]
Eventually, note that u0, t and args remain unchanged and you can again call scipy.integrate.odeint(fun, u0, t, args).
A word about the correctness of this approach. The performance of the approximation of the numerical integration is affected, I don't know precisely how (no theoretical guarantees) but here is a simple example which works:
import matplotlib.pyplot as plt
import numpy as np
import scipy as sp
import scipy.integrate
tmax = 10.0
def a(t):
if t < tmax / 2.0:
return ((tmax / 2.0) - t) / (tmax / 2.0)
else:
return 1.0
def func(x, t, a):
return - (x - a(t))
x0 = 0.8
t = np.linspace(0.0, tmax, 1000)
args = (a,)
y = sp.integrate.odeint(func, x0, t, args)
fig = plt.figure()
ax = fig.add_subplot(111)
h1, = ax.plot(t, y)
h2, = ax.plot(t, [a(s) for s in t])
ax.legend([h1, h2], ["y", "a"])
ax.set_xlabel("t")
ax.grid()
plt.show()
I Hope this will help you.
No, that is not possible in the literal sense of
"I currently have a list of "a"s for every time-step which I would like to insert at every time-step"
as the solver has adaptive step size control, that is, it will use internal time steps that you have no control over, and each time step uses several evaluations of the function. Thus there is no connection between the solver time steps and the data time steps.
In the extended sense that the given data defines a piecewise constant step function however, there are several approaches to get to a solution.
You can integrate from jump point to jump point, using the ODE function with the constant parameter for this time segment. After that use numpy array operations like concatenate to assemble the full solution.
You can use interpolation functions like numpy.interp or scipy.interpolate.interp1d. The first gives a piecewise linear interpolation, which may not be desired here. The second returns a function object that can be configured to be a "zero-order hold", which is a piecewise constant step function.
You could implement your own logic to go from the time t to the correct values of those parameters. This mostly applies if there is some structure to the data, for instance, if they have the form f(int(t/h)).
Note that the approximation order of the numerical integration is not only bounded by the order of the RK (solve_ivp) or multi-step (odeint) method, but also by the differentiability order of the (parts of) the differential equation. If the ODE is much less smooth than the order of the method, the implicit assumptions for the step size control mechanism are violated, which may result in a very small step size requiring a huge number of integration steps.
I also encountered similar problem. In my case, parameters a, b, and c are not in direct function with time, but determined by x, y, and z at that time. So I have to get x, y, z at time t, and calculate a, b, c for the integration calculation for x, y, z at t+dt. It turns out that if I change dt value, the whole integration result will change dramatically, even to something unreasonable.
I have a simple x,y data set to fit, at least at first glance. The issue is that scipy.optimize.curve_fit gives back a very large value for one of the parameters fitted and I don't know if this is mathematically correct or if there's something wrong with how I'm fitting the data.
The figure below shows both the data points and the best fit obtained in blue. The curve used (func in the MWE below) has four parameters a, b, c, d being fitted:
a gives approximately the x value where the curve attains it's half-maximum.
b represents the x value where the curve stabilizes. This func value is given by the d parameter, ie: func(b) = d
c is related to the maximum value of the curve at the origin: func(0) = c*constant + d
d is where the curve stabilizes (black line in the figure).
The b parameter is the one I'm having issues with (see end of question), it's also the one I'm most interested in assigning a reasonable value.
The MWE shows the function being fitted and the results:
import numpy as np
from scipy.optimize import curve_fit
import matplotlib.pyplot as plt
# Function to be fitted.
def func(x, a, b, c, d):
return c * (1 / np.sqrt(1 + (np.asarray(x) / a) ** 2) -
1 / np.sqrt(1 + (b / a) ** 2)) ** 2 + d
# Define x,y data.
x_list = [12.5, 37.5, 62.5, 87.5, 112.5, 137.5, 162.5, 187.5, 212.5, 237.5,
262.5, 287.5, 312.5, 337.5, 362.5, 387.5, 412.5, 437.5, 462.5, 487.5,
512.5]
y_list = [0.008, 0.0048, 0.0032, 0.00327, 0.0023, 0.00212, 0.00187,
0.00086, 0.00070, 0.00100, 0.00056, 0.00076, 0.00052, 0.00077, 0.00067,
0.00048, 0.00078, 0.00067, 0.00069, 0.00061, 0.00047]
# Initial guess for the 4 parameters.
guess = (50., 200., 80. / 10000., 6. / 10000.)
# Fit curve to x,y data.
f_prof, f_err = curve_fit(func, x_list, y_list, guess)
# Values for the a,b,c,d fitted parameters.
print f_prof
# Errors (standard deviations) for the fitted parameters.
print np.sqrt(f_err[0][0]), np.sqrt(f_err[1][1]), np.sqrt(f_err[2][2]),\
np.sqrt(f_err[3][3])
# Generate plot.
plt.scatter(x_list, y_list)
plt.plot(x_list, func(x_list, f_prof[0], f_prof[1], f_prof[2], f_prof[3]))
plt.hlines(y=f_prof[3], xmin=0., xmax=max(x_list))
plt.show()
The results I get are:
# a, b, c, d
52.74, 2.52e+09, 7.46e-03, 5.69e-04
# errors
11.52, 1.53e+16, 0.0028, 0.00042
The b parameter has a huge value and also a huge error. By looking at the data plotted in the figure, one would estimate by eye that the value for b (ie: the x value where the data set stabilizes) should be around x=300. Why am I getting such a large value for b and its error?
I don't know if this is intentional or a mistake but it looks to me like 'b' will be correlated strongly with 'a' and 'd' and has no "interaction" with the independent variable 'x'. If b/a is large enough, you could approximate 1/np.sqrt(1 + (b / a) ** 2)) ** 2 as a/b, so that your function becomes
c * function_of(x, a) - a/b + d
Your 'a' and 'x' values are large enough that this becomes very nearly c*a/x - a/b + d.
As pointed by behzad.nouri, curve_fit can be slightly unstable compared to other minimizers, and always minimizes the least-squares. But it does return the full covariance matrix including correlations between variables (off-diagonal elements of your f_err). Use these!!
If you're sure 'b' has a value around 300, or are interested in easily switching between fmin and the levenberg-marquardt algorithm, you might find the lmfit package (http://lmfit.github.io/lmfit-py/) useful. It allows you to put bounds on parameters, easily switch between fitting algorithms, and also to do a more brute-force exploration of the confidence intervals for the parameters.
you can use a penalty value for the norm of parameters, and use fmin:
from scipy.optimize import fmin
def func(x, a, b, c, d):
return c * (1 / np.sqrt(1 + (x / a) ** 2) - 1 / np.sqrt(1 + (b / a) ** 2)) ** 2 + d
def errfn(params, xs, ys, lm, ord=1):
'''
lm: penalty maltiplier
ord: order in norm calculation
'''
from numpy.linalg import norm
a, b, c, d = params
err = func(xs, a, b, c, d) - ys
return norm(err) + lm * norm(params, ord)
params = fmin(errfn, guess, args=(xs, ys, 1e-6, 2))
above I am using a small penalty of 1e-6 and the fit result are
[6.257e+01 3.956e+02 9.926e-03 7.550e-04]
with a decent fit:
edit: playing with the penalty function and the norm order, it gives a very good fit at
params = [ 1.479e+01 -3.344e+00 -8.781e-03 8.347e-03]
From a quick look, it seems that a large b will eliminate the second term of func():
When b/a goes to infinity, 1 / np.sqrt(1 + (b / a) ** 2)) ** 2 goes to zero.
This suggests to me that this part of the function is not needed in the model and is doing more damage than good.
Just set func to be:
c * (1 / np.sqrt(1 + (np.asarray(x) / a) ** 2) + d