I didn't find a way to perform optimize.minimize from scipy with a multidimensional function. In nearly all examples an analytical function is optimized while my function is interpolated. The test data set looks like this:
x = np.array([2000,2500,3000,3500])
y = np.array([10,15,25,50])
z = np.array([10,12,17,19,13,13,16,20,17,60,25,25,8,35,15,20])
data = np.array([x,y,z])
While the function is like F(x,y) = z
What I want to know is what happens at f(2200,12) and what is the global maximum in the range of x (2000:3500) and y (10:50). The interpolation works fine. But finding the global maximum doesn't work so far.
The interpolation
self.F2 = interp2d(xx, -yy, z, kind, bounds_error=False)
yields
<scipy.interpolate.interpolate.interp2d object at 0x0000000002C3BBE0>
I tried to optimize via:
x0 = [(2000,3500),(10,50)]
res = scipy.optimize.minimize(self.F2, x0, method='Nelder-Mead')
An exception is thrown:
TypeError: __call__() missing 1 required positional argument: 'y'
I think that the optimizer can't handle the object from the interpolation. In the examples the people used lambda to get values from their function. What do I have to do in my case?
Best,
Alex
First, to find global maximum (instead of minimum) you need to interpolate your function with opposite sign:
F2 = interp2d(x, y, -z)
Second, the callable in minimize takes a tuple of arguments, and interp2d object needs input coordinates to be given as separate positional arguments. Therefore, we cannot use interp2d object in minimize directly; we need a wrapper that will unpack a tuple of arguments from minimize and feed it to interp2d:
f = lambda x: F2(*x)
And third, to use minimize you need to specify an initial guess for minimum (and bounds, in your case). Any reasonable point will do:
x0 = (2200, 12)
bounds = [(2000,3500),(10,50)]
print minimize(f, x0, method='SLSQP', bounds=bounds)
This yields:
status: 0
success: True
njev: 43
nfev: 243
fun: array([-59.99999488])
x: array([ 2500.00002708, 24.99999931])
message: 'Optimization terminated successfully.'
jac: array([ 0.07000017, 1. , 0. ])
nit: 43
One more possible solution (hope you get the idea):
One more function is created (f), and the minimized values are sent as arguments to this function.
from scipy.optimize import minimize
x = data.Height.values
y = data.Weight.values
def f(params):
w0, w1 = params
return mse(w0, w1, x, y)
optimum = minimize(f, (0,0), method = 'L-BFGS-B', bounds = ((-100, 100), (-5,5)) )
w0 = optimum.x[0]
w1 = optimum.x[1]
Also tried implementation with lambda function, but had no luck.
Related
I have a system of ODEs which assume the form
In essence, I have the solutions B_1(t) and B_2(t) for t=5 and I am interested in finding the unknown parameters rho_1 and rho_2. The approach I took entailed: 1) define the function corresponding to the system above; 2) integrate using solve_ivp and deduct the result from the true values of B_1(t) and B_2(t); 3) finally use fsolve to find the appropriate values of rho_1 and rho_2, such that the difference between the true parameters B_1(t) and B_2(t) and the ones obtained using the tuned parameters of rho_1 and rho_2 is a zero vector. The code I have implemented for this purpose is the following:
t_eval = np.arange(0, 5)
def fun(t, s, rho_1, rho_2):
return np.dot(np.array([0.775416, 0,0, 0.308968]).reshape(2,2), s) + np.array([rho_1, rho_2]).reshape(2,1)
def fun2(t, rho_1, rho_2):
res = solve_ivp(fun, [0, 5], y0 = [0, 0], t_eval=t_eval, args = (rho_1, rho_2), vectorized = True)
sol = res.y[:,4]-np.array([0.01306365, 0.00589119])
return sol
root = fsolve(fun2, [0, 0])
However, I am not sure whether fsolve is not appropriate for this purpose tor there is something wrong with my code, as I get the following error:
fun2() missing 2 required positional arguments: 'rho_1' and 'rho_2'
Running the scipy.minimize function "I get TypeError: 'numpy.float64' object is not callable". Specifically during the execution of:
.../scipy/optimize/optimize.py", line 292, in function_wrapper
return function(*(wrapper_args + args))
I already looked at previous similar topics here and usually this problem occurs due to the fact that as first input parameter of .minimize is not a
function. I have difficulties in figure it out, because "a" is function.
What do you think?
### "data" is a pandas data frame of float values
### "w" is a numpy float array i.e. [0.11365704 0.00886848 0.65302202 0.05680696 0.1676455 ]
def a(data, w):
### Return a negative float value from position [2] of an numpy array of float values calculated via the "b" function i.e -0.3632965490830499
return -b(data, w)[2]
constraint = ({'type': 'eq', 'fun': lambda x: np.sum(x) - 1})
### i.e ((0, 1), (0, 1), (0, 1), (0, 1), (0, 1))
bound = tuple((0, 1) for x in range (len(symbols)))
opts = scipy.minimize(a(data, w), len(symbols) * [1. / len(symbols),], method = 'SLSQP', bounds = bound, constraints = constraint)
Short answer
It should instead be:
opts = scipy.minimize(a, len(symbols) * [1. / len(symbols),], args=(w,), method='SLSQP', bounds=bound, constraints=constraint)
Details
a(data, w) is not a function, it's a function call. In other words a(data, w) effectively has the value and type of the return value of the function a. minimize needs the actual function without the call (ie without the parentheses (...) and everything in-between), as its first parameter.
From the scipy.optimize.minimize docs:
scipy.optimize.minimize(fun, x0, args=(), method=None, jac=None, hess=None, hessp=None, bounds=None, constraints=(), tol=None, callback=None, options=None)
...
fun : callable
The objective function to be minimized. Must be in the form f(x, *args). The optimizing argument, x, is a 1-D array of points, and args is a tuple of any additional fixed parameters needed to completely specify the function.
...
args : tuple, optional
Extra arguments passed to the objective function...
So, assuming w is fixed (at least with respect to your desired minimization), you would pass it to minimize via the args parameter, as I've done above.
You're not passing the function, but the evaluated result to minimize.
opts = scipy.minimize(a, len(symbols) * [1. / len(symbols),], method = 'SLSQP', bounds = bound, constraints = constraint, args = (data,w))
Should work.
Edit: Fixed stupid syntax error.
I would like to fit my surface equation to some data. I already tried scipy.optimize.leastsq but as I cannot specify the bounds it gives me an unusable results. I also tried scipy.optimize.least_squares but it gives me an error:
ValueError: too many values to unpack
My equation is:
f(x,y,z)=(x-A+y-B)/2+sqrt(((x-A-y+B)/2)^2+C*z^2)
parameters A, B, C should be found so that the equation above would be as close as possible to zero when the following points are used for x,y,z:
[
[-0.071, -0.85, 0.401],
[-0.138, -1.111, 0.494],
[-0.317, -0.317, -0.317],
[-0.351, -2.048, 0.848]
]
The bounds would be A > 0, B > 0, C > 1
How I should obtain such a fit? What is the best tool in python to do that. I searched for examples on how to fit 3d surfaces but most of examples involving function fitting is about line or flat surface fits.
I've edited this answer to provide a more general example of how this problem can be solved with scipy's general optimize.minimize method as well as scipy's optimize.least_squares method.
First lets set up the problem:
import numpy as np
import scipy.optimize
# ===============================================
# SETUP: define common compoments of the problem
def our_function(coeff, data):
"""
The function we care to optimize.
Args:
coeff (np.ndarray): are the parameters that we care to optimize.
data (np.ndarray): the input data
"""
A, B, C = coeff
x, y, z = data.T
return (x - A + y - B) / 2 + np.sqrt(((x - A - y + B) / 2) ** 2 + C * z ** 2)
# Define some training data
data = np.array([
[-0.071, -0.85, 0.401],
[-0.138, -1.111, 0.494],
[-0.317, -0.317, -0.317],
[-0.351, -2.048, 0.848]
])
# Define training target
# This is what we want the target function to be equal to
target = 0
# Make an initial guess as to the parameters
# either a constant or random guess is typically fine
num_coeff = 3
coeff_0 = np.ones(num_coeff)
# coeff_0 = np.random.rand(num_coeff)
This isn't strictly least squares, but how about something like this?
This solution is like throwing a sledge hammer at the problem. There probably is a way to use least squares to get a solution more efficiently using an SVD solver, but if you're just looking for an answer scipy.optimize.minimize will find you one.
# ===============================================
# FORMULATION #1: a general minimization problem
# Here the bounds and error are all specified within the general objective function
def general_objective(coeff, data, target):
"""
General function that simply returns a value to be minimized.
The coeff will be modified to minimize whatever the output of this function
may be.
"""
# Constraints to keep coeff above 0
if np.any(coeff < 0):
# If any constraint is violated return infinity
return np.inf
# The function we care about
prediction = our_function(coeff, data)
# (optional) L2 regularization to keep coeff small
# (optional) reg_amount = 0.0
# (optional) reg = reg_amount * np.sqrt((coeff ** 2).sum())
losses = (prediction - target) ** 2
# (optional) losses += reg
# Return the average squared error
loss = losses.sum()
return loss
general_result = scipy.optimize.minimize(general_objective, coeff_0,
method='Nelder-Mead',
args=(data, target))
# Test what the squared error of the returned result is
coeff = general_result.x
general_output = our_function(coeff, data)
print('====================')
print('general_result =\n%s' % (general_result,))
print('---------------------')
print('general_output = %r' % (general_output,))
print('====================')
The output looks like this:
====================
general_result =
final_simplex: (array([[ 2.45700466e-01, 7.93719271e-09, 1.71257109e+00],
[ 2.45692680e-01, 3.31991619e-08, 1.71255150e+00],
[ 2.45726858e-01, 6.52636219e-08, 1.71263360e+00],
[ 2.45713989e-01, 8.06971686e-08, 1.71260234e+00]]), array([ 0.00012404, 0.00012404, 0.00012404, 0.00012404]))
fun: 0.00012404137498459109
message: 'Optimization terminated successfully.'
nfev: 431
nit: 240
status: 0
success: True
x: array([ 2.45700466e-01, 7.93719271e-09, 1.71257109e+00])
---------------------
general_output = array([ 0.00527974, -0.00561568, -0.00719941, 0.00357748])
====================
I found in the documentation that all you need to do to adapt this to actual least squares is to specify the function that computes the residuals.
# ===============================================
# FORMULATION #2: a special least squares problem
# Here all that is needeed is a function that computes the vector of residuals
# the optimization function takes care of the rest
def least_squares_residuals(coeff, data, target):
"""
Function that returns the vector of residuals between the predicted values
and the target value. Here we want each predicted value to be close to zero
"""
A, B, C = coeff
x, y, z = data.T
prediction = our_function(coeff, data)
vector_of_residuals = (prediction - target)
return vector_of_residuals
# Here the bounds are specified in the optimization call
bound_gt = np.full(shape=num_coeff, fill_value=0, dtype=np.float)
bound_lt = np.full(shape=num_coeff, fill_value=np.inf, dtype=np.float)
bounds = (bound_gt, bound_lt)
lst_sqrs_result = scipy.optimize.least_squares(least_squares_residuals, coeff_0,
args=(data, target), bounds=bounds)
# Test what the squared error of the returned result is
coeff = lst_sqrs_result.x
lst_sqrs_output = our_function(coeff, data)
print('====================')
print('lst_sqrs_result =\n%s' % (lst_sqrs_result,))
print('---------------------')
print('lst_sqrs_output = %r' % (lst_sqrs_output,))
print('====================')
The output here is:
====================
lst_sqrs_result =
active_mask: array([ 0, -1, 0])
cost: 6.197329866927735e-05
fun: array([ 0.00518416, -0.00564099, -0.00710112, 0.00385024])
grad: array([ -4.61826888e-09, 3.70771396e-03, 1.26659198e-09])
jac: array([[-0.72611025, -0.27388975, 0.13653112],
[-0.74479565, -0.25520435, 0.1644325 ],
[-0.35777232, -0.64222767, 0.11601263],
[-0.77338046, -0.22661953, 0.27104366]])
message: '`gtol` termination condition is satisfied.'
nfev: 13
njev: 13
optimality: 4.6182688779976278e-09
status: 1
success: True
x: array([ 2.46392438e-01, 5.39025298e-17, 1.71555150e+00])
---------------------
lst_sqrs_output = array([ 0.00518416, -0.00564099, -0.00710112, 0.00385024])
====================
import numpy as np
import scipy.optimize as spo
def function(x,y):
return (np.sin(x*y+y)*np.exp(-1*(x**2+y**2)))**-1
xi=[0,0]
answer=spo.fmin(function,xi)
print 'the answer is', answer
I am trying to minimise this function. However running it brings up
TypeError: function() takes exactly 2 arguments (1 given)
scipy.optimize.fmin(func, x0, args=(), xtol=0.0001, ftol=0.0001, maxiter=None, maxfun=None, full_output=0, disp=1, retall=0, callback=None)
parameter func is callable func(x,*args)
In this case fmin call function with one parameter - x (which is xi). The second parameter must be passed as args parameter.
xi = 0
args = (0,)
answer = spo.fmin(function, x0=xi, args=args)
http://docs.scipy.org/doc/scipy-0.16.0/reference/generated/scipy.optimize.fmin.html
Is your intent to minimize over 2 variables ('x','y'), or over just one (with 'y' as an extra parameter)?
def fn1(x, y):
# x is minimization variable
# y is extra argument
return (np.sin(x*y+y)*np.exp(-1*(x**2+y**2)))**-1
def fn2(xy):
# xy is minimization variable; assumed to be 2 elements
x,y = xy
return (np.sin(x*y+y)*np.exp(-1*(x**2+y**2)))**-1
fmin with 1 variable; fails
In [35]: optimize.fmin(fn1, x0=0, args=(0,))
Warning: Maximum number of function evaluations has been exceeded.
Out[35]: array([ 0.])
fmin with 2 element array (x0 and function); returns 2 element array.
In [38]: optimize.fmin(fn2, x0=np.array([0,0]))
Optimization terminated successfully.
Current function value: 2.227274
Iterations: 64
Function evaluations: 121
Out[38]: array([ 0.29782369, 0.62167083])
I am trying to optimise a function using the fminbound function of the scipy.optimize module. I want to set parameter bounds to keep the answer physically sensible (e.g. > 0).
import scipy.optimize as sciopt
import numpy as np
The arrays:
x = np.array([[ 1247.04, 1274.9 , 1277.81, 1259.51, 1246.06, 1230.2 ,
1207.37, 1192. , 1180.84, 1182.76, 1194.76, 1222.65],
[ 589. , 581.29, 576.1 , 570.28, 566.45, 575.99,
601.1 , 620.6 , 637.04, 631.68, 611.79, 599.19]])
y = np.array([ 1872.81, 1875.41, 1871.43, 1865.94, 1854.8 , 1839.2 ,
1827.82, 1831.73, 1846.68, 1856.56, 1861.02, 1867.15])
I managed to optimise the linear function within the parameter bounds when I use only one parameter:
fp = lambda p, x: x[0]+p*x[1]
e = lambda p, x, y: ((fp(p,x)-y)**2).sum()
pmin = 0.5 # mimimum bound
pmax = 1.5 # maximum bound
popt = sciopt.fminbound(e, pmin, pmax, args=(x,y))
This results in popt = 1.05501927245
However, when trying to optimise with multiple parameters, I get the following error message:
fp = lambda p, x: p[0]*x[0]+p[1]*x[1]
e = lambda p, x, y: ((fp(p,x)-y)**2).sum()
pmin = np.array([0.5,0.5]) # mimimum bounds
pmax = np.array([1.5,1.5]) # maximum bounds
popt = sciopt.fminbound(e, pmin, pmax, args=(x,y))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/dist-packages/scipy/optimize/optimize.py", line 949, in fminbound
if x1 > x2:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
I have tried to vectorize e (np.vectorize) but the error message remains the same. I understand that fminbound expects a float or array scalar as bounds. Is there another function that would work for this problem?
fminbound is only for optimizing functions of one variable.
For the multivariate case, you should use scipy.optimize.minimize, for example,
from scipy.optimize import minimize
p_guess = (pmin + pmax)/2
bounds = np.c_[pmin, pmax] # [[pmin[0],pmax[0]], [pmin[1],pmax[1]]]
sol = minimize(e, p_guess, bounds=bounds)
print(sol)
if not sol.success:
raise RuntimeError("Failed to solve")
popt = sol.x