I want to use scipy.optimize.minimize to solve for a set of parameters by minimizing an error function.
The function called "error" returns the squared error for the function that I am trying to find z1,z2,z3(the parameters) for.
I have an array of x(called "b" in the function) and y(called "real" in the function) values.
The code below works fine if I set x and y to some integer, but not if I try to pass in an array of x and y values, to act as the variable "b" and "real" in the equation to be minimized.
Trying to pass in an array of X and Y values results in the error pasted below.
Is there a way to pass in arrays to act as a variable in an equation for the minimize function, instead of just a single integer?
Here is what my code looks like:
import numpy as np
import pandas as pd
from scipy.optimize import minimize
#dataset file, with a column named x,y with 1000 rows
f = pd.read_csv('Test_Data_.txt', sep='\t')
#initial guess
x0 = [1, 2, 3]
#f['x'] and f['y'] are columns with 1000 rows
#x = f['x'].values
#y = f['y'].values
x = 1 #these parameters work fine
y = 4
#a function called inside the function to be minimized
def est(z1, z2, z3, b):
return z1 * b**2 + z2 * b + z3
#function to minimize
def error(x, real, b):
return (real - est(x[0], x[1], x[2], b))**2
print(minimize(error, x0, args = ( x, y), method='BFGS', tol=1e-6))
Feeding in the array of x and y values produces the error:
Traceback (most recent call last):
File "problem1.py", line 24, in <module>
minimize(error, x0, args = ( np.array(list(f['y'].values)), np.array(list(f['x'].values))), method='BFGS', tol=1e-6)
File "/usr/local/lib/python3.5/dist-packages/scipy/optimize/_minimize.py", line 595, in minimize
return _minimize_bfgs(fun, x0, args, jac, callback, **options)
File "/usr/local/lib/python3.5/dist-packages/scipy/optimize/optimize.py", line 970, in _minimize_bfgs
gfk = myfprime(x0)
File "/usr/local/lib/python3.5/dist-packages/scipy/optimize/optimize.py", line 300, in function_wrapper
return function(*(wrapper_args + args))
File "/usr/local/lib/python3.5/dist-packages/scipy/optimize/optimize.py", line 730, in approx_fprime
return _approx_fprime_helper(xk, f, epsilon, args=args)
File "/usr/local/lib/python3.5/dist-packages/scipy/optimize/optimize.py", line 670, in _approx_fprime_helper
grad[k] = (f(*((xk + d,) + args)) - f0) / d[k]
ValueError: setting an array element with a sequence.
Related
I am currently trying to implement the following optimization problem in python (in order to resolve it with scipy.optimize.minimize).
Please note that alpha is given,T is the number of generated random values (i.e. via Monte Carlo simulation, also given), z is an array of artificial variables (ignore the last constraint). The function f(x,y) is equal to -y.T * x (y is an array of nT random values). Variable val is a pandas data frame with all the observed data. Variable r is the realization of the random events (randomly generated using MonteCarlo technique - fitting a distribution, pandas nT).
Unfortunately I am facing different problems while truing to solve it. Can anyone be so kind to help me to code it correctly?
EDIT: following the modified code with correct init and constraints. I am not able to figured out how to write correctly the bounds (for x[0] bounds should be (0, None), for x1 and x[2] bounds should be (None, None). Can anyone be so kind to suggest me the correct way?
def objective(x, alpha, t):
#
return x[1] + (1 / (1 - alpha) * t) * np.sum(x[2])
def problem(val, t = 10, alpha = 0.9):
#
y = []
for simbolo in val.columns:
loc, scale = sts.gumbel_l.fit(val[simbolo])
y.append(sts.gumbel_l.rvs(loc, scale, t))
init = np.array(([1 / len(val.columns)] * len(val.columns), [1] * t, [0] * t))
constraints = [
{"type": "ineq", "fun": lambda x: x[2]},
{"type": "ineq", "fun": lambda x: np.dot(x[0].T, np.asarray(y)) + x[1] + x[2]}
]
bounds = ((0, None),) * len(val.columns)
args = (alpha, t)
res = opt.minimize(
objective,
x0 = init,
args = args,
bounds = bounds,
constraints = constraints
)
return res['x']
I tried to write the bounds as follow:
bounds = (((0, None),) * len(var.columns), ((None, None),) * len(var.columns), ((None, None),) * len(var.columns))
by I get the following error:
File "main.py", line 250, in <module>
problem(r)
File "port.py", line 152, in vanilla_cvar
res = opt.minimize(
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/scipy/optimize/_minimize.py", line 625, in minimize
return _minimize_slsqp(fun, x0, args, jac, bounds,
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/scipy/optimize/slsqp.py", line 315, in _minimize_slsqp
new_bounds = old_bound_to_new(bounds)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/scipy/optimize/_constraints.py", line 316, in old_bound_to_new
lb, ub = zip(*bounds)
ValueError: too many values to unpack (expected 2)
How can I fix code with error 'The array returned by a function changed size between calls' while using lmfit for minimization?
Please find my code below:
import numpy as np
import pandas as pd
import lmfit as lf
#model needs to be fitted
x0 = 75
def func(params, x, Tsky):
A = params['amp']
w = params['width']
t = params['thickness']
v0 = params['mid_freq']
b0 = params['b0']
b1 = params['b1']
b2 = params['b2']
b3 = params['b3']
b4 = params['b4']
B = (4 * (x - v0)**2. / w**2.) * np.log(-1./t * np.log((1 + np.exp(-t))/2))
T21 = -A * (1 - np.exp(-t * np.exp(B)))/(1 - np.exp(-t))
model = T21 + b0 * ((x/x0)**(-2.5 + b1 + b2 * np.log(x/x0))) * np.exp(-b3*(x/x0)**-2.) + b4 * (x/x0)**-2.
return (Tsky-model)
#read the data
df = pd.read_csv('figure1_plotdata.csv')
data_list = df.T.values.tolist()
xdata = np.array(data_list[0])
Tsky = np.array(data_list[2])
#initial value of the parameters
params = lf.Parameters()
params.add('amp', value=0.2)
params.add('width', value=10)
params.add('thickness', value=5)
params.add('mid_freq', value=70)
params.add('b0', value=500)
params.add('b1', value=-0.5)
params.add('b2', value=-0.5)
params.add('b3', value=-0.5)
params.add('b4', value=500)
#minimize the function
out = lf.minimize(func, params, args=(xdata, Tsky), method='leastsq', kws=None, iter_cb=None, scale_covar=True, nan_policy='omit', calc_covar=True)
print(lf.fit_report(out))
Here is the error message:
File "/home/ankita/Dropbox/Python/Bowman_work/min.py", line 81, in <module>
out = lf.minimize(func, params, args=(xdata, Tsky), method='leastsq', kws=None, iter_cb=None, scale_covar=True, nan_policy='omit', calc_covar=True)
File "/home/ankita/anaconda3/lib/python3.7/site-packages/lmfit-0.9.13-py3.7.egg/lmfit/minimizer.py", line 2300, in minimize
return fitter.minimize(method=method)
File "/home/ankita/anaconda3/lib/python3.7/site-packages/lmfit-0.9.13-py3.7.egg/lmfit/minimizer.py", line 1949, in minimize
return function(**kwargs)
File "/home/ankita/anaconda3/lib/python3.7/site-packages/lmfit-0.9.13-py3.7.egg/lmfit/minimizer.py", line 1492, in leastsq
lsout = scipy_leastsq(self.__residual, variables, **lskws)
File "/home/ankita/anaconda3/lib/python3.7/site-packages/scipy/optimize/minpack.py", line 394, in leastsq
gtol, maxfev, epsfcn, factor, diag)
**ValueError: The array returned by a function changed size between calls**
If you had used
out = lf.minimize(func, params,...,nan_policy='raise')
you would have seen an exception being raised telling you that there are NaNs. When you use nan_policy='omit', any such NaNs that are generated by your model are removed from the residual array, and so the size of the array changes between calls. The fitting cannot handle NaNs or changes to the size of the arrays -- you have to eliminate them.
In particular, np.log(x) is NaN when x<0. You have np.log() with a complicated argument that depends on the value of the parameter t. If that argument goes below 0 for some value of t, the model has NaNs and makes no sense.
You will have to ensure that this argument cannot be below 0. It might be that
use
params.add('thickness', value=5, min=0)
is sufficient. But you should examine your model in more detail and determine if that makes sense.
Your model looks quite complicated to me. I cannot guess where such a model derives from. Taking multiple np.exp() and np.log() is sort of asking for numerical instabilities. So, I don't know that simply forcing t to be positive will give a good fit, but it might point you in the right direction.
Just in case you arrived at this question and you are not using lf.minimize: in my case, the problem was that I was forgetting to define a variable as list when using np.vectorize.
When you vectorize a function, you need to define explicitly which variables are lists (or arrays or numpy arrays). If you pass a list to the vectorized function without defining it as such (that is, defining it as a parameter in the function that is vectorized), it will give you the error 'The array returned by a function changed size between calls'.
My code was:
my_fn = np.vectorize(lambda x1, x2, x3: _my_fn_(a, b, c,
x1, x2, x3, x4))
return my_fn(x1, x2, x3)
But x4 was a list (not a constant), so I needed to define it as a parameter in the lambda:
my_fn = np.vectorize(lambda x1, x2, x3, x4: _my_fn_(a, b, c,
x1, x2, x3, x4))
return my_fn(x1, x2, x3, x4)
I am trying to minimize (A_m-B*C_m)**2 where A is fixed matrix and B and C are variable matrices. The matrices have shapes (3,3,512,512), (3,3,512,1), and (1,1,512,512), respectively.
Any advice on where I am going wrong, or any advice in general on how to perform this optimization is appreciated.
Code
# objective function
def f(params, A):
total = 0
B, C = params
for k in range(0,512):
for m in range(0,512):
for n in range(0,9):
total += (A[k*m*n] - B[m*n] * C[k*m])**2
print(total)
return total
# create matrices
A = np.random.rand(3,3,512,512)
B = np.random.rand(3,3,512,1)
C = np.random.rand(1,1,512,512)
# flatten into 1d vectors
A = A.ravel()
B = B.ravel()
C = C.ravel()
# perform minimization
x = [B, C]
result = optimize.minimize(f, x, args=(A))
Output I get numbers for 'total' for a few iterations, and then it crashes:
407783.32968906895
407783.3247324555
407783.3244573258
407783.32968906895
2988640.705408569
2988640.705408569
2988640.797336188
2988640.7918476323
358476.57749339886
358476.57749339886
358476.57633670256
358476.5758499606
Traceback (most recent call last):
File "minimize.py", line 148, in <module>
result = optimize.minimize(f, x, args=(A))
File "/anaconda3/lib/python3.6/site-packages/scipy/optimize/_minimize.py", line 481, in minimize
return _minimize_bfgs(fun, x0, args, jac, callback, **options)
File "/anaconda3/lib/python3.6/site-packages/scipy/optimize/optimize.py", line 994, in _minimize_bfgs
rhok = 1.0 / (numpy.dot(yk, sk))
ValueError: operands could not be broadcast together with shapes (4608,) (262144,)
I'm having some trouble with python's complex_ode solver.
I'm trying to solve the following equation:
dy/dt = -iAy - icos(Omegat)By
where A and B are NxN arrays and the unknown y is an Nx1 array, i is the imaginary unit and Omega is a parameter.
Here's my code:
import numpy as np
from scipy.integrate import ode,complex_ode
N = 3 #linear matrix dim
Omega = 1.0 #parameter
# define symmetric matrices A and B
A = np.random.ranf((N,N))
A = (A + A.T)/2.0
B = np.random.ranf((N,N))
B = (B + B.T)/2.0
# define RHS of ODE
def f(t,y,Omega,A,B):
return -1j*A.dot(y)-1j*np.cos(Omega*t)*B.dot(y)
# define list of parameter
params=[Omega,A,B]
# choose solver: need complex_ode for this ODE
#solver = ode(f)
solver = complex_ode(f)
solver.set_f_params(*params)
solver.set_integrator("dop853")
# set initial value
v0 = np.zeros((N,),dtype=np.float64)
v0[0] = 1.0
# check that the function f works properly
print f(0,v0,Omega,A,B)
# solve-check the ODE
solver.set_initial_value(v0)
solver.integrate(10.0)
print solver.successful()
Running this script produces the error
capi_return is NULL
Call-back cb_fcn_in___user__routines failed.
Traceback (most recent call last):
File "ode_test.py", line 37, in <module>
solver.integrate(10.0)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/integrate/_ode.py", line 515, in integrate
y = ode.integrate(self, t, step, relax)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/integrate/_ode.py", line 388, in integrate
self.f_params, self.jac_params)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/integrate/_ode.py", line 946, in run
tuple(self.call_args) + (f_params,)))
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/integrate/_ode.py", line 472, in _wrap
f = self.cf(*((t, y[::2] + 1j * y[1::2]) + f_args))
TypeError: f() takes exactly 5 arguments (2 given)
If instead I use solver = ode(f), ie. the real-valued solver, it runs fine. Except that it doesn't solve the ODE I want which is complex-valued :(
I then tried to reduce the number of parameters by making the matrices A and B global variables. This way the only parameter the function f accepts is Omega. The error changes to
capi_return is NULL
Call-back cb_fcn_in___user__routines failed.
Traceback (most recent call last):
File "ode_test.py", line 37, in <module>
solver.integrate(10.0)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/integrate/_ode.py", line 515, in integrate
y = ode.integrate(self, t, step, relax)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/integrate/_ode.py", line 388, in integrate
self.f_params, self.jac_params)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/integrate/_ode.py", line 946, in run
tuple(self.call_args) + (f_params,)))
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/integrate/_ode.py", line 472, in _wrap
f = self.cf(*((t, y[::2] + 1j * y[1::2]) + f_args))
TypeError: 'float' object has no attribute '__getitem__'
where I figured out that float refers to the parameter Omega [by trying an integer]. Again, "ode" alone works in this case as well.
Last, I tried the same complex valued equation, but now A and B are just numbers. I tried to pass them both as parameters, i.e. params = [Omega,A,B], as well as making them global variables in which case params=[Omega]. The error is the
TypeError: 'float' object has no attribute '__getitem__'
error - the full error is the same as above. And once again this problem does not occur for the real-valued "ode".
I know zvode is an alternative, but it appears to become quite slow for large N. In the real problem I have, A is a diagonal matrix but B is a non-sparse full matrix.
Any insights are much appreciated! I'm interested both in (i) alternative ways to solve this complex-valued ODE with array-valued parameters, and (ii) how to get complex_ode to run :)
Thanks!
It seems like the link that Reti43 posted contains the answer, so let me put it here for the benefit of future users:
from scipy.integrate import complex_ode
import numpy as np
N = 3
Omega = 1.0;
class myfuncs(object):
def __init__(self, f, fargs=[]):
self._f = f
self.fargs=fargs
def f(self, t, y):
return self._f(t, y, *self.fargs)
def f(t, y, Omega,A,B):
return -1j*(A+np.cos(Omega*t)*B).dot(y)
A = np.random.ranf((N,N))
A = (A + A.T)/2.0
B = np.random.ranf((N,N))
B = (B + B.T)/2.0
v0 = np.zeros((N,),dtype=np.float64)
v0[0] = 1.0
t0 = 0
case = myfuncs(f, fargs=[Omega, A, B] )
solver = complex_ode(case.f)
solver.set_initial_value(v0, t0)
solver.integrate([10.0])
print solver.successful()
"""
t1 = 10
dt = 1
while solver.successful() and solver.t < t1:
solver.integrate(solver.t+dt)
print(solver.t, solver.y)
"""
Could maybe someone comment on why this trick does the job?
I have a set of numbers which cause error when I try to do a curve fitting to them. I'm quite certain I managed to do this before with the same numbers (I'm certain I did with other data sets). What is causing this error then?
The X, Y, Err values are (by order of appearance)
[0.0, 0.6931471805599453, 1.3862943611198906]
[-5.354761064902713, -6.190455611580044, -6.558604540577015]
[0.0014079400762288246, 0.0006083544693643583, 0.0002989970199491765]
and kappa is equal to 8
This is the function I try to fit (largely a + 2*x)
out = []
for x in X:
y = log(kappa)
y += 4*log(pi)
y += 2*x
y -= 2*log(2)
out.append(-y)
return np.array(out)
this is how I call curve_fit
popt,pcov = curve_fit(fitFunc1,self.X[0:3],self.Y[0:3],sigma=self.Err[0:3],p0=kappa)
and this is the error I get
popt,pcov = curve_fit(fitFunc1,self.X[0:3],self.Y[0:3],sigma=self.Err[0:3],p0=kappa)
File "/usr/lib/python2.7/dist-packages/scipy/optimize/minpack.py", line 506, in curve_fit
res = leastsq(func, p0, args=args, full_output=1, **kw)
File "/usr/lib/python2.7/dist-packages/scipy/optimize/minpack.py", line 355, in leastsq
gtol, maxfev, epsfcn, factor, diag)
minpack.error: Error occurred while calling the Python function named _weighted_general_function
edit 1
Added kappa value (8)
edit 2
Here is a minimal working example of this
#!/usr/bin/python
import numpy as np
from scipy.optimize import curve_fit
from math import log,pi
X = [0.0, 0.6931471805599453, 1.3862943611198906]
Y = [-5.354761064902713, -6.190455611580044, -6.558604540577015]
Err = [0.0014079400762288246, 0.0006083544693643583, 0.0002989970199491765]
kappa = 8
def func(X,kappa):
out = []
for x in X:
y = log(kappa)
y += 4*log(pi)
y += 2*x
y -= 2*log(2)
out.append(-y)
return np.array(out)
popt,pcov = curve_fit(func,X,Y,sigma=Err,p0=kappa)
First, convert your input arrays to numpy arrays. This allows you to use broadcasting in your func. In addition, you should check if kappa<=0 and return a bad fit value to prevent from evaluating outside the domain you are probably interested in:
import numpy as np
from scipy.optimize import curve_fit
X = np.array([0.0, 0.6931471805599453, 1.3862943611198906])
Y = np.array([-5.354761064902713, -6.190455611580044, -6.558604540577015])
Err = np.array([0.0014079400762288246, 0.0006083544693643583, 0.0002989970199491765])
kappa = 8.0
def func(X,kappa):
if kappa <=0: return np.inf
return -(np.log(kappa) + 4*np.log(np.pi) + 2*X - 2*np.log(2))
popt,pcov = curve_fit(func,X,Y,sigma=Err,p0=kappa)