I am trying to minimize (A_m-B*C_m)**2 where A is fixed matrix and B and C are variable matrices. The matrices have shapes (3,3,512,512), (3,3,512,1), and (1,1,512,512), respectively.
Any advice on where I am going wrong, or any advice in general on how to perform this optimization is appreciated.
Code
# objective function
def f(params, A):
total = 0
B, C = params
for k in range(0,512):
for m in range(0,512):
for n in range(0,9):
total += (A[k*m*n] - B[m*n] * C[k*m])**2
print(total)
return total
# create matrices
A = np.random.rand(3,3,512,512)
B = np.random.rand(3,3,512,1)
C = np.random.rand(1,1,512,512)
# flatten into 1d vectors
A = A.ravel()
B = B.ravel()
C = C.ravel()
# perform minimization
x = [B, C]
result = optimize.minimize(f, x, args=(A))
Output I get numbers for 'total' for a few iterations, and then it crashes:
407783.32968906895
407783.3247324555
407783.3244573258
407783.32968906895
2988640.705408569
2988640.705408569
2988640.797336188
2988640.7918476323
358476.57749339886
358476.57749339886
358476.57633670256
358476.5758499606
Traceback (most recent call last):
File "minimize.py", line 148, in <module>
result = optimize.minimize(f, x, args=(A))
File "/anaconda3/lib/python3.6/site-packages/scipy/optimize/_minimize.py", line 481, in minimize
return _minimize_bfgs(fun, x0, args, jac, callback, **options)
File "/anaconda3/lib/python3.6/site-packages/scipy/optimize/optimize.py", line 994, in _minimize_bfgs
rhok = 1.0 / (numpy.dot(yk, sk))
ValueError: operands could not be broadcast together with shapes (4608,) (262144,)
Related
Using scipy.optimise (code below) - for a battery optimisation problem
Getting this error:
TypeError: numpy boolean subtract, the - operator, is not supported, use the bitwise_xor, the ^ operator, or the logical_xor function instead.
Which came from the minimize function directly, so I'm not sure exactly where its coming from.
line 70, in <module>
sol = minimize(objective, B0, method='SLSQP', \
File "AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\scipy\optimize\_minimize.py", line 708, in minimize
res = _minimize_slsqp(fun, x0, args, jac, bounds,
File "AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\scipy\optimize\_slsqp_py.py", line 418, in _minimize_slsqp
a = _eval_con_normals(x, cons, la, n, m, meq, mieq)
File "AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\scipy\optimize\_slsqp_py.py", line 486, in _eval_con_normals
a_eq = vstack([con['jac'](x, *con['args'])
File "AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\scipy\optimize\_slsqp_py.py", line 486, in <listcomp>
a_eq = vstack([con['jac'](x, *con['args'])
File "AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\scipy\optimize\_slsqp_py.py", line 301, in cjac
return approx_derivative(fun, x, method='2-point',
File "AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\scipy\optimize\_numdiff.py", line 505, in approx_derivative
return _dense_difference(fun_wrapped, x0, f0, h,
File "AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\scipy\optimize\_numdiff.py", line 576, in _dense_difference
df = fun(x) - f0
TypeError: numpy boolean subtract, the `-` operator, is not supported, use the bitwise_xor, the `^` operator, or the logical_xor function instead.
TIME = 2
MAX_BATTERY_CHARGE_RATE = 4 #kwh
MAX_BATTERY_CAPACITY = 12 #kw
INITIAL_BATTERY_CHARGE = 5
BUY_RATE = [0.1, 0.3]
SELL_RATE = [0, 0]
L = [2, 5]
S = [1, 1]
def objective(B):
cost = 0
for i in range(TIME):
isum = (L[i] - S[i] + B[i])
if isum > 0 :
cost += BUY_RATE[i] * isum
else:
cost += SELL_RATE[i] * isum
return cost
# BOUNDS
# cannot exceed charge rate
b = (-1 * MAX_BATTERY_CHARGE_RATE, MAX_BATTERY_CHARGE_RATE)
bnds = [b for i in range(TIME)]
print(bnds)
# CONSTRAINTS
# Sum of B up to any point in time cannot be less than 0
# or greater than battery capacity
def constraint1(x):
for i in range(TIME + 1):
array = x[:i]
print(array)
if (np.sum(array) < 0) or (np.sum(array) > MAX_BATTERY_CAPACITY):
return False
return True
con1 = {'type': 'ineq', 'fun': constraint1}
# Battery charge at initial time is set
def constraint2(x):
return x[0] == INITIAL_BATTERY_CHARGE
con2 = {'type': 'eq', 'fun': constraint2}
cons = [con1, con2]
# SOLUTION
B0 = np.ones(TIME)
sol = minimize(objective, B0, method='SLSQP', \
bounds=bnds, constraints=cons)
print(sol)
Is the SLSQP method compatible with functions that return boolean values true and false? I would rework function constraint1 to return real values instead of boolean values. If SLSQP is compatible with boolean functions, could you point me to the documentation that states this?
When I saw this message, "df = fun(x) - f0 TypeError: numpy boolean subtract," I found this odd because why would the computation of the derivative involve substraction of boolean values. After looking through your code, I noticed that the function constraint1 is returning boolean values true and false. Traditionally, the objective and constraint functions return real values. I would rework function constraint1 to return real values.
im trying to estimate the gjr garch model using this code below and the dataset im using is the bitcoin returns (daily) total of 1600 observations. im getting this error ''index 1 is out of bounds for axis 0 with size 1''. and above it it says :
File "C:\Users\georgios\Downloads\untitled1.py", line 97, in
estimates = fmin_slsqp(gjr_garch_likelihood, startingVals,
File "C:\Users\georgios\Nieuwe map\lib\site-packages\scipy\optimize\slsqp.py", line 207, in fmin_slsqp
res = _minimize_slsqp(func, x0, args, jac=fprime, bounds=bounds,
File "C:\Users\georgios\Nieuwe map\lib\site-packages\scipy\optimize\slsqp.py", line 375, in _minimize_slsqp
sf = _prepare_scalar_function(func, x, jac=jac, args=args, epsilon=eps,
File "C:\Users\georgios\Nieuwe map\lib\site-packages\scipy\optimize\optimize.py", line 261, in _prepare_scalar_function
sf = ScalarFunction(fun, x0, args, grad, hess,
File "C:\Users\georgios\Nieuwe map\lib\site-packages\scipy\optimize_differentiable_functions.py", line 136, in init
self._update_fun()
File "C:\Users\georgios\Nieuwe map\lib\site-packages\scipy\optimize_differentiable_functions.py", line 226, in _update_fun
self._update_fun_impl()
File "C:\Users\georgios\Nieuwe map\lib\site-packages\scipy\optimize_differentiable_functions.py", line 133, in update_fun
self.f = fun_wrapped(self.x)
File "C:\Users\georgios\Nieuwe map\lib\site-packages\scipy\optimize_differentiable_functions.py", line 130, in fun_wrapped
return fun(x, *args)
File "C:\Users\georgios\Downloads\untitled1.py", line 21, in gjr_garch_likelihood
sigma2[t] = (omega + alpha * eps[t-1]**2
File "C:\Users\georgios\Nieuwe map\lib\site-packages\pandas\core\series.py", line 977, in setitem
values[key] = value
IndexError: index 1 is out of bounds for axis 0 with size 1
My dataset is fine from
What I see , I only have one excel file with only one column with the returns
And the garch model is Univariate
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from numpy import size, log, pi, sum, array, zeros, diag, mat, asarray, sqrt, \
copy
from numpy.linalg import inv
from scipy.optimize import fmin_slsqp
def gjr_garch_likelihood(parameters, data, sigma2, out=None):
''' Returns negative log-likelihood for GJR-GARCH(1,1,1) model.'''
mu = parameters[0]
omega = parameters[1]
alpha = parameters[2]
gamma = parameters[3]
beta = parameters[4]
T = size(data,0)
eps = data - mu
# Data and sigma2 are T by 1 vectors
for t in range(1,T):
sigma2[t] = (omega + alpha * eps[t-1]**2
+ gamma * eps[t-1]**2 * (eps[t-1]<0) + beta * sigma2[t-1])
logliks = 0.5*(log(2*pi) + log(sigma2) + eps**2/sigma2)
loglik = sum(logliks)
if out is None:
return loglik
else:
return loglik, logliks, copy(sigma2)
def gjr_constraint(parameters, data, sigma2, out=None):
''' Constraint that alpha+gamma/2+beta<=1'''
alpha = parameters[2]
gamma = parameters[3]
beta = parameters[4]
return array([1-alpha-gamma/2-beta])
def hessian_2sided(fun, theta, args):
f = fun(theta, *args)
h = 1e-5*np.abs(theta)
thetah = theta + h
h = thetah - theta
K = size(theta,0)
h = np.diag(h)
fp = zeros(K)
fm = zeros(K)
for i in range(K):
fp[i] = fun(theta+h[i], *args)
fm[i] = fun(theta-h[i], *args)
fpp = zeros((K,K))
fmm = zeros((K,K))
for i in range(K):
for j in range(i,K):
fpp[i,j] = fun(theta + h[i] + h[j], *args)
fpp[j,i] = fpp[i,j]
fmm[i,j] = fun(theta - h[i] - h[j], *args)
fmm[j,i] = fmm[i,j]
hh = (diag(h))
hh = hh.reshape((K,1))
hh = hh # hh.T
H = zeros((K,K))
for i in range(K):
for j in range(i,K):
H[i,j] = (fpp[i,j] - fp[i] - fp[j] + f
+ f - fm[i] - fm[j] + fmm[i,j])/hh[i,j]/2
H[j,i] = H[i,j]
return H
# Import data
FTSEreturn = pd.read_csv('1.csv')
# Starting values
startingVals = array([FTSEreturn.mean(),
FTSEreturn.var() * .01,
.03, .09, .90])
# Estimate parameters
finfo = np.finfo(np.float64)
bounds = [(-10*FTSEreturn.mean(), 10*FTSEreturn.mean()),
(finfo.eps, 2*FTSEreturn.var() ),
(0.0,1.0), (0.0,1.0), (0.0,1.0)]
T = FTSEreturn.shape[0]
sigma2 = T * FTSEreturn.var()
# Pass a NumPy array, not a pandas Series
args = (np.asarray(FTSEreturn), sigma2)
estimates = fmin_slsqp(gjr_garch_likelihood, startingVals,
f_ieqcons=gjr_constraint, bounds = bounds,
args = args)
loglik, logliks, sigma2final = gjr_garch_likelihood(estimates, FTSEreturn,
sigma2, out=True)
step = 1e-5 * estimates
scores = zeros((T,5))
for i in range(5):
h = step[i]
delta = np.zeros(5)
delta[i] = h
loglik, logliksplus, sigma2 = gjr_garch_likelihood(estimates + delta, \
np.asarray(FTSEreturn), sigma2, out=True)
loglik, logliksminus, sigma2 = gjr_garch_likelihood(estimates - delta, \
np.asarray(FTSEreturn), sigma2, out=True)
scores[:,i] = (logliksplus - logliksminus)/(2*h)
I = (scores.T # scores)/T
J = hessian_2sided(gjr_garch_likelihood, estimates, args)
J = J/T
Jinv = mat(inv(J))
vcv = Jinv*mat(I)*Jinv/T
vcv = asarray(vcv)
output = np.vstack((estimates,sqrt(diag(vcv)),estimates/sqrt(diag(vcv)))).T
print('Parameter Estimate Std. Err. T-stat')
param = ['mu','omega','alpha','gamma','beta']
for i in range(len(param)):
print('{0:<11} {1:>0.6f} {2:0.6f} {3: 0.5f}'.format(param[i],
output[i,0], output[i,1], output[i,2]))
any help would be appreciated because im stuck on this one month now and i cant solve it
The error says there's an indexing error during a setitem in
sigma2[t] = (omega + alpha * eps[t-1]**2
In other words, t is too large for the array sigma2.
That's in the gjr_garch_likelihood function. t is an iterator that starts at 1. sigma2 is an argument. So we need to look at how the function is called, and the corresponding argument is.
That's more complicated, since it's a function used by fmin_slsqp. So the next step is to review that function's docs, to understand how it calls the func, and especially what arguments it provides to your gjr.... I won't do that for you!
But it's a good idea when encountering errors like this to add some diagnostic prints to your gjr... function. You need to clearly understand what gets passed to it, paying particular attention to array shapes. Don't make assumptions. Verify.
You may need to change gjr ... to accomodate the arguments, or you may need to modify how you call fmin....
Question - how much of this did you already figure out in the past month of struggle? If you already figured this out, you should have included that information in the question.
I want to use scipy.optimize.minimize to solve for a set of parameters by minimizing an error function.
The function called "error" returns the squared error for the function that I am trying to find z1,z2,z3(the parameters) for.
I have an array of x(called "b" in the function) and y(called "real" in the function) values.
The code below works fine if I set x and y to some integer, but not if I try to pass in an array of x and y values, to act as the variable "b" and "real" in the equation to be minimized.
Trying to pass in an array of X and Y values results in the error pasted below.
Is there a way to pass in arrays to act as a variable in an equation for the minimize function, instead of just a single integer?
Here is what my code looks like:
import numpy as np
import pandas as pd
from scipy.optimize import minimize
#dataset file, with a column named x,y with 1000 rows
f = pd.read_csv('Test_Data_.txt', sep='\t')
#initial guess
x0 = [1, 2, 3]
#f['x'] and f['y'] are columns with 1000 rows
#x = f['x'].values
#y = f['y'].values
x = 1 #these parameters work fine
y = 4
#a function called inside the function to be minimized
def est(z1, z2, z3, b):
return z1 * b**2 + z2 * b + z3
#function to minimize
def error(x, real, b):
return (real - est(x[0], x[1], x[2], b))**2
print(minimize(error, x0, args = ( x, y), method='BFGS', tol=1e-6))
Feeding in the array of x and y values produces the error:
Traceback (most recent call last):
File "problem1.py", line 24, in <module>
minimize(error, x0, args = ( np.array(list(f['y'].values)), np.array(list(f['x'].values))), method='BFGS', tol=1e-6)
File "/usr/local/lib/python3.5/dist-packages/scipy/optimize/_minimize.py", line 595, in minimize
return _minimize_bfgs(fun, x0, args, jac, callback, **options)
File "/usr/local/lib/python3.5/dist-packages/scipy/optimize/optimize.py", line 970, in _minimize_bfgs
gfk = myfprime(x0)
File "/usr/local/lib/python3.5/dist-packages/scipy/optimize/optimize.py", line 300, in function_wrapper
return function(*(wrapper_args + args))
File "/usr/local/lib/python3.5/dist-packages/scipy/optimize/optimize.py", line 730, in approx_fprime
return _approx_fprime_helper(xk, f, epsilon, args=args)
File "/usr/local/lib/python3.5/dist-packages/scipy/optimize/optimize.py", line 670, in _approx_fprime_helper
grad[k] = (f(*((xk + d,) + args)) - f0) / d[k]
ValueError: setting an array element with a sequence.
I'm having some trouble with python's complex_ode solver.
I'm trying to solve the following equation:
dy/dt = -iAy - icos(Omegat)By
where A and B are NxN arrays and the unknown y is an Nx1 array, i is the imaginary unit and Omega is a parameter.
Here's my code:
import numpy as np
from scipy.integrate import ode,complex_ode
N = 3 #linear matrix dim
Omega = 1.0 #parameter
# define symmetric matrices A and B
A = np.random.ranf((N,N))
A = (A + A.T)/2.0
B = np.random.ranf((N,N))
B = (B + B.T)/2.0
# define RHS of ODE
def f(t,y,Omega,A,B):
return -1j*A.dot(y)-1j*np.cos(Omega*t)*B.dot(y)
# define list of parameter
params=[Omega,A,B]
# choose solver: need complex_ode for this ODE
#solver = ode(f)
solver = complex_ode(f)
solver.set_f_params(*params)
solver.set_integrator("dop853")
# set initial value
v0 = np.zeros((N,),dtype=np.float64)
v0[0] = 1.0
# check that the function f works properly
print f(0,v0,Omega,A,B)
# solve-check the ODE
solver.set_initial_value(v0)
solver.integrate(10.0)
print solver.successful()
Running this script produces the error
capi_return is NULL
Call-back cb_fcn_in___user__routines failed.
Traceback (most recent call last):
File "ode_test.py", line 37, in <module>
solver.integrate(10.0)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/integrate/_ode.py", line 515, in integrate
y = ode.integrate(self, t, step, relax)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/integrate/_ode.py", line 388, in integrate
self.f_params, self.jac_params)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/integrate/_ode.py", line 946, in run
tuple(self.call_args) + (f_params,)))
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/integrate/_ode.py", line 472, in _wrap
f = self.cf(*((t, y[::2] + 1j * y[1::2]) + f_args))
TypeError: f() takes exactly 5 arguments (2 given)
If instead I use solver = ode(f), ie. the real-valued solver, it runs fine. Except that it doesn't solve the ODE I want which is complex-valued :(
I then tried to reduce the number of parameters by making the matrices A and B global variables. This way the only parameter the function f accepts is Omega. The error changes to
capi_return is NULL
Call-back cb_fcn_in___user__routines failed.
Traceback (most recent call last):
File "ode_test.py", line 37, in <module>
solver.integrate(10.0)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/integrate/_ode.py", line 515, in integrate
y = ode.integrate(self, t, step, relax)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/integrate/_ode.py", line 388, in integrate
self.f_params, self.jac_params)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/integrate/_ode.py", line 946, in run
tuple(self.call_args) + (f_params,)))
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/integrate/_ode.py", line 472, in _wrap
f = self.cf(*((t, y[::2] + 1j * y[1::2]) + f_args))
TypeError: 'float' object has no attribute '__getitem__'
where I figured out that float refers to the parameter Omega [by trying an integer]. Again, "ode" alone works in this case as well.
Last, I tried the same complex valued equation, but now A and B are just numbers. I tried to pass them both as parameters, i.e. params = [Omega,A,B], as well as making them global variables in which case params=[Omega]. The error is the
TypeError: 'float' object has no attribute '__getitem__'
error - the full error is the same as above. And once again this problem does not occur for the real-valued "ode".
I know zvode is an alternative, but it appears to become quite slow for large N. In the real problem I have, A is a diagonal matrix but B is a non-sparse full matrix.
Any insights are much appreciated! I'm interested both in (i) alternative ways to solve this complex-valued ODE with array-valued parameters, and (ii) how to get complex_ode to run :)
Thanks!
It seems like the link that Reti43 posted contains the answer, so let me put it here for the benefit of future users:
from scipy.integrate import complex_ode
import numpy as np
N = 3
Omega = 1.0;
class myfuncs(object):
def __init__(self, f, fargs=[]):
self._f = f
self.fargs=fargs
def f(self, t, y):
return self._f(t, y, *self.fargs)
def f(t, y, Omega,A,B):
return -1j*(A+np.cos(Omega*t)*B).dot(y)
A = np.random.ranf((N,N))
A = (A + A.T)/2.0
B = np.random.ranf((N,N))
B = (B + B.T)/2.0
v0 = np.zeros((N,),dtype=np.float64)
v0[0] = 1.0
t0 = 0
case = myfuncs(f, fargs=[Omega, A, B] )
solver = complex_ode(case.f)
solver.set_initial_value(v0, t0)
solver.integrate([10.0])
print solver.successful()
"""
t1 = 10
dt = 1
while solver.successful() and solver.t < t1:
solver.integrate(solver.t+dt)
print(solver.t, solver.y)
"""
Could maybe someone comment on why this trick does the job?
I have the following function definition of a 2D Gaussian:
# Return a gaussian distribution at an angle alpha from the x-axis
# from astroML for use with curve_fit
def mult_gaussFun_Fit((x,y),*m):
A,x0,y0,varx,vary,rho,alpha = m
X,Y = np.meshgrid(x,y)
assert rho != 1
a = 1/(2*(1-rho**2))
Z = A*np.exp(-a*((X-x0)**2/(varx)+(Y-y0)**2/(vary)-(2*rho/(np.sqrt(varx*vary)))*(X-x0)*(Y-y0)))
return Z.ravel()
I use the following code to attempt a curve_fit of data drawn from a bivariate gaussian that is converted to a 2D histogram. I am receiving broadcast errors and I am not sure as to why this is happening.
import numpy as np
from scipy.optimize import curve_fit
import matplotlib.pyplot as plt
import gauss
import plotutils
# Produce a number of points in x-y from 1 distribution.
mean = [0,0]
cov = [[3,0],[0,1]]
N = 3000
x,y = np.random.multivariate_normal(mean,cov,N).T
# Prep bins for histogram
bin_size = 0.2
max_edge = 2.5*(np.sqrt(cov[0][0])+np.sqrt(cov[1][1]))
min_edge = -max_edge
bin_num = (max_edge-min_edge)/bin_size
bin_numPlus1 = bin_num + 1
bins = np.linspace(min_edge,max_edge,bin_numPlus1)
# Produce 2D histogram
H,xedges,yedges = np.histogram2d(x,y,bins,normed=False)
bin_centers_x = (xedges[:-1]+xedges[1:])/2.0
bin_centers_y = (yedges[:-1]+yedges[1:])/2.0
# Initial Guess
p0 = (H.max(),mean[0],mean[1],cov[0][0],cov[1][1],0.5,np.pi/4)
# Curve Fit parameters
coeff, var_matrix = curve_fit(gauss.mult_gaussFun_Fit,(bin_centers_x,bin_centers_y),H,p0=p0)
The error is:
Traceback (most recent call last):
File "/home/luis/Documents/SRC2014/galsim_work/2D_Gaussian_Estimate.py", line 44, in <module>
coeff, var_matrix = curve_fit(gauss.mult_gaussFun_Fit,(bin_centers_x,bin_centers_y),H,p0=p0)
File "/usr/local/lib/python2.7/dist-packages/scipy/optimize/minpack.py", line 555, in curve_fit
res = leastsq(func, p0, args=args, full_output=1, **kw)
File "/usr/local/lib/python2.7/dist-packages/scipy/optimize/minpack.py", line 369, in leastsq
shape, dtype = _check_func('leastsq', 'func', func, x0, args, n)
File "/usr/local/lib/python2.7/dist-packages/scipy/optimize/minpack.py", line 20, in _check_func
res = atleast_1d(thefunc(*((x0[:numinputs],) + args)))
File "/usr/local/lib/python2.7/dist-packages/scipy/optimize/minpack.py", line 445, in _general_function
return function(xdata, *params) - ydata
ValueError: operands could not be broadcast together with shapes (4624) (68,68)
I simply needed to perform
H = H.ravel()
and that solves it.