pasing arguments in global optimization in scipy.optimize - python

I am trying to solve an optimization problem with scipy global optimization. I am using differential evolution.
code
def objective(x,*args):
x = np.append(x,args)
res = MLmodel.predict(x)
return res
fun_history = []
x_values = []
def callback(x,convergence):
fobj = objective(x)
x_values.append(x)
fun_history.append(fobj)
bounds = [(5.5,8.8),(29,40)]
load = (50,)
res = optimize.differential_evolution(objective,bounds,args=load,disp=True,callback = callback)
My objective function takes these three parameters as input and gives an output. I want to optimize on only the first two parameters. So the third parameter I am passing as an argument.
When I run the optimizer I get an error after the first run, which says
ValueError: operands could not be broadcast together with shapes (1,2) (3,) (1,2)
I guess the argument is not being appended to the x value from the second run.
Can someone help me with how to solve this problem?

Related

Parameter Optimization Using Minimize in Python

I have written the following two functions to calibrate a model :
The main function is:
def function_Price(para,y,t,T,tau,N,C):
# y= price array
# C = Auto and cross correlation array
# a= paramters need to be calibrated
a=para[0:]
temp=0
for j in range(N):
price_j = a[j]*C[j]*P[t:T-tau,j]
temp=temp+price_j
Price=temp
return Price
The objective function is :
def GError_function_Price(para,y,k,t,T,tau,N,C):
# k is the price need to be fitted
return sum((function_Price(para,y,t,T,tau,N,C)-k[t+tau:T]) ** 2)
Now, I am calling these two functions to do the optimization of the model:
import numpy as np
from scipy.optimize import minimize
# Prices (example)
y = np.array([[1,2,3,4,5,4], [4,5,6,7,8,9], [6,7,8,7,8,6], [13,14,15,11,12,19]])
# Correaltion (example)
Corr= np.array([[1,2,3,4,5,4], [4,5,6,7,8,9], [6,7,8,7,8,6], [13,14,15,11,12,19],[1,2,3,4,5,4],[6,7,8,7,8,6]])
# Define
tau=1
Size = y.shape
N = Size[1]
T = Size[0]
t=0
# initial Values
para=np.zeros(N)
# Bounds
B = np.zeros(shape=(N,2))
for n in range(N):
B[n][0]= float('-inf')
B[n][1]= float('inf')
# Calibration
A = np.zeros(shape=(N,N))
for i in range (N):
k=y[:,i] #fitted one
C=Corr[i,:]
parag=minimize(GError_function_Price,para,args=(y,Y,t,T,tau,N,C),method='SLSQP',bounds=B)
A[i,:]=parag.x
Once, I run the model, It should produce an N by N array of optimized values of paramters. But, except for the first column, it keeps zeros for the rest. Something is wrong.
Can you help me fix the problem, please?
I know how to do it in Matlab.
The following is Matlab Code :
main function
function Price=function_Price(para,P,t,T,tau,N,C)
a=para(:,:);
temp=0;
for j=1:N
price_j = a(j).*C(j).*P(t:T-tau,j);
temp=temp+price_j;
end
Price=temp;
end
The objective function:
function gerr=GError_function_Price(para,P,Y,t,T,tau,N,C)
gerr=sum((function_Price(para,P,t,T,tau,N,C)-Y(t+tau:T)).^2);
end
Now, I call these two functions in the following way:
P = [1,2,3,4,5,4;4,5,6,7,8,9;6,7,8,7,8,6;13,14,15,11,12,19];
AutoAndCrossCorr= [1,2,3,4,5,4;4,5,6,7,8,9;6,7,8,7,8,6;13,14,15,11,12,19;1,2,3,4,5,4;6,7,8,7,8,6];
tau=1;
Size = size(P);
N =6;
T =4;
t=1;
for i=1:N
Y=P(:,i); % fitted one
C=AutoAndCrossCorr(i,:);
para=zeros(1,N);
lb= repmat(-inf,N,1);
ub= repmat(inf ,N,1);
parag=fminsearchbnd(#(para)abs(GError_function_Price(para,P,Y,t,T,tau,N,C)),para,lb,ub);
a(i,:)=parag;
end
The problem seems to be that you're passing the result of a function call to minimize, rather than the function itself. The arguments get passed by the args parameter. So instead of:
minimize(GError_function_Price(para,y,k,t,T,tau,N,C),para,method='SLSQP',bounds=B)
the following should work
minimize(GError_function_Price,para,args=(y,k,t,T,tau,N,C),method='SLSQP',bounds=B)

How to include adjustable parameter in fsolve?

I'm getting familiar with fsolve in Python and I am having trouble including adjustable parameters in my system of nonlinear equations. This link seems to answer my question but I still get errors. Below is my code:
import scipy.optimize as so
def test(x,y,z):
eq1 = x**2+y**2-z
eq2 = 2*x+1
return [eq1,eq2]
z = 1 # Ajustable parameter
sol = so.fsolve(test , [-1,2] ,args=(z)) # Solution Array
print(sol) # Display Solution
The output gives
TypeError: test() missing 1 required positional argument: 'z'
When z is clearly defined as an argument. How do I include this adjustable parameter?
So before posting here I should have spent a little bit more time playing with it. Here is what I found
import scipy.optimize as so
import numpy as np
def test(variables,z): #Define function of variables and adjustable arg
x,y = variables #Declare variables
eq1 = x**2+y**2-1-z #Equation to solve #1
eq2 = 2*x+1 #Equation to solve #2
return [eq1,eq2] #Return equation array
z = 1 #Ajustable parameter
initial = [1,2] #Initial condition list
sol = so.fsolve(test , initial, args = (z)) #Call fsolve
print(np.array(sol)) #Display sol
With an output
[-0.5 1.32287566]
I'm not the best at analyzing code, but I think my issue was that I had mixed up my variables and arguments in test(x,y,z) such that it didn't know what I was trying to apply the initial guess to.
Anyway, I hope this helped someone.
edit: While I'm here, the test function is a circle of adjustable radius and a line that intersects it at two points.
If you wanted to find the positive and negative solutions, you'd need to pass your initial guess in as an array (someone asked a similar question here). Here is the updated version
z = 1
initial = [[-2,-1],[2,1]]
sol = []
for i in range(len(initial)):
sol.append(so.fsolve(test , initial[i], args = (z)))
print(np.array(sol))
The output is
[[-0.5 -1.32287566]
[-0.5 1.32287566]]

Solving an ODE with scipy.integrate.solve_bvp, the return of my function is not able to be broadcast into the appropriate array shape for solve_bvp

I am trying to solve a second order ODE with solve_bvp. I have split the second order ODE into a system of tow first oder ODEs. I have a changing set of constants depending on the x (mesh) value. So I am passing these as an array of shape (N,) into my function numdens. While trying to run solve_bvp I get the error that the returns have different shapes namely (N,) and (N-1,) and thus cannot be broadcast into one array. But when I check each return back manually outside of the function it has the shape (N,).
If I run the solver without my changing constants I get a solution akin to the right one.
import numpy as np
from scipy.integrate import solve_bvp,odeint
import matplotlib.pyplot as plt
E_0 = 1 * 0.0000016021773 #erg: gcm^2/s^2
m_H = 1.6*10**(-24) #g
c = 3e11 #cm
sigma_c = 2*10**(-23)
n_0 = 1*10**(20) #1/cm^3
v_0 = (2*E_0/m_H)**(0.5) #cm/s
T = 10**7
b = 20.3
n_eq = b*T**3
n_s = 2.03*10**(19)
Q = 1
def velocity(v,x):
dvdx = -sigma_c*n_0*v_0*((8*v_0*v-7*v**2-v_0**2)/(2*v*c))
return dvdx
n_num = 100
x_num = np.linspace(-1*10**(6),3*10**(6), n_num)
sol_velo = odeint(velocity,0.999999999999*v_0,x_num)
sol_new = np.reshape(sol_velo,n_num)
def constants(v):
D1 = (c*v/(3*n_0*v_0*sigma_c))
D2 = ((v**2-8*v_0*v+v_0**2)/(6*v))
D3 = sigma_c*n_0*v_0*((8*v_0*v-7*v**2-v_0**2)/(2*v*c))
return D1,D2,D3
def numdens(x,y):
v = sol_new
D1,D2,D3 = constants(v)
return np.vstack((y[1],(-D2*y[1]-D3*y[0]+Q*((1-y[0])/n_eq))/(D1)))
def bc_num(ya, yb):
return np.array([ya[0]-n_s,yb[0]-n_eq])
y_num = np.array([np.linspace(n_s, n_eq, n_num),np.linspace(n_s, n_eq, n_num)])
sol_num = solve_bvp(numdens, bc_num, x_num, y_num)
plt.plot(sol_num.x, sol_num.y[0], label='$n(x)$')
plt.plot(x_num, sol_velo-v_0/7, label='$v(x)$')
plt.yscale('log')
plt.grid(alpha=0.5)
plt.legend(framealpha=1)
plt.show()
You need to take into account that the BVP solver uses an adaptive mesh. That is, after refining the initial guess on the initial grid the solver identifies regions with overly large errors and creates new mesh nodes there. As far as I have seen, the opposite is not implemented, even if it may be in some applications sensible to reduce the number of mesh nodes on especially "nice" segments.
Thus what you are doing the the numdens function is incomprehensible, it has to function exactly like any other function that you would pass to an ODE solver. If I had to propose some fast fix, and without knowing what the underlying problem is that you want to solve, I would change the assignment of v to
v = np.interp(x,x_num,sol_velo)
as that should at least produce an array of the correct format.

Scipy odeint giving index out of bounds errors

I am trying to solve a differential equation in python using Scipy's odeint function. The equation is of the form dy/dt = w(t) where w(t) = w1*(1+A*sin(w2*t)) for some parameters w1, w2, and A. The code I've written works for some parameters, but for others I get given index out of bound errors.
Here's some example code that works
import numpy as np
import scipy.integrate as integrate
t = np.arange(1000)
w1 = 2*np.pi
w2 = 0.016*np.pi
A = 1.0
w = w1*(1+A*np.sin(w2*t))
def f(y,t0):
return w[t0]
y = integrate.odeint(f,0,t)
Here's some example code that doesn't work
import numpy as np
import scipy.integrate as integrate
t = np.arange(1000)
w1 = 0.3*np.pi
w2 = 0.005*np.pi
A = 0.15
w = w1*(1+A*np.sin(w2*t))
def f(y,t0):
return w[t0]
y = integrate.odeint(f,0,t)
The only thing that changes between these is that the three parameters w1, w2, and A are smaller in the second, but the second one always gives me the following error
line 13, in f
return w[t0]
IndexError: index 1001 is out of bounds for axis 0 with size 1000
This error continues even after restarting python and running the second code first. I've tried with other parameters, some seem to work, but others give me different index out of bounds errors. Some say 1001 is out of bounds, some say 1000, some say 1008, ect.
Changing the initial condition on y (the second input for odeint, which I have as 0 on the above codes) also changes the number on the index error, so it might be that I'm misunderstanding what to put here. I wasn't told what the initial conditions should be other than that y is used as a phase of a signal, so I presumed it to be initially 0.
What you want to do is
def w(t):
return w1*(1+A*np.sin(w2*t))
def f(y,t0):
return w(t0)
Array indices are typically integers, time arguments and values of solutions of differential equations are typically real numbers. Thus there is some conceptual difficulty in invoking w[t0].
You might also try to integrate directly the function w, there is no inherent difficulty in this example.
As for coupled systems, you solve them as coupled systems.
def w(t):
return w1*(1+A*np.sin(w2*t))
def f(y,t):
wt = w(t)
return np.array([ wt, wt*sin(y[1]-y[0]) ])

scipy fmin operands could not be broadcast together with shapes

i'm trying to learn about optimization in Python so i've written some code to test out the fmin function.
However i keep receiving the following error:
ValueError: operands could not be broadcast together with shapes (1,2) (100,)
I can tell the issue is to do with the dimensions of my arguments but I'm not sure how to rectify it. Rather than a lambda function I also tried to def a function but I still get the same error.
I'm sure it's something pretty basic but I can't seem to understand it. Any help wold be greatly appreciated!
import numpy as np
import pandas as pd
from scipy.stats.distributions import norm
from scipy.optimize import fmin
x = np.random.normal(size=100)
norm_1 = lambda theta,x: -(np.log(norm.pdf(x,theta[0],theta[1]))).sum()
def norm_2(theta,x):
mu = theta[0]
sigma = theta[1]
ll = np.log(norm.pdf(x,mu,sigma)).sum()
return -ll
fmin(norm_1,np.array([0,1]),x)
fmin(norm_2,np.array([0,1]),x)
The docs for fmin say:
Definition: fmin(func, x0, args=(), xtol=0.0001, ftol=0.0001, maxiter=None, maxfun=None, full_output=0, disp=1, retall=0, callback=None)
...
args : tuple, optional
Extra arguments passed to func, i.e. ``f(x,*args)``.
Therefore, the third argument, args, should be a tuple:
In [45]: fmin(norm_1,np.array([0,1]),(x,))
Warning: Maximum number of function evaluations has been exceeded.
Out[45]: array([-0.02405078, 1.0203125 ])
(x, ) is a tuple containing one element, x.
The docs say f(x, *args) gets called. Which mean in your case
norm_1(np.array([0,1]), *(x,))
will get called, which is equivalent to
norm_1(np.array([0,1]), x)

Categories