I need help on minimizing function in python using three variable constraints.
I have posted the code which is giving me error. If you would like I can post the entire code to to show the mathematical calculation.:
# the time-series data.
coeff = [0.2, 0.3, 0.4]
x =[146, 96, 59, 133, 192, 127, 79, 186, 272, 155, 98, 219]
test = y(x,coeff)
print("x : ", x)
print("y : ",test)
result = minimize(mape, coeff, (x,), bounds =[(0,1),(0,1), (0,1)], method='SLSQP')
opt = result.x
print("opt : ", result.x)
This is my code:
from __future__ import division
import numpy as np
from scipy.optimize import minimize
#coeffList[0] = alpha
#coeffList[1] = beta
#coeffList[2] =gamma
def mape(x, coeffList):
diff = abs(y(x,coeffList)-x)
print("np.mean(diff/x) : ", np.mean(diff/x))
return np.mean(diff/x)
#Holt Winters-Multiplicative
def y(x, coeffList , debug=True):
c =4
#Compute initial b and intercept using the first two complete c periods.
xlen =len(x)
print("xlen : ", xlen)
#if xlen % c !=0:
# return None
fc =float(c)
xbar2 =sum([x[i] for i in range(c, 2 * c)])/ fc
print("xbar2 : ",xbar2)
xbar1 =sum([x[i] for i in range(c)]) / fc
print("xbar1 : ", xbar1)
b0 =(xbar2 - xbar1) / fc
if debug: print ("b0 = ", b0)
#Compute for the level estimate a0 using b0 above.
tbar =sum(i for i in range(1, c+1)) / fc
print("tbar : ",tbar)
a0 =xbar1 - b0 * tbar
if debug: print ("a0 = ", a0)
#Compute for initial indices - seasonality
I =[x[i] / (a0 + (i+1) * b0) for i in range(0, xlen)]
if debug: print ("Initial indices = ", I)
S=[0] * (xlen+ c)
for i in range(c):
S[i] =(I[i] + I[i+c]) / 2.0
print ("S[",i,"]=", S[i])
#Normalize so S[i] for i in [0, c) will add to c.
tS =c / sum([S[i] for i in range(c)])
print("tS : ", tS)
for i in range(c):
S[i] *=tS
if debug: print ("Normalized S[",i,"]=", S[i])
# Holt - winters proper ...
if debug: print( "Use Holt Winters formulae")
At =a0
Bt =b0
#y =[0] * (xlen)
y = np.empty(len(x),float)
for i in range(xlen):
Atm1 =At
Btm1 =Bt
At =coeffList[0] * x[i] / S[i] + (1.0-coeffList[0]) * (Atm1 + Btm1)
Bt =coeffList[1] * (At - Atm1) + (1- coeffList[1]) * Btm1
S[i+c] =coeffList[2] * x[i] / At + (1.0 - coeffList[2]) * S[i]
y[i]=(a0 + b0 * (i+1)) * S[i]
#print ("i=", i+1, "y=", y[i], "S=", S[i], "(level)Atm1=", Atm1, "(trend)Btm1=",Btm1, "(level)At=", At, "Bt=", Bt, "S[i+c]=", S[i+c], "y[i]=", y[i])
print ("i=", i+1, "y=", y[i], "S=", S[i], "(level)At=", At, "Bt=", Bt, "y[i]=", y[i])
#coeffList[0] = alpha
#coeffList[1] = beta
#coeffList[2] =gamma
return y
#print (i,y[i], F[i])
#Forecast for next c periods:
#for m in range(c):
#print( "forecast:", (At + Bt* (m+1))* S[ylen + m])
# the time-series data.
coeff = [0.2, 0.3, 0.4]
x =[146, 96, 59, 133, 192, 127, 79, 186, 272, 155, 98, 219]
bnds = ((0,1), (0,1), (0,1))
coeff = [0.2, 0.3, 0.4]
test = y(x,coeff)
print("x : ", x)
print("y : ",test)
#cons = ({'type' :'alpha', 'fun' :lambda x: np.array(x[0]<=1 and x[0]>=0)})
result = minimize(mape, coeff, (x,), method ="L-BFGS-B", bounds =bnds)
opt = result.x(0)
print("opt : ", result.x)
This is the error message. The function without minimization function works just fine.
Traceback (most recent call last):
File "C:\Users\gelalmp\Desktop\Bibha Gelal_SD\testing_Optimization_HWM.py", line 100, in <module>
result = minimize(mape, coeff, (x,), method ="L-BFGS-B", bounds =bnds)
File "C:\Python27\lib\site-packages\scipy\optimize\_minimize.py", line 380, in minimize
callback=callback, **options)
File "C:\Python27\lib\site-packages\scipy\optimize\lbfgsb.py", line 314, in _minimize_lbfgsb
f, g = func_and_grad(x)
File "C:\Python27\lib\site-packages\scipy\optimize\lbfgsb.py", line 258, in func_and_grad
f = fun(x, *args)
File "C:\Users\gelalmp\Desktop\Bibha Gelal_SD\testing_Optimization_HWM.py", line 12, in mape
diff = abs(y(x,coeffList)-x)
File "C:\Users\gelalmp\Desktop\Bibha Gelal_SD\testing_Optimization_HWM.py", line 30, in y
xbar2 =sum([x[i] for i in range(c, 2 * c)])/ fc
IndexError: index out of bounds
Change your last 4 lines to:
M=lambda p1, p2: mape(p2, p1)
result = minimize(M, coeff, (x,), method ="L-BFGS-B", bounds =bnds)
opt = result['x']
print("opt : ", result['x'])
And it should work now, need explanations? I get the optimization result ('opt : ', array([ 0.45330204, 0.26761714, 0. ]))
The lambda function reverses the order of how the parameters are supplied to mape. As you are attempting to find the coeff that minimize the mape() given a fixed x, the target function should take coeff first and x second, which is not the case for mape.
To your comment question: I thought you are using L-BFGS-B in your code. The difference are explained here: http://docs.scipy.org/doc/scipy/reference/tutorial/optimize.html#tutorial-sqlsp. I have to admit I don't too much detailed of SLSQP as that was long time ago in graduate school. BFGS is more common and every textbook explains it. L-BFGS-B supports bound constrained minimization. SLSQP supports bounds, as well as equality and inequality constraints. So, SLSQP can function when L-BFGS-B can't. See, http://scipy-lectures.github.io/advanced/mathematical_optimization/index.html?utm_source=twitterfeed&utm_medium=twitter.
Related
I want to use scipy.optimize.minimize on a function with an array type variable input. This is what I hope to do.
I have the following signal,
import numpy as np
time = np.linspace(0, 1, 501)
data = np.cos(2 * np.pi * 4 * time) + np.cos(2 * np.pi * 9 * time) + np.cos(2 * np.pi * 20 * time)
noise = np.sqrt(1 / 25) * np.random.randn(501)
signal = data + noise
and I am hoping to find a curve fit for this signal. Since I created the data myself, I know that a sum of cosine functions will work for this. So the function that I hope to optimize is the following:
def cos_sum(x, P):
assert isinstance(P, np.ndarray)
assert P.shape[0] == P.shape[1]
sums = []
for param in P:
a, b, c = param
sums.append(a * np.cos(b * (x - c)))
sums = np.array(sums)
return np.sum(sums, axis=0)
In order to use minimize to find the correct parameters, I create this residual function.
def resid(params, x):
assert isinstance(params, np.ndarray)
fit = cos_sum(x, params)
residual = np.sqrt(np.mean(np.abs(fit - signal)) ** 2)
return residual
I now need to create a guess. Since I already know how the signal was created, I made the following guess:
guess_A = np.random.normal(1, .2, size=3)
guess_B = 2 * np.pi * np.array([4, 9, 20], dtype=float)
guess_C = np.random.normal(0, .2, size=3)
guess = np.array([guess_A, guess_B, guess_C]).T
However, I am unable to run the following:
from scipy.optimize import minimize
optimization = minimize(resid, guess, args=(time))
and I receive the following error:
Traceback (most recent call last):
File "/Users/nickeisenberg/GitRepos/Python_Misc/Misc/minimize_curvefit_vector_variables.py", line 70, in <module>
optimization = minimize(resid, guess, args=(time))
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/scipy/optimize/_minimize.py", line 676, in minimize
res = _minimize_bfgs(fun, x0, args, jac, callback, **options)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/scipy/optimize/_optimize.py", line 1296, in _minimize_bfgs
sf = _prepare_scalar_function(fun, x0, jac, args=args, epsilon=eps,
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/scipy/optimize/_optimize.py", line 263, in _prepare_scalar_function
sf = ScalarFunction(fun, x0, args, grad, hess,
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/scipy/optimize/_differentiable_functions.py", line 158, in __init__
self._update_fun()
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/scipy/optimize/_differentiable_functions.py", line 251, in _update_fun
self._update_fun_impl()
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/scipy/optimize/_differentiable_functions.py", line 155, in update_fun
self.f = fun_wrapped(self.x)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/scipy/optimize/_differentiable_functions.py", line 137, in fun_wrapped
fx = fun(np.copy(x), *args)
File "/Users/nickeisenberg/GitRepos/Python_Misc/Misc/minimize_curvefit_vector_variables.py", line 53, in resid
fit = cos_sum(x, params)
File "/Users/nickeisenberg/GitRepos/Python_Misc/Misc/minimize_curvefit_vector_variables.py", line 30, in cos_sum
assert P.shape[0] == P.shape[1]
IndexError: tuple index out of range
Is this possible to do?
The problem is that minimize treats parameters as 1D array. Adding print just before failing assert shows, that it reshaped guess array to 1D array.
Changing cos_sum so that it accepts 1D array of params fixes this problem. Here is the code.
def cos_sum(x, P):
assert isinstance(P, np.ndarray)
sums = []
for i in range(0, P.shape[0], 3):
a, b, c = P[i], P[i+1], P[i+2]
sums.append(a * np.cos(b * (x - c)))
sums = np.array(sums)
return np.sum(sums, axis=0)
Result:
optimization = minimize(resid, guess, args=(time))
print(optimization.x)
[ 0.96805816 25.1679919 0.25020317 1.00261543 56.44511497
0.77872223 1.00966167 125.71622787 0.55004217]
plt.figure(figsize=(8, 4))
plt.plot(data)
plt.plot([cos_sum(t, optimization.x) for t in time], 'C1--')
plt.show()
I am doing Runge-Kutta method, with controlling step by stability, accuracy, and from other numerical method - to prevent transition beyond the switching point.
When the step is small(by accuracy or other method), integrating lasts long, and I got memory error, like this
line 712, in _getitem_RepMatrix
return self._rep.getitem_sympy(index_(i), index_(j))
TypeError: 'slice' object cannot be interpreted as an integer
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
line 157, in extract
row2[j2] = row1_j1
MemoryError
So, it happens, when I need accuracy about 1E-5 or 1E-6 and more.
The main built of method
def rkf2stepcontrol(
matrix_size,
dydx, A, x, b, dxdt, hn, t, tout, eps, predicate_func, MatrixForYacobian, hstabilitygetting, halgebraic):
k1 = zeros(matrix_size, 1)
k2 = zeros(matrix_size, 1)
k3 = zeros(matrix_size, 1)
k2k1norm = zeros(matrix_size, 1)
while ((predicate_func == False) or (tout - t > eps)):
print('Current time', t)
b = t
iterations += 1
print("I am in point ", x)
xprint_s = xprint_s.col_insert(1, Matrix([x]))
k1 = hn * dxdt(A, x, b)
print(k1, 'k1')
for i in (range(matrix_size)):
k2[i, 0] = hn * dxdt(A, x + k1[i, 0] * sympy.ones(*x.shape), b)[i, 0]
xold = x
for i in (range(matrix_size)):
x[i, 0] = x[i, 0] + 0.5 * (k1[i, 0] + k2[i, 0])
Then, controlling step by accuracy solving the equation 0.5‖k1-k2 ‖≤eps, where k2 and k1 - method steps and eps - needed accuracy. And controlling stability by getting max(abs(Jacobian Matrix.eigenvalues)
All the values will be shown on plot, so I must contain them.
Switching to numpy gives some speed up, but still too long, if you use accuracy about 1E-5 or 1E-6. With 1E-4, it takes 55 minutes to integrate 2.37 seconds, so about an hour to make 2.5 seconds. And, you need to recalculate accuracy only by condition 0.5‖k1-k2 ‖≤eps.
Code rewrited as
...
k1 = numpy.zeros((matrix_size, 1))
k2 = numpy.zeros((matrix_size, 1))
k3 = numpy.zeros((matrix_size, 1))
k2k1norm = numpy.zeros((matrix_size, 1))
xprint_s = numpy.c_[xprint_s, x]
k1 = hn * dxdt(A, x, b)
for i in (range(matrix_size)):
k2[i, 0] = hn * dxdt(A, x + k1[i, 0] * numpy.ones(x.shape,dtype=float,order='C'), b)[i, 0]
for j in range(matrix_size):
k2k1norm[j, 0] = float(dxdt(A, x + k1[i, 0] * numpy.ones(x.shape,dtype=float,order='C'), b)[j, 0]) - \
float(dxdt(A, x, b)[j, 0])
...
return iterations, endtime, xprint_s
And to make the plot
yprint_s_rkeiler_angle_step2 = numpy.delete(yprint_s_rkeiler_angle_step2, 0, -1)
argument1 = list((yprint_s_rkeiler_angle_step2[0]))
argument2 = list((yprint_s_rkeiler_angle_step2[1]))
title('Runge Kutta Esposito Based on Eiler')
plot(t, argument1, '-o', linewidth=2)
plot(t, argument2, '-o', linewidth=2)
legend(["y", "Vy"], loc ="upper right")
print()
ylabel("argument")
xlabel("t")
grid(True)
show() # display
I am doing a project, where I want to use Euler's Method to show a solution to this Second Order Different equation
0=y''+y'+9.81y
So I started by changing the second-order into a system of first-order equations
y'=u, u'=f(t,y,u)
With initial condition
y(0)=180, u(0)=0
So I get two equation in the end
y[n + 1] = y[n] + u[n] * (t[n + 1] - t[n]), u[n + 1] = u[n] + f(u[0], y[n], t[0]) * (t[n + 1] - t[n])
This is my code
import numpy as np
import matplotlib.pyplot as plt
def odeEuler(f, y0, u0, t):
y = np.zeros(len(t))
u = np.zeros(len(t))
y[0] = y0
u[0] = u0
for n in range(0, len(t) - 1):
y[n + 1] = y[n] + u[n] * (t[n + 1] - t[n])
u[n + 1] = u[n] + f(u[0], y[n], t[0]) * (t[n + 1] - t[n])
return y, u
t = np.linspace(0, 100)
y0 = 180
u0 = 0
f = lambda u, y, t: -9.81 * y - u
y = odeEuler(f, y0, u0, t)
plt.plot(t, y, 'b.-')
plt.legend(['Euler'])
plt.axis([0, 100, 0, 200])
plt.grid(True)
plt.show()
However, when I run the code, it give me the error
Traceback (most recent call last):
File "/Users/huangy15/PycharmProjects/Draft/Damped Driven Pendulum/Praying this works.py", line 22, in <module>
plt.plot(t, y, 'b.-')
File "/Users/huangy15/PycharmProjects/Draft/Damped Driven Pendulum/venv/lib/python3.7/site-packages/matplotlib/pyplot.py", line 3021, in plot
**({"data": data} if data is not None else {}), **kwargs)
File "/Users/huangy15/PycharmProjects/Draft/Damped Driven Pendulum/venv/lib/python3.7/site-packages/matplotlib/axes/_axes.py", line 1605, in plot
lines = [*self._get_lines(*args, data=data, **kwargs)]
File "/Users/huangy15/PycharmProjects/Draft/Damped Driven Pendulum/venv/lib/python3.7/site-packages/matplotlib/axes/_base.py", line 315, in __call__
yield from self._plot_args(this, kwargs)
File "/Users/huangy15/PycharmProjects/Draft/Damped Driven Pendulum/venv/lib/python3.7/site-packages/matplotlib/axes/_base.py", line 501, in _plot_args
raise ValueError(f"x and y must have same first dimension, but "
ValueError: x and y must have same first dimension, but have shapes (50,) and (2, 50)
Can anyone help me check if my idea works, and if not, what other approaches I can take? Thanks!
Your odeEuler function is returning two variables: y and u, you need to select just one of them to plot.
Store the output of the function in two different variables will solve the problem:
y, u = odeEuler(f, y0, u0, t)
I am revisiting a school project, which I did not complete to my satisfaction. Namely, I wrote an algorithm that takes an ALMOST arbitrary size set of equations and solves them iteratively. The problem being the "almost" part. Essentially, it must have at least two equations, and will not solve for a single one. This is because, I believe, that I don't understand how to use positional arguments correctly.
Below, in the main method, I define two functions y_prime and z_prime. If I pass them both, I get a beautiful graph of my solutions. But, if I only pass y_prime along with its initial conditions and the solution vector to the rungekutta() function, things go haywire:
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
def rungekutta(dt, y, t, *funcs):
"""
The following code was written in order to
reproduce the classic 4th order Runge-Kutta numerical
method of solving a system of differential equations.
The aim was to not only apply this to the budworm deforestation
model developed by Ludwig et al, but also to create an algorithm
that is generic enough to accept a wide range of ODEs and
systems of ODEs.
:param dt: time step "Delta t"
:param y: The solution vector at the last time step
:param t: The time at the last time step
:param funcs: the vector field dy/dt = f(t,y)
:return: The solution vector for the next time step
"""
k1 = [dt * f(*y, t) for f in funcs]
args = [y_n + 0.5 * k_1 for y_n, k_1 in zip((*y, t), (*k1, dt))]
k2 = [dt * f(*args) for f in funcs]
args = [y_n + 0.5 * k_2 for y_n, k_2 in zip((*y, t), (*k2, dt))]
k3 = [dt * f(*args) for f in funcs]
args = [y_n + k_3 for y_n, k_3 in zip((*y, t), (*k3, dt))]
k4 = [dt * f(*args) for f in funcs]
return [y_n + (k_1 + 2 * k_2 + 2 * k_3 + k_4) / 6 for y_n, k_1, k_2, k_3, k_4 in
zip(y, k1, k2, k3, k4)]
if __name__ == '__main__':
def y_prime(y, z, t):
return -t * y
def z_prime(y, z, t):
return z
t_0 = -10
t_n = 10
dt = .05
steps = int((t_n - t_0) / dt)
y_soln = [0] * steps
z_soln = [0] * steps
time = np.arange(t_0, t_n, dt)
y_soln[0] = 1.928749848e-22
z_soln[0] = .0000453999297625
for i in np.arange(1, steps):
y_soln[i] = rungekutta(dt, y_soln[i-1], time[i-1], y_prime)
The first error I received, when trying to pass a single equation was:
Traceback (most recent call last):
File "C:/Users/wesle/PycharmProjects/Budworms/RK4v2.py", line 57, in <module>
y_soln[i] = rungekutta(dt, y_soln[i-1], time[i-1], y_prime, z_prime)
File "C:/Users/wesle/PycharmProjects/Budworms/RK4v2.py", line 23, in rungekutta
k1 = [dt * f(*y, t) for f in funcs]
File "C:/Users/wesle/PycharmProjects/Budworms/RK4v2.py", line 23, in <listcomp>
k1 = [dt * f(*y, t) for f in funcs]
TypeError: y_prime() argument after * must be an iterable, not float
This was because, I think, I have "y_soln" as a positional argument, but now there is only one and it is no longer iterable. So, I made it a tuple of 1 when I passed it in the main method:
for i in np.arange(1, steps):
y_soln[i] = rungekutta(dt, (y_soln[i-1],), time[i-1], y_prime)
That bit me in the butt, however, because now I am passing a tuple into my y_prime equation, when what it really needs is a float:
Traceback (most recent call last):
File "C:/Users/wesle/PycharmProjects/Budworms/RK4v2.py", line 57, in <module>
y_soln[i] = rungekutta(dt, (y_soln[i-1],), time[i-1], y_prime)
File "C:/Users/wesle/PycharmProjects/Budworms/RK4v2.py", line 23, in rungekutta
k1 = [dt * f(*y, t) for f in funcs]
File "C:/Users/wesle/PycharmProjects/Budworms/RK4v2.py", line 23, in <listcomp>
k1 = [dt * f(*y, t) for f in funcs]
File "C:/Users/wesle/PycharmProjects/Budworms/RK4v2.py", line 38, in y_prime
return -t * y
TypeError: can't multiply sequence by non-int of type 'numpy.float64'
My only work-around so far has been to solve an extra, random equation like $y= y'$ in addition to whatever equation I'm interested in. This seems pretty inefficient though.
So, it seems like I'm damned if I do, or damned if I don't. Is there any remedy to this?
EDIT If you want to see the code actually work, replace this:
for i in np.arange(1, steps):
y_soln[i] = rungekutta(dt, (y_soln[i-1],), time[i-1], y_prime)
with the instance where I pass both equations and their solution vectors to the function:
for i in np.arange(1, steps):
y_soln[i], z_soln[i] = rungekutta(dt, (y_soln[i-1], z_soln[i-1]), time[i-1], y_prime, z_prime)
My solution ended up being to convert all lists to numpy arrays, which allowed me to take advantage of the built in element-wise scalar addition and multiplication. This made computing the "k" values much less cumbersome and convoluted:
def rk4(dt, t, field, y_0):
"""
:param dt: float - the timestep
:param t: array - the time mesh
:param field: method - the vector field y' = f(t, y)
:param y_0: array - contains initial conditions
:return: ndarray - solution
"""
# Initialize solution matrix. Each row is the solution to the system
# for a given time step. Each column is the full solution for a single
# equation.
y = np.asarray(len(t) * [y_0])
for i in np.arange(len(t) - 1):
k1 = dt * field(t[i], y[i])
k2 = dt * field(t[i] + 0.5 * dt, y[i] + 0.5 * k1)
k3 = dt * field(t[i] + 0.5 * dt, y[i] + 0.5 * k2)
k4 = dt * field(t[i] + dt, y[i] + k3)
y[i + 1] = y[i] + (k1 + 2 * k2 + 2 * k3 + k4) / 6
return y
I have some difficulty in running the code. When I run it, this kind of error always appears:
Traceback (most recent call last):
File "C:\Users\zhangjq058\workspace\22222222\src\222.py", line 20, in <module>
x, y = meshgrid(fftfreq(th.shape[0], dx), fftfreq(th.shape[1], dx))
File "C:\Python27\lib\site-packages\numpy\fft\helper.py", line 153, in fftfreq
assert isinstance(n,types.IntType) or isinstance(n, integer)
AssertionError
The code is:
from pylab import *
from numpy import *
N = 100 #lattice points per axis
dt = 1 #time step
dx = 1 #lattice spacing
t = arange(0, 10000*dt, dt) #time
a = 1 #cofficient
epsilon = 100 #cofficient
M = 1.0 #cofficient
every = 100 #dump an image every
phi_0 = 0.5 #initial mean value of the order parameter
noise = 0.1 #initial amplitude of thermal fluctuations in the order parameter
th = phi_0*ones((N, N)) + noise*(rand(N, N) - 0.5) #initial condition
x, y = meshgrid(fftfreq(th.shape[0], dx), fftfreq(th.shape[1], dx))
k2 = (x*x + y*y) #k is a victor in the Fourier space, k2=x^2+y^2
g = lambda th, a: 4*a*th*(1-th)*(1-2*th) #function g
def update(th, dt, a, k2):
return ifft2((fft2(th)-dt*M*k2*fft2(g(th,a)))/(1+2*epsilon*M*dt*k2**2))
for i in range(size(t)):
print t[i]
if mod(i, every)==0:
imshow(abs(th), vmin=0.0, vmax=1.0)
colorbar()
savefig('t'+str(i/every).zfill(3)+'.png', dpi=100)
clf()
th=update(th, dt, a, k2)
You are under Windows 64-bit i assume. There is a bug in your numpy version, where
windows Long type is not recognized as int. Either update numpy or cast your shapes
manually:
fftfreq(int(th.shape[0]), dx)