I'm trying to maximize a sum of two monotonically increasing functions f(x) = f1(x) + f2(x) within the given bounds, say x = 0 to 6. The curves of the two functions are:
To solve this I'm using basinhopping function from scipy package.
I would like to specify a constraint for using the bounds. Specifically, I want the summation of bounds to be lesser than or equal to a constant value. i.e. In my implementation below, I want x[0] + x[1] <= C where C = 6.
In the above figure, for C = 6, approximately x[0] = 2 and x[1] = 4 (4 + 2 =<=6) will yield the maximum value. My question is how to specify this constraint? If it's not possible, is there another optimization function that is better suited for this problem?
from scipy.optimize import basinhopping
from math import tanh
def f(x):
return -(f1(x[0]) + f2(x[1])) # -ve sign for maximization
def f1(x):
return tanh(x)
def f2(x):
return (10 ** (0.2 * x))
# Starting point
x0 = [1., 1.]
# Bounds
xmin = [1., 1.]
xmax = [6., 6.]
# rewrite the bounds in the way required by L-BFGS-B
bounds = [(low, high) for low, high in zip(xmin, xmax)]
minimizer_kwargs = dict(method="L-BFGS-B", bounds=bounds)
result = basinhopping(f,
x0,
minimizer_kwargs=minimizer_kwargs,
niter=200)
print("global max: x = [%.4f, %.4f], f(x0) = %.4f" % (result.x[0], result.x[1], result.fun))
This is possible with COBYLA constrained optimization, though I don't know how to do it with L-BFGS-B. You can add constraints like the following:
def constraint1(x):
return 6 - x[0]
def constraint2(x):
return 6 - x[0] - x[1]
def constraint3(x):
return x[0] - 1
To add these constraints to the minimizer, use:
c1={"type":"ineq","fun":constraint1}
c2={"type":"ineq","fun":constraint2}
c3={"type":"ineq","fun":constraint3}
minimizer_kwargs = dict(method="COBYLA",constraints=(c1,c2,c3))
When I run this example, I get the result global max: x = [1.0000, 5.0000], f(x0) = -10.7616
Note, I haven't tested whether you need to add a constraint4 that x[1]>1.
For more reading, see the documentation for cobyla:
https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_cobyla.html
Related
I have this python code for finding intersections in the function "f(x)=x**2+x-2" with "g(x)=6-x"
import math
#brute force the functions with numbers until his Y values match, and then, do this for the other point.
def funcs():
for X in range(-100, 100):
funcA = (X**2)+X-2
funcB = 6 - X
if funcA == funcB:
print("##INTERSECTION FOUND!!")
print(f"({X},{funcA})")
print(f"({X},{funcB})")
else:
pass
funcs()
But my problem is the script only works with THAT SPECIFIC MATH FUNCTION, if I try to change the math function a little bit, the code won't work.
The code just checks when the Y values of the f(x) and the g(x) match together, and do the same for the other point.
Here it is the output:
##INTERSECTION FOUND!!
(-4,10)
(-4,10)
##INTERSECTION FOUND!!
(2,4)
(2,4)
In general, this is a root finding problem.
Define h(x) = f(x) - g(x).
The intersection point x implies f(x)=g(x) or h(x)=0.
For root-finding problems, there are many methods, say, bisection method, newton's method.
Here is a numerical example with bisection method.
def f(x):
return x ** 2 + x - 2
def g(x):
return 6 - x
def h(x):
return f(x) - g(x)
def bisection(a, b):
eps = 10 ** -10
ha = h(a)
hb = h(b)
if ha * hb > 0:
raise ValueError("Bad input")
for i in range(1000): # fix iteration number
ha = h(a)
midpoint = (a + b) / 2
hm = h(midpoint)
if abs(hm) < eps:
return midpoint
if hm * ha >= 0:
a = midpoint
else:
b = midpoint
raise RuntimeError("Out of iterations")
if __name__ == '__main__':
print(bisection(0, 100))
print(bisection(-100, 0))
Output:
1.999999999998181
-3.999999999996362
Why the numbers so close but not exact? because the problem is solved numerically. Other answers that utilize the sympy package solves the problem symbolically, which give the exact answer. But they only work with simple problems.
Why [0, 100] and [-100, 0]? this is because I sketched the graph somehow and know there is a root within the interval. In practice, the bisection method requires the interval [a,b] such that h(a) * h(b) < 0. Given a big interval [-100,100] and, thus, h(-100) * h(100) > 0, bisection method does not work this case. The big interval is partitioned such that some of the sub-intervals [a,b] satisfy the condition h(a) * h(b) < 0, say, [-100, 0] and [0, 100]
Why abs(hm) < eps? This tests whether hm is close to 0. In computers, we consider two floating-point numbers equal if the absolute value of their difference abs(hm) is smaller than a threshold eps. eps is usually 10 ** -10 to 10 ** -15 because there are usually 15 decimal significant digits for float numbers in Python or computer.
Newton's method will give you one of the outputs depending on the initial point.
For further study, search root finding problem or numerical root finding.
As you want the intersection, hence you are looking for a solution for f(x) - g(x) = 0. So, you can use fsolve in python to find the root of f(x) - g(x):
from scipy.optimize import fsolve
def func(X):
funcA = (X ** 2) + X - 2
funcB = 6 - X
return (funcA - funcB)
x = fsolve(func,0)
print(x)
You could employ sympy, Python's symbolic math library:
from sympy import symbols, Eq, solve
X = symbols('X', real=True)
funcA = (X ** 2) + X - 2
funcB = 6 - X
sol = solve(Eq(funcA, funcB))
print(sol) # --> [-4, 2]
To obtain the corresponding values for funcA and funcB
for s in sol:
print(f'X={s} funcA({s})={funcA.subs(X, s)} funcB({s})={funcB.subs(X, s)} ')
# X=-4 funcA(-4)=10 funcB(-4)=10
# X=2 funcA(2)=4 funcB(2)=4
For some functions, the result could still be symbolically, as that is the most exact form. .evalf() obtains a numeric approximation. For example:
funcA = X ** 2 + X - 2
funcB = - 2*X ** 2 + X + 7
sol = solve(Eq(funcA, funcB))
for s in sol:
print(f'X={s} funcA(X)={funcA.subs(X, s)} funcB(X)={funcB.subs(X, s)}')
print(f'X={s.evalf()} funcA(X)={funcA.subs(X, s).evalf()} funcB(X)={funcB.subs(X, s).evalf()}')
Output:
X=-sqrt(3) funcA(X)=1 - sqrt(3) funcB(X)=1 - sqrt(3)
X=-1.73205080756888 funcA(X)=-0.732050807568877 funcB(X)=-0.732050807568877
X=sqrt(3) funcA(X)=1 + sqrt(3) funcB(X)=1 + sqrt(3)
X=1.73205080756888 funcA(X)=2.73205080756888 funcB(X)=2.73205080756888
Given this function:
def f(x):
return (1-x**2)**m * ((1-x)/2)**n
where m and n are constants, let's say both 0.5 for the sake of an example.
I'm trying to use functions from scipy.optimize to solve for x given a value of y. I'm only interested in xvalues from -1 to 1. Plotting the function with
x = numpy.arange(0, 1, 0,1)
matplotlib.pyplot.plot(x, f(x))
shows that the function is a kind of distorted parabola covering the range about 0 to 0.65. So lets try solving it for y = 0.3:
def f(x):
return (1 - x**2)**m * ((1-x)/2)**n - 0.3
print(scipy.optimize.newton_krylov(f, 0.5))
0.6718791645800665
This looks about right for one of the possible solutions. But there are two. The second should be around -0.9. Try what I might for an initial guess, I can't get it to find this second solution. The Newton-Krylov method gives no convergence at all for xin < 0 but none of the solvers can find this second solution.
Am I missing something? What am I doing wrong?
The method converges at least for x=-0.9:
scipy.optimize.newton_krylov(f, -0.9)
#array(-0.9527983).
It diverges for x approximately in [-0.85...0.06].
This is because, newton_krylov uses the Jacobian of the function. This makes it a gradient decent method consequently your solutions always converge to a local minima. Furthermore, because your function is parabolic you have a very interesting option!
The first is to find the maxima of f(x) and split your search domain into to. Next you can make an initial guess in each domain and solve with newton_krylov.
def f(x):
# Here is our function
return (1-x**2)**m * ((1-x)/2)**n
def minf(x):
# Here is where we find an optima and split the domain
return -f(x)
def fy(x):
# This is where you want your y value target defined
return abs(f(x) - .3)
if __name__ == "__main__":
x = numpy.arange(-1., 1., 1e-3, dtype=float)
# pyplot.plot(x, f(x))
# pyplot.show()
minx = minimize(minf, 0.0)['x']
# Make an initial guess in each domain
a1 = minx - 1.6 * minx
a2 = minx + 1.6 * minx
print(newton_krylov(fy, a1))
print(newton_krylov(fy, a2))
The output then is:
[0.67187916]
[-0.95279992]
I have the same problem as in this question but don't want to add only one but several constraints to the optimization problem.
So e.g. I want to maximize x1 + 5 * x2 with the constraints that the sum of x1 and x2 is smaller than 5 and x2 is smaller than 3 (needless to say that the actual problem is far more complicated and cannot just thrown into scipy.optimize.minimize as this one; it just serves to illustrate the problem...).
I can to an ugly hack like this:
from scipy.optimize import differential_evolution
import numpy as np
def simple_test(x, more_constraints):
# check wether all constraints evaluate to True
if all(map(eval, more_constraints)):
return -1 * (x[0] + 5 * x[1])
# if not all constraints evaluate to True, return a positive number
return 10
bounds = [(0., 5.), (0., 5.)]
additional_constraints = ['x[0] + x[1] <= 5.', 'x[1] <= 3']
result = differential_evolution(simple_test, bounds, args=(additional_constraints, ), tol=1e-6)
print(result.x, result.fun, sum(result.x))
This will print
[ 1.99999986 3. ] -16.9999998396 4.99999985882
as one would expect.
Is there a better/ more straightforward way to add several constraints than using the rather 'dangerous' eval?
An example is something like this::
additional_constraints = [lambda(x): x[0] + x[1] <= 5., lambda(x):x[1] <= 3]
def simple_test(x, more_constraints):
# check wether all constraints evaluate to True
if all(constraint(x) for constraint in more_constraints):
return -1 * (x[0] + 5 * x[1])
# if not all constraints evaluate to True, return a positive number
return 10
There is a proper solution to the problem described in the question, to enforce multiple nonlinear constraints with scipy.optimize.differential_evolution.
The proper way is by using the scipy.optimize.NonlinearConstraint function.
Here below I give a non-trivial example of optimizing the classic Rosenbrock function inside a region defined by the intersection of two circles.
import numpy as np
from scipy import optimize
# Rosenbrock function
def fun(x):
return 100*(x[1] - x[0]**2)**2 + (1 - x[0])**2
# Function defining the nonlinear constraints:
# 1) x^2 + (y - 3)^2 < 4
# 2) (x - 1)^2 + (y + 1)^2 < 13
def constr_fun(x):
r1 = x[0]**2 + (x[1] - 3)**2
r2 = (x[0] - 1)**2 + (x[1] + 1)**2
return r1, r2
# No lower limit on constr_fun
lb = [-np.inf, -np.inf]
# Upper limit on constr_fun
ub = [4, 13]
# Bounds are irrelevant for this problem, but are needed
# for differential_evolution to compute the starting points
bounds = [[-2.2, 1.5], [-0.5, 2.2]]
nlc = optimize.NonlinearConstraint(constr_fun, lb, ub)
sol = optimize.differential_evolution(fun, bounds, constraints=nlc)
# Accurate solution by Mathematica
true = [1.174907377273171, 1.381484428610871]
print(f"nfev = {sol.nfev}")
print(f"x = {sol.x}")
print(f"err = {sol.x - true}\n")
This prints the following with default parameters:
nfev = 636
x = [1.17490808 1.38148613]
err = [7.06260962e-07 1.70116282e-06]
Here is a visualization of the function (contours) and the feasible region defined by the nonlinear constraints (shading inside the green line). The constrained global minimum is indicated by the yellow dot, while the magenta one shows the unconstrained global minimum.
This constrained problem has an obvious local minimum at (x, y) ~ (-1.2, 1.4) on the boundary of the feasible region which will make local optimizers fail to converge to the global minimum for many starting locations. However, differential_evolution consistently finds the global minimum as expected.
I'm trying to implement euler's method to approximate the value of e in python. This is what I have so far:
def Euler(f, t0, y0, h, N):
t = t0 + arange(N+1)*h
y = zeros(N+1)
y[0] = y0
for n in range(N):
y[n+1] = y[n] + h*f(t[n], y[n])
f = (1+(1/N))^N
return y
However, when I try to call the function, I get the error "ValueError: shape <= 0". I suspect this has something to do with how I defined f? I tried inputting f directly when euler is called, but gave me errors related to variables not being defined. I also tried defining f as its own function, which gave me a division by 0 error.
def f(N):
for n in range(N):
return (1+(1/n))^n
(not sure if N was the appropriate variable to use here...)
The formula you are trying to use is not Euler's method, but rather the exact value of e as n approaches infinity wiki,
$n = \lim_{n\to\infty} (1 + \frac{1}{n})^n$
Euler's method is used to solve first order differential equations.
Here are two guides that show how to implement Euler's method to solve a simple test function: beginner's guide and numerical ODE guide.
To answer the title of this post, rather than the question you are asking, I've used Euler's method to solve usual exponential decay:
$\frac{dN}{dt} = -\lambda N$
Which has the solution,
$N(t) = N_0 e^{-\lambda t}$
Code:
import numpy as np
import matplotlib.pyplot as plt
from __future__ import division
# Concentration over time
N = lambda t: N0 * np.exp(-k * t)
# dN/dt
def dx_dt(x):
return -k * x
k = .5
h = 0.001
N0 = 100.
t = np.arange(0, 10, h)
y = np.zeros(len(t))
y[0] = N0
for i in range(1, len(t)):
# Euler's method
y[i] = y[i-1] + dx_dt(y[i-1]) * h
max_error = abs(y-N(t)).max()
print 'Max difference between the exact solution and Euler's approximation with step size h=0.001:'
print '{0:.15}'.format(max_error)
Output:
Max difference between the exact solution and Euler's approximation with step size h=0.001:
0.00919890254720457
Note: I'm not sure how to get LaTeX displaying properly.
Are you sure you are not trying to implement the Newton's method? Because Newton's method is used to approximate the roots.
In case you decide to go with Newton's method, here is a slightly changed version of your code that approximates the square-root of 2. You can change f(x) and fp(x) with the function and its derivative you use in your approximation to the thing you want.
import numpy as np
def f(x):
return x**2 - 2
def fp(x):
return 2*x
def Newton(f, y0, N):
y = np.zeros(N+1)
y[0] = y0
for n in range(N):
y[n+1] = y[n] - f(y[n])/fp(y[n])
return y
print Newton(f, 1, 10)
gives
[ 1. 1.5 1.41666667 1.41421569 1.41421356 1.41421356
1.41421356 1.41421356 1.41421356 1.41421356 1.41421356]
which are the initial value and the first ten iterations to the square-root of two.
Besides this a big problem was the usage of ^ instead of ** for powers which is a legal but a totally different (bitwise) operation in python.
Given inputs of:
present value = 11, SUMMATIONS of future values that = 126, and n = 7 (periods of change)
how can I solve for a rate of chain that would create a chain that sums into being the FV? This is different from just solving for a rate of return between 11 and 126. This is solving for the rate of return that allows the summation to 126. Ive been trying different ideas and looking up IRR and NPV functions but the summation aspect is stumping me.
In case the summation aspect isn't clear, if I assume a rate of 1.1, that would turn PV = 11 into a list like so (that adds up to nearly the FV 126), how can I solve for r only knowing n, summation fv and pv?:
11
12.1
13.31
14.641
16.1051
17.71561
19.487171
21.4358881
total = 125.7947691
Thank you.
EDIT:
I attempted to create a sort of iterator, but it's hanging after the first loop...
for r in (1.01,1.02,1.03,1.04,1.05,1.06,1.07,1.08,1.09,1.10,1.11,1.12):
print r
test = round(11* (1-r**8) / (1 - r),0)
print test
while True:
if round(126,0) == round(11* (1-r**8) / (1 - r),0):
answer = r
break
else:
pass
EDIT 2:
IV = float(11)
SV = float(126)
N = 8
# sum of a geometric series: (SV = IV * (1 - r^n) / (1 - r )
# r^n - (SV/PV)*r + ((SV - IV)/IV) = 0
# long form polynomial to be solved, with an n of 3 for example:
# 1r^n + 0r^n + 0r^n + -(SV/PV)r + ((SV - IV)/IV)
# each polynomial coefficient can go into numpy.roots to solve
# for the r that solves for the abcd * R = 0 above.
import numpy
array = numpy.roots([1.,0.,0.,0.,0.,0.,0.,(-SV)/IV,(SV-IV)/IV])
for i in array:
if i > 1:
a = str(i)
b = a.split("+")
answer = float(b[0])
print answer
I'm getting a ValueError that my string "1.10044876702" cant be converted to float. any ideas?
SOLVED: i.real gets the real part of it. no need for split or string conversion ie:
for i in array:
if i > 1:
a = i.real
answer = float(a)
print answer
Sum of a geometric series
Subbing in,
126 = 11 * (1 - r**8) / (1 - r)
where we need to solve for r. After rearranging,
r**8 - (126/11)*r + (115/11) = 0
then using NumPy
import numpy as np
np.roots([1., 0., 0., 0., 0., 0., 0., -126./11, 115./11])
gives
array([-1.37597528+0.62438671j, -1.37597528-0.62438671j,
-0.42293755+1.41183514j, -0.42293755-1.41183514j,
0.74868844+1.1640769j , 0.74868844-1.1640769j ,
1.10044877+0.j , 1.00000000+0.j ])
where the first six roots are imaginary and the last is invalid (gives a div-by-0 in the original equation), so the only useable answer is r = 1.10044877.
Edit:
Per the Numpy docs, np.root expects an array-like object (aka a list) containing the polynomial coefficients. So the parameters above can be read as 1.0*r^8 + 0.*r^7 + 0.*r^6 + 0.*r^5 + 0.*r^4 + 0.*r^3 + 0.*r^2 + -126./11*r + 115./11, which is the polynomial to be solved.
Your iterative solver is pretty crude; it will get you a ballpark answer, but the calculation time is exponential with the desired degree of accuracy. We can do much better!
No general analytic solution is known for an eighth-order equation, so some numeric method is needed.
If you really want to code your own solver from scratch, the simplest is Newton-Raphson method - start with a guess, then iteratively evaluate the function and offset your guess by the error divided by the first derivative to hopefully converge on a root - and hope that your initial guess is a good one and your equation has real roots.
If you care more about getting good answers quickly, np.root is hard to beat - it computes the eigenvectors of the companion matrix to simultaneously find all roots, both real and complex.
Edit 2:
Your iterative solver is hanging because of your while True clause - r never changes in the loop, so you will never break. Also, else: pass is redundant and can be removed.
After a good bit of rearranging, your code becomes:
import numpy as np
def iterative_test(rng, fn, goal):
return min(rng, key=lambda x: abs(goal - fn(x)))
rng = np.arange(1.01, 1.20, 0.01)
fn = lambda x: 11. * (1. - x**8) / (1. - x)
goal = 126.
sol = iterative_test(rng, fn, goal)
print('Solution: {} -> {}'.format(sol, fn(sol)))
which results in
Solution: 1.1 -> 125.7947691
Edit 3:
Your last solution is looking much better, but you must keep in mind that the degree of the polynomial (and hence the length of the array passed to np.roots) changes as the number of periods changes.
import numpy as np
def find_rate(present_value, final_sum, periods):
"""
Given the initial value, sum, and number of periods in
a geometric series, solve for the rate of growth.
"""
# The formula for the sum of a geometric series is
# final_sum = sum_i[0..periods](present_value * rate**i)
# which can be reduced to
# final_sum = present_value * (1 - rate**(periods+1) / (1 - rate)
# and then rearranged as
# rate**(periods+1) - (final_sum / present_value)*rate + (final_sum / present_value - 1) = 0
# Build the polynomial
poly = [0.] * (periods + 2)
poly[ 0] = 1.
poly[-2] = -1. * final_sum / present_value
poly[-1] = 1. * final_sum / present_value - 1.
# Find the roots
roots = np.roots(poly)
# Discard unusable roots
roots = [rt for rt in roots if rt.imag == 0. and rt.real != 1.]
# Should be zero or one roots left
if len(roots):
return roots[0].real
else:
raise ValueError('no solution found')
def main():
pv, fs, p = 11., 126., 7
print('Solution for present_value = {}, final_sum = {}, periods = {}:'.format(pv, fs, p))
print('rate = {}'.format(find_rate(pv, fs, p)))
if __name__=="__main__":
main()
This produces:
Solution for present_value = 11.0, final_sum = 126.0, periods = 7:
rate = 1.10044876702
Solving the polynomial roots is overkill. This computation is usually made with a solver such as the Newton method directly applied to the exponential formula. Works for fractional durations too.
For example, https://math.stackexchange.com/questions/502976/newtons-method-annuity-due-equation