We Keep Coding

Minimizing a function in scipy with three parameters returns the initial guess

A set of points with coordinates x and y look like this. I want to construct a curve in the region below y = 0 of the form - a - np.exp(-(x - b)/c), where parameters a, b and c are found by the condition that 90% of the points below y = 0 are enclosed by this line and the function in question.
I've written the following code to do this, but the minimize function gives the initial guess as a result and I don't know what I'm missing.
from scipy.optimize import minimize
import numpy as np
def enclosed_points(params):
a, b, c = params
den = (y < 0).sum() # Calculate the number of points with y coordinate below y0
func = - a - np.exp(-(x - b)/c) # Calculate the value of the function for each x
num = ((y < 0) & (y > func)).sum() # Calculate the number of points with y coordinate
# below y0 and above the function
return np.abs(num/den - 0.9) # Return the absolute value of the difference between
# the ratio of num and den and the target number (0.9)
initial_guess = [0.1, 0.2, 1] # Dummy initial guess
result = minimize(enclosed_points, initial_guess)
Edit. Here I have uploaded a random sample of the whole data in npy format.

Well, I tried some different methods and change some parts of your code:
def func(x, a, b, c):
return - a - np.exp(-(x - b)/c)
def enclosed_points(params):
a, b, c = params
loss1 = y[np.argwhere( y > 0)]
loss2 = loss1[np.argwhere( loss1 < func(x[np.argwhere( y > 0)], a, b, c) )]
loss = ((loss2.sum() / len(y)) - 0.9)**2
return loss
initial_guess = [-0.1, 0.2, 1]
result = minimize(enclosed_points, initial_guess, method='SLSQP', options={'eps': 1e-2})
The loss1 and loss2 do the same work as your loss function, but I change abs to the power of 2 (because it's more common in my opinion) (also added "method='SLSQP', options={'eps': 1e-2}" to your minimizer based on another post on StackOverflow; try to read them carefully and get familiar with their issues about this minimizer too). However, I think the main issue is that your problem is non-convex, and minimize function tries to find a local minimum. Please see this post for a comprehensive description.
In the end, I would say that with [-0.1, 0.2, 1] initial point I could find a solution (try different negative value for a, and may you can find another solutions :) )
Good luck

Scipy optimisation routines only finding one answer

Given this function:
def f(x):
return (1-x**2)**m * ((1-x)/2)**n
where m and n are constants, let's say both 0.5 for the sake of an example.
I'm trying to use functions from scipy.optimize to solve for x given a value of y. I'm only interested in xvalues from -1 to 1. Plotting the function with
x = numpy.arange(0, 1, 0,1)
matplotlib.pyplot.plot(x, f(x))
shows that the function is a kind of distorted parabola covering the range about 0 to 0.65. So lets try solving it for y = 0.3:
def f(x):
return (1 - x**2)**m * ((1-x)/2)**n - 0.3
print(scipy.optimize.newton_krylov(f, 0.5))
0.6718791645800665
This looks about right for one of the possible solutions. But there are two. The second should be around -0.9. Try what I might for an initial guess, I can't get it to find this second solution. The Newton-Krylov method gives no convergence at all for xin < 0 but none of the solvers can find this second solution.
Am I missing something? What am I doing wrong?

The method converges at least for x=-0.9:
scipy.optimize.newton_krylov(f, -0.9)
#array(-0.9527983).
It diverges for x approximately in [-0.85...0.06].

This is because, newton_krylov uses the Jacobian of the function. This makes it a gradient decent method consequently your solutions always converge to a local minima. Furthermore, because your function is parabolic you have a very interesting option!
The first is to find the maxima of f(x) and split your search domain into to. Next you can make an initial guess in each domain and solve with newton_krylov.
def f(x):
# Here is our function
return (1-x**2)**m * ((1-x)/2)**n
def minf(x):
# Here is where we find an optima and split the domain
return -f(x)
def fy(x):
# This is where you want your y value target defined
return abs(f(x) - .3)
if __name__ == "__main__":
x = numpy.arange(-1., 1., 1e-3, dtype=float)
# pyplot.plot(x, f(x))
# pyplot.show()
minx = minimize(minf, 0.0)['x']
# Make an initial guess in each domain
a1 = minx - 1.6 * minx
a2 = minx + 1.6 * minx
print(newton_krylov(fy, a1))
print(newton_krylov(fy, a2))
The output then is:
[0.67187916]
[-0.95279992]

Solving Inequalities in Sympy

I tried to solve the following inequality in sympy:
(10000 / x) - 1 < 0
So I issued the command:
solve_poly_inequality( Poly((10000 / x) - 1 ), '<')
However, I got:
[Interval.open(-oo, 1/10000)]
However, my manual computations give either x < 0 or x > 10000.
What am I missing? Due to the -1, I cannot represent it as a rational function.
Thanks in advance!

You are using a low-level solving routine. I would recommend using the higher-level routines solve or solveset, e.g.
>>> solveset((10000 / x) - 1 < 0, x, S.Reals)
(−∞, 0) ∪ (10000, ∞)
The reason that your attempt is right but looks wrong is that you did not specify the generator to use so Poly used 1/x as its variable (let's call it g) so it solved the problem 1000*g - 1 < 0...which is true when g is less than 1/1000 as you found.
You can see this generator identification by writing
>>> Poly(1000/x - 1)
Poly(1000*1/x - 1, 1/x, domain='ZZ')

10000/x-1 is not a polynomial in x but a polynomial in 1/x. Rather, 10000/x-1 is a rational function in x. While you may try to put Poly(1000*1/x - 1, x, domain='ZZ'), there will be errors
PolynomialError: 1/x contains an element of the generators set
because by definition 10000/x-1 cannot be a polynomial in x. Thus, you just cannot do the computation with this.
You can also try to following or other solvers.
from sympy.solvers.inequalities import reduce_rational_inequalities
from sympy import Poly
from sympy.abc import x
reduce_rational_inequalities([[10000/x - 1 < 0]], x)
((-oo < x) & (x < 0)) | ((10000 < x) & (x < oo))

How to add several constraints to differential_evolution?

I have the same problem as in this question but don't want to add only one but several constraints to the optimization problem.
So e.g. I want to maximize x1 + 5 * x2 with the constraints that the sum of x1 and x2 is smaller than 5 and x2 is smaller than 3 (needless to say that the actual problem is far more complicated and cannot just thrown into scipy.optimize.minimize as this one; it just serves to illustrate the problem...).
I can to an ugly hack like this:
from scipy.optimize import differential_evolution
import numpy as np
def simple_test(x, more_constraints):
# check wether all constraints evaluate to True
if all(map(eval, more_constraints)):
return -1 * (x[0] + 5 * x[1])
# if not all constraints evaluate to True, return a positive number
return 10
bounds = [(0., 5.), (0., 5.)]
additional_constraints = ['x[0] + x[1] <= 5.', 'x[1] <= 3']
result = differential_evolution(simple_test, bounds, args=(additional_constraints, ), tol=1e-6)
print(result.x, result.fun, sum(result.x))
This will print
[ 1.99999986 3. ] -16.9999998396 4.99999985882
as one would expect.
Is there a better/ more straightforward way to add several constraints than using the rather 'dangerous' eval?

An example is something like this::
additional_constraints = [lambda(x): x[0] + x[1] <= 5., lambda(x):x[1] <= 3]
def simple_test(x, more_constraints):
# check wether all constraints evaluate to True
if all(constraint(x) for constraint in more_constraints):
return -1 * (x[0] + 5 * x[1])
# if not all constraints evaluate to True, return a positive number
return 10

There is a proper solution to the problem described in the question, to enforce multiple nonlinear constraints with scipy.optimize.differential_evolution.
The proper way is by using the scipy.optimize.NonlinearConstraint function.
Here below I give a non-trivial example of optimizing the classic Rosenbrock function inside a region defined by the intersection of two circles.
import numpy as np
from scipy import optimize
# Rosenbrock function
def fun(x):
return 100*(x[1] - x[0]**2)**2 + (1 - x[0])**2
# Function defining the nonlinear constraints:
# 1) x^2 + (y - 3)^2 < 4
# 2) (x - 1)^2 + (y + 1)^2 < 13
def constr_fun(x):
r1 = x[0]**2 + (x[1] - 3)**2
r2 = (x[0] - 1)**2 + (x[1] + 1)**2
return r1, r2
# No lower limit on constr_fun
lb = [-np.inf, -np.inf]
# Upper limit on constr_fun
ub = [4, 13]
# Bounds are irrelevant for this problem, but are needed
# for differential_evolution to compute the starting points
bounds = [[-2.2, 1.5], [-0.5, 2.2]]
nlc = optimize.NonlinearConstraint(constr_fun, lb, ub)
sol = optimize.differential_evolution(fun, bounds, constraints=nlc)
# Accurate solution by Mathematica
true = [1.174907377273171, 1.381484428610871]
print(f"nfev = {sol.nfev}")
print(f"x = {sol.x}")
print(f"err = {sol.x - true}\n")
This prints the following with default parameters:
nfev = 636
x = [1.17490808 1.38148613]
err = [7.06260962e-07 1.70116282e-06]
Here is a visualization of the function (contours) and the feasible region defined by the nonlinear constraints (shading inside the green line). The constrained global minimum is indicated by the yellow dot, while the magenta one shows the unconstrained global minimum.
This constrained problem has an obvious local minimum at (x, y) ~ (-1.2, 1.4) on the boundary of the feasible region which will make local optimizers fail to converge to the global minimum for many starting locations. However, differential_evolution consistently finds the global minimum as expected.

python nodal/piecewise-linear power-law generator

I need efficient python code that returns (not fits) a piecewise-linear (or, actually, piecewise power-law) continuous function for an arbitrary number of nodes (/poles/control points) defined by their positions plus (preferably) slopes rather than amplitudes. For example, for three pieces (four nodes) I have:
def powerLawFuncTriple(x,C,alpha,beta,gamma,xmin,xmax,x0,x1):
"""
Extension of the two-power-law version described at
http://en.wikipedia.org/wiki/Power_law#Broken_power_law
"""
if x <= xmin or x > xmax:
return 0.0
elif xmin < x <= x0:
n = C * (x ** alpha)
elif x0 < x <= x1:
n = C * x0**(alpha-beta) * (x ** beta)
elif x1 < x <= xmax:
n = C * x0**(alpha-beta) * x1**(beta-gamma) * (x ** gamma)
return n
Do any helpful functions already exist, otherwise what's an efficient way to code up code to generate these functions? Maybe this amounts to evaluating rather than fitting one of the scipy built-ins.
Somewhat related: Fitting piecewise function in Python
One possible answer may be:
def piecewise(x,n0,posns=[1.0,2.0,3.0],alpha=[-2.0,-1.5]):
if x <= posns[0] or x > posns[-1]: return 0.0
n=n0*x**alpha[0]
np=len(posns)
for ip in range(np):
if posns[ip] < x <= posns[ip+1]: return n
n *= (posns[ip+1]/float(x))**(alpha[ip]-alpha[ip+1])
return n
But this must get slower as x increases. Would list comprehension or anything else speed up the loop?
Thanks!

In the end I've gone with the excellent CosmoloPy module:
http://pydoc.net/Python/CosmoloPy/0.1.103/cosmolopy.utils
class PiecewisePowerlaw()
"""
You can specify the intervals and power indices, and this class
will figure out the coefficients needed to make the function
continuous and normalized to unit integral.
"""
Seems to work swimmingly.