I wrote a script in Python that finds the zero of a fairly complicated function using fsolve. The way it works is as follows. There is a class that simply stores the parameter of the function. The class has an evaluate method that returns a value based on the stored parameter and another method (inversion) that finds the parameter at which the function takes the supplied output.
The inversion method updates the parameter of the function at each iteration and it keeps on doing so until the mismatch between the value returned by the evaluate method and the supplied value is zero.
The issue that I am having is that while the value returned by the inversion method is correct, the parameter, that is part of the object, is always 0 after the inversion method terminates. Oddly enough, this issue disappears if I use root instead of fsolve. As far as I know, fsolve is just a wrapper for root with some settings on the solver algorithm and some other things enforced.
Is this a known problem with fsolve or am I doing something dumb here? The script below demonstrates the issue I am having on the sine function.
from scipy.optimize import fsolve, root
from math import sin, pi
class invertSin(object):
def __init__(self,x):
self.x = x
def evaluate(self):
return sin(self.x)
def arcsin_fsolve(self,y):
def errorfunc(xnew):
self.x = xnew
return self.evaluate() - y
soln = fsolve(errorfunc, 0.1)
return soln
def arcsin_root(self,y):
def errorfunc(xnew):
self.x = xnew
return self.evaluate() - y
soln = root(errorfunc, 0.1, method = 'anderson')
return soln
myobject = invertSin(pi/2)
x0 = myobject.arcsin_fsolve(0.5) #find x s.t. sin(x) = 0.5 using fsolve
print(x0) #this prints pi/6
x0obj = myobject.x
print(x0obj) #this always prints 0 no matter which function I invert
myobject2 = invertSin(pi/2)
x1 = myobject2.arcsin_root(0.5) #find x s.t. sin(x) = 0.5 using root
print(x1) #this prints pi/6
x1obj = myobject2.x
print(x1obj) #this prints pi/6
If you add print statements for xnew in the errorfunc then you will see that fsolve works with a list (of one element). This means that the function is re-interpreted that way, not the original function. Somehow the type information is lost after exiting the solver so that then the address/reference to that list is interpreted as floating point data, which gives the wrong value.
Setting self.x = xnew[0] there restores the desired behavior.
Related
I am trying to solve a non linear system. Here is the code for a toy problem.
import collections
import numpy as np
import scipy
def flat(x):
''' flattens a shallow list
ex: [[1,2,3],[4,5],[6]] ----> flattens to [1,2,3,4,5]
numpy flatten does not work on lists.
'''
if isinstance(x, collections.Iterable):
return [a for i in x for a in flat(i)]
else:
return [x]
def func(X):
'''setups the matrix dynamic equation and the set of constraints
'''
A = [[0,1,0,1],[2,1,0,4],[1,4,1,3],[3, 2, 1,0]]
A1 = [[1,0,1,-1], [0,-1,2,1],[1,2,0,1],[1,2,0,-2]]
x = X[:-1]
alpha = X[-1]
x0 = [1,2,3,4]
y = x - x0
# x[0] = 0.5
# x[3] = 0.3
dyneqn = np.dot(A,y) + alpha * np.dot(A1,x)
cons = (1/2.0)*np.dot(x.T,np.dot(A1,x)) + np.dot([-1,1,2,-3], x) + 0.5
return flat([dyneqn, cons])
sol = scipy.optimize.root(func,[1,-1,2,0,-1])
sol.x
Problem Statement
The argument X of the objective function f has five unknowns that we are solving for. I want to set the first parameter, i.e., X[0]=0.5and the fourth parameter i.e., X[3] = 0.3 and solve for the remaining 3 unknowns. Let us assume for simplicity that such a solution exists and my initial guess is somehow a good one.
Attempt:
I know I should probably pass these arguments to the args=() argument in scipy.optimize.root. I tried setting
args = (X[0]=0.5, X[3]=0.3)
init_guess = [0.5,-1,2,0.3,-1]
scipy.optimize.root(func,init_guess, args=args)
This is obviously wrong.
Question? How can I fix this?.
Note: I added the flat function so that the code is self contained. It has nothing to do with this question.
Typically with scipy functions like root, minimize, etc
root(func, x0, args=(a, b, c, ...))
requires a func that accepts:
func(x0, a, b, c, ...)
# do something those arguments
return value
x0 is the value that root varies, a,b,c are args value that are passed unchanged to your function. Depending of the problem x0 may be an array. The nature of the args is entirely up to you.
From your example I reconstruct that you want to solve for the second and third component of some vector x as well as the parameter alpha. With the args keyword of scipy.optmize.root that would look something like
def func(x_solve, x0, x3):
#x_solve.size should be 3
x = np.empty(4)
x[0], x[3] = x0, x3
x[1:3] = x_solve[:2]
alpha = x_solve[2]
...
scipy.optimize.root(func, [-1,2,-1], args=(.5, .3))
As Azat and kazemakase pointed out, I'm also not sure if you actually want to use root, but the usage of scipy.optimize.minimize is pretty much the same.
Edit: It should be possible to have a flexible set of fixed variables by using a dictionary as an additional argument which specifies those:
def func(x_solve, fixed):
x = x_solve[:-1] # last value is alpha
for idx in fixed.keys(): # overwrite fixed entries
x[idx] = fixed[idx]
alpha = x_solve[-1]
# fixed variables, key is the index
fixed_vars = {0:.5, 3:.3}
# find roots
scipy.optimize.root(func,
[.5, -1, 2, .3, -1],
args=(fixed_vars,))
That way, when the optimizer in root numerically evaluates the Jacobian it obtains zero for the fixed variables and should therefore leave those invariant. However, that might lead to complications in the convergence of the algorithm.
I am trying to optimise the output of a function using the scipy basinhopping algorithm.
def acceptance_criteria(self,**kwargs):
print "kwargs "
print kwargs
x = kwargs["x_new"]
beta = x[0]
alpha = [x[1],x[2],x[3],x[4],x[5],x[6]]
print x
inputnow= raw_input()
beta_gamma_pass = beta != self.gamma
beta_zero_pass = beta >= 0.0
alpha1_pass = alpha[0] > 0.0
alpha2_pass = alpha[1] > 0.0
alpha3_pass = alpha[2] > 0.0
alpha4_pass= alpha[3] > 0.0
alpha5_pass= alpha[4] > 0.0
alpha6_pass= alpha[5] > 0.0
return beta_gamma_pass,beta_zero_pass,alpha1_pass,alpha2_pass,alpha3_pass,alpha4_pass,alpha5_pass,alpha6_pass
def variational_calculation(self):
minimizer_kwargs = {"method": "BFGS"}
initial_paramater_guesses = [2,1.0,1.0/2.0,1.0/3.0,1.0/4.0,1.0/5.0,1.0/6.0]
ret = basinhopping(self.Calculate, initial_paramater_guesses, minimizer_kwargs=minimizer_kwargs, niter=200, accept_test=self.acceptance_criteria)
I am getting problems with Nans and infs in my calculate function.
This is due to invalid parameter values being used.
I have attempted to prevent this by using acceptance criteria.
But the basinhopping routine does not call the accept_test function.
Thus the criteria remain unimplemted.
Can anyone help me out as to why basinhopping isn't calling the accept_test function?
Thanks
EDIT:
in response to #sascha's comment,
There are fractional powers of parameters, and 1/parameter terms in the function.
Not limiting the range of the allowed parameter values gives complex and inf values in this case.
It is actually an eigenvalue problem, where I am trying to minimise the trace of the eigenvalues of a set of 18*18 matrices.
The matrix elements depend on the 7 parameters in a complex way with dozens of non linear terms.
I have never worked on anything more complex than polynomial regression before, so I am not familiar with the algorithms or their applicability at all.
However, the function/s that I am trying to minimise are smooth as long as you avoid parameter values near poles; caused by 1/parameter and 1/(paramter^n -constant) terms.
EDIT2:
QUESTION CLARIFICATION
The question here is nothing to do with the applicability of the basinhopping algorithm.
It is why the specific implementation of it, in the 2.7 version of python and scipy, does not call the accept_test function?
I can't say why your example doesn't work, but here's a similar but minimal example where it does call the accept_test, maybe you can spot the difference
import scipy
import numpy as np
from scipy.optimize import basinhopping
class MyClass:
def Calculate(self, x):
return np.dot(x, x)
def acceptance_criteria(self, **kwargs):
print("in accept test")
return True
def run(self):
minimizer_kwargs = {"method": "BFGS"}
initial_paramater_guesses = [2,1.0,1.0/2.0,1.0/3.0,1.0/4.0,1.0/5.0,1.0/6.0]
ret = basinhopping(self.Calculate,
initial_paramater_guesses,
minimizer_kwargs=minimizer_kwargs,
niter=200,
accept_test=self.acceptance_criteria)
my_class = MyClass()
my_class = my_class.run()
I know this post is old, but it still shows up on Googling this question.
I was having the same issue, so I ran a test by modifying the code here a bit and adding a counter. My code minimizes 5 variables, but requires all values to be greater than 0.5
import numpy as np
from scipy.optimize import basinhopping
n = 0
def acceptance_criteria(**kwargs):
print("in accept test")
X = kwargs['x_new']
for x in X:
if x < .5:
print('False!')
return False
return True
def f(x):
global n
print(n)
n += 1
return (x[0]**2-np.sin(x[1])*4+np.cos(x[2]**2)+np.sin(x[3]*5.0)-(x[4]**2 -3*x[4]**3))**2
if __name__ == '__main__':
res = basinhopping(f,[.5]*5,accept_test=acceptance_criteria)
It took about 100 iterations before entering the acceptance_criteria function.
If you are optimizing a function that takes a long time to run (as I was), then you might just need to give it more time to enter into the acceptance_test.
I want to apply a function f to a data X, beign X a numpy array. The problem is that f is a sort of "linear combination" of functions, let's say f_i, and each of this functions depends also of another parameter, let's say:
param = 1.0 #same param for every f_i call.
def f(x):
for xi in range(len(x)):
cummulate sum of f_i(x, xi, param)
return the result of the loop, which depends of (x)
Any help with this? I tried sympy but the f_i are not trivial math function, but a combination of them.
You've got a few approaches here.
First and easiest is to pass in params as an argument, which could be an array of extra arguments for each function:
def f(x, params):
for i in len(x):
# Pass in params[i] to f_i
In the case that you need f to only accept one argument, you can do the second approach using a closure:
def f_creator(params):
def closure(x):
for i in len(x):
# Pass in params[i] to f_i
return closure
f = f_creator(... params for f_is go in here...)
# Use f for any special calculations that you need
Finally if these parameters are constant and don't change over the course of your program, you can set them to global constants. This approach isn't recommended because it makes it difficult to test things and makes the code less robust to change.
params = ....
def f(x):
for i in len(x):
# Calculate f_i using global params
Say I have the following code
def myfunc(x):
return monsterMathExpressionOf(x)
and I would like to find numerically the solution of myfunc(x) == y for diverse values of y. If y == 0 then there are a lot of root finding procedures available, e.g. from scipy. However, if I'd like to find the solution for e.g. y==1 it seems I have to define a new function
def myfunc1(x):
return myfunc(x) - 1
and then find it's root using available procedures. This way does not work for me as I will need to find a lot of solution by running a loop, and I don't want to redefine the function in each step of the loop. Is there a neater solution?
You don't have to redefine a function for every value of y: just define a single function of y that returns a function of x, and use that function inside your loop:
def wrapper(y):
def myfunc(x):
return monsterMathExpressionOf(x) - y
return myfunc
for y in y_values:
f = wrapper(y)
find_root(f, starting_point, ...)
You can also use functools.partial, which may be more to your liking:
def f(x, y):
return monsterMathExpressionOf(x) - y
for y in y_values:
g = partial(f, y=y)
find_root(g, starting_point, ...)
Read the documentation to see how partial is roughly implemented behind the scenes; you'll see it may not be too different compared to the first wrapper implementation.
#Evert's answer shows how you can do this by using either a closure or by using functools.partial, which are both fine solutions.
Another alternative is provided by many numerical solvers. Consider, for example, scipy.optimize.fsolve. That function provides the args argument, which allows you to pass additional fixed arguments to the function to be solved.
For example, suppose myfunc is x**3 + x
def myfunc(x):
return x**3 + x
Define one additional function that includes the parameter y as an argument:
def myfunc2(x, y):
return myfunc(x) - y
To solve, say, myfunc(x) = 3, you can do this:
from scipy.optimize import fsolve
x0 = 1.0 # Initial guess
sol = fsolve(myfunc2, x0, args=(3,))
Instead of defining myfunc2, you could use an anonymous function as the first argument of fsolve:
sol = fsolve(lambda x, y: myfunc(x) - y, x0, args=(3,))
But then you could accomplish the same thing using
sol = fsolve(lambda x: myfunc(x) - 3, x0)
I create a function which returns an value specified by an variable. Like
y = 1.
def f(x):
return y
I need this function as an function object to create another object
dist = dist.distribution(f, other_variables)
this works fine. But if i want to create several different distribution objects (with different functions f in the sense that y changes) Like
dist = dist.distribution(f, other_variables)
y = 2.
dist2 = dist.distribution(f, other_variables)
Then all distribution objects only return the last specified value y. I.e.
dist.f()(1.)
>>>> 2.
dist2.f()(1.)
>>>> 2.
Instead of the expected
dist.f()(1.)
>>>> 12.
dist2.f()(1.)
>>>> 2.
The problem clearly is, that the function f accesses the variable only when it is called and not one time initially.
Is there a way around it?
What I want at the end is:
A function with one variable only (x, although this doesnt do anything in this case, it is needed in others), which returns the value of y of the moment, when the distribution is created. So in principle I want that at the initialisation of the distribution, the given function is deepcopyed, in the sense, that it is no longer influenced by any change of variables.
Is this possible?
Don't use globals for this. There is no need to 'deepcopy' the function either; the y global is not part of the function state at all.
Use a function factory that provides a scoped value instead, or use a functools.partial() to provide a default argument to your function.
Function factory:
def produce_f(y):
def f(x):
return y
return f
dist = dist.distribution(produce_f(1.), other_variables)
Now y is a scoped value for f, produce_f() returns a new f every time it is called, and y is stored as a cell variable for f.
Demo:
>>> f1 = produce_f(12.)
>>> f2 = produce_f(42.)
>>> f1('foo')
12.0
>>> f2('foo')
42.0
Using functools.partial():
from functools import partial
def f(y, x):
return y
dist = dist.distribution(partial(f, 1.), other_variables)
Here partial(f, 1.) produces a new callable that will call f(1., ...) whenever called, appending any extra arguments passed in.
Demo:
>>> f1 = partial(f, 12.)
>>> f2 = partial(f, 42.)
>>> f1('foo')
12.0
>>> f2('foo')
42.0