I am trying to optimise the output of a function using the scipy basinhopping algorithm.
def acceptance_criteria(self,**kwargs):
print "kwargs "
print kwargs
x = kwargs["x_new"]
beta = x[0]
alpha = [x[1],x[2],x[3],x[4],x[5],x[6]]
print x
inputnow= raw_input()
beta_gamma_pass = beta != self.gamma
beta_zero_pass = beta >= 0.0
alpha1_pass = alpha[0] > 0.0
alpha2_pass = alpha[1] > 0.0
alpha3_pass = alpha[2] > 0.0
alpha4_pass= alpha[3] > 0.0
alpha5_pass= alpha[4] > 0.0
alpha6_pass= alpha[5] > 0.0
return beta_gamma_pass,beta_zero_pass,alpha1_pass,alpha2_pass,alpha3_pass,alpha4_pass,alpha5_pass,alpha6_pass
def variational_calculation(self):
minimizer_kwargs = {"method": "BFGS"}
initial_paramater_guesses = [2,1.0,1.0/2.0,1.0/3.0,1.0/4.0,1.0/5.0,1.0/6.0]
ret = basinhopping(self.Calculate, initial_paramater_guesses, minimizer_kwargs=minimizer_kwargs, niter=200, accept_test=self.acceptance_criteria)
I am getting problems with Nans and infs in my calculate function.
This is due to invalid parameter values being used.
I have attempted to prevent this by using acceptance criteria.
But the basinhopping routine does not call the accept_test function.
Thus the criteria remain unimplemted.
Can anyone help me out as to why basinhopping isn't calling the accept_test function?
Thanks
EDIT:
in response to #sascha's comment,
There are fractional powers of parameters, and 1/parameter terms in the function.
Not limiting the range of the allowed parameter values gives complex and inf values in this case.
It is actually an eigenvalue problem, where I am trying to minimise the trace of the eigenvalues of a set of 18*18 matrices.
The matrix elements depend on the 7 parameters in a complex way with dozens of non linear terms.
I have never worked on anything more complex than polynomial regression before, so I am not familiar with the algorithms or their applicability at all.
However, the function/s that I am trying to minimise are smooth as long as you avoid parameter values near poles; caused by 1/parameter and 1/(paramter^n -constant) terms.
EDIT2:
QUESTION CLARIFICATION
The question here is nothing to do with the applicability of the basinhopping algorithm.
It is why the specific implementation of it, in the 2.7 version of python and scipy, does not call the accept_test function?
I can't say why your example doesn't work, but here's a similar but minimal example where it does call the accept_test, maybe you can spot the difference
import scipy
import numpy as np
from scipy.optimize import basinhopping
class MyClass:
def Calculate(self, x):
return np.dot(x, x)
def acceptance_criteria(self, **kwargs):
print("in accept test")
return True
def run(self):
minimizer_kwargs = {"method": "BFGS"}
initial_paramater_guesses = [2,1.0,1.0/2.0,1.0/3.0,1.0/4.0,1.0/5.0,1.0/6.0]
ret = basinhopping(self.Calculate,
initial_paramater_guesses,
minimizer_kwargs=minimizer_kwargs,
niter=200,
accept_test=self.acceptance_criteria)
my_class = MyClass()
my_class = my_class.run()
I know this post is old, but it still shows up on Googling this question.
I was having the same issue, so I ran a test by modifying the code here a bit and adding a counter. My code minimizes 5 variables, but requires all values to be greater than 0.5
import numpy as np
from scipy.optimize import basinhopping
n = 0
def acceptance_criteria(**kwargs):
print("in accept test")
X = kwargs['x_new']
for x in X:
if x < .5:
print('False!')
return False
return True
def f(x):
global n
print(n)
n += 1
return (x[0]**2-np.sin(x[1])*4+np.cos(x[2]**2)+np.sin(x[3]*5.0)-(x[4]**2 -3*x[4]**3))**2
if __name__ == '__main__':
res = basinhopping(f,[.5]*5,accept_test=acceptance_criteria)
It took about 100 iterations before entering the acceptance_criteria function.
If you are optimizing a function that takes a long time to run (as I was), then you might just need to give it more time to enter into the acceptance_test.
Related
I'm implementing a Markov Chain Montecarlo with metropolis and barkes alphas for numerical integration. I've created a class called MCMCIntegrator(). I've loaded it with some attributes, one of then is the pdf of the function (a lambda) we're trying to integrate called g.
import numpy as np
import scipy.stats as st
class MCMCIntegrator:
def __init__(self):
self.g = lambda x: st.gamma.pdf(x, 0, 1, scale=1 / 1.23452676)*np.abs(np.cos(1.123454156))
self.size = 10000
self.std = 0.6
self.real_int = 0.06496359
There are other methods in this class, the size is the size of the sample that the class must generate, std is the standard deviation of the Normal Kernel, which you will see in a few seconds. The real_int is the value of the integral from 1 to 2 of the function we're integrating. I've generated it with a R script. Now, to the problem.
def _chain(self, method=None):
"""
Markov chain heat-up with burn-in
:param method: Metrpolis or barker alpha
:return: np.array containing the sample
"""
old = 0
sample = np.zeros(int(self.size * 1.5))
i = 0
if method:
def alpha(a, b): return min(1, self.g(b) / self.g(a))
else:
def alpha(a, b): return self.g(b) / (self.g(a) + self.g(b))
while i != len(sample):
new = st.norm(loc=old, scale=self.std).rvs()
new = abs(new)
al = alpha(old, new)
u = st.uniform.rvs()
if al > u:
sample[i] = new
old = new
i += 1
return np.array(sample)
Below this method is an integrate() method that calculates the proportion of numbers in the [1, 2] interval:
def integrate(self, method=None):
"""
Integration step
"""
sample = self._chain(method=method)
# discarding 30% of the sample for the burn-in
ind = int(len(sample)*0.3)
sample = sample[ind:]
setattr(self, "sample", sample)
sample = [1 if 1 < v < 2 else 0 for v in sample]
return np.mean(sample)
This is the main function:
def main():
print("-- RESULTS --".center(20), end='\n')
mcmc = MCMCIntegrator()
print(f"\t{mcmc.integrate()}", end='\n')
print(f"\t{np.abs(mcmc.integrate() - mcmc.real_int) / mcmc.real_int}")
if __name__ == "__main__":
main()
I'm stuck in an infinity while loop and I have no idea why this is happening.
Couple things... You are hung up in the chain method because the alpha computation is returning NaN, because g() is returning NaN. Take a look at the print statements I inserted into your code and run it...
tips:
Make g() a class function just like chain.
Test g() on some test values, something is clearly amiss
Don't conditionally define a function like alpha. Wayyy confusing and error prone/tough to troubleshoot. Just pass alpha what it needs and you also can make it a class function alpha(a, b, method=None)
Take a look at where you are updating i in the `_chain' function.... You realize you are risking a long looping process because you only update i conditionally!
You are set up for disaster with your use of numpy array. You may have a bunch of trailing zeros after your actual data because you are over-writing the large zeros list. You do NOT need numpy array here, just use a python empty list and append new values to it, either zero or one...based on whatever.
Add a couple print statements in when you are troubleshooting (or unit test your functions). Try my adds to your function below... it is what I used to figure out what was going on
def _chain(self, method=None, verbose=True):
"""
Markov chain heat-up with burn-in
:param method: Metrpolis or barker alpha
:return: np.array containing the sample
"""
old = 0
sample = np.zeros(int(self.size * 1.5))
i = 0
if method:
def alpha(a, b): return min(1, self.g(b) / self.g(a))
else:
def alpha(a, b):
if verbose: print(f'g(a): {self.g(a)}, g(b): {self.g(b)}')
return self.g(b) / (self.g(a) + self.g(b))
while i != len(sample):
new = st.norm(loc=old, scale=self.std).rvs()
new = abs(new)
al = alpha(old, new)
u = st.uniform.rvs()
if verbose: print(f'old: {old:.3f} new: {new:.3f} alpha: {al:.3f} u: {u:.3f}')
if al > u:
sample[i] = new
old = new
i += 1 # do you really want to conditionally update i?
sys.exit(-1) # to breakout of infinite loop...
return np.array(sample)
I wrote a script in Python that finds the zero of a fairly complicated function using fsolve. The way it works is as follows. There is a class that simply stores the parameter of the function. The class has an evaluate method that returns a value based on the stored parameter and another method (inversion) that finds the parameter at which the function takes the supplied output.
The inversion method updates the parameter of the function at each iteration and it keeps on doing so until the mismatch between the value returned by the evaluate method and the supplied value is zero.
The issue that I am having is that while the value returned by the inversion method is correct, the parameter, that is part of the object, is always 0 after the inversion method terminates. Oddly enough, this issue disappears if I use root instead of fsolve. As far as I know, fsolve is just a wrapper for root with some settings on the solver algorithm and some other things enforced.
Is this a known problem with fsolve or am I doing something dumb here? The script below demonstrates the issue I am having on the sine function.
from scipy.optimize import fsolve, root
from math import sin, pi
class invertSin(object):
def __init__(self,x):
self.x = x
def evaluate(self):
return sin(self.x)
def arcsin_fsolve(self,y):
def errorfunc(xnew):
self.x = xnew
return self.evaluate() - y
soln = fsolve(errorfunc, 0.1)
return soln
def arcsin_root(self,y):
def errorfunc(xnew):
self.x = xnew
return self.evaluate() - y
soln = root(errorfunc, 0.1, method = 'anderson')
return soln
myobject = invertSin(pi/2)
x0 = myobject.arcsin_fsolve(0.5) #find x s.t. sin(x) = 0.5 using fsolve
print(x0) #this prints pi/6
x0obj = myobject.x
print(x0obj) #this always prints 0 no matter which function I invert
myobject2 = invertSin(pi/2)
x1 = myobject2.arcsin_root(0.5) #find x s.t. sin(x) = 0.5 using root
print(x1) #this prints pi/6
x1obj = myobject2.x
print(x1obj) #this prints pi/6
If you add print statements for xnew in the errorfunc then you will see that fsolve works with a list (of one element). This means that the function is re-interpreted that way, not the original function. Somehow the type information is lost after exiting the solver so that then the address/reference to that list is interpreted as floating point data, which gives the wrong value.
Setting self.x = xnew[0] there restores the desired behavior.
import math
class Vector():
vA = [3.183, 7.627]
def magnitude(vector):
sum = 0
i = 0
while i < len(vector):
sum += vector[i]
i += 1
return math.sqrt(sum)
def unitVector(vector):
print( 1 / (magnitude(vA) * vA))
I'm attempting to code some linear algebra and calculate the unit vectors for the vector 'vA' stated above. When I run the code I get NameError: global name 'magnitude' is not defined. I do not understand why I am having an issue with simply calling one function from another.
I'm a beginner with python and I'm assuming I have a misunderstanding about classes and functions, but I have looked through the documentation and cannot find the answer I am looking for.
You have several errors in your code:
def magnitude(vector) should be def magnitude(self, vector)
def unitVector(vector) should be def unitVector(self, vector)
magnitude(vA) should be self.magnitude(vA)
EDIT:
A better way of writing your class would be to use OOP concepts in Python, so you do not have to pass vector as a function argument if you make it an instance variable.
Your class can be rewrote like this:
import math
class Vector():
def __init__(self, vector):
self.vector = vector
def magnitude(self):
sum = 0
i = 0
while i < len(self.vector):
sum += self.vector[i]
i += 1
return math.sqrt(sum)
def unitVector(self):
print( 1 / (self.magnitude() * self.vector))
vA = [3.183, 7.627]
vec = Vector(vA)
vec.unitVector()
Be aware that it does not work. Because in unitVector, Python doesn't know how to multiply a float by a list (self.magnitude() returns a float and self.vector is a list). You probably want to rework this part.
I have solved a previous problem and now I am stuck with this one.
It is asking for an improvement of the previous solution which I posted below, but can't quite understand what the problem is asking. (Also, can't really figure out how to solve it.)
Please help
thanks.
Problem:
Elena complains that the recursive newton function in Project 2 includes
an extra argument for the estimate. The function’s users should not have to
provide this value, which is always the same, when they call this function.
Modify the definition of the function so that it uses a keyword parameter
with the appropriate default value for this argument, and call the function
without a second argument to demonstrate that it solves this problem.
Here is my code:
def newtonSquare(x, estimate):
if abs(x-estimate ** 2) <= 0.000001:
return estimate
else:
return newtonSquare(x, (estimate + x / estimate) / 2)
def main():
num = int(raw_input('Enter a positive number >> '))
print newtonSquare(num, 1.0)
main()
The problem is a complaint that when somebody calls newtonSquare, they always do so with an initial estimate of 1.0. To set it as the default value, unless you explicitly provide it, you only have to modify the definition of the function to:
def newtonSquare(x, estimate = 1.0):
An alternative to the default value solution of Michael's is defining the recursive function local to the newtonSquare function:
def newtonSquare(x):
def f(x, estimate):
if abs(x-estimate ** 2) <= 0.000001:
return estimate
else:
return f(x, (estimate + x / estimate) / 2)
return f(x, 1.0)
def main():
num = int(raw_input('Enter a positive number >> '))
print newtonSquare(num)
main()
In this case having the default value is nicer and more flexible though, but this can still be a useful pattern.
I am currently solving a system of differential equation under python using odeint to simulate charged particles in a field (the source comes from this package):
time = np.linspace(0, 5, 1000)
def sm(x, t):
return np.array([x[1], eta*Ez0(x[0])])
traj = odeint(sm,[0,1.], time)
It works fine but I would like to stop the calculation as soon as x[0] < 0. For the moment I just block the evolution of the sytem:
def sm1(x, t):
if x[0] < 0:
return np.array([0, 0])
else:
return np.array([x[1], eta*Ez0(x[0])])
traj = odeint(sm1,[0,1.],time)
but I gess there are better solutions. I've found this but is seems to me that it fixes the number of steps, which is regrettable.
Any suggestion appreciated.
If you write a custom extension of the odeint function, you could have your function raise a particular exception when it's finished. Doing it in Python might make it substantially slower, but I think you write the same thing in C or Cython. Note that I haven't tested the following.
class ThatsEnoughOfThat(Exception):
pass
def custom_odeint(func, y0, t): # + whatever parameters you need
for timestep in t:
try:
# Do stuff. Call odeint/other scipy functions?
except ThatsEnoughOfThat:
break
return completedstuff
def sm2(x, t):
if x[0] < 0:
raise ThatsEnoughOfThat
return np.array([x[1], eta*Ez0(x[0])])