I just wanted to ask you all about what is fitfunc, errfunc followed by scipy.optimize.leastsq is intuitively. I am not really used to python but I would like to understand this. Here is the code that I am trying to understand.
def optimize_parameters2(p0,mz):
fitfunc = lambda p,p0,mz: calculate_sp2(p, p0, mz)
errfunc = lambda p,p0,mz: exp-fitfunc(p,p0,mz)
return scipy.optimize.leastsq(errfunc, p0, args=(p0,mz))
Can someone please explain what this code is saying narratively word by word?
Sorry for being so specific but I really do have trouble understanding what it's saying.
This particular code snippet is implementing nonlinear least-squares regression to find the parameters of a curve function (this is the fitfunc, here) that best fit a set of data (exp, probably an abbreviation for "experimental data"). leastsq() is a somewhat more general routine for doing nonlinear least-squares optimization, not just curve-fitting. It requires a function (named errfunc, here) that is given a vector of parameters (p) and returns an array. It will attempt to find the parameter vector that minimizes the square of the returned array. In order to implement "fitting a curve to data" with leastsq, you have to provide an errfunc that evaluates the curve (fitfunc) at the given trial parameter vector and then subtracts it from the data (i.e. calculate the "error" or sometimes called the "residuals").
Just to be clear, none of these names are important. I'm just using them to refer to specific parts of the code snippet you provided. You will find other code that uses leastsq() for curve-fitting that names and organizes the code a little bit differently, but now that you know the general scheme, you should be able to follow along.
Python supports the creation of anonymous functions (i.e. functions that are not bound to a name) at runtime, using a construct called lambda. In your example, fitfunc and errfunc are two such lambda functions.
I believe calculate_sp2 and exp_fitfunc are simply two functions which are in the code but you didn't provide their code in the example. So, in short fitfunc actually calls the calculate_sp2 function with 3 parameters (p, p0, mz) and returns the value which is returned by calculate_sp2. errfunc also works in the same manner.
As mentioned in official documentation of scipy.optimize.leastsq, leastsq() minimizes the sum of squares of a set of equations. You can learn about the parameters of leastsq() from the official documentation.
I am giving a simple example to illustrate how lambda function works.
def add(x,y):
return x + y
def subtract(x,y):
return x-y if x > y else y-x
def main(x,y):
addition = lambda x,y: add(x,y)
subtraction = lambda x,y: subtract(x,y)
return addition(x,y) * subtraction(x,y)
print(main(7,4)) # prints 33 which is equal to (7+4)*(7-4)
Related
I have an assignment where I need to make a single defined function that runs newtons method and then be able plug in other defined functions to it and it will solve them all. I wrote one that works for equations that have 1 variable, and I only need to solve for one variable from the system but I don't know how to do that in code without solving for all four of them.
the function I wrote to run newtons method is this:
def fnewton(function, dx, x, n):
#defined the functions that need to be evaluated so that this code can be applied to any function I call
def f(x):
f=eval(function)
return f
#eval is used to evaluate whatever I put in the function place when I recall fnewton
#this won't work without eval to run the functions
def df(x):
df=eval(dx)
return df
for intercept in range(1,n):
i= x-(f(x)/df(x))
x= i
#this is literally just newtons method
#to find an intercept you can literally input intercept in a for loop and it'll do it for you
#I just found this out
#putting n in the range makes it count iterations
print ('root is at')
print (x)
print ('after this many iterations:')
print (n)
my current system of equations function looks like this:
def c(x):
T=x[0]
y=x[1]
nl=x[2]
nv=x[3]
RLN=.63*Antoine(An,Bn,Cn,T)-y*760
RLH=(1-.63)*Antoine(Ah,Bh,Ch,T)-(1-y)*760
N=.63*nl+y*nv-50
H=(1-.63)*nl+(1-y)*nv-50
return[RLN,RLH,N,H]
To use my function to solve this I've entered multiple variations of:
fnewton("c(x)","dcdx(x)", (2,2,2,2), 10)
Do I need to change the system of equations into 1 equation somehow or something I don't know how to manipulate my code to make this work and also work for equations with only 1 variable.
To perform Newton's method in multiple dimensions, you must replace the simple derivative by a Jacobian matrix, that is, a matrix that has the derivatives of all components of your multidimensional function with respect to all given variables. This is decribed here: https://en.wikipedia.org/wiki/Newton%27s_method#Systems_of_equations,
(or, perhaps more helpful, here: https://web.mit.edu/18.06/www/Spring17/Multidimensional-Newton.pdf in Sec. 1.4)
Instead of f(x)/f'(x), you need to work with the inverse of the Jacobian matrix times the vector function f. So the formula is actually quite similar!
I had no idea how to phrase the title of this question, so apologies for any confusion there. I am using the pymanopt package for optimization and would like to be able to create some sort of a function/method that allows for a generalized input (variable amount of input arrays). To use pymanopt, one has to provide a cost function defined in terms of array that are to be optimized to minimize the cost.
For example, a cost function could be:
#pymanopt.function.Autograd
def f(A,B):
return ((X - A#B.T)**2).sum()
To do the optimization, the variable X is defined prior to f, then f is supplied as the cost function to the pymanopt solver. Optimization is done with respect to the arguments of f and these arrays are returned by pymanopt with values that minimize the cost function.
Ideally, I would like to be able to do this definition more dynamically. So instead of defining a function in terms of hard coded arrays, to be able to supply a list of variables to be optimized. So if my cost function was instead:
#pymanopt.function.Autograd
def f(L):
return ((X - np.linalg.multi_dot(L)**2).sum()
Where the arrays A,B,...,C would be stored in a list, L. However, as far as I can tell, the variables to be optimized have to be directly defined as individual arrays in the cost function supplied to the solver.
The only thing I can think of doing is to define the cost function by creating a string that contains the 'hard coded' function and executing it via exec() with something like this:
args = ','.join(['A{}'.format(i) for i in range(len(L))])
exec('#pymanopt.function.Autograd\ndef({}):\n\treturn ((X-np.linalg.multi_dot({}))**2).sum()'.format(args,args))
but I understand that using this method should be avoided if possible. Any advice for navigating this sort of problem is greatly appreciated - thanks! Please let me know if anything is unclear/doesn't make sense.
I'm working through some exercises on improper integrals and I've stumbled across an issue I can't resolve. I'm attempting to use the limit() function on the following problem:
Here N(x) is the cumulative distribution function of the standard normal variable.
The limit() function so far hasn't caused any problems, including problems which require L'Hôpital's rule be applied. However, I'm struggling to get compute the correct answer for this particular problem and can't work out why. The following code yields an incorrect answer
from sympy import *
x, y = symbols('x y')
init_printing(use_unicode=False) #Print the answers in unicode characters
cum_distribution = (1/sqrt(2*pi)*(integrate(exp(-y**2/2), (y, -oo, x))))
func = (cum_distribution -(1/2)-(x/sqrt(2*pi)))/(x**3)
limit(func, x, 0)
If I apply L'Hôpital's rule, i get the correct
l_hopital = diff((cum_distribution -(1/2)-(x/sqrt(2*pi))), x)/diff(x**3, x)
limit(l_hopital, x, 0)
I looked through the limit() function source code and my understanding is that L'Hôpital's rule isn't applied? In this case, can this problem be solved using the limit() function without applying this rule?
At present, a limit involving the function erf (known as the error function, related to normal CDF) can only be evaluated when the argument of erf tends to positive infinity. Limits at other places are either not evaluated, or evaluated incorrectly. (Related PR). This includes the limit
limit(-(sqrt(2)*x - sqrt(pi)*erf(sqrt(2)*x/2))/(2*sqrt(pi)*x**3), x, 0)
which returns unevaluated (though I would not call this incorrect). As a workaround, you can compute the Taylor series of this function with one term (the constant term), which gives the correct value of the limit:
series(func, x, 0, 1).removeO()
returns -sqrt(2)/(12*sqrt(pi)).
As in calculus practice, L'Hopital's rule is inferior to power series techniques when it comes to algorithmic computations, and SymPy relies primarily on the latter. The algorithm it uses is devised and explained in On Computing Limits in a Symbolic Manipulation System by Dominik Gruntz.
I'm trying to use scipy.optimize.minimize to minimize a complicated function. I noticed in hindsight that the minimize function takes the objective and derivative functions as separate arguments. Unfortunately, I've already defined a function which returns the objective function value and first-derivative values together -- because the two are computed simultaneously in a for loop. I don't think there is a good way to separate my function into two without the program essentially running the same for loop twice.
Is there a way to pass this combined function to minimize?
(FYI, I'm writing an artificial neural network backpropagation algorithm, so the for loop is used to loop over training data. The objective and derivatives are accumulated concurrently.)
Yes, you can pass them in a single function:
import numpy as np
from scipy.optimize import minimize
def f(x):
return np.sin(x) + x**2, np.cos(x) + 2*x
sol = minimize(f, [0], jac=True, method='L-BFGS-B')
Something that might work is: you can memoize the function, meaning that if it gets called with the same inputs a second time, it will simply return the same outputs corresponding to those inputs without doing any actual work the second time. What is happening behind the scenes is that the results are getting cached. In the context of a nonlinear program, there could be thousands of calls which implies a large cache. Often with memoizers(?), you can specify a cache limit and the population will be managed FIFO. IOW you still benefit fully for your particular case because the inputs will be the same only when you are needing to return function value and derivative around the same point in time. So what I'm getting at is that a small cache should suffice.
You don't say whether you are using py2 or py3. In Py 3.2+, you can use functools.lru_cache as a decorator to provide this memoization. Then, you write your code like this:
#functools.lru_cache
def original_fn(x):
blah
return fnvalue, fnderiv
def new_fn_value(x):
fnvalue, fnderiv = original_fn(x)
return fnvalue
def new_fn_deriv(x):
fnvalue, fnderiv = original_fn(x)
return fnderiv
Then you pass each of the new functions to minimize. You still have a penalty because of the second call, but it will do no work if x is unchanged. You will need to research what unchanged means in the context of floating point numbers, particularly since the change in x will fall away as the minimization begins to converge.
There are lots of recipes for memoization in py2.x if you look around a bit.
Did I make any sense at all?
I am having trouble sovling the optical bloch equation, which is a first order ODE system with complex values. I have found scipy may solve such system, but their webpage offers too little information and I can hardly understand it.
I have 8 coupled first order ODEs, and I should generate a function like:
def derv(y):
compute the time dervative of elements in y
return answers as an array
then do complex_ode(derv）
My questions are:
my y is not a list but a matrix, how can i give a corrent output
fits into complex_ode()?
complex_ode() needs a jacobian, I have no idea how to start constructing one
and what type it should be?
Where should I put the initial conditions like in the normal ode and
time linspace?
this is scipy's complex_ode link:
http://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.complex_ode.html
Could anyone provide me with more infomation so that I can learn a bit more.
I think we can at least point you in the right direction. The optical
bloch equation is a problem which is well understood in the scientific
community, although not by me :-), so there are already solutions on the internet
to this particular problem.
http://massey.dur.ac.uk/jdp/code.html
However, to address your needs, you spoke of using complex_ode, which I suppose
is fine, but I think just plain scipy.integrate.ode will work just fine as well
according to their documentation:
from scipy import eye
from scipy.integrate import ode
y0, t0 = [1.0j, 2.0], 0
def f(t, y, arg1):
return [1j*arg1*y[0] + y[1], -arg1*y[1]**2]
def jac(t, y, arg1):
return [[1j*arg1, 1], [0, -arg1*2*y[1]]]
r = ode(f, jac).set_integrator('zvode', method='bdf', with_jacobian=True)
r.set_initial_value(y0, t0).set_f_params(2.0).set_jac_params(2.0)
t1 = 10
dt = 1
while r.successful() and r.t < t1:
r.integrate(r.t+dt)
print r.t, r.y
You also have the added benefit of an older more established and better
documented function. I am surprised you have 8 and not 9 coupled ODE's, but I'm
sure you understand this better than I. Yes, you are correct, your function
should be of the form ydot = f(t,y), which you call def derv() but you're
going to need to make sure your function takes at least two parameters
like derv(t,y). If your y is in matrix, no problem! Just "reshape" it in
the derv(t,y) function like so:
Y = numpy.reshape(y,(num_rows,num_cols));
As long as num_rows*num_cols = 8, your number of ODE's you should be fine. Then
use the matrix in your computations. When you're all done, just be sure to return
a vector and not a matrix like:
out = numpy.reshape(Y,(8,1));
The Jacobian is not required, but it will likely allow the computation to proceed
much more quickly. If you do not know how to compute this you may want to consult
wikipedia or a calculus text book. It's pretty simple, but can be time consuming.
As far as initial conditions, you should probably already know what those should
be, whether it's complex or real valued. As long as you select values that are
within reason, it shouldn't matter much.