This is my first time posting a question here, so please bear with me. Now, I was "playing" with the minimize function and I came across something odd.
The set up is simple, all I want to do is to minimize the euclidian norm of the form:
sqrt[(x1 -a1)^2+(x2 -a2)^2+(x3 -a3)^2+(x4 -a4)^2]
Where x's are variables and a's are some fixed numbers, constants. Of course the solution is simple, xi=ai for i=1,2,3,4; and the value of the function is zero.
Now, when I do this code in python, everything works fine:
import numpy as np
from scipy.optimize import minimize
def w(x):
a = np.array([[0,1,1,0]]).T
return np.sqrt((x[0]-a[0])**2 + (x[1]-a[1])**2 + (x[2]-a[2])**2+(x[3]-a[3])**2)
np.random.seed(1)
x0 = np.random.random((4,1))
min_fun1 = minimize(w,x0,method='nelder-mead',options={'xatol': 1e-8, 'disp': True})
print(min_fun1.fun)
print(min_fun1.x)
As I said, this works just fine. The value of the function is zero and xi=ai. But, when I try to set up the problem using the np.linlag.norm function, it gets weird. For example, when I run this code:
def y(x):
a = np.array([[0,1,1,0]]).T
return np.linalg.norm(x-a)
min_fun0 = minimize(y,x0,method='nelder-mead',options={'xatol': 1e-8, 'disp': True})
min_fun0.fun
min_fun0.x
It seems like the optimization gets stuck, the value of the function is 2 and xi=0.5, which obviously it is not correct.
Even more, if I try to do it with the dot product, the result is different but still incorrect:
def v(x):
a = np.array([[0,1,1,0]]).T
return np.sqrt(np.dot((x-a).T,(x-a)))
min_fun2 = minimize(v,x0,method='nelder-mead',options={'xatol': 1e-8, 'disp': True})
min_fun2.fun
min_fun2.x
What I was able to understand is that it has something to do with the "type" of the object. For example, no.linalg.norm returns a float64 and for the example that works is ndarray with size 1. So, I change the "type" for no.linalg.norm result to match ndarray size 1, but the problem persists.
Looks like you're trying to minimize a 1d array, but np.array([[0,1,1,0]]) is actually 2d (shape is (1, 4)).
If you convert your array to 1d the minimisation seems to work:
def y(x):
a = np.array([0,1,1,0])
return np.linalg.norm(x-a)
min_fun0 = minimize(y,x0,method='nelder-mead',options={'xatol': 1e-8, 'disp': True})
min_fun0.fun
min_fun0.x
>>> array([-1.87741758e-09, 1.00000000e+00, 1.00000000e+00, -3.04858536e-09])
Related
I'm trying to find global maximum of a Python function with a lot of parameters (500+). Unfortunately I'm unable to make a derivative of this function - what the function does is basically it loops several times over np.array of shape ~ (150000,50) and then does some calculations with the data.
So far I was using scipy.optimize.minimize, method=Powell which seemed to give the best results out of the scipy.optimize.minimize methods.
At first I thought the output of the minimize function was final and the best result that could be found - but then I found out that in some cases - when I save the coefficients and run the minimize function again with the coefficients from the previous run as starting values it gives higher values than the previous run. So what I'm basically doing is the following:
import numpy as np
from scipy.optimize import minimize
from numba import jit
#jit (nopython=True)
def MyFunction (coefs, data):
# do some calculations
return value*-1
data = np.load('myData.npy', allow_pickle=True)
coefs = np.random.sample(500)
res = minimize(MyFunction, coefs, args = (data), method='Powell', options={'maxiter': 100000000, 'disp': True, 'return_all': True})
for a in range (0, 10000):
res = minimize(MyFunction, coefs, args = (data), method='Powell', options={'maxiter': 100000000, 'disp': True, 'return_all': True})
coefs = res.x
Is there some more effective way?
How can I speed up the code? Yes I already made the code much faster with using jit. But what about threading? Or some other idea?
Following this tutorial, I have created the following churn.py file:
import numpy as np
import scipy as sp
import scipy.stats as stats
#duration of alive subscriptions
censored = np.array([419,513, ... ,316,14])
#duration of completed subscriptions
uncensored = np.array([389,123,340, ... ,56,31])
#Log likelihoods for censored data
def log_likelihood_lomax(args):
shape, scale = args
val = stats.lomax.logpdf(uncensored, shape, loc=0, scale=scale).sum() + stats.lomax.logsf(censored, shape, loc=0, scale=scale).sum()
return -val
res_lomax = sp.optimize.minimize(log_likelihood_lomax, [1, 1], bounds=((0.001, 1000000), (0.001, 1000000)))
print("lomax shape", res_lomax.x[0], ", scale=", res_lomax.x[1])
print("lomax mean", stats.lomax.mean(res_lomax.x[0], scale=res_lomax.x[1]))
print("lomax median", stats.lomax.median(res_lomax.x[0], scale=res_lomax.x[1]))
Note: the ... in the censored & uncensored arrays are here for confidentiality purposes. In the actual script, I have included real values instead.
When I run this script with python3 churn.py, I get the following results:
lomax shape 0.36948878639375643 , scale= 1440.4384891101636
lomax mean inf
lomax median 7961.447172364986
I know for a fact that the value returned for the median is incorrect.
But most importantly, I don't understand why the lomar mean returns inf.
Is there anything wrong in my script?
Your result shows
lomax shape 0.36948878639375643
That is, using scipy's notation, the shape parameter c is 0.36948878639375643 (in the wikipedia article, c is α).
For c ≤ 1, the mean of the distribution is infinite (that is, the integral that defines the mean diverges).
You asked "Is there anything wrong in my script?" There is one important change that I recommend: after you call minimize, check that res_lomax.success is True before you use the values in res_lomax.x. Something like this:
res_lomax = sp.optimize.minimize(log_likelihood_lomax, [1, 1], bounds=((0.001, 1000000), (0.001, 1000000)))
if res_lomax.success:
print("lomax shape", res_lomax.x[0], ", scale=", res_lomax.x[1])
print("lomax mean", stats.lomax.mean(res_lomax.x[0], scale=res_lomax.x[1]))
print("lomax median", stats.lomax.median(res_lomax.x[0], scale=res_lomax.x[1]))
else:
print("minimization failed:", res_lomax.message)
I am trying to utilise scipy.optimise.fsolve for solving a function. I noticed that the function is evaluated with the same value multiple times in the beginning and the end of the iteration steps. For example when the following code is evaluated:
from scipy.optimize import fsolve
def yy(x):
print(x)
return x**2+9*x+20
y = fsolve(yy,22.)
print(y)
The following output is obtained:
[ 22.]
[ 22.]
[ 22.]
[ 22.00000033]
[ 8.75471707]
[ 4.34171812]
[ 0.81508685]
[-1.16277103]
[-2.42105811]
[-3.17288066]
[-3.61657372]
[-3.85653348]
[-3.96397335]
[-3.99561793]
[-3.99984826]
[-3.99999934]
[-4.]
[-4.]
[-4.]
Therefore the function is evaluated with 22. three times, which is unnecessary.
This is especially annoying when the function requires substantial evaluation time. Could anyone please explain this and suggest how to avoid this issue?
The first evaluation is done only to check the shape and data type of the output of the function. Specifically, fsolve calls _root_hybr which contains the line
shape, dtype = _check_func('fsolve', 'func', func, x0, args, n, (n,))
Naturally, _check_func calls the function:
res = atleast_1d(thefunc(*((x0[:numinputs],) + args)))
Since only the shape and data type are retained from this evaluation, the solver will be calling the function with the value x0 again when actual root finding process begins.
The above accounts for one extraneous call (out of two). I did not track down the other one, but it's conceivable that the FORTRAN code does some kind of preliminary check of its own. This sort of thing happens when algorithms written long ago get wrapped over and over again.
If you really want to save these two evaluations of expensive function yy, one way is to compute the value yy(x0) separately and store it. For example:
def yy(x):
if x == x0 and y0 is not None:
return y0
print(x)
return x**2+9*x+20
x0 = 22.
y0 = None
y0 = yy(x0)
y = fsolve(yy, x0)
I realised an important reason for this issue is that fsolve is not meant for such a problem: Solvers should be chosen wisely :)
multivariate: fmin, fmin_powell, fmin_cg, fmin_bfgs, fmin_ncg
nonlinear: leastsq
constrained: fmin_l_bfgs_b, fmin_tnc, fmin_cobyla
global: basinhopping, brute, differential_evolution
local: fminbound, brent, golden, bracket
n-dimensional: fsolve
one-dimensional: brenth, ridder, bisect, newton
scalar: fixed_point
I want to adjust the parameters of a model to a given set of data.
I'm trying to use scipy's function curve_fit in Sage, but I keep getting
TypeError: only length-1 arrays can be converted to Python scalars
Here´s my code:
from numpy import cos,exp,pi
f = lambda x: exp( - 1 / cos(x) )
import numpy as np
def ang(time): return (time-12)*pi/12
def temp(x,maxtemp):
cte=(273+maxtemp)/f(0)**(1/4)
if 6<x and x<18:
return float(cte*f(ang(x))**(1/4)-273)
else:
return -273
lT=list(np.linspace(15,40,1+24*2))
lT=[float(num) for num in lT] #list of y data
ltimes=np.linspace(0,24,6*24+1)[1:]
ltimes=list(ltimes) #list of x data
u0=lT[0]
def u(time,maxtemp,k): #the function I want to fit to the data
def integ(t): return k*exp(k*t)*temp(t,maxtemp)
return exp(-k*time)*( numerical_integral(integ, 0, time)[0] + u0 )
import scipy.optimize as optimization
print optimization.curve_fit(u, ltimes, lT,[1000,0.0003])
scipy.optimize.curve_fit expects the model function to be vectorized: that is, it must be able to receive an array (ndarray, to be precise), and return an array of values. You can see the problem right away by adding a debug printout
def u(time,maxtemp,k):
print time % for debugging
def integ(t): return k*exp(k*t)*temp(t,maxtemp)
return exp(-k*time)*( numerical_integral(integ, 0, time)[0] + u0 )
The output of print will be the entire array ltimes you are passing to curve_fit. This is something numerical_integral is not designed to handle. You need to give it values one by one.
Like this:
def u(time,maxtemp,k):
def integ(t): return k*exp(k*t)*temp(t,maxtemp)
return [exp(-k*time_i)*( numerical_integral(integ, 0, time_i)[0] + u0) for time_i in time]
This will take care of the “only length-1 arrays can be converted" error. You will then have another one, because your lists ltimes and lT are of different length, which doesn't make sense since lT is supposed to be the target outputs for inputs ltimes. You should revise the definitions of these arrays to figure out what size you want.
I am trying to solve a differential equation in python using Scipy's odeint function. The equation is of the form dy/dt = w(t) where w(t) = w1*(1+A*sin(w2*t)) for some parameters w1, w2, and A. The code I've written works for some parameters, but for others I get given index out of bound errors.
Here's some example code that works
import numpy as np
import scipy.integrate as integrate
t = np.arange(1000)
w1 = 2*np.pi
w2 = 0.016*np.pi
A = 1.0
w = w1*(1+A*np.sin(w2*t))
def f(y,t0):
return w[t0]
y = integrate.odeint(f,0,t)
Here's some example code that doesn't work
import numpy as np
import scipy.integrate as integrate
t = np.arange(1000)
w1 = 0.3*np.pi
w2 = 0.005*np.pi
A = 0.15
w = w1*(1+A*np.sin(w2*t))
def f(y,t0):
return w[t0]
y = integrate.odeint(f,0,t)
The only thing that changes between these is that the three parameters w1, w2, and A are smaller in the second, but the second one always gives me the following error
line 13, in f
return w[t0]
IndexError: index 1001 is out of bounds for axis 0 with size 1000
This error continues even after restarting python and running the second code first. I've tried with other parameters, some seem to work, but others give me different index out of bounds errors. Some say 1001 is out of bounds, some say 1000, some say 1008, ect.
Changing the initial condition on y (the second input for odeint, which I have as 0 on the above codes) also changes the number on the index error, so it might be that I'm misunderstanding what to put here. I wasn't told what the initial conditions should be other than that y is used as a phase of a signal, so I presumed it to be initially 0.
What you want to do is
def w(t):
return w1*(1+A*np.sin(w2*t))
def f(y,t0):
return w(t0)
Array indices are typically integers, time arguments and values of solutions of differential equations are typically real numbers. Thus there is some conceptual difficulty in invoking w[t0].
You might also try to integrate directly the function w, there is no inherent difficulty in this example.
As for coupled systems, you solve them as coupled systems.
def w(t):
return w1*(1+A*np.sin(w2*t))
def f(y,t):
wt = w(t)
return np.array([ wt, wt*sin(y[1]-y[0]) ])