I'm writing this program where I have to do a bunch of optimizations. Some with only 1 variable, some with 2. At first I was using the basinhopping algorithm from the scipy.optimize library, but I figured that the normal minimize algorithm should do the job. The basinhopping optimization was working, more or less, but it was extremely time-consuming. Now, I'm using the normal minimize optimization and I've already figured out how to do it for 1 variable. The code for this is given below. I'm using the COBYLA method here, since this one seems to be the only one working. (Nelder-Mead and Powell also work, but sometimes they give back a negative x, which I can't have. And since both of these methods are unconstrained, I can't use them). Hence my first question: what is the difference between all the methods and why do some of the methods converge for my function and others don't?
x0 = [50]
func = lambda x: calculate_score(Sector(center, x, rot), im, msk)
ret = op.minimize(func, x0, method='COBYLA')
print(ret)
The code that I use for the optimization for 2 variables is quite identical to the one for 1 variable, but somehow it gives me the wrong results. Does this have to do with the method I'm using? Or what could be the problem here?
x0 = [50, 50]
func = lambda x: calculate_score(Triangle(center, x[0], x[1], rot), im, msk)
ret = op.minimize(func, x0, method='COBYLA')
print(ret.x[0], ret.x[1])
For the sake of completeness, below is my code for the calculate_score function. I was maybe thinking to calculate the gradient of this function, so that given this gradient the BFGS or L-BFGS-B methods would work, but I'm not quite sure how to do this.
def calculate_score(sect, im, msk):
# find the circle in the image
center, radius = find_circle(im)
# variable to count the score
score = 0
# Loop over all pixels of the detected circle
# This is more time efficient than looping over all pixels
for i in range(0 - radius, radius):
for j in range(0 - radius, radius):
pixel = Point(center.x + i, center.y + j)
# Check if pixel is in given sector
if sect.in_sector(pixel):
# Check if pixel is white
if msk[pixel.y, pixel.x]:
score -= 1 # Decrement score
else:
score += 1 # Increment score
print(score)
return score # Return score as result
In short, what I would like to know is:
Was it a good idea to switch from basinhopping to minimize? (I just thought basinhopping was extremely slow)
Is the method COBYLA I'm using the best one for this specific case?
Why is my result for 1 variable correct, while my result for 2 variables isn't?
Related
I am trying to use TDOA to find the sound source location in a coordinate system.
We have a costfunction looks like this:
def costfunction(point, meas_list, dims=2):
#### Cost function to be fed into the scipy.optimize.minimize function
#### For a candidate point, calculates the time differences, compares it
#### to data
error = 0
for m in meas_list:
actualdiff = m.delta
calcdiff = (
m.soundpointB.dist(point, dims=dims)
- m.soundpointA.dist(point, dims=dims)
)
error += (actualdiff - calcdiff) ** 2
return(error)
soundpointB.dist #---> is to calculate the distance between sensors
our sensor location is
b = [-2, 2,0]
c = [2,-2,0]
d = [2,2,0]
e = [0,0,2]
we simulate random source in range -50 to 50 in x,y and 0 to -20 in z
the simulation would generate time difference from the source to each sensor.
so we have the cost function to calculate the mean square error
Then we use scipy minimize to find out the best solution
startpoint = np.array([0,0,0])
dims = 3
result = minimize(
fun=costfunction,
x0=startpoint,
args=(meas_list, dims),
method='BFGS',
options={
'gtol': 1e-06,
'return_all':True,
'norm':inf},
jac='2-point')
For most of the calculation are good, the %error between the actual position and TDOA calculated position is +-10%
However for some location like [-30.0, 0.5, -6.0], the TDOA calculation return [-325.07,5.49,-64.73]
When we check all result from the scipy minimize. it seems it already found the location but it overshoot.
Does anyone know how to make this better? or something need to be corrected?
BFGS will not guarantee anything but a local minimum. If you know that your sensors will always be between -50 and 50 and -2 and 2, then I would suggest:
As a first step, try a bounded local minimization - using L-BFGS-B, SLSQP, etc... and give your variables those bounds
If the above is not satisfactory enough, bring out the big artillery and try global optimization algorithms, such as SHGO, DifferentialEvolution, DualAnnealing. They are all available in SciPy, might be slower but you’ll be much more confident in your results.
I am coding a program to find how far a bullet shoots at a 20 degree angle with various parameters with and without drag. For some reason my program immediately calculates the y value of the bullet as being below zero and I think it may be a problem with the physics equations I have used. I don't have the strongest background in physics so after trying my best to figure out where I went wrong I figured I would ask for help. In the graph below I am attempting to display the different distances associated with different timesteps.
import math
import matplotlib.pyplot as plt
rho = 1.2754
Af = math.pi*(16*0.0254)**2/4
g = -9.8 #m/s^2
V = 250.0 #m/s
#equation of motion
def update(r,V,a,dt):
return r+V*dt +.5*a*dt*dt, V + a*dt
#drag force from velocity vector return x and y
def drag(Vx,Vy):
Vmag = math.sqrt(Vx**2+Vy**2)
Ff = .5*Cd*rho*Af*Vmag**2
return -Ff * Vx/Vmag, -Ff *Vy/Vmag
def getTrajectory(dt, th):
#initialize problem [time,x,y,Vx,Vy]
state = [[0.0,0.0,0.0, V*math.cos(th), V*math.sin(th)]]
while state[-1][2] >= 0 : # while y > 0
time = state[-1][0] + dt
Fd = drag(state[-1][3],state[-1][4]) # Vx,Vy
ax = Fd[0]/m # new x acceleration
ay = (Fd[1]/m) +g # new y acceleration
nextX, nextVx = update(state[-1][1], state[-1][3],ax,dt) #x,Vx,acceleration x, dt
nextY, nextVy = update(state[-1][2], state[-1][4],ay,dt) #y,Vy, acceleration y, dt
state.append([time,nextX,nextY,nextVx,nextVy])
# linear interpolation
dtf = -state[-1][2]*(state[-2][0]-state[-1][0])/(state[-2][2]-state[-1][2])
xf,vxf = update(state[-1][1],state[-1][3], ax, dtf)
print(xf,vxf)
yf,vyf = update(state[-1][2],state[-1][4], ay, dtf)
print(yf,vyf)
state[-1] = [time+dtf,xf,yf,vxf,vyf]
return state
delt = 0.01 #timestep
th = 20.0 * math.pi/180.0 #changing to radians
m = 2400 #kg
Cd = 1.0
tsteps = [0.001,0.01,0.1,1.0,3.0] # trying different timesteps to find the optimal one
xdistance = [getTrajectory(tstep,th)[-1][1] for tstep in tsteps]
plt.plot(tsteps,xdistance)
plt.show()
A print of the state variable which contains all of the values of time,x,y,Vx, and Vy after one timestep yields:
[[0.0, 0.0, 0.0, 234.9231551964771, 85.50503583141717]
,[0.0, 0.0, -9.540979117872439e-18, 234.9231551964771, 85.50503583141717]]
The first array is the initial conditions with the Vx and Vy broken up into their vectors calculated by the 20 degree angle. The second array should have the bullet flying off with a positive x and y but for some reason the y value is calculated to be -9.54e-18 which quits the getTrajectory method as it is only supposed to continue while y > 0
In order to simplify my problem, I think it may help to look specifically at my update function.
def update(initial,V,a,dt): #current x or y, Velocity in the x or y direction, acceleration in x or y, timestep
return initial+V*dt +.5*a*dt*dt, V + a*dt
This function is supposed to return an updated x,Vx or an updated y,Vy depending on what is passed. I think this may be where the problem is in the physics equations but I'm not completely sure.
I think I see what's going on: in your while loop, you first make a forward time step to compute the kinematic variables at, say, t = 0.001 from their values at t = 0, and then you set dtf to a negative value and make a backward time step to compute new values at t = 0 from the values you just computed at t = 0.001 (or whatever). I'm not sure why you do that, but at any rate your update() function is using the Euler method which is numerically unstable - as you take more and more steps, it often diverges more and more from the true solution in somewhat unpredictable ways. In this case, that turns up as a negative y position, instead of setting it back to 0 as you might expect.
No method of numerical integration is going to give you perfect results, so you can always expect a little bit of inaccuracy, which may be positive or negative. Just because something is a little bit negative isn't necessarily cause for concern. (But it could be, depending on the situation.) That being said, you can get better results by using a more robust method of integration, like Runge-Kutta 4th-order (a.k.a. RK4) or a symplectic integrator. But those are somewhat advanced techniques, so I wouldn't worry about them too much for now.
As for the problem at hand, I wonder if you might have a sign error somewhere in your equations, possibly in the line that sets dtf. I can't say for sure, though, because I really don't understand how getTrajectory() is supposed to work.
This seems to be an indentation error. The part of determining the location of the zero crossing of y starting with the linear interpolation is surely not meant to be part of the integration loop, especially not the return statement. Repairing this indentation level gives the output
(3745.3646817336303, 205.84399509886356)
(6.278677479133594e-07, -81.7511764639779)
(3745.2091465034314, 205.8432621147464)
(8.637877372419503e-05, -81.75094436152747)
(3743.6311931495325, 205.83605594288176)
(0.01056407140434586, -81.74756057157798)
(3727.977248738945, 205.75827166721402)
(0.1316764390634169, -81.72155620161116)
(3669.3437231150106, 205.73704583954634)
(10.273469073526238, -80.52193274789204)
I'm trying to solve numerically a parabolic type of Partial differential equation(PDE):
u't=u''xx-u(1-u)(0.3-u)
with Neumann boundary conditions and a step like function as an initial condition.
Expected plot result is here
It may be a bit misleading: red - is an initial step-like function and smooth lines are different iteration in time after (like t=50,100, 150..))
The idea is based on the method of lines: make the space (x) discrete, use finite difference methods (second order is written by the symmetric-centered approximation), standard Python scipy odeint(which can use implicit methods if I'm not mistaken)and then to solve the obtained set of ODEs. I think this is pretty standard way of how to deal with such ordinary PDEs(correct me if I'm wrong).
Below are code snippets I'm running.
In a nutshell, I want to represent the above continuous equation in a matrix based form:
[u]'t=A*[u]+f([u])
where A is a matrix for second order symmetric-cantered approximation of u''xx(after multiplying by [u])
Unfortunately, there is almost nothing qualitatively significant happening with initial conditions after running the code, so I'm confused whether there is a problem with my script or in an approach itself.
Defining the right side u(1-u)(0.3-u):
def f(u, h):
puff = []
for i in u:
puff.append(i*(1. - i)*(0.3 - i))
puff[0] = (h/2.)*puff[0] #this is to include
puff[len(puff)-1] = (h/2.)*puff[len(puff)-1]
return puff
Defining spatial resolution:
h = 10.
x = np.linspace(0., 100., num=h)
This is to define second derivative by the means of finite difference(A*[u]):
diag = [-2]*(len(x))
diag[0] = -h
diag[len(x)-1] = h**2
diag_up = [1]*(len(x)-1)
diag_up[0] = h
diag_down = [1]*(len(x)-1)
diag_down[len(x)-2] = 0
diagonals = [diag, diag_up, diag_down]
A = diags(diagonals, [0, 1, -1]).toarray()
A = A/h**2
Defining step-like initial condition:
init_cond = [0]*(len(x))
for i in range(int(round(0.2*len(init_cond)))):
init_cond[i] = 1
Solving set of ODEs with a time frame defined.
t = np.linspace(0, 100, 200)
def pde(u, t, A, h):
dudt = A.dot(u) - f(u, h)
return dudt
sol = odeint(pde, init_cond, t, args=(A, h))
print(sol[len(sol)-1])
The code may be flawed as I'm rather new to Python. If there is anything I can improve, please let me know. The final result is here
Initial conditions did not change at all. Can anyone give me a hint where I could make a mistake? Thank you in advance!
Given a 2D point p, I'm trying to calculate the smallest distance between that point and a functional curve, i.e., find the point on the curve which gives me the smallest distance to p, and then calculate that distance. The example function that I'm using is
f(x) = 2*sin(x)
My distance function for the distance between some point p and a provided function is
def dist(p, x, func):
x = np.append(x, func(x))
return sum([[i - j]**2 for i,j in zip(x,p)])
It takes as input, the point p, a position x on the function, and the function handle func. Note this is a squared Euclidean distance (since minimizing in Euclidean space is the same as minimizing in squared Euclidean space).
The crucial part of this is that I want to be able to provide bounds for my function so really I'm finding the closest distance to a function segment. For this example my bounds are
bounds = [0, 2*np.pi]
I'm using the scipy.optimize.minimize function to minimize my distance function, using the bounds. A result of the above process is shown in the graph below.
This is a contour plot showing distance from the sin function. Notice how there appears to be a discontinuity in the contours. For convenience, I've plotted a few points around that discontinuity and the "closet" points on the curve that they map to.
What's actually happening here is that the scipy function is finding a local minimum (given some initial guess), but not a global one and that is causing the discontinuity. I know finding the global minimum of any function is impossible, but I'm looking for a more reliable way to find the global minimum.
Possible methods for finding a global minimum would be
Choose a smart initial guess, but this amounts to knowing approximately where the global minimum is to begin with, which is using the solution of the problem to solve it.
Use a multiple initial guesses and choose the answer which gets to the best minimum. This however seems like a poor choice, especially when my functions get more complicated (and higher dimensional).
Find the minimum, then perturb the solution and find the minimum again, hoping that I may have knocked it into a better minimum. I'm hoping that maybe there is some way to do this simply without evoking some complicated MCMC algorithm or something like that. Speed counts for this process.
Any suggestions about the best way to go about this, or possibly directions to useful functions that may tackle this problem would be great!
As suggest in a comment, you could try a global optimization algorithm such as scipy.optimize.differential_evolution. However, in this case, where you have a well-defined and analytically tractable objective function, you could employ a semi-analytical approach, taking advantage of the first-order necessary conditions for a minimum.
In the following, the first function is the distance metric and the second function is (the numerator of) its derivative w.r.t. x, that should be zero if a minimum occurs at some 0<x<2*np.pi.
import numpy as np
def d(x, p):
return np.sum((p-np.array([x,2*np.sin(x)]))**2)
def diff_d(x, p):
return -2 * p[0] + 2 * x - 4 * p[1] * np.cos(x) + 4 * np.sin(2*x)
Now, given a point p, the only potential minimizers of d(x,p) are the roots of diff_d(x,p) (if any), as well as the boundary points x=0 and x=2*np.pi. It turns out that diff_d may have more than one root. Noting that the derivative is a continuous function, the pychebfun library offers a very efficient method for finding all the roots, avoiding cumbersome approaches based on the scipy root-finding algorithms.
The following function provides the minimum of d(x, p) for a given point p:
import pychebfun
def min_dist(p):
f_cheb = pychebfun.Chebfun.from_function(lambda x: diff_d(x, p), domain = (0,2*np.pi))
potential_minimizers = np.r_[0, f_cheb.roots(), 2*np.pi]
return np.min([d(x, p) for x in potential_minimizers])
Here is the result:
How can I find the point where the first derivative of my equation equals 0 using scipy.integrate.ode?
I set up this function, which gets the answer, but I'm not sure about accuracy and it can't be the most efficient way to do this.
Basically I am using this function to find the time a projectile with initial velocity stops moving. With systems of ODEs, is there a better way to solve for this answer?
import numpy as np
from scipy import integrate
def find_nearest(array,value):
idx=(np.abs(array-value)).argmin()
return array[idx], idx
def deriv(x,t):
# This function sets up the following relations
# dx/dt = v , dv/dt = -(Cp/m)*(4+v^2)
return np.array([ x[1], -(0.005/0.1) * (4+ (x[1]**2)) ])
def findzero(start, stop, v0):
time = np.linspace(start, stop, 100000)
#xinit are initial vaules of equation
xinit = np.array([0.0,v0])
x = integrate.odeint(deriv,xinit,time)
# find nearest velocity value nearest to 0
value, num = find_nearest(x[:,1],0.0001)
print 'closest value ',
print value
print 'reaches zero at time ',
print time[num]
return time[num]
# from 0 to 20 seconds with initial velocity of 100 m/s
b = findzero(0.0,20.0,100.0)
In general, a good approach to solve this sort of problem is to rewrite your equations so that velocity is the independent variable and time and distance are the dependent variables. Then, you simply have to integrate the equations from v=v0 to v=0.
However, in the example you give it is not even necessary to resort to scipy.integrate at all. The equations can be easily solved with pencil and paper (separation of variables followed by a standard integral). The result is
t = (m/(2 Cp)) arctan(v0/2)
where v0 is the initial velocity and the result of arctan must be taken in radians.
For an initial velocity of 100 m/s, the answer is 15.5079899282 seconds.
I would use something like scipy.optimize.fsolve() to find the roots of the derivative. Using this, one can work backwards to find the time taken to reach a root.