I'm trying to code Euler's approximation method to solve the harmonic oscillator problem, and then compare it on a plot with the exact solution. However, I'm finding that my approximation seems to be too good (even for larger step sizes), especially since I know the error is supposed to increase as time goes on. Am I doing something wrong in my Euler's method code? It's all written below.
import numpy as np
import matplotlib.pyplot as plt
w = 1.0 #frequency of oscillation
x_0 = 1.0 #initial position
h = 0.6 #step size
t_0 = 0 #starting time
n = 20
t = np.arange(t_0, n, h) #t = t_0 + nh (h has units of seconds)
y = np.array([[0.0, 0.0]]*len(t))
def x(t, x_0, w): #exact solution
return x_0*np.cos(w*t)
def x_prime(t, x_0, w): #first derivative of solution (which is also v)
return -x_0*w*np.sin(w*t)
y[0] = [x_0, 0.0] #starting velocity is 0
for i in range(1, len(t)):
# Euler's method
y[i] = y[i-1] + np.array([x_prime(t[i-1], x_0, w), -w**2 * x(t[i-1], x_0, w)]) * h
xlist = [item[0] for item in y] #list of x values at each t value
vlist = [item[1] for item in y] #list of velocity values at each t value
plt.plot(t, xlist, label='Euler approximation')
plt.plot(t, x(t, x_0, w), label='Exact solution')
plt.xlabel('t')
plt.ylabel('x(t)')
plt.legend(loc='upper right')
And my plot for this code outputs as:
Your problem is that you compute the slopes for the Euler method from the exact solution. You should use the previous approximation instead,
# Euler's method
y[i] = y[i-1] + np.array([y[i-1,1], -w**2 * y[i-1,0]]) * h
or use a function
def f(t,y): x, dotx = y; return np.array([ dotx, -w**2*x])
...
for i in range(1, len(t)):
# Euler's method
y[i] = y[i-1] + f(t[i-1],y[i-1]) * h
Then you will get the expected rapidly diverging behavior, here for h=0.1:
Btw., it might be simpler to write
xlist, vlist = y.T
Related
Is there a performant way to directly solve for the most likely intersection point (X, Y) of several multivariable Gaussians?
I've seen a few posts here that have asked how to solve for the intersection between two Gaussians - the concept is familiar to me. Right now it's not obvious to me aside from iterating and solving for two distributions at a time.
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
mus = [np.array([[0.3],[0.7]]),
np.array([[0.3],[0.2]]),
np.array([[1.5],[0.6]])]
covs = [np.array([[0.85, 0.3], [0.3, 0.25]]),
np.array([[0.7, -0.41], [-0.41, 0.25]]),
np.array([[0.5, 0.15], [0.15, 0.15]])]
cmaps = ["Reds", "Blues", "Greens"]
for m, cov, c in zip(mus, covs, cmaps):
cov_inv = np.linalg.inv(cov)
cov_det = np.linalg.det(cov)
x = np.linspace(-3, 3)
y = np.linspace(-3, 3)
X,Y = np.meshgrid(x,y)
coe = 1.0 / ((2 * np.pi)**2 * cov_det)**0.5
Z = coe * np.e ** (-0.5 * (cov_inv[0,0]*(X-m[0])**2 + (cov_inv[0,1] + cov_inv[1,0])*(X-m[0])*(Y-m[1]) + cov_inv[1,1]*(Y-m[1])**2))
plt.contour(X,Y,Z, cmap = c)
You can do a LOT better than iterating between 2 solutions at a time. Realize that at every (x, y) point, you have a Z value for all 3 curves, and at the 3-way intersecting point, they are all equal (or within tolerance). And at other points, if you take the lowest Z of the curves, and move towards the center (mu_x, mu_y) of that curve, you are moving in an improving direction.
The below is an iterative algorithm that does that. There is certainly some meat on the bone in terms of possible enhancements. Notably, you could incorporate a "tolerance" for stopping conditions easily, or do some weighted average of the 2 lower z values instead of just the lowest to get the movement vector, or tinker with a larger step size.
Anyhow, this converges very rapidly for many different test starting points.
Code:
import numpy as np
import matplotlib.pyplot as plt
class Curve:
# a convenience so we can avoid recomputations
def __init__(self, mu, cov_inv, cov_det):
self.mu = mu
self.cov_inv = cov_inv
self.cov_det = cov_det
self.coe = 1.0 / ((2 * np.pi)**2 * cov_det)**0.5
def z(self, x, y):
Z = self.coe * np.e ** (-0.5 * (self.cov_inv[0,0]*(x-self.mu[0])**2 + \
(self.cov_inv[0,1] + self.cov_inv[1,0])*(x-self.mu[0])*(y-self.mu[1]) + self.cov_inv[1,1]*(y-self.mu[1])**2))
return Z
mus = [np.array([[0.3],[0.7]]),
np.array([[0.3],[0.2]]),
np.array([[1.5],[0.6]])]
covs = [np.array([[0.85, 0.3], [0.3, 0.25]]),
np.array([[0.7, -0.41], [-0.41, 0.25]]),
np.array([[0.5, 0.15], [0.15, 0.15]])]
cmaps = ["Reds", "Blues", "Greens"]
curves = []
for m, cov, c in zip(mus, covs, cmaps):
cov_inv = np.linalg.inv(cov)
cov_det = np.linalg.det(cov)
x = np.linspace(-3, 3)
y = np.linspace(-3, 3)
X,Y = np.meshgrid(x,y)
coe = 1.0 / ((2 * np.pi)**2 * cov_det)**0.5
Z = coe * np.e ** (-0.5 * (cov_inv[0,0]*(X-m[0])**2 + (cov_inv[0,1] + cov_inv[1,0])*(X-m[0])*(Y-m[1]) + cov_inv[1,1]*(Y-m[1])**2))
plt.contour(X,Y,Z, cmap = c)
curves.append(Curve(m, cov_inv, cov_det))
# iterative algorithm...
pos = np.array((-1,2))
step_size = 0.1
num_steps = 100
footprints = [pos,]
for step in range(num_steps):
zs = [ (curves[i].z(*pos), i) for i in range(len(curves))]
zs.sort() # sort by z value, lowest will be first
c = curves[zs[0][1]] # the curve to move toward
vec = c.mu.T - pos
move_vec = vec * (step_size/np.linalg.norm(vec))
print(f'move: {move_vec} towards curve {zs[0][1]}')
pos = pos + move_vec
pos = pos.flatten()
# check to see if we have backtracked, if so, shorten the step
if len(footprints) > 1 and np.linalg.norm(pos - footprints[-2]) < step_size:
#print(f'norm: {np.linalg.norm(pos-footprints[-2])}')
step_size *= 0.5
footprints.append(pos)
plt.plot([t[0] for t in footprints], [t[1] for t in footprints], c='k', lw=2)
plt.show()
Plot:
Practicing finite difference implementation and I cannot figure out why my solution looks so strange. Code taken from: http://people.bu.edu/andasari/courses/numericalpython/Week9Lecture15/PythonFiles/FTCS_DirichletBCs.py.
Note: I'm using this lecture example for the heat equation not the reaction-diffusion equation!
I haven't learned the relevant mathematics so this could be why!
My code:
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import cm
import math as mth
from mpl_toolkits.mplot3d import Axes3D
import pylab as plb
import scipy as sp
import scipy.sparse as sparse
import scipy.sparse.linalg
# First start with diffusion equation with initial condition u(x, 0) = 4x - 4x^2 and u(0, t) = u(L, t) = 0
# First discretise the domain [0, L] X [0, T]
# Then discretise the derivatives
# Generate algorithm:
# 1. Compute initial condition for all i
# 2. For all n:
# 2i. Compute u_i^{n + 1} for internal space points
# 2ii. Set boundary values for i = 0 and i = N_x
M = 40 # number of grid points for space interval
N = 70 # '' '' '' '' '' time ''
x0 = 0
xL = 1 # unit grid differences
dx = (xL - x0) / (M - 1) # space step
t0 = 0
tF = 0.2
dt = (tF - t0) / (N - 1)
D = 0.3 # thermal diffusivity
a = D * dt / dx**2
# Create grid
tspan = np.linspace(t0, tF, N)
xspan = np.linspace(x0, xL, M)
# Initial matrix solution
U = np.zeros((M, N))
# Initial condition
U[:, 0] = 4*xspan - 4*xspan**2
# Boundary conditions
U[0, :] = 0
U[-1, 0] = 0
# Discretised derivative formula
for k in range(0, N-1):
for i in range(1, M-1):
U[i, k+1] = a * U[i-1, k] + (1 - 2 * a) * U[i, k] + a * U[i + 1, k]
X, T = np.meshgrid(tspan, xspan)
fig = plt.figure()
ax = fig.gca(projection='3d')
surf = ax.plot_surface(X, T, U, cmap=cm.coolwarm,
linewidth=0, antialiased=False)
ax.set_xticks([0, 0.05, 0.1, 0.15, 0.2])
ax.set_xlabel('Space')
ax.set_ylabel('Time')
ax.set_zlabel('U')
plt.tight_layout()
plt.show()
edit: Changed therm diff value to correct one.
The main problem is the time step length. If you look at the differential equation, the numerics become unstable for a>0.5. Translated this means for you that roughly N > 190. I get a nice picture if I increase your N to such value.
However, I thing somewhere the time and space axes are swapped (if you try to interpret the graph then, i.e. boundary conditions and expected dampening of profile over time). I cannot figure out right now why.
Edit: Actually, you swap T and X when you do meshgrid. This should work:
N = 200
...
...
T, X = np.meshgrid(tspan, xspan)
...
surf = ax.plot_surface(T, X, U, cmap=cm.coolwarm,
linewidth=0, antialiased=False)
...
ax.set_xlabel('Time')
ax.set_ylabel('Space')
I've written a code which looks at projectile motion of an object with drag. I'm using odeint from scipy to do the forward Euler method. The integration runs until a time limit is reached. I would like to stop integrating either when this limit is reached or when the output for ry = 0 (i.e. the projectile has landed).
def f(y, t):
# takes vector y(t) and time t and returns function y_dot = F(t,y) as a vector
# y(t) = [vx(t), vy(t), rx(t), ry(t)]
# magnitude of velocity calculated
v = np.sqrt(y[0] ** 2 + y[1] **2)
# define new drag constant k
k = cd * rho * v * A / (2 * m)
return [-k * y[0], -k * y[1] - g, y[0], y[1]]
def solve_f(v0, ang, h0, tstep, tmax=1000):
# uses the forward Euler time integration method to solve the ODE using f function
# create vector for time steps based on args
t = np.linspace(0, tmax, tmax / tstep)
# convert angle from degrees to radians
ang = ang * np.pi / 180
return odeint(f, [v0 * np.cos(ang), v0 * np.sin(ang), 0, h0], t)
solution = solve_f(v0, ang, h0, tstep)
I've tried several loops and similar to try to stop integrating when ry = 0. And found this question below but have not been able to implement something similar with odeint. solution[:,3] is the output column for ry. Is there a simple way to do this with odeint?
Does scipy.integrate.ode.set_solout work?
Checkout scipy.integrate.ode here. It is more flexible than odeint and helps with what you want to do.
A simple example using a vertical shot, integrated until it touches ground:
from scipy.integrate import ode, odeint
import scipy.constants as SPC
def f(t, y):
return [y[1], -SPC.g]
v0 = 10
y0 = 0
r = ode(f)
r.set_initial_value([y0, v0], 0)
dt = 0.1
while r.successful() and r.y[0] >= 0:
print('time: {:.3f}, y: {:.3f}, vy: {:.3f}'.format(r.t + dt, *r.integrate(r.t + dt)))
Each time you call r.integrate, r will store current time and y value. You can pass them to a list if you want to store them.
Let's solve this as the boundary value problem that it is. We have the conditions x(0)=0, y(0)=h0, vx(0)=0, vy(0)=0 and y(T)=0. To have a fixed integration interval, set t=T*s, which means that dx/ds=T*dx/dt=T*vx etc.
def fall_ode(t,u,p):
vx, vy, rx, ry = u
T = p[0]
# magnitude of velocity calculated
v = np.hypot(vx, vy)
# define new drag constant k
k = cd * rho * v * A / (2 * m)
return np.array([-k * vx, -k * vy - g, vx, vy])*T
def solve_fall(v0, ang, h0):
# convert angle from degrees to radians
ang = ang * np.pi / 180
vx0, vy0 = v0*np.cos(ang), v0*np.sin(ang)
def fall_bc(y0, y1, p): return [ y0[0]-vx0, y0[1]-vy0, y0[2], y0[3]-h0, y1[3] ]
t_init = [0,1]
u_init = [[0,0],[0,0],[0,0], [h0,0]]
p_init = [1]
res = solve_bvp(fall_ode, fall_bc, t_init, u_init, p_init)
print res.message
if res.success:
print "time to ground: ", res.p[0]
# res.sol is a dense output, evaluation interpolates the computed values
return res.sol
sol = solve_fall(300, 30, 20)
s = np.linspace(0,1,501)
u = sol(s)
vx, vy, rx, ry = u
plt.plot(rx, ry)
I am trying to mimic the gradient descent algorithm for linear regression from Andrew NG's Machine learning course to Python, but for some reason my implementation is not working correctly.
Here's my implementation in Octave, it works correctly:
function [theta, J_history] = gradientDescent(X, y, theta, alpha, num_iters)
J_history = zeros(num_iters, 1);
for iter = 1:num_iters
prediction = X*theta;
margin_error = prediction - y;
gradient = 1/m * (alpha * (X' * margin_error));
theta = theta - gradient;
J_history(iter) = computeCost(X, y, theta);
end
end
However, when I translate this to Python for some reason it is not giving me accurate results. The cost seems to be going up rather than descending.
Here's my implementation in Python:
def gradientDescent(x, y, theta, alpha, iters):
m = len(y)
J_history = np.matrix(np.zeros((iters,1)))
for i in range(iters):
prediction = x*theta.T
margin_error = prediction - y
gradient = 1/m * (alpha * (x.T * margin_error))
theta = theta - gradient
J_history[i] = computeCost(x,y,theta)
return theta,J_history
My code is compiling and there isn't anything wrong. Please note this is theta:
theta = np.matrix(np.array([0,0]))
Alpha and iters is set to this:
alpha = 0.01
iters = 1000
When I run it, opt_theta, cost = gradientDescent(x, y, theta, alpha, iters) and print out opt_theta, I get this:
matrix([[ 2.36890383e+16, -1.40798902e+16],
[ 2.47503758e+17, -2.36890383e+16]])
when I should get this:
matrix([[-3.24140214, 1.1272942 ]])
What am I doing wrong?
Edit:
Cost function
def computeCost(x, y, theta):
# Get length of data set
m = len(y)
# We get theta transpose because we are working with a numpy array [0,0] for example
prediction = x * theta.T
J = 1/(2*m) * np.sum(np.power((prediction - y), 2))
return J
Look there:
>>> A = np.matrix([3,3,3])
>>> B = np.matrix([[1,1,1], [2,2,2]])
>>> A-B
matrix([[2, 2, 2],
[1, 1, 1]])
Matrices are broadcasted together.
"it's because np.matrix inherits from np.array. np.matrix overrides multiplication, but not addition and subtraction"
In yours situation theta(1x2) subtract gradient(2x1) and in result you have got 2x2. Try to transpose gradient before subtracting.
theta = theta - gradient.T
I have implemented the following explicit euler method in python:
def explicit_euler(df, x0, h, N):
"""Solves an ODE IVP using the Explicit Euler method.
Keyword arguments:
df - The derivative of the system you wish to solve.
x0 - The initial value of the system you wish to solve.
h - The step size.
N - The number off steps.
"""
x = np.zeros(N)
x[0] = x0
for i in range(0, N-1):
x[i+1] = x[i] + h * df(x[i])
return x
Following the article on wikipedia I can plot the function and verify that I get the same plot: . I believe that here the method I have written is working correctly.
Next I tried to use it to solve the last system given on this page and instead of the plot shown there I obtain this:
I am not sure why my plot doesn't match the one shown on the webpage. The explicit euler method seems to work fine when I use it to solve systems where the slope doesn't change, but for an oscillating function it never seems to mimic it at all. Not even showing the expected error gain as indicated on the linked webpage. I am not sure what is wrong with the method I have implemented.
Here is the code used for plotting and the derivative:
def g(t):
return -0.5 * np.exp(t * 0.5) * np.sin(5 * t) + 5 * np.exp(t * 0.5)
* np.cos(5 * t)
h = 0.001
x0 = 0
tn = 4
N = int(tn / h)
x = ee.explicit_euler(f, x0, h, N)
t = np.arange(0, tn, h)
fig = plt.figure()
plt.plot(t, x, label="Explicit Euler")
plt.plot(t, (np.exp(0.5 * t) * np.sin(5 * t)), label="Analytical
solution")
#plt.plot(t, np.exp(0.5 * t), label="Analytical solution")
plt.xlabel('Timesteps t')
plt.ylabel('x(t)=e^(0.5*t) * sin(5*t)')
plt.legend()
plt.grid()
plt.show()
Edit:
As requested here is the current equation I am applying the method to:
y'-y=-0.5*e^(t/2)*sin(5t)+5e^(t/2)*cos(5t)
Where y(0)=0.
I would like to make clear however that this behaviour doesn't occur just for this equation but all equations where the slope has a change in sign, or oscillating behaviour.
Edit 2:
Ok thanks. Yes the code below does indeed work. But I have one further question. In the simple example I had for the exponential function, I had defined a method:
def f(x):
return x
for the system f'(x)=x. This gave the output of my first graph which looks correct. I then defined another function:
def k(x):
return cos(x)
for the system f'(x)=cos(x), this does not give expected output. But when I change the function definition to
def k(t, x):
return cos(t)
I get the expected output. If I change my function
def f(t, x):
return t
I get an incorrect output. Am I always actually evaluating the function at a time step and is it just by chance for the system x'=x that at each time step the value is just the value of x?
I had understood that the Euler method used the value of the previously calculated value in order to get the next value. But if I run code for my function k(x)=cos(x), I get output pictured below, which must be incorrect. This now uses the updated code you provided.
def k(t, x):
return np.cos(x)
h = 0.1 # Step size
x0 = (0, 0) # Initial point of iteration
tn = 10 # Time step to iterate to
N = int(tn / h) # Number of steps
x = ee.explicit_euler(k, x0, h, N)
t = np.arange(0, tn, h)
The problem is that you have incorrectly raised the function g, you want to solve the equation:
From where we observe that:
y' = y -0.5*e^(t/2)*sin(5t)+5e^(t/2)*cos(5t)
Then we define the function f(t, y) = y -0.5*e^(t/2)*sin(5t)+5e^(t/2)*cos(5t) as:
def f(t, y):
return y -0.5 * np.exp(t * 0.5) * np.sin(5 * t) + 5 * np.exp(t * 0.5) * np.cos(5 * t)
The initial point of iteration is f0=(t(0), y(0)):
f0 = (0, 0)
Then from Euler's equations:
def explicit_euler(df, x0, h, N):
"""Solves an ODE IVP using the Explicit Euler method.
Keyword arguments:
df - The derivative of the system you wish to solve.
x0 - The initial value of the system you wish to solve.
h - The step size.
N - The number off steps.
"""
x = np.zeros(N)
t, x[0] = x0
for i in range(0, N-1):
x[i+1] = x[i] + h * df(t ,x[i])
t += h
return x
Complete Code:
def explicit_euler(df, x0, h, N):
"""Solves an ODE IVP using the Explicit Euler method.
Keyword arguments:
df - The derivative of the system you wish to solve.
x0 - The initial value of the system you wish to solve.
h - The step size.
N - The number off steps.
"""
x = np.zeros(N)
t, x[0] = x0
for i in range(0, N-1):
x[i+1] = x[i] + h * df(t ,x[i])
t += h
return x
def df(t, y):
return -0.5 * np.exp(t * 0.5) * np.sin(5 * t) + 5 * np.exp(t * 0.5) * np.cos(5 * t) + y
h = 0.001
f0 = (0, 0)
tn = 4
N = int(tn / h)
x = explicit_euler(df, f0, h, N)
t = np.arange(0, tn, h)
fig = plt.figure()
plt.plot(t, x, label="Explicit Euler")
plt.plot(t, (np.exp(0.5 * t) * np.sin(5 * t)), label="Analytical solution")
#plt.plot(t, np.exp(0.5 * t), label="Analytical solution")
plt.xlabel('Timesteps t')
plt.ylabel('x(t)=e^(0.5*t) * sin(5*t)')
plt.legend()
plt.grid()
plt.show()
Screenshot:
Dump y' and what is on the right side is what you should place in the df function.
We will modify the variables to maintain the same standard for the variables, and will y be the dependent variable, and t the independent variable.
Equation 2: In this case the equation f'(x)=cos(x) will be rewritten to:
y'=cos(t)
Then:
def df(t, y):
return np.cos(t)
In conclusion, if we have an equation of the following form:
y' = f(t, y)
Then:
def df(t, y):
return f(t, y)