We Keep Coding

Explicit (Forward) and Implicit (Backward) Euler Methods in Python

I am trying to implement both the explicit and implicit Euler methods to approximate a solution for the following ODE:
dx/dt = -kx, where k = cos(2 pi t), and x(0) = 1
Euler's methods use finite differencing to approximate a derivative:
dx/dt = (x(t+dt) - x(t)) / dt
The forward method explicitly calculates x(t+dt) based on a previous solution x(t):
x(t+dt) = x(t) + f(x,t)dt
The backwards method is implicit, and finds the solution x(t+dt) by solving an equation involving the current state of the system x(t) and the later one x(t+dt):
x(t) = x(t+dt) - f(x,(t+dt))dt
My code for approximating a solution to dx/dt = -kx, x(0) = 1 and plotting it alongside the actual solution is given below:
### Import Necessary Packages
import matplotlib.pyplot as plt
import numpy as np
%config InlineBackend.figure_format = 'retina'
plt.rcParams['figure.figsize'] = (10.0, 6.0)
### Defining Basic Data
t0 = 0 # initial t
tf = 4*np.pi # final t
N = 1000 # factor affecting time step size
dt = (4*np.pi)/N # time step size (as a factor of N)
t = np.linspace(t0,tf,N) # defining a vector of t values from t0 to tf
x0 = 1 # initial x
x = np.zeros([N]) # initializing array for x values
f = lambda x,t: -np.cos(2*np.pi*t)*x # defining f(x,t) on RHS on ODE
### Define a Function for Euler's Forward Method ###
def ForwardEuler(f,x0,t):
x[0] = x0
for i in range(1,N-1):
x[i+1] = x[i] + (f(x[i],t[i]))*dt
return x
# Plot Solution
forward = ForwardEuler(f,x0,t)
actual = 1/np.exp((1/(2*np.pi))*np.sin(2*np.pi*t))
plt.plot(t,actual,'r-',t,forward,'b-')
plt.legend(['Actual','Backward Euler'])
plt.xlabel('Time')
plt.ylabel('Amplitude')
plt.title("Solution to $x'=-kx$, $x$(0)=1")
plt.grid(True);plt.show()
My question lies in how to adapt the for-loop section of the code to display the backward Euler method instead of forward Euler method. I am having trouble with this since the equations require you to know x[i+1] in order to solve x[i+1].
I believe the backwards for-loop would be what is given below, but I am unsure:
def BackwardEuler(f,x0,t):
x[0] = x0
for i in range(1,N-1):
x[i] = x[i+1] - (f(x[i+1],t[i+1]))*dt
return x
I have found very few resources online and am at a complete loss. Any help on this would be appreciated! Thank you!

Usually, for Backward Euler and Trapezoidal Rule, you write the expression as a equation (or a system of equations), then solve it (i.e. find its zeros). The zeros represent the value of x[i+1].
For example, for Backward Euler, the system is:
x[i+1] = x[i] + (f(x[i+1],t[i+1]))*dt
Which you can rewrite as:
x[i+1] - x[i] - dt*f(x[i+1], t[i+1]) = 0
The values x[i] and t[i+1] are known. The only unknown is x[i+1]. You can solve this system numerically (using something like fsolve), and the solution would be your x[i+1]. It is possible, of course, that you get more than one solution. You have to select the one that fits your problem (i.e. x cannot be an imaginary number, or x cannot be negative, etc.)
The same technique can be applied for Trapezoidal Rule, with the system being:
x[i+1] - x[i] - (f(x[i],t[i]) + f(x[i+1],t[i+1]))*(dt/2) = 0
PS: Check out Computational Science StackExchange. It is more suitable for question related to numerical and computational methods.

Solving two coupled second order boundary value problems

I have solved a single second order differential equation with two boundary conditions using the module solve_bvp. However, now I am trying to solve the system of two second order differential equations;
U'' + a*B' = 0
B'' + b*U' = 0
with the boundary conditions U(+/-0.5) = +/-0.01 and B(+/-0.5) = 0. I have split this into a system of first ordinary differential equations and I am trying to use solve_bvp to solve them numerically. However, I am just getting arrays full of zeros for my solution. I believe I am implementing the boundary conditions wrong. It is not clear to me how to handle more than two equations from the documentation. My attempt is below
import numpy as np
from scipy.integrate import solve_bvp
import matplotlib.pyplot as plt
%matplotlib inline
from scipy.integrate import solve_bvp
alpha = 1E-8
zeta = 8E-3
C_k = 0.05
sigma = 0.01
def fun(x, y):
return np.vstack((y[1],-((alpha)/(C_k*sigma))*y[2],y[2], -(1/(C_k*zeta))*y[1]))
def bc(ya, yb):
return np.array([ya[0]+0.001, yb[0]-0.001,ya[0]-0, yb[0]-0])
x = np.linspace(-0.5, 0.5, 5000)
y = np.zeros((4, x.size))
print(y)
sol = solve_bvp(fun, bc, x, y)
print(sol)
In my question I have just relabeled a and b, but they're just parameters that I input. I have the analytic solution for this set of equations so I know one exists that is non-trivial. Any help would be greatly appreciated.

It is most times really helpful if you state at least once in a comment or by assignment to specifically named variables how you want to compose the state vector.
By the form of the derivative return vector, I would think you intend
U, U', B, B'
which means that U=y[0], U'=y[1] and B=y[2],B'=y[3], so that your derivatives vector should correctly be
return y[1], -((alpha)/(C_k*sigma))*y[3], y[3], -(1/(C_k*zeta))*y[1]
and the boundary conditions
return ya[0]+0.001, yb[0]-0.001, ya[2]-0, yb[2]-0
Especially your boundary condition should throw the algorithm in the first step because of a singular Jacobian, always check the .success field and the .message field of the solution structure.
Note that by default the absolute and relative tolerance of the experimental solve_bvp is 1e-3, and the number of nodes is limited to 500.
Setting the initial node number to 50 (5000 is much too much, the solver refines where necessary), and the tolerance to 1-6, I get the following solution plots that visibly satisfy the boundary conditions.

Newton method for transcendental equation

Transcendental equation:
tan(x)/x + b = 0, where b is any real number.
I need to introduce n and give me n solutions of this equation.
My code (Python):
from math import tan, cos, pi, sqrt, sin,exp
import numpy as np
from matplotlib.figure import Figure
import matplotlib.pyplot as plt
def f(x,b):
return tan(x)/x + b
def f1(x,b):
return (x/(cos(x)*cos(x)) - tan(x))/(x**2)
e = 0.00000000001
def newtons_method(x0, f, f1, e):
x0 = float(x0)
while True:
x1 = x0 - (f(x0,b) / f1(x0,b))
if abs(x1 - x0) < e:
return x1
x0 = x1
result = []
n = int(input("Input n: "))
b = float(input("Input b: "))
for i in range(2,4*n,1):
result.append(newtons_method(i, f , f1, e))
lambda_result = sorted(list(set(result)))
print(len(lambda_result))
I change the initial approximation with step 1.The roots are repeated with a period of ~pi, so the second argument 4*n. I exclude repeated solutions through set. If n is 50 then he finds only 18 solution. What needs to be fixed to make it work? Help me, please.

In your cross-post https://math.stackexchange.com/q/2942400/115115 it was strongly recommended by Yves Daoust to base your Newton method on the function
f(x)=sin(x) + b*x*cos(x)
or
f(x)=sin(x)/x + b*cos(x)
as these functions do not have poles or other singularities (if x is not close to 0).
The solutions are, at least for large c, close to the initial values (i+0.5)*pi for i in range(n). The result does then not require sorting or shortening of the result.
One could use that at the roots of the cosine the sine term has alternating sign. This makes a perfect situation to apply a bracketing method like regula falsi (illinois), Dekker's fzeroin, Brent's method,... These methods are almost as fast as the Newton method and are guaranteed to find the root inside the interval.
The only complication is the interval (0,pi/2) as there will be a non-zero root for b<-1. One has to remove the root-finding process from x=0 which is non-trivial for b close to -1.
Newton's method only zero's in to the root reliably when the initial point is close enough to the root. If the point is farther away, close to an extremum of the function, the root of the tangent may be very far away. Thus to successfully apply the Newton method, one needs to find good root approximations from the start. To that end one may use globally convergent fixed-point iterations or structurally simple approximations of the function under consideration.
Using the contracting fixed-point iteration: the solution around k*pi is also a root of the equivalent equation x+arctan(b*x)=k*pi. This gives the approximate solution x=g(k*pi)=k*pi-arctan(b*k*pi). As the arcus tangent is rather flat even for small k, this gives a good approximation.
If b<-1 there is a positive root for k=0, that is in the interval (0,pi/2). The previous method does not work in this case as the arcus tangent has a slope around 1 in this interval. The root is due to the higher order, non-linear terms of the equation, so one needs a non-linear approximation of one of the equivalent forms of the equation. Approximating tan(x)=x/(1-(2*x/pi)^2) gives the same poles at +-pi/2 and is sufficiently close in between. Inserting this approximation into the given equation and solving gives the initial root approximation at x=pi/2*sqrt(1+1/b).
In the implementation one has to shift the root set for b<-1 to include the additional first solution.
from math import tan, cos, pi, sqrt, sin, atan
def f(x,b):
return sin(x)/x+b*cos(x)
def f1(x,b):
return cos(x)/x-(b+x**-2)*sin(x)
e = 1e-12
def newtons_method(x0, f, f1, e):
x0 = float(x0)
while True:
x1 = x0 - (f(x0,b) / f1(x0,b))
if abs(x1 - x0) < e:
return x1
x0 = x1
result = []
n = int(input("Input n: "))
b = float(input("Input b: "))
for i in range(n):
k=i;
if b >= -1: k=k+1
x0 = pi/2*sqrt(1+1/b) if k==0 else k*pi-atan(b*k*pi)
result.append(newtons_method(x0, f , f1, e))
lambda_result = sorted(list(set(result)))
print(len(result), len(lambda_result))

On ordinary differential equations (ODE) and optimization, in Python

I want to solve this kind of problem:
dy/dt = 0.01*y*(1-y), find t when y = 0.8 (0<t<3000)
I've tried the ode function in Python, but it can only calculate y when t is given.
So are there any simple ways to solve this problem in Python?
PS: This function is just a simple example. My real problem is so complex that can't be solve analytically. So I want to know how to solve it numerically. And I think this problem is more like an optimization problem:
Objective function y(t) = 0.8, Subject to dy/dt = 0.01*y*(1-y), and 0<t<3000
PPS: My real problem is:
objective function: F(t) = 0.85,
subject to: F(t) = sqrt(x(t)^2+y(t)^2+z(t)^2),
x''(t) = (1/F(t)-1)*250*x(t),
y''(t) = (1/F(t)-1)*250*y(t),
z''(t) = (1/F(t)-1)*250*z(t)-10,
x(0) = 0, y(0) = 0, z(0) = 0.7,
x'(0) = 0.1, y'(0) = 1.5, z'(0) = 0,
0<t<5

This differential equation can be solved analytically quite easily:
dy/dt = 0.01 * y * (1-y)
rearrange to gather y and t terms on opposite sides
100 dt = 1/(y * (1-y)) dy
The lhs integrates trivially to 100 * t, rhs is slightly more complicated. We can always write a product of two quotients as a sum of the two quotients * some constants:
1/(y * (1-y)) = A/y + B/(1-y)
The values for A and B can be worked out by putting the rhs on the same denominator and comparing constant and first order y terms on both sides. In this case it is simple, A=B=1. Thus we have to integrate
1/y + 1/(1-y) dy
The first term integrates to ln(y), the second term can be integrated with a change of variables u = 1-y to -ln(1-y). Our integrated equation therefor looks like:
100 * t + C = ln(y) - ln(1-y)
not forgetting the constant of integration (it is convenient to write it on the lhs here). We can combine the two logarithm terms:
100 * t + C = ln( y / (1-y) )
In order to solve t for an exact value of y, we first need to work out the value of C. We do this using the initial conditions. It is clear that if y starts at 1, dy/dt = 0 and the value of y never changes. Thus plug in the values for y and t at the beginning
100 * 0 + C = ln( y(0) / (1 - y(0) )
This will give a value for C (assuming y is not 0 or 1) and then use y=0.8 to get a value for t. Note that because of the logarithm and the factor 100 multiplying t y will reach 0.8 within a relatively short range of t values, unless the initial value of y is incredibly small. It is of course also straightforward to rearrange the equation above to express y in terms of t, then you can plot the function as well.
Edit: Numerical integration
For a more complexed ODE which cannot be solved analytically, you will have to try numerically. Initially we only know the value of the function at zero time y(0) (we have to know at least that in order to uniquely define the trajectory of the function), and how to evaluate the gradient. The idea of numerical integration is that we can use our knowledge of the gradient (which tells us how the function is changing) to work out what the value of the function will be in the vicinity of our starting point. The simplest way to do this is Euler integration:
y(dt) = y(0) + dy/dt * dt
Euler integration assumes that the gradient is constant between t=0 and t=dt. Once y(dt) is known, the gradient can be calculated there also and in turn used to calculate y(2 * dt) and so on, gradually building up the complete trajectory of the function. If you are looking for a particular target value, just wait until the trajectory goes past that value, then interpolate between the last two positions to get the precise t.
The problem with Euler integration (and with all other numerical integration methods) is that its results are only accurate when its assumptions are valid. Because the gradient is not constant between pairs of time points, a certain amount of error will arise for each integration step, which over time will build up until the answer is completely inaccurate. In order to improve the quality of the integration, it is necessary to use more sophisticated approximations to the gradient. Check out for example the Runge-Kutta methods, which are a family of integrators which remove progressive orders of error term at the cost of increased computation time. If your function is differentiable, knowing the second or even third derivatives can also be used to reduce the integration error.
Fortunately of course, somebody else has done the hard work here, and you don't have to worry too much about solving problems like numerical stability or have an in depth understanding of all the details (although understanding roughly what is going on helps a lot). Check out http://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.ode.html#scipy.integrate.ode for an example of an integrator class which you should be able to use straightaway. For instance
from scipy.integrate import ode
def deriv(t, y):
return 0.01 * y * (1 - y)
my_integrator = ode(deriv)
my_integrator.set_initial_value(0.5)
t = 0.1 # start with a small value of time
while t < 3000:
y = my_integrator.integrate(t)
if y > 0.8:
print "y(%f) = %f" % (t, y)
break
t += 0.1
This code will print out the first t value when y passes 0.8 (or nothing if it never reaches 0.8). If you want a more accurate value of t, keep the y of the previous t as well and interpolate between them.

As an addition to Krastanov`s answer:
Aside of PyDSTool there are other packages, like Pysundials and Assimulo which provide bindings to the solver IDA from Sundials. This solver has root finding capabilites.

Use scipy.integrate.odeint to handle your integration, and analyse the results afterward.
import numpy as np
from scipy.integrate import odeint
ts = np.arange(0,3000,1) # time series - start, stop, step
def rhs(y,t):
return 0.01*y*(1-y)
y0 = np.array([1]) # initial value
ys = odeint(rhs,y0,ts)
Then analyse the numpy array ys to find your answer (dimensions of array ts matches ys). (This may not work first time because I am constructing from memory).
This might involve using the scipy interpolate function for the ys array, such that you get a result at time t.
EDIT: I see that you wish to solve a spring in 3D. This should be fine with the above method; Odeint on the scipy website has examples for systems such as coupled springs that can be solved for, and these could be extended.

What you are asking for is a ODE integrator with root finding capabilities. They exist and the low-level code for such integrators is supplied with scipy, but they have not yet been wrapped in python bindings.
For more information see this mailing list post that provides a few alternatives: http://mail.scipy.org/pipermail/scipy-user/2010-March/024890.html
You can use the following example implementation which uses backtracking (hence it is not optimal as it is a bolt-on addition to an integrator that does not have root finding on its own): https://github.com/scipy/scipy/pull/4904/files

3d integral, python, integration set constrained

I wanted to compute the volume of the intersect of a sphere and infinite cylinder at some distance b, and i figured i would do it using a quick and dirty python script. My requirements are a <1s computation with >3 significant digits.
My thinking was as such:
We place the sphere, with radius R, such that its center is at the origin, and we place the cylinder, with radius R', such that its axis is spanned in z from (b,0,0). We integrate over the sphere, using a step function that returns 1 if we are inside the cylinder, and 0 if not, thus integrating 1 over the set constrained by being inside both sphere and cylinder, i.e. the intersect.
I tried this using scipy.intigrate.tplquad. It did not work out. I think its because of the discontinuity of the step function as i get warnings such the following. Of course, i might just be doing this wrong. Assuming i have not made some stupid mistake, I could attempt to formulate the ranges of the intersect, thus removing the need for the step function, but i figured i might try and get some feedback first. Can anyone spot any mistake, or point towards some simple solution.
Warning: The maximum number of
subdivisions (50) has been achieved.
If increasing the limit yields no
improvement it is advised to analyze
the integrand in order to determine
the difficulties. If the position of
a local difficulty can be
determined (singularity,
discontinuity) one will probably
gain from splitting up the interval
and calling the integrator on the
subranges. Perhaps a special-purpose
integrator should be used.
Code:
from scipy.integrate import tplquad
from math import sqrt
def integrand(z, y, x):
if Rprim >= (x - b)**2 + y**2:
return 1.
else:
return 0.
def integral():
return tplquad(integrand, -R, R,
lambda x: -sqrt(R**2 - x**2), # lower y
lambda x: sqrt(R**2 - x**2), # upper y
lambda x,y: -sqrt(R**2 - x**2 - y**2), # lower z
lambda x,y: sqrt(R**2 - x**2 - y**2), # upper z
epsabs=1.e-01, epsrel=1.e-01
)
R=1
Rprim=1
b=0.5
print integral()

Assuming you are able to translate and scale your data such a way that the origin of the sphere is in [0, 0, 0] and its radius is 1, then a simple stochastic approximation may give you a reasonable answer fast enough. So, something along the lines could be a good starting point:
import numpy as np
def in_sphere(p, r= 1.):
return np.sqrt((p** 2).sum(0))<= r
def in_cylinder(p, c, r= 1.):
m= np.mean(c, 1)[:, None]
pm= p- m
d= np.diff(c- m)
d= d/ np.sqrt(d** 2).sum()
pp= np.dot(np.dot(d, d.T), pm)
return np.sqrt(((pp- pm)** 2).sum(0))<= r
def in_sac(p, c, r_c):
return np.logical_and(in_sphere(p), in_cylinder(p, c, r_c))
if __name__ == '__main__':
n, c= 1e6, [[0, 1], [0, 1], [0, 1]]
p= 2* np.random.rand(3, n)- 2
print (in_sac(p, c, 1).sum()/ n)* 2** 3

Performing a triple adaptive numerical integrations on a discontinuous function that is constant over two domains is a terribly poor idea, especially if you wish to see either speed or accuracy.
I would suggest a far better idea is to reduce the problem analytically.
Align the cylinder with an axis, by transformation. This translates the sphere to some point that is not at the origin.
Now, find the limits of intersection of the sphere with the cylinder along that axis.
Integrate over that axis variable. The area of intersection at any fixed value along the axis is simply the area of intersection of two circles, which in turn is simply computable using trigonometry and a little effort.
In the end, you will have an exact result, with almost no computation time needed.

I solved it using a simple MC integration, as suggested by eat, but my implementation was to slow. My requirements had increased. I therefore reformulated the problem mathematically, as suggested by woodchips.
Basically i formulated the limits of x as a function of z and y, and y as a function of z. Then i, in essence, integrated f(z,y,z)=1 over the intersection, using the limits. I did this because of the speed increase, allowing me to plot volume vs b, and because it allows me to integrate more complex functions with relative minor modification.
I include my code in case anyone is interested.
from scipy.integrate import quad
from math import sqrt
from math import pi
def x_max(y,r):
return sqrt(r**2-y**2)
def x_min(y,r):
return max(-sqrt(r**2 - y**2), -sqrt(R**2 - y**2) + b)
def y_max(r):
if (R<b and b-R<r) or (R>b and b-R>r):
return sqrt( R**2 - (R**2-r**2+b**2)**2/(4.*b**2) )
elif r+R<b:
return 0.
else: #r+b<R
return r
def z_max():
if R>b:
return R
else:
return sqrt(2.*b*R - b**2)
def delta_x(y, r):
return x_max(y,r) - x_min(y,r)
def int_xy(z):
r = sqrt(R**2 - z**2)
return quad(delta_x, 0., y_max(r), args=(r))
def int_xyz():
return quad(lambda z: int_xy(z)[0], 0., z_max())
R=1.
Rprim=1.
b=0.5
print 4*int_xyz()[0]

First off: You can calculate the volume of the intersection by hand. If you don't want to (or can't) do that, here's an alternative:
I'd generate a tetrahedral mesh for the domain and then add up the cell volumes. An example with pygalmesh and meshplex (both authored by myself):
import pygalmesh
import meshplex
import numpy
ball = pygalmesh.Ball([0, 0, 0], 1.0)
cyl = pygalmesh.Cylinder(-1, 1, 0.7, 0.1)
u = pygalmesh.Intersection([ball, cyl])
mesh = pygalmesh.generate_mesh(u, cell_size=0.05, edge_size=0.1)
points = mesh.points
cells = mesh.cells["tetra"]
# kick out unused vertices
uvertices, uidx = numpy.unique(cells, return_inverse=True)
cells = uidx.reshape(cells.shape)
points = points[uvertices]
mp = meshplex.MeshTetra(points, cells)
print(sum(mp.cell_volumes))
This gives you
and prints 2.6567890958740463 as volume. Decrease cell or edge sizes for higher precision.