I was given two equations one for the growth healthy people in the population,
dh/dt=-.05*h*s+.0003*h,
and the other equation is for the infection rate of sick people
ds/dt=.05*h*s-.01*s.
assume that after 10 days of being infected the people die.
for initial variables h=9000 and s=100
using the differential equations generate a figure predicting the impact of the out come of the infection on the population. It was suggested that eulers method be used, how would I use Eulers method using multiple differential equations? or is there a better method you would suggest and how would you use it?
In python you would use, for instance, scipy.integrate.odeint and compute
def odesys(u,t):
h, s = u
return [ -.05*h*s+.0003*h, .05*h*s-.01*s]
h0, s0 = 9000, 100
t0, tf = 0, 0.10
t = linspace(t0, tf, 301)
sol = odeint(odesys, [h0, s0], t)
h, s = sol.T
plot(t,h, label="healthy")
plot(t,s, label="sick")
And if you have to use Euler, using the same interface if would look like
def odeinteuler(f, y0, tspan):
y = zeros([len(tspan),len(y0)])
y[0,:]=y0
for k in range(1, len(tspan)):
y[k,:] = y[k-1,:]+(t[k]-t[k-1])*array(f(y[k-1], t[k-1]))
return y
sol = odeint(odesys, [h0, s0], t)
Related
I have two datasets with the same features and would like to estimate the "distance of distributions" between the two datasets. I had the idea to estimate a gaussian KDE in each of the datasets and computing the KL-divergence between the estimated KDEs. However, I am struggling to compute the "distance" between the distributions. This is what I have so far:
import numpy as np
from scipy import stats
from scipy.stats import entropy
dataset1 = np.random.rand(50)
dataset2 = np.random.rand(49)
kernel1 = stats.gaussian_kde(dataset1)
kernel2 = stats.gaussian_kde(dataset2)
I know I can use entropy(pk, qk) to calculate the kl-divergence but I don't understand how do that starting from the kernels. I thought about generating some random points and using entropy(kernel1.pdf(points),kernel2.pdf(points)) but the pdf function outputs some weird number (higher than 1 sometimes, does it mean it assigns more than 100% of prob??), and I am not sure the output I get is correct.
If anyone knows how to calculate the distance between the 2 gaussian kde kernels I would be very thankful.
There is no closed form solution for KL between two mixtures of gaussians.
KL(p, q) := -E_p log [p(x)/q(x)]
so you can use MC estimator:
def KL_mc(p, q, n=100):
points = p.resample(n)
p_pdf = p.pdf(points)
q_pdf = q.pdf(points)
return np.log(p_pdf / q_pdf).mean()
Note:
you might need to add some clipping to avoid 0s and infinities
depending on the dimensionality of the space this can require quite large n
(higher than 1 sometimes, does it mean it assigns more than 100% of prob??)
PDF is not a probability. Not for continuous distributions. It is a probability density. It is a relative measure. Probability assigned to any single value is always 0, but probability of sampling an element in a given set/interval equals integral of pdf over this set/integral (and thus pointwise it can have a weight >1, but over a "small enough" set)
More general solution
Overall, unless you really need KL for theoretical reasons, there are divergences that are better suited to deal with gaussian mixtures (e.g. such that have closed form solutions), for example Cauchy-Schwarz Divergence.
In particular you can look at Maximum Entropy Linear Manifold which is based exactly on computing CS divergences between KDEs of points. You can see python implementation in melm/dcsk.py in value(v) function on github. In your case you do not want a projection so just put v = identity matrix.
def value(self, v):
# We need matrix, not vector
v = v.reshape(-1, self.k)
ipx0 = self._ipx(self.x0, self.x0, v)
ipx1 = self._ipx(self.x1, self.x1, v)
ipx2 = self._ipx(self.x0, self.x1, v)
return np.log(ipx0) + np.log(ipx1) - 2 * np.log(ipx2)
def _f1(self, X0, X1, v):
Hxy = self.gamma * self.gamma * self._H(X0, X1)
vHv = v.T.dot(Hxy).dot(v)
return 1.0 / (X0.shape[0] * X1.shape[0] * np.sqrt(la.det(vHv)) * (2 * np.pi) ** (self.k / 2))
def _f2(self, X0, X1, v):
Hxy = self.gamma * self.gamma * self._H(X0, X1)
vHv = v.T.dot(Hxy).dot(v)
vHv_inv = la.inv(vHv)
vx0 = X0.dot(v)
vx1 = X1.dot(v)
vx0c = vx0.dot(vHv_inv)
vx1c = vx1.dot(vHv_inv)
ret = 0.0
for i in range(X0.shape[0]):
ret += np.exp(-0.5 * ((vx0c[i] - vx1c) * (vx0[i] - vx1)).sum(axis=1)).sum()
return ret
def _ipx(self, X0, X1, v):
return self._f1(X0, X1, v) * self._f2(X0, X1, v)
Main difference between CS and KL is that KL requires your to compute integral of a logarithm of a pdf and CS computes logarithm of the integral. It happens, that with gaussian mixtures it is this integration of the logarithm that is a problem, without the logarithm everything is easy, and thus DCS is preferable.
Background
I've written a python implementation of this answer to my recent question over on Math SE. In short, the problem is as follows:
I have an experimental setup consisting of three receivers, with known rectangular coordinates [xi, yi, zi], and a transmitter with unknown coordinates [x,y,z] emitting a signal at unknown time t with velocity c and arriving at receiver i at known time ti.
This information is insufficient to uniquely determine the rectangular coordinates of the transmitter. We can, however, estimate the angle to the transmitter (i.e. the transmitter's phi and theta in spherical coordinates). This may be done by solving the system of equations given in the linked post.
My goal is to solve these equations for real, experimental data.
Problem
I have been able to write an effective python implementation of the approach described in the linked post. For both simulated and experimental data, this gives satisfactory estimates of the angle to the transmitter.
I now wish to use this in practice. Unfortunately, however, my code runs too slowly to be useful in my desired application.
Ideally, we'd like to be able to solve for on the order of 1 million datapoints per hour in
a near-real-time application. Currently, this takes several hours. While I recognize that, particularly with Python, dramatic performance improvements are not to be expected, any improvement would be helpful.
In short, I'd like to reduce the execution time of this algorithm. Due to the minimal Linux install (and my lack of control over it) being used on the host machine, I'd like to do so by improving my code, and without use of additional modules/external libraries/etc if possible.
Code
import math
import numpy as np
import ROOT
from scipy.optimize import root
from dataclasses import dataclass
from ROOT import TFile, TNtuple
c = 299792
#dataclass
class Vertexer:
roc: list
def fun(self, var, dat):
f0 = var.dot(self.roc[0] - self.roc[1]) - c * (dat[1] - dat[0])
f1 = var.dot(self.roc[1] - self.roc[2]) - c * (dat[2] - dat[1])
n = np.linalg.norm(var) - 1
return [f0, f1, n]
def find(self, dat):
result = root(
self.fun,
(0, 0, 0),
method="lm",
args=dat,
options={
"col_deriv": 1,
"xtol": 1.49012e-08,
"ftol": 1.49012e-08,
"gtol": 0.0,
"maxiter": 0,
"eps": 0.0,
"factor": 100,
"diag": None,
},
)
if result.success:
return result.x
def main():
myVertexer = Vertexer(
[
np.array([3.0085470085470085, 3.7642857142857116, -0.06]),
np.array([2.0034188034188034, 2.0142857142857133, -0.19]),
np.array([1.0324786324786326, 0.27142857142857135, -0.19]),
]
)
data = ROOT.RDataFrame("D", "2018_filtered.root")
colns = data.AsNumpy(columns=["ai", "aj", "ak"])
f = TFile("theta_phi_2.root", "RECREATE")
ntuple = TNtuple("N", "N", "i:j:k:ai:aj:ak")
for (ai, aj, ak) in zip(colns["ai"], colns["aj"], colns["ak"]):
v = myVertexer.find([ai, aj, ak])
if v.any() != None:
ntuple.Fill(v[0], v[1], v[2], ai, aj, ak)
ntuple.Write()
f.Write()
main()
Step-Through
The Vertexer class contains a fairly standard and straightforward SciPy based system of equations solution "algorithm". The function func() contains the system of equations described in the linked post, and find() solves the system given the times of arrival (dat) and the receiver coordinates (roc, provided upon instantiation). I'm using a Levenberg Marquardt solver with coln_deriv set to True in order to improve solution speed. I see similar solution accuracy across all solvers. All other settings are set to default.
The main() function reads time-of-arrival data from a .root file (see) into a dict of NumPy arrays, which I loop over, feed to the algorithm, and record the result in another .root file.
If you like, you may replace the main() function with the following crude "source simulation" code I've written, which simply produces a random point, computes the arrival time of a signal from that point to three randomly-placed "receivers", and feeds those times to the algorithm:
x0 = random.randrange(0,1000); y0 = random.randrange(0,1000); z0 = random.randrange(0,1000)
x1 = random.randrange(0,50); x2 = random.randrange(0,50); x3 = random.randrange(0,50);
y1 = random.randrange(0,50); y2 = random.randrange(0,50); y3 = random.randrange(0,50);
z1 = random.randrange(0,50); z2 = random.randrange(0,50); z3 = random.randrange(0,50);
t1 = math.sqrt((x0-x1)**2 + (y0-y1)**2 + (z0-z1)**2)/c
t2 = math.sqrt((x0-x2)**2 + (y0-y2)**2 + (z0-z2)**2)/c
t3 = math.sqrt((x0-x3)**2 + (y0-y3)**2 + (z0-z3)**2)/c
myVertexer = Vertexer([[x1,y1,z1], [x2,y2,z2], [x3,y3,z3]])
result = myVertexer.find([t1,t2,t3])
You can solve your equations analytically rather than numerically (probably someone will say that this is not the place for this).
Let's analyze your function
def fun(self, var, dat):
f0 = var.dot(self.roc[0] - self.roc[1]) - c * (dat[1] - dat[0])
f1 = var.dot(self.roc[1] - self.roc[2]) - c * (dat[2] - dat[1])
n = np.linalg.norm(var) - 1
return [f0, f1, n]
The solutions of f0 = 0 lie in a plane normal to self.roc[0] - self.roc[1].
The solutions of f1 = 0 lie in a plane normal to self.roc[1] - self.roc[2].
The solutions of n = 0 lies in a sphere.
The intersection of two planes is a line going on the direction give by the cross product between the normal of the two planes.
The intersection between a line and a sphere will either be (a) one tangent point; (b) two points, one entering, one leaving the sphere; (c) no solution if the line passes is far from the sphere.
I have a bit of a simple question regarding solving a differential equation I'm hoping someone can help me answer. I have a differential equation for a simple neuron model, with varying input (i.e. integrate and fire model of a neuron, with a "square" input):
def impulse_func(t, V):
if 100 <= t <= 200:
return 0.25
return 0
def passive_membrane(t, V, El, R, tau, I_app):
dVdt = (-(V - El) + R*I_app(t, V))/tau
return dVdt
t = np.arange(0, 400, 0.1)
V0 = np.array([-60.])
V = solve_ivp(fun = passive_membrane,
t_span = [0, 400],
t_eval = t,
y0=V0,
args = (-60., 100., 10., impulse_func),
dense_output=True,
rtol = 1e-8, atol = 1e-8)
In this case, the values of El, R, and tau will be remaining constant. This works just as expected, with the plot being My problem is that I now need to create a "spike" in the solution which is dependent on the solution.
Specifically, if the solution passes my "threshold" value of -37, for a single unit of time I would like to set the solution value to 60 and then return to my "resting" which is -65 in the next unit of time. I've tried modifying the impulse_func and the passive_membrane functions, but it just led to an exploding solution. I'm able to do this when I write my own solver, but I'm lost on how to implement it with solve_ivp, which I'd like to use since it has much more functionality than my own poorly optimized RK2 method and because I want to be able to pass variable thresholds, times, etc.
I want to use Hawkes process to model some data. I could not find whether PyMC supports Hawkes process. More specifically I want an observed variable with Hawkes Process and learn a posterior on its params.
If it is not there, then could I define it in PyMC in some way e.g. #deterministic etc.??
It's been quite a long time since your question, but I've worked it out on PyMC today so I'd thought I'd share the gist of my implementation for the other people who might get across the same problem. We're going to infer the parameters λ and α of a Hawkes process. I'm not going to cover the temporal scale parameter β, I'll leave that as an exercise for the readers.
First let's generate some data :
def hawkes_intensity(mu, alpha, points, t):
p = np.array(points)
p = p[p <= t]
p = np.exp(p - t)
return mu + alpha * np.sum(p)
def simulate_hawkes(mu, alpha, window):
t = 0
points = []
lambdas = []
while t < window:
m = hawkes_intensity(mu, alpha, points, t)
s = np.random.exponential(scale=1/m)
ratio = hawkes_intensity(mu, alpha, points, t + s)
t = t + s
if t < window:
points.append(t)
lambdas.append(ratio)
else:
break
points = np.sort(np.array(points, dtype=np.float32))
lambdas = np.array(lambdas, dtype=np.float32)
return points, lambdas
# parameters
window = 1000
mu = 8
alpha = 0.25
points, lambdas = simulate_hawkes(mu, alpha, window)
num_points = len(points)
We just generated some temporal points using some functions that I adapted from there : https://nbviewer.jupyter.org/github/MatthewDaws/PointProcesses/blob/master/Temporal%20points%20processes.ipynb
Now, the trick is to create a matrix of size (num_points, num_points) that contains the temporal distance of the ith point from all the other points. So the (i, j) point of the matrix is the temporal interval separating the ith point to the jth. This matrix will be used to compute the sum of the exponentials of the Hawkes process, ie. the self-exciting part. The way to create this matrix as well as the sum of the exponentials is a bit tricky. I'd recommend to check every line yourself so you can see what they do.
tile = np.tile(points, num_points).reshape(num_points, num_points)
tile = np.clip(points[:, None] - tile, 0, np.inf)
tile = np.tril(np.exp(-tile), k=-1)
Σ = np.sum(tile, axis=1)[:-1] # this is our self-exciting sum term
We have points and we have a matrix containg the sum of the excitations term.
The duration between two consecutive events of a Hawkes process follow an exponential distribution of parameter λ = λ0 + ∑ excitation. This is what we are going to model, but first we have to compute the duration between two consecutive points of our generated data.
interval = points[1:] - points[:-1]
We're now ready for inference:
with pm.Model() as model:
λ = pm.Exponential("λ", 1)
α = pm.Uniform("α", 0, 1)
lam = pm.Deterministic("lam", λ + α * Σ)
interarrival = pm.Exponential(
"interarrival", lam, observed=interval)
trace = pm.sample(2000, tune=4000)
pm.plot_posterior(trace, var_names=["λ", "α"])
plt.show()
print(np.mean(trace["λ"]))
print(np.mean(trace["α"]))
7.829
0.284
Note: the tile matrix can become quite large if you have many data points.
I currently have a system of odes with a time-dependent constant. E.g.
def fun(u, t, a, b, c):
x = u[0]
y = u[1]
z = u[2]
dx_dt = a * x + y * z
dy_dt = b * (y-z)
dz_dt = -x*y+c*y-z
return [dx_dt, dy_dt, dz_dt]
The constants are "a", "b" and "c". I currently have a list of "a"s for every time-step which I would like to insert at every time-step, when using the scipy ode solver...is this possible?
Thanks!
Yes, this is possible. In the case where a is constant, I guess you called scipy.integrate.odeint(fun, u0, t, args) where fun is defined as in your question, u0 = [x0, y0, z0] is the initial condition, t is a sequence of time points for which to solve for the ODE and args = (a, b, c) are the extra arguments to pass to fun.
In the case where a depends on time, you simply have to reconsider a as a function, for example (given a constant a0):
def a(t):
return a0 * t
Then you will have to modify fun which computes the derivative at each time step to take the previous change into account:
def fun(u, t, a, b, c):
x = u[0]
y = u[1]
z = u[2]
dx_dt = a(t) * x + y * z # A change on this line: a -> a(t)
dy_dt = b * (y - z)
dz_dt = - x * y + c * y - z
return [dx_dt, dy_dt, dz_dt]
Eventually, note that u0, t and args remain unchanged and you can again call scipy.integrate.odeint(fun, u0, t, args).
A word about the correctness of this approach. The performance of the approximation of the numerical integration is affected, I don't know precisely how (no theoretical guarantees) but here is a simple example which works:
import matplotlib.pyplot as plt
import numpy as np
import scipy as sp
import scipy.integrate
tmax = 10.0
def a(t):
if t < tmax / 2.0:
return ((tmax / 2.0) - t) / (tmax / 2.0)
else:
return 1.0
def func(x, t, a):
return - (x - a(t))
x0 = 0.8
t = np.linspace(0.0, tmax, 1000)
args = (a,)
y = sp.integrate.odeint(func, x0, t, args)
fig = plt.figure()
ax = fig.add_subplot(111)
h1, = ax.plot(t, y)
h2, = ax.plot(t, [a(s) for s in t])
ax.legend([h1, h2], ["y", "a"])
ax.set_xlabel("t")
ax.grid()
plt.show()
I Hope this will help you.
No, that is not possible in the literal sense of
"I currently have a list of "a"s for every time-step which I would like to insert at every time-step"
as the solver has adaptive step size control, that is, it will use internal time steps that you have no control over, and each time step uses several evaluations of the function. Thus there is no connection between the solver time steps and the data time steps.
In the extended sense that the given data defines a piecewise constant step function however, there are several approaches to get to a solution.
You can integrate from jump point to jump point, using the ODE function with the constant parameter for this time segment. After that use numpy array operations like concatenate to assemble the full solution.
You can use interpolation functions like numpy.interp or scipy.interpolate.interp1d. The first gives a piecewise linear interpolation, which may not be desired here. The second returns a function object that can be configured to be a "zero-order hold", which is a piecewise constant step function.
You could implement your own logic to go from the time t to the correct values of those parameters. This mostly applies if there is some structure to the data, for instance, if they have the form f(int(t/h)).
Note that the approximation order of the numerical integration is not only bounded by the order of the RK (solve_ivp) or multi-step (odeint) method, but also by the differentiability order of the (parts of) the differential equation. If the ODE is much less smooth than the order of the method, the implicit assumptions for the step size control mechanism are violated, which may result in a very small step size requiring a huge number of integration steps.
I also encountered similar problem. In my case, parameters a, b, and c are not in direct function with time, but determined by x, y, and z at that time. So I have to get x, y, z at time t, and calculate a, b, c for the integration calculation for x, y, z at t+dt. It turns out that if I change dt value, the whole integration result will change dramatically, even to something unreasonable.