We Keep Coding

finding probability in multivariate normal distribution

I am using python. I know that to find probability in multivariate normal distribution I have to use following:
fx(x1,…,xk) = (1/√(2π)^k|Σ|) * exp(−1/2(x−μ)T* Σ^-1 *(x−μ))
where x = [x1, x2]
I have different values of x1 and x2.
but here I have to find probability for:
0.5< x1<1.5 and 4.5< x2<5.5
I know how to use this formula for single value of x1 and x2. But I am confused in this case. Please help.

What you need to so is find the area beneath the function for the rectangle bounded by 0.5 < x1 < 1.5 and 4.5 < x2 < 5.5.
As a quick and dirty solution, you could use this code to do a two-variable Reimann sum to estimate the integral. A Reimann sum just divides the rectangle into small regions and approximates the area under each region as if the function was flat.
Provided you've defined your distribution as the function f.
x1Low = 0.5
x1Hi = 1.5
x2Low = 4.5
x2Hi = 5.5
x1steps = 1000
x2steps = 1000
x1resolution = (x1Hi-x1Low)/x1steps
x2resolution = (x2Hi-x2Low)/x2steps
area = x1resolution*x2resolution
x1vals = [x1Low + i*x1resolution for i in range(x1steps)]
x2vals = [x2Low + i*x2resolution for i in range(x2steps)]
sum = 0;
for i in range(len(x1vals-1)):
for j in range(len(x2vals-1)):
sum += area * f(x1vals[i],x2vals[j])
print sum
Keep in mind that this sum is only an approximation, and not a great one either. It will seriously over- or under-estimate the integral in regions where the function changes too quickly.
If you need more accuracy, you can try implementing triangle rule or simpsons's rule, or look into scipy's numerical integration tools.

Monte Carlo simulations by large no of trials

Consider that following program.
import math
import random
def inside_unit_circle(point):
"""
Compute distance of point from origin
"""
distance = math.sqrt(point[0] ** 2 + point[1] ** 2)
return distance < 1
def estimate_mystery(num_trials):
"""
Main function
"""
num_inside = 0
for dumm_idx in range(num_trials):
new_point = [2 * random.random() - 1, 2 * random.random() - 1]
if inside_unit_circle(new_point):
num_inside += 1
return float(num_inside) / num_trials
print estimate_mystery(10000)
This program uses random.random() to generates a random set of points that are uniformly distributed over the square with corners at
(1, 1) (−1, 1)
(1,−1) (−1,−1)
Here, being uniformly distribution means that each point in the square has an equal chance of being generated. The method then tests whether these points lie inside a unit circle.
As one increases the number of trials, the value returned by estimate_mystery tends towards a specific value that has a simple expression involving a well-known constant. Enter this value as a math expression below. (Do not enter a floating point number.)

So you need to run estimate_mystery with increasingly higher numbers of trials. As you do so, it will become clear that the value increases to the following simple expression:
(\sum_{k=1}^{\infty} \frac{e^{i\pi(k+1)}}{2k-1})
It should be noted, however, that this is not the only correct answer. The following would have been valid too, where \zeta is the Riemann zeta function:
However, this does not include the well-known constant e.
I'm not sure why this is confusing. It's quite clear that the sum expression is correct, and it's written quite clearly: the code below the image is very standard LaTeX formatting for mathematical expressions. But to illustrate its correctness, here's a plot showing the convergence when taking the sum to n, and running estimate_mystery up to n as well:
Hrmm... maybe this wasn't what your question wanted? It should also converge to the following, where \gamma is a unit circle around z=0 on the complex plane:
(-i\oint_\gamma z^{-3}e^{\frac{z}{2}}dz)

If you try estimate_mystery() method with different inputs such as with, 100, 1000, 10000, 100000), you will see that the result will be 0.81, 0.781 0.7807 0.7855, accordingly.
It means, the more you increase the trial number, the result is getting closer ( converges ) to 0.7855. This number can be defined with Pi.
You can find it just by simple calculation. Pi * x = 0.7855. From this equation we can find that x ~ 0.25. Therefore, 0.7855 can be described with Pi/4.

On ordinary differential equations (ODE) and optimization, in Python

I want to solve this kind of problem:
dy/dt = 0.01*y*(1-y), find t when y = 0.8 (0<t<3000)
I've tried the ode function in Python, but it can only calculate y when t is given.
So are there any simple ways to solve this problem in Python?
PS: This function is just a simple example. My real problem is so complex that can't be solve analytically. So I want to know how to solve it numerically. And I think this problem is more like an optimization problem:
Objective function y(t) = 0.8, Subject to dy/dt = 0.01*y*(1-y), and 0<t<3000
PPS: My real problem is:
objective function: F(t) = 0.85,
subject to: F(t) = sqrt(x(t)^2+y(t)^2+z(t)^2),
x''(t) = (1/F(t)-1)*250*x(t),
y''(t) = (1/F(t)-1)*250*y(t),
z''(t) = (1/F(t)-1)*250*z(t)-10,
x(0) = 0, y(0) = 0, z(0) = 0.7,
x'(0) = 0.1, y'(0) = 1.5, z'(0) = 0,
0<t<5

This differential equation can be solved analytically quite easily:
dy/dt = 0.01 * y * (1-y)
rearrange to gather y and t terms on opposite sides
100 dt = 1/(y * (1-y)) dy
The lhs integrates trivially to 100 * t, rhs is slightly more complicated. We can always write a product of two quotients as a sum of the two quotients * some constants:
1/(y * (1-y)) = A/y + B/(1-y)
The values for A and B can be worked out by putting the rhs on the same denominator and comparing constant and first order y terms on both sides. In this case it is simple, A=B=1. Thus we have to integrate
1/y + 1/(1-y) dy
The first term integrates to ln(y), the second term can be integrated with a change of variables u = 1-y to -ln(1-y). Our integrated equation therefor looks like:
100 * t + C = ln(y) - ln(1-y)
not forgetting the constant of integration (it is convenient to write it on the lhs here). We can combine the two logarithm terms:
100 * t + C = ln( y / (1-y) )
In order to solve t for an exact value of y, we first need to work out the value of C. We do this using the initial conditions. It is clear that if y starts at 1, dy/dt = 0 and the value of y never changes. Thus plug in the values for y and t at the beginning
100 * 0 + C = ln( y(0) / (1 - y(0) )
This will give a value for C (assuming y is not 0 or 1) and then use y=0.8 to get a value for t. Note that because of the logarithm and the factor 100 multiplying t y will reach 0.8 within a relatively short range of t values, unless the initial value of y is incredibly small. It is of course also straightforward to rearrange the equation above to express y in terms of t, then you can plot the function as well.
Edit: Numerical integration
For a more complexed ODE which cannot be solved analytically, you will have to try numerically. Initially we only know the value of the function at zero time y(0) (we have to know at least that in order to uniquely define the trajectory of the function), and how to evaluate the gradient. The idea of numerical integration is that we can use our knowledge of the gradient (which tells us how the function is changing) to work out what the value of the function will be in the vicinity of our starting point. The simplest way to do this is Euler integration:
y(dt) = y(0) + dy/dt * dt
Euler integration assumes that the gradient is constant between t=0 and t=dt. Once y(dt) is known, the gradient can be calculated there also and in turn used to calculate y(2 * dt) and so on, gradually building up the complete trajectory of the function. If you are looking for a particular target value, just wait until the trajectory goes past that value, then interpolate between the last two positions to get the precise t.
The problem with Euler integration (and with all other numerical integration methods) is that its results are only accurate when its assumptions are valid. Because the gradient is not constant between pairs of time points, a certain amount of error will arise for each integration step, which over time will build up until the answer is completely inaccurate. In order to improve the quality of the integration, it is necessary to use more sophisticated approximations to the gradient. Check out for example the Runge-Kutta methods, which are a family of integrators which remove progressive orders of error term at the cost of increased computation time. If your function is differentiable, knowing the second or even third derivatives can also be used to reduce the integration error.
Fortunately of course, somebody else has done the hard work here, and you don't have to worry too much about solving problems like numerical stability or have an in depth understanding of all the details (although understanding roughly what is going on helps a lot). Check out http://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.ode.html#scipy.integrate.ode for an example of an integrator class which you should be able to use straightaway. For instance
from scipy.integrate import ode
def deriv(t, y):
return 0.01 * y * (1 - y)
my_integrator = ode(deriv)
my_integrator.set_initial_value(0.5)
t = 0.1 # start with a small value of time
while t < 3000:
y = my_integrator.integrate(t)
if y > 0.8:
print "y(%f) = %f" % (t, y)
break
t += 0.1
This code will print out the first t value when y passes 0.8 (or nothing if it never reaches 0.8). If you want a more accurate value of t, keep the y of the previous t as well and interpolate between them.

As an addition to Krastanov`s answer:
Aside of PyDSTool there are other packages, like Pysundials and Assimulo which provide bindings to the solver IDA from Sundials. This solver has root finding capabilites.

Use scipy.integrate.odeint to handle your integration, and analyse the results afterward.
import numpy as np
from scipy.integrate import odeint
ts = np.arange(0,3000,1) # time series - start, stop, step
def rhs(y,t):
return 0.01*y*(1-y)
y0 = np.array([1]) # initial value
ys = odeint(rhs,y0,ts)
Then analyse the numpy array ys to find your answer (dimensions of array ts matches ys). (This may not work first time because I am constructing from memory).
This might involve using the scipy interpolate function for the ys array, such that you get a result at time t.
EDIT: I see that you wish to solve a spring in 3D. This should be fine with the above method; Odeint on the scipy website has examples for systems such as coupled springs that can be solved for, and these could be extended.

What you are asking for is a ODE integrator with root finding capabilities. They exist and the low-level code for such integrators is supplied with scipy, but they have not yet been wrapped in python bindings.
For more information see this mailing list post that provides a few alternatives: http://mail.scipy.org/pipermail/scipy-user/2010-March/024890.html
You can use the following example implementation which uses backtracking (hence it is not optimal as it is a bolt-on addition to an integrator that does not have root finding on its own): https://github.com/scipy/scipy/pull/4904/files

Model I-V in Python

Model I-V.
Method:
Perform an integral, as a function of E, which outputs Current for each Voltage value used. This is repeated for an array of v_values. The equation can be found below.
Although the limits in this equation range from -inf to inf, the limits must be restricted so that (E+eV)^2-\Delta^2>0 and E^2-\Delta^2>0, to avoid poles. (\Delta_1 = \Delta_2). Therefore there are currently two integrals, with limits from -inf to -gap-e*v and gap to inf.
However, I keep returning a math range error although I believe I have excluded the troublesome E values by using the limits stated above. Pastie of errors: http://pastie.org/private/o3ugxtxai8zbktyxtxuvg
Apologies for the vagueness of this question. But, can anybody see obvious mistakes or code misuse?
My attempt:
from scipy import integrate
from numpy import *
import scipy as sp
import pylab as pl
import numpy as np
import math
e = 1.60217646*10**(-19)
r = 3000
gap = 400*10**(-6)*e
g = (gap)**2
t = 0.02
k = 1.3806503*10**(-23)
kt = k*t
v_values = np.arange(0,0.001,0.0001)
I=[]
for v in v_values:
val, err = integrate.quad(lambda E:(1/(e*r))*(abs(E)/np.sqrt(abs(E**2-g)))*(abs(E+e*v)/(np.sqrt(abs((E+e*v)**2-g))))*((1/(1+math.exp((E+e*v)/kt)))-(1/(1+math.exp(E/k*t)))),-inf,(-gap-e*v)*0.9)
I.append(val)
I = array(I)
I2=[]
for v in v_values:
val2, err = integrate.quad(lambda E:(1/(e*r))*(abs(E)/np.sqrt(abs(E**2-g)))*(abs(E+e*v)/(np.sqrt(abs((E+e*v)**2-g))))*((1/(1+math.exp((E+e*v)/kt)))-(1/(1+math.exp(E/k*t)))),gap*0.9,inf)
I2.append(val2)
I2 = array(I2)
I[np.isnan(I)] = 0
I[np.isnan(I2)] = 0
pl.plot(v_values,I,'-b',v_values,I2,'-b')
pl.show()

This question is better suited for the Computational Science site. Still here are some points for you to think about.
First, the range of integration is the intersection of (-oo, -eV-gap) U (-eV+gap, +oo) and (-oo, -gap) U (gap, +oo). There are two possible cases:
if eV < 2*gap then the allowed energy values are in (-oo, -eV-gap) U (gap, +oo);
if eV > 2*gap then the allowed energy values are in (-oo, -eV-gap) U (-eV+gap, -gap) U (gap, +oo).
Second, you are working in a very low temperature region. With t equal to 0.02 K, the denominator in the Boltzmann factor is 1.7 µeV, while the energy gap is 400 µeV. In this case the value of the exponent is huge for positive energies and it soon goes off the limits of the double precision floating point numbers, used by Python. As this is the minimum possible positive energy, things would not get any better at higher energies. With negative energies the value would always be very close to zero. Note that at this temperature, the Fermi-Dirac distribution has a very sharp edge and resembles a reflected theta function. At E = gap you would have exp(E/kT) of approximately 6.24E+100. You would run out of range when E/kT > 709.78 or E > 3.06*gap.
Yet it makes no sense to go to such energies since at that temperature the difference between the two Fermi functions very quickly becomes zero outside the [-eV, 0] interval which falls entirely inside the gap for the given temperature when V < (2*gap)/e (0.8 mV). That's why one would expect that the current would be very close to zero when the bias voltage is less than 0.8 mV. When it is more than 0.8 mV, then the main value of the integral would come from the integrand in (-eV+gap, -gap), although some non-zero value would come from the region near the singularity at E = gap and some from the region near the singularity at E = -eV-gap. You should not avoid the singularities in the DoS, otherwise you would not get the expected discontinuities (vertical lines) in the I(V) curve (image taken from Wikipedia):
Rather, you have to derive equivalent approximate expressions in the vicinity of each singularity and integrate them instead.
As you can see, there are many special cases for the value of the integrand and you have to take them all into account when computing numerically. If you don't want to do that, you should probably turn to some other mathematical package like Maple or Mathematica. These have much more sophisticated numerical integration routines and might be able to directly handle your formula.
Note that this is not an attempt to answer your question but rather a very long comment that would not fit in any comment field.

The reason for the math range error is that your exponential goes to infinity. Taking v = 0.0009 and E = 5.18e-23, the expression exp((E + e*v) / kt) (I corrected the typo pointed out by Hristo Liev in your Python expression) is exp(709.984..) which is beyond the range you can represent with double precision numbers (up to ca. 1E308).
Two additional notes:
As noted by others, you should probably rescale your equation by using a unit system which delivers numbers in a smaller range. Maybe, atomic units are a possible choice as it would set e = 1, but I did not try to convert your equation into it. (Probably, your timestep would then become quite large, as in atomic units the time unit is about is 1/40 fs).
Usually, one uses the exponential notation for float point numbers: e = 1.60217E-19 instead of e = 1.60217*10**(-19).

The best way to approach this problem in the end was to use a heaviside function to preventE variable from exceeding \Delta variable.

Generate random integer weighted toward median

Where in the range 10-20 there would be twice the probability of 15 being returned than either extreme.

You can use random.triangular() with Python >= 2.6:
n = random.triangular(10, 20)
n will be a floating point value, so you need to convert it to int.

As pointed out by Blender, you really need to be more specific. But in the simplest case you can generate a Triangular Distribution from a uniform variate.

Try and see if this works (sorry if it's not very readable):
import random
def randIntWeight(min, max):
distanceFromMedian = random.uniform(0, (max - min) / 2.0)
return (max - min) / 2.0 + distanceFromMedian * (-1) ** (random.randrange(-1, 0))
I'm still brushing up on my Probability Theory, so please correct me if this isn't right.

Another built-in function is [numpy.random.normal][1]
numpy.random.normal(loc=0.0, scale=1.0, size=None)
Draw random samples from a normal (Gaussian) distribution.
You can specify loc=15.0 to set the mean and scale=2 to 5 to make the range of possible values narrower or broader. The scale is the number of standard deviations +/- of your mean (15) that is likely. It doesn't let you define a specific range, but you can always take the output and re-roll it if it falls outside of some range. This gives you a more nuanced way to get values around a certain value.
set size=None to return one value.
From https://docs.scipy.org/doc/numpy-1.15.0/reference/generated/numpy.random.normal.html
mu, sigma = 0, 0.1 # mean and standard deviation
s = np.random.normal(mu, sigma, 1000)
... (see link below for full code)
plt.show()