As a learning exercise for myself, I am trying to estimate the regression parameters using the MLE method in Python.
From what I have gathered, the log-likelihood function boils down to maximizing the following:
So I need to take partial derivatives with respect to the intercept and the slope, setting each to zero, and this should give me the coefficients.
I have been trying to approach this using sympy as follows:
from sympy import *
b = Symbol('b')# beta1
a = Symbol('a')# intercept
x = Symbol('x', integer=True)
y = Symbol('y', integer=True)
i = symbols('i', cls=Idx)
x_values = [2,3,2]
y_values = [1,2,3]
n = len(x_values)-1
function = summation((Indexed('y',i) - a+b*Indexed('x',i))**2, (i, 0, n))
partial_intercept = function.diff(a)
print(partial_intercept)
# 6*a - 2*b*x[0] - 2*b*x[1] - 2*b*x[2] - 2*y[0] - 2*y[1] - 2*y[2]
intercept_f = lambdify([x, y], partial_intercept)
inter = solve(intercept_f(x_values, y_values), a)
print(inter)
# [7*b/3 + 2]
I would have expected a single value for the slope, such that the 'b' variable is gone. However, I see that this wouldn't be possible given that the variable b is still there in my derivative equation.
Does anyone have any advice on where I am going wrong?
Thanks!
Edit : Fixed a typo in the codeblock
The expression 7*b/2 + 2 at the end tell you that we have to satisfies a = 7*b/2 + 2, it depends on the quantity of b.
You should solve for both a and b as a system simultaneously.
In the following code, I find the relationship that a and b has to satisfies and solve them simultaneously.
from sympy import *
b = Symbol('b')# beta1
a = Symbol('a')# intercept
x = Symbol('x', integer=True)
y = Symbol('y', integer=True)
i = symbols('i', cls=Idx)
x_values = [2,3,2]
y_values = [1,2,3]
n = len(x_values)-1
function = summation((Indexed('y',i) - a+b*Indexed('x',i))**2, (i, 0, n))
partial_intercept = function.diff(a)
print(partial_intercept)
# 6*a - 2*b*x[0] - 2*b*x[1] - 2*b*x[2] - 2*y[0] - 2*y[1] - 2*y[2]
intercept_f = lambdify([x, y], partial_intercept)
inter = solve(intercept_f(x_values, y_values), a)
print(inter)
#[7*b/3 + 2]
partial_gradient = function.diff(b)
print(partial_gradient)
# 6*a - 2*b*x[0] - 2*b*x[1] - 2*b*x[2] - 2*y[0] - 2*y[1] - 2*y[2]
intercept_f = lambdify([x, y], partial_gradient)
inter2 = solve(intercept_f(x_values, y_values), b)
print(inter2)
ans = solve([a-inter[0], b-inter2[0]])
print(ans)
Here are the outputs:
6*a - 2*b*x[0] - 2*b*x[1] - 2*b*x[2] - 2*y[0] - 2*y[1] - 2*y[2]
[7*b/3 + 2]
2*(-a + b*x[0] + y[0])*x[0] + 2*(-a + b*x[1] + y[1])*x[1] + 2*(-a + b*x[2] + y[2])*x[2]
[7*a/17 - 14/17]
{a: 2, b: 0}
a should be set to be 2 and b should be set to be 0.
Related
This is more of a computational physics problem, and I've asked it on physics stack exchange, but no answers on there. This is, I suppose, a mix of the disciplines on here and there (and maybe even mathematics stack exchange), so finding the right place to post is a task in of itself apparently...
I'm attempting to use Crank-Nicolson scheme to solve the TDSE in 1D. The initial wave is a real Gaussian that has been normalised wrt its probability density. As the solution evolves, a depression grows in the central peak of the real part of the wave, and the imaginary part's central trough is perhaps a bit higher than I expect (image below).
Does this behaviour seem reasonable? I have searched around and not seen questions/figures that are similar. I've tested another person's code from Github and it exhibits the same behaviour, which makes me feel a bit better. But I still think the center peak should just decrease in height and increase in width. The likelihood of me getting a physics-based explanation is relatively low here I'd assume, but a computational-based explanation on errors I may have made is more likely.
I'm happy to give more information, for example my code, or the matrices used in the scheme, etc. Thanks in advance!
Here's a link to GIF of time evolution:
And the part of my code relevant to solving the 1D TDSE:
(pretty much the entire thing except the plotting)
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.animation import FuncAnimation
# Define function for norm.
def normf(dxc, uc, ic):
return sum(dxc * np.square(np.abs(uc[ic, :])))
# Define function for expectation value of position.
def xexpf(dxc, xc, uc, ic):
return sum(dxc * xc * np.square(np.abs(uc[ic, :])))
# Define function for expectation value of squared position.
def xexpsf(dxc, xc, uc, ic):
return sum(dxc * np.square(xc) * np.square(np.abs(uc[ic, :])))
# Define function for standard deviation.
def sdaf(xexpc, xexpsc, ic):
return np.sqrt(xexpsc[ic] - np.square(xexpc[ic]))
# Time t: t0 =< t =< tf. Have N steps at which to evaluate the CN scheme. The
# time interval is dt. decp: variable for plotting to certain number of decimal
# places.
t0 = 0
tf = 20
N = 200
dt = tf / N
t = np.linspace(t0, tf, num = N + 1, endpoint = True)
decp = str(dt)[::-1].find('.')
# Initialise array for filling with norm values at each time step.
norm = np.zeros(len(t))
# Initialise array for expectation value of position.
xexp = np.zeros(len(t))
# Initialise array for expectation value of squared position.
xexps = np.zeros(len(t))
# Initialise array for alternate standard deviation.
sda = np.zeros(len(t))
# Position x: -a =< x =< a. M is an even number. There are M + 1 total discrete
# positions, for the points to be symmetric and centred at x = 0.
a = 100
M = 1200
dx = (2 * a) / M
x = np.linspace(-a, a, num = M + 1, endpoint = True)
# The gaussian function u diffuses over time. sd sets the width of gaussian. u0
# is the initial gaussian at t0.
sd = 1
var = np.power(sd, 2)
mu = 0
u0 = np.sqrt(1 / np.sqrt(np.pi * var)) * np.exp(-np.power(x - mu, 2) / (2 * \
var))
u = np.zeros([len(t), len(x)], dtype = 'complex_')
u[0, :] = u0
# Normalise u.
u[0, :] = u[0, :] / np.sqrt(normf(dx, u, 0))
# Set coefficients of CN scheme.
alpha = dt * -1j / (4 * np.power(dx, 2))
beta = dt * 1j / (4 * np.power(dx, 2))
# Tridiagonal matrices Al and AR. Al to be solved using Thomas algorithm.
Al = np.zeros([len(x), len(x)], dtype = 'complex_')
for i in range (0, M):
Al[i + 1, i] = alpha
Al[i, i] = 1 - (2 * alpha)
Al[i, i + 1] = alpha
# Corner elements for BC's.
Al[M, M], Al[0, 0] = 1 - alpha, 1 - alpha
Ar = np.zeros([len(x), len(x)], dtype = 'complex_')
for i in range (0, M):
Ar[i + 1, i] = beta
Ar[i, i] = 1 - (2 * beta)
Ar[i, i + 1] = beta
# Corner elements for BC's.
Ar[M, M], Ar[0, 0] = 1 - 2*beta, 1 - beta
# Thomas algorithm variables. Following similar naming as in Wiki article.
a = np.diag(Al, -1)
b = np.diag(Al)
c = np.diag(Al, 1)
NT = len(b)
cp = np.zeros(NT - 1, dtype = 'complex_')
for n in range(0, NT - 1):
if n == 0:
cp[n] = c[n] / b[n]
else:
cp[n] = c[n] / (b[n] - (a[n - 1] * cp[n - 1]))
d = np.zeros(NT, dtype = 'complex_')
dp = np.zeros(NT, dtype = 'complex_')
# Iterate over each time step to solve CN method. Maintain boundary
# conditions. Keep track of standard deviation.
for i in range(0, N):
# BC's.
u[i, 0], u[i, M] = 0, 0
# Find RHS.
d = np.dot(Ar, u[i, :])
for n in range(0, NT):
if n == 0:
dp[n] = d[n] / b[n]
else:
dp[n] = (d[n] - (a[n - 1] * dp[n - 1])) / (b[n] - (a[n - 1] * \
cp[n - 1]))
nc = NT - 1
while nc > -1:
if nc == NT - 1:
u[i + 1, nc] = dp[nc]
nc -= 1
else:
u[i + 1, nc] = dp[nc] - (cp[nc] * u[i + 1, nc + 1])
nc -= 1
norm[i] = normf(dx, u, i)
xexp[i] = xexpf(dx, x, u, i)
xexps[i] = xexpsf(dx, x, u, i)
sda[i] = sdaf(xexp, xexps, i)
# Fill in final norm value.
norm[N] = normf(dx, u, N)
# Fill in final position expectation value.
xexp[N] = xexpf(dx, x, u, N)
# Fill in final squared position expectation value.
xexps[N] = xexpsf(dx, x, u, N)
# Fill in final standard deviation value.
sda[N] = sdaf(xexp, xexps, N)
I am trying my hand on interpolating data using a simple linear function, Lagrange Interpolating Polynomial. I have managed to get the required equations, however, I am not able to figure out how to plot it piece-wise. I do understand using sympy is not the best way forward, but I am a noob and I wanted to see how my equations look.
How can I make it plot in matplotlib without having the need to manually type the equations at the end?
import numpy as np
import matplotlib.pyplot as plt
import sympy as sym
x = sym.Symbol('x')
year = np.arange(1960,2020,10)
pop = [179323,203302,226542,249633,281422,308746]
def lgn(a,b): #this ideally should be taking a value of x where you'd like to interpolate.
result = []
for i in range(1,len(a)):
L0 = (x - a[i])/(a[i-1] - a[i])
L1 = (x - a[i-1])/(a[i] - a[i-1])
temp = (L0 * b[i-1]) + (L1 * b[i])
result.append(temp)
return result
lgn(year,pop) #result gives a list of linear equations between each year value.
[23979*x/10 - 4520561,
2324*x - 4374978,
23091*x/10 - 4345476,
31789*x/10 - 6076378,
13662*x/5 - 5183378]
#plotting for each interval. this is what I am trying to code.
x1 = np.linspace(year[0],year[1],10)
y1 = 23979 * x1/10 - 4520561
x2 = np.linspace(year[1],year[2],10)
y2 = 2324*x2 - 4374978
x3 = np.linspace(year[2],year[3],10)
y3 = 23091*x3/10 - 4345476
x4 = np.linspace(year[3],year[4],10)
y4 = 31789*x4/10 - 6076378
x5 = np.linspace(year[4],year[5],10)
y5 = 13662*x5/5 - 5183378
plt.plot(year,pop,'ro',x1,y1,x2,y2,x3,y3,x4,y4,x5,y5 )
To convert a sympy expression to a numerical value with expr.subs(x, 123).evalf(). Note that this only works for substituting a single value. To work with arrays, sym.lambdify() can convert the expression to a function that understands numpy arrays, which then can be used for plotting.
Here is some example code:
import numpy as np
import matplotlib.pyplot as plt
import sympy as sym
def lgn(a, b):
result = []
for i in range(1, len(a)):
L0 = (x - a[i]) / (a[i - 1] - a[i])
L1 = (x - a[i - 1]) / (a[i] - a[i - 1])
temp = (L0 * b[i - 1]) + (L1 * b[i])
result.append(temp)
return result
x = sym.Symbol('x')
year = np.arange(1960, 2020, 10)
pop = [179323, 203302, 226542, 249633, 281422, 308746]
equations = lgn(year, pop)
for i in range(1, len(year)):
xi = np.linspace(year[i - 1], year[i], 10)
yi_np = sym.lambdify(x, equations[i - 1])
yi = yi_np(xi)
plt.plot(xi, yi)
plt.plot(year, pop, 'ro')
plt.show()
Here is an approach using sympy's Piecewise:
import numpy as np
import matplotlib.pyplot as plt
import sympy as sym
def line_2points(a0, b0, a1, b1):
L0 = (x - a1) / (a0 - a1)
L1 = (x - a0) / (a1 - a0)
return L0 * b0 + L1 * b1
x = sym.Symbol('x')
year = np.arange(1960, 2020, 10)
pop = [179323, 203302, 226542, 249633, 281422, 308746]
eq = 0
for i in range(1, len(year)):
eq = sym.Piecewise( (line_2points(year[i-1], pop[i-1], year[i], pop[i]), (x >= year[i-1] ) & (x <= year[i] ) ),
(eq, True) )
# sym.plot(eq, (x, year[0], year[-1])) # this also works, but the visualization is much harder to customize
eq_np = sym.lambdify(x, eq)
xs = np.linspace(year[0], year[-1], 200)
plt.plot(xs, eq_np(xs))
plt.plot(year, pop, 'ro')
plt.show()
I'm trying to solve the following coupled equations:
x = 1;
y - 0.5*y - 0.7*v = 0;
w - 0.7*x - 0.5*x = 0;
v = 1.
(I know that the equations = 1 seem unnecessary but I need them for a later generalization of the code).
My code is the following:
import numpy as np
from scipy.optimize import fsolve
def myFunction(z):
x = z[0]
y = z[1]
w = z[2]
v = z[3]
F = np.empty((4))
F[0] = 1
F[1] = y - 0.5*y - 0.7*v
F[2] = w - 0.7*x - 0.5*w
F[3] = 1
return F
zGuess = np.array([1,2.5,2.5,1])
z = fsolve(myFunction,zGuess)
print(z)
The answer I get is [-224.57569869, -314.40597772, -314.40597817, -224.57569837], but I would expect [1, 1.4, 1.4, 1]. Why does fsolve is unable to find the answer to this simple set of equations? What's more: why the values of x and v, which are not modified at any point, are not the same as those of the initial guess?
To convert the requirement x = 1 to code, rewrite it as x - 1 = 0. That says that the line F[0] = 1 should be changed to F[0] = x - 1. Similarly, the line F[3] = 1 should be F[3] = v - 1.
I would like to compute the integral of discrete signal segment Y (Y=[y1,y2,...y50]) and X is the following datetime64[ns] object:
x=['2018-01-24T13:41:25.057000000' '2018-01-24T13:41:25.069000000'
'2018-01-24T13:41:25.077000000' '2018-01-24T13:41:25.090000000'
'2018-01-24T13:41:25.097000000' '2018-01-24T13:41:25.111000000'
'2018-01-24T13:41:25.117000000' '2018-01-24T13:41:25.130000000'
'2018-01-24T13:41:25.138000000' '2018-01-24T13:41:25.150000000'
'2018-01-24T13:41:25.158000000' '2018-01-24T13:41:25.170000000'
'2018-01-24T13:41:25.178000000' '2018-01-24T13:41:25.199000000'
'2018-01-24T13:41:25.200000000' '2018-01-24T13:41:25.211000000'
'2018-01-24T13:41:25.218000000' '2018-01-24T13:41:25.231000000'
'2018-01-24T13:41:25.238000000' '2018-01-24T13:41:25.250000000'
'2018-01-24T13:41:25.258000000' '2018-01-24T13:41:25.269000000'
'2018-01-24T13:41:25.278000000' '2018-01-24T13:41:25.290000000'
'2018-01-24T13:41:25.298000000' '2018-01-24T13:41:25.311000000'
'2018-01-24T13:41:25.317000000' '2018-01-24T13:41:25.331000000'
'2018-01-24T13:41:25.338000000' '2018-01-24T13:41:25.350000000'
'2018-01-24T13:41:25.358000000' '2018-01-24T13:41:25.370000000'
'2018-01-24T13:41:25.378000000' '2018-01-24T13:41:25.390000000'
'2018-01-24T13:41:25.398000000' '2018-01-24T13:41:25.411000000'
'2018-01-24T13:41:25.418000000' '2018-01-24T13:41:25.430000000'
'2018-01-24T13:41:25.437000000' '2018-01-24T13:41:25.450000000'
'2018-01-24T13:41:25.469000000' '2018-01-24T13:41:25.469000000'
'2018-01-24T13:41:25.479000000' '2018-01-24T13:41:25.493000000'
'2018-01-24T13:41:25.502000000' '2018-01-24T13:41:25.510000000'
'2018-01-24T13:41:25.517000000' '2018-01-24T13:41:25.530000000'
'2018-01-24T13:41:25.537000000' '2018-01-24T13:41:25.550000000']
My attempt is using the following python code:
Case A, with the above x:
f_x_tot_acc = Y
x_n = x[-1]
x_0 = x[0]
h = (x_n - x_0) / (len(f_x_tot_acc) - 1)
print('trapz [' + str(integrate.trapz(f_x_tot_acc, dx=h)) + ']')
The Result is: trapz [6203255447 nanoseconds]
Case B,
x = range(0,50)
f_x_tot_acc = Y
x_n = x[-1]
x_0 = x[0]
h = (x_n - x_0) / (len(f_x_tot_acc) - 1)
print('trapz [' + str(integrate.trapz(f_x_tot_acc, dx=h)) + ']')
The Result is: trapz [616.5507767408332].
Which is the correct one?
Many Thanks in advance,
Best Regards,
Carlo
--- using python 3 ---
Following the equations here, I tried to find all real roots of an arbitrary third-order-polynomial. Unfortunatelly, my implementation does not yield the correct result and I cannot find the error. Maybe you are able to spot it within a blink of an eye and tell me.
(As you notice, only the roots of the green curve are wrong.)
With best regards
import numpy as np
def find_cubic_roots(a,b,c,d):
# with ax³ + bx² + cx + d = 0
a,b,c,d = a+0j, b+0j, c+0j, d+0j
all_ = (a != np.pi)
Q = (3*a*c - b**2)/ (9*a**2)
R = (9*a*b*c - 27*a**2*d - 2*b**3) / (54 * a**3)
D = Q**3 + R**2
S = (R + np.sqrt(D))**(1/3)
T = (R - np.sqrt(D))**(1/3)
result = np.zeros(tuple(list(a.shape) + [3])) + 0j
result[all_,0] = - b / (3*a) + (S+T)
result[all_,1] = - b / (3*a) - (S+T) / 2 + 0.5j * np.sqrt(3) * (S - T)
result[all_,2] = - b / (3*a) - (S+T) / 2 - 0.5j * np.sqrt(3) * (S - T)
return result
The example where you see it does not work:
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
a = np.array([2.5])
b = np.array([-5])
c = np.array([0])
x = np.linspace(-2,3,100)
for i, d in enumerate([-8,0,8]):
d = np.array(d)
roots = find_cubic_roots(a,b,c,d)
ax.plot(x, a*x**3 + b*x**2 + c*x + d, label = "a = %.3f, b = %.3f, c = %.3f, d = %.3f"%(a,b,c,d), color = colors[i])
print(roots)
ax.plot(x, x*0)
ax.scatter(roots,roots*0, s = 80)
ax.legend(loc = 0)
ax.set_xlim(-2,3)
plt.show()
Output:
[[ 2.50852567+0.j -0.25426283+1.1004545j -0.25426283-1.1004545j]]
[[ 2.+0.j 0.+0.j 0.-0.j]]
[[ 1.51400399+1.46763129j 1.02750817-1.1867528j -0.54151216-0.28087849j]]
Here is my stab at the solution. Your code fails for the case where R + np.sqrt(D) or R - np.sqrt(D) is negative. The reason is in this post. Basically if you do a**(1/3) where a is negative, numpy returns a complex number. However, we infact, want S and T to be real since cube root of a negative real number is simply a negative real number (let's ignore De Moivre's theorem for now and focus on the code and not the math). The way to work around it is to check if S is real, cast it to real and pass S to the function from scipy.special import cbrt. Similarly for T.
Example code:
import numpy as np
import pdb
import math
from scipy.special import cbrt
def find_cubic_roots(a,b,c,d, bp = False):
a,b,c,d = a+0j, b+0j, c+0j, d+0j
all_ = (a != np.pi)
Q = (3*a*c - b**2)/ (9*a**2)
R = (9*a*b*c - 27*a**2*d - 2*b**3) / (54 * a**3)
D = Q**3 + R**2
S = 0 #NEW CALCULATION FOR S STARTS HERE
if np.isreal(R + np.sqrt(D)):
S = cbrt(np.real(R + np.sqrt(D)))
else:
S = (R + np.sqrt(D))**(1/3)
T = 0 #NEW CALCULATION FOR T STARTS HERE
if np.isreal(R - np.sqrt(D)):
T = cbrt(np.real(R - np.sqrt(D)))
else:
T = (R - np.sqrt(D))**(1/3)
result = np.zeros(tuple(list(a.shape) + [3])) + 0j
result[all_,0] = - b / (3*a) + (S+T)
result[all_,1] = - b / (3*a) - (S+T) / 2 + 0.5j * np.sqrt(3) * (S - T)
result[all_,2] = - b / (3*a) - (S+T) / 2 - 0.5j * np.sqrt(3) * (S - T)
#if bp:
#pdb.set_trace()
return result
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
a = np.array([2.5])
b = np.array([-5])
c = np.array([0])
x = np.linspace(-2,3,100)
for i, d in enumerate([-8,0,8]):
d = np.array(d)
if d == 8:
roots = find_cubic_roots(a,b,c,d, True)
else:
roots = find_cubic_roots(a,b,c,d)
ax.plot(x, a*x**3 + b*x**2 + c*x + d, label = "a = %.3f, b = %.3f, c = %.3f, d = %.3f"%(a,b,c,d))
print(roots)
ax.plot(x, x*0)
ax.scatter(roots,roots*0, s = 80)
ax.legend(loc = 0)
ax.set_xlim(-2,3)
plt.show()
DISCLAIMER: The output root gives some warning, which you can probably ignore. The output is correct. However, the plotting shows an extra root for some reasons. This is likely due to your plotting code. The printed roots look fine though.