All the examples I've seen on pykalman documentation works on a given dataset, I was wandering how it could be used by feeding a single observation at the time while taking into account the time delta.
From the documentation:
from pykalman import KalmanFilter
import numpy as np
kf = KalmanFilter(transition_matrices = [[1, 1], [0, 1]], observation_matrices = [[0.1, 0.5], [-0.3, 0.0]])
measurements = np.asarray([[1,0], [0,0], [0,1]]) # 3 observations
kf = kf.em(measurements, n_iter=5)
(filtered_state_means, filtered_state_covariances) = kf.filter(measurements)
(smoothed_state_means, smoothed_state_covariances) = kf.smooth(measurements)
Related
I want to try to use pykalman to apply a kalman filter to data from sensor variables. Now, I have a doubt with the data of the observations. In the example, the 3 observations are two variables measured in three instants of time or are 3 variables measured in a moment of time
from pykalman import KalmanFilter
>>> import numpy as np
>>> kf = KalmanFilter(transition_matrices = [[1, 1], [0, 1]], observation_matrices = [[0.1, 0.5], [-0.3, 0.0]])
>>> measurements = np.asarray([[1,0], [0,0], [0,1]]) # 3 observations
>>> kf = kf.em(measurements, n_iter=5)
>>> (filtered_state_means, filtered_state_covariances) = kf.filter(measurements)
>>> (smoothed_state_means, smoothed_state_covariances) = kf.smooth(measurements)
Let's see:
transition_matrices = [[1, 1], [0, 1]]
means
So your state vector consists of 2 elements, for example:
observation_matrices = [[0.1, 0.5], [-0.3, 0.0]]
means
The dimension of an observation matrix should be [n_dim_obs, n_dim_state].
So your measurement vector also consists of 2 elements.
Conclusion: the code has 3 observations of two variables measured at 3 different points in time.
You can change the given code so it can process each measurement at a time step. You use kf.filter_update() for each measurement instead of kf.filter() for all measurements at once:
from pykalman import KalmanFilter
import numpy as np
kf = KalmanFilter(transition_matrices = [[1, 1], [0, 1]], observation_matrices = [[0.1, 0.5], [-0.3, 0.0]])
measurements = np.asarray([[1,0], [0,0], [0,1]]) # 3 observations
kf = kf.em(measurements, n_iter=5)
filtered_state_means = kf.initial_state_mean
filtered_state_covariances = kf.initial_state_covariance
for m in measurements:
filtered_state_means, filtered_state_covariances = (
kf.filter_update(
filtered_state_means,
filtered_state_covariances,
observation = m)
)
print(filtered_state_means);
Output:
[-1.69112511 0.30509999]
The result is slightly different as when using kf.filter() because this function does not perform prediction on the first measurement, but I think it should.
I am using the following example from :
from scipy import spatial
x, y = np.mgrid[0:5, 2:8]
tree = spatial.KDTree(list(zip(x.ravel(), y.ravel())))
pts = np.array([[0, 0], [2.1, 2.9]])
idx = tree.query(pts)[1]
data = tree.data[??????????]
If I input two arbitrary points (see variable pts), I am looking to return all pairs of coordinates that lie within the rectangle defined by the two points (KDTree finds the closest neighbour). So in this case:
array([[0, 0],
[0, 1],
[0, 2],
[1, 0],
[1, 1],
[1, 2],
[2, 0],
[2, 1],
[2, 2]])
How can I achieve that from the tree data?
Seems that I found a solution:
from scipy import spatial
import numpy as np
x, y = np.mgrid[0:5, 0:5]
tree = spatial.KDTree(list(zip(x.ravel(), y.ravel())))
pts = np.array([[0, 0], [2.1, 2.2]])
idx = tree.query(pts)[1]
data = tree.data[[idx[0], idx[1]]]
rectangle = tree.data[np.where((tree.data[:,0]>=min(data[:,0])) & (tree.data[:,0]<=max(data[:,0])) & (tree.data[:,1]>=min(data[:,1])) & (tree.data[:,1]<=max(data[:,1])))]
However, I would love to see a solution using the query option!
I have the following input data [-5, 10,2], [-2, -3,3], [-4, -9,1], [7, 11,-3], [12, 6,-1], [13, 4,5] on hand and would like to use PCA to convert from 3D array to 1D array. I typed with the following code:
import numpy as np
input = np.array([[-5, 10,2], [-2, -3,3], [-4, -9,1], [7, 11,-3], [12, 6,-1], [13, 4,5]])
mean_x = np.mean(input[0,:])
mean_y = np.mean(input[1,:])
mean_z = np.mean(input[2,:])
scaled_vector = np.array([input[0,:]-[mean_x],input[1,:]-[mean_y],input[2,:]-[mean_z]])
data=np.vstack((scaled_vector)).T
scatter_matrix=np.dot(np.transpose(data),data)
eig_val, eig_vec = np.linalg.eig(scatter_matrix)
eig_pairs = [(np.abs(eig_val[i]), eig_vec[:,i]) for i in range(len(eig_val))]
eig_pairs.sort(reverse=True)
feature=eig_pairs[0][1][2]
new_data_reduced=np.dot(data,np.transpose(feature))
print(new_data_reduced)
I also use the sklearn.decomposition import PCA to do as verification.
import numpy as np
from sklearn.decomposition import PCA
X = np.array([[-5, 10,2], [-2, -3,3], [-4, -9,1], [7, 11,-3], [12, 6,-1], [13, 4,5]])
pca = PCA(n_components=1)
pca.componrnt = True
newX = pca.fit_transform(X)
print (newX)
The results from sklearn is
[[ 1.81922968]
[ 8.34080915]
[ 13.64517202]
[ -8.17114609]
[ -8.37254693]
[ -7.26151783]]
I am not sure if this results is correct or not. However, when I use my own PCA, I find that the results are extremely different. Therefore, how can I correct it?
First, you subtract the mean along rows instead of column. Then, after computing the eigenvectors, you make several steps that are unnecessary for PCA. A reduced version of your code is:
import numpy as np
input = np.array([[-5, 10,2], [-2, -3,3], [-4, -9,1],
[7, 11,-3], [12, 6,-1], [13, 4,5]])
data = input - np.mean(input, axis=0)
scatter_matrix = np.dot(data, data.T)
eig_val, eig_vec = np.linalg.eig(scatter_matrix)
new_reduced_data = np.sqrt(eig_val[0]) * eig_vec.T[0].reshape(-1,1)
print(new_reduced_data)
which seems to give the correct result.
I was trying to get the same result fitting lasso using Python's scikit-learn and R's glmnet. A helpful link
If I specify "normalize =True" in Python and "standardize = T" in R, they gave me the same result.
Python:
from sklearn.linear_model import Lasso
X = np.array([[1, 1, 2], [3, 4, 2], [6, 5, 2], [5, 5, 3]])
y = np.array([1, 0, 0, 1])
reg = Lasso(alpha =0.01, fit_intercept = True, normalize =True)
reg.fit(X, y)
np.hstack((reg.intercept_, reg.coef_))
Out[95]: array([-0.89607695, 0. , -0.24743375, 1.03286824])
R:
reg_glmnet = glmnet(X, y, alpha = 1, lambda = 0.02,standardize = T)
coef(reg_glmnet)
4 x 1 sparse Matrix of class "dgCMatrix"
s0
(Intercept) -0.8960770
V1 .
V2 -0.2474338
V3 1.0328682
However, if I don't want to standardize variables and set normalize =False and standardize = F, they gave me quite different results.
Python:
from sklearn.linear_model import Lasso
Z = np.array([[1, 1, 2], [3, 4, 2], [6, 5, 2], [5, 5, 3]])
y = np.array([1, 0, 0, 1])
reg = Lasso(alpha =0.01, fit_intercept = True, normalize =False)
reg.fit(Z, y)
np.hstack((reg.intercept_, reg.coef_))
Out[96]: array([-0.88 , 0.09384212, -0.36159299, 1.05958478])
R:
reg_glmnet = glmnet(X, y, alpha = 1, lambda = 0.02,standardize = F)
coef(reg_glmnet)
4 x 1 sparse Matrix of class "dgCMatrix"
s0
(Intercept) -0.76000000
V1 0.04441697
V2 -0.29415542
V3 0.97623074
What's the difference between "normalize" in Python's Lasso and "standardize" in R's glmnet?
Currently, with regard to the normalize parameter the docs state "If you wish to standardize, please use StandardScaler before calling fit on an estimator with normalize=False.''
So evidently normalize and standardize are not the same with sklearn.linear_model.Lasso. Having read the StandardScaler docs I fail to understand the difference, but the fact that there is one is implied by the provided description of the normalize parameter.
When I study Python SKlearn, the first example that I come across is Generalized Linear Models.
Code of its very first example:
from sklearn import linear_model
reg = linear_model.LinearRegression()
reg.fit([[0, 0], [1, 1], [2,2]], [0, 1,2])
reg.fit
reg.coef_
array([ 0.5, 0.5])
Here I assume [[0, 0], [1, 1], [2,2]] represents a data.frame containing x1 = c(0,1,2) and x2 = c(0,1,2) and y = c(0,1,2) as well.
Immediately, I begin to think that array([ 0.5, 0.5]) are the coeffs for x1 and x2.
But, are there standard errors for those estimates? What about t tests p values, R2 and other figures?
Then I try to do the same thing in R.
X = data.frame(x1 = c(0,1,2),x2 = c(0,1,2),y = c(0,1,2))
lm(data=X, y~x1+x2)
Call:
lm(formula = y ~ x1 + x2, data = X)
#Coefficients:
#(Intercept) x1 x2
# 1.282e-16 1.000e+00 NA
Obviously x1 and x2 are completely linearly dependent so the OLS will fail. Why the SKlearn still works and gives this results? Am I getting sklearn in a wrong way? Thanks.
Both solutions are correct (assuming that NA behaves like a zero). Which solution is favored depends on the numerical solver used by the OLS estimator.
sklearn.linear_model.LinearRegression is based on scipy.linalg.lstsq which in turn calls the LAPACK gelsd routine which is described here:
http://www.netlib.org/lapack/lug/node27.html
In particular it says that when the problem is rank deficient it seeks the minimum norm least squares solution.
If you want to favor the other solution, you can use a coordinate descent solver with a tiny bit of L1 penalty as implemented in th Lasso class:
>>> from sklearn.linear_model import Lasso
>>> reg = Lasso(alpha=1e-8)
>>> reg.fit([[0, 0], [1, 1], [2, 2]], [0, 1, 2])
Lasso(alpha=1e-08, copy_X=True, fit_intercept=True, max_iter=1000,
normalize=False, positive=False, precompute=False, random_state=None,
selection='cyclic', tol=0.0001, warm_start=False)
>>> reg.coef_
array([ 9.99999985e-01, 3.97204719e-17])