We Keep Coding

Principle Component Analysis, add a line to the 3d graph showing the first principal component

I am conducting PCA on a dataset. I am attempting to add a line in my 3d graph which shows the first principal component. I have tried a few methods but have not been able to display the first principal component as a line in my 3d graph. Any help is greatly appreciated. My code is as follows:
import numpy as np
np.set_printoptions (suppress=True, precision=5, linewidth=150)
import pandas as pd
from sklearn.decomposition import PCA
from sklearn.preprocessing import LabelEncoder
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
file_name = 'C:/Users/data'
input_data = pd.read_csv (file_name + '.csv', header=0, index_col=0)
A = input_data.A.values.astype(float)
B = input_data.B.values.astype(float)
C = input_data.C.values.astype(float)
D = input_data.D.values.astype(float)
E = input_data.E.values.astype(float)
F = input_data.F.values.astype(float)
X = np.column_stack((A, B, C, D, E, F))
ncompo = int (input ("Number of components to study: "))
print("")
pca = PCA (n_components = ncompo)
pcafit = pca.fit(X)
cov_mat = np.cov(X, rowvar=0)
eig_vals, eig_vecs = np.linalg.eig(cov_mat)
perc = pcafit.explained_variance_ratio_
perc_x = range(1, len(perc)+1)
plt.plot(perc_x, perc)
plt.xlabel('Components')
plt.ylabel('Percentage of Variance Explained')
plt.show()
#3d Graph
plt.clf()
le = LabelEncoder()
le.fit(input_data.Grade)
number = le.transform(input_data.Grade)
colormap = np.array(['green', 'blue', 'red', 'yellow'])
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.scatter(D, E, F, c=colormap[number])
ax.set_xlabel('D')
ax.set_ylabel('E')
ax.set_zlabel('F')
plt.title('PCA')
plt.show()

Some remarks to begin with:
You are computing PCA twice! To compute PCA is to compute eigen values and eigen vectors of the covariance matrix. So, either you use the sklearn function pca.fit, either you do it yourself. But you don't need to do both, unless you want to discover pca.fit and see for yourself that it does exactly what you expect it to do (if this is what you wanted, fine. It is a good thing to do that king of checking. I did this once also). Of course pca.fit has another advantage: once you have it, it also provides pca.predict to project points in the components space. But that also is simply a base change using eigenvectors matrix (that is matrix to change base)
pca object let you get the eigenvectors (pca.components_) and eigen values (pca.explained_variance_)
pca.fit is a 'inplace' method. It does not return a new PCA object. It just fit the one you have. So, no need to get pcafit and use it.
This is not a minimal reproducible exemple as required on SO. We should be able to copy and paste it, and run it, so see exactly your problem. Not to guess what kind of secret data you have. And in the meantime, it should be minimal. So, contains data example generation (it doesn't matter if those data doesn't make sense. Sometimes it is even better, since it allows some testing. In my following code, I generate my own noisy data along an axis, which allow me to verify that, indeed, I am able to "guess" what was that axis). Plus, since your problem concerns only 3d plot, there is no need to include ploting of explained variance here. That part is not part of your question.
Now, to print the principal component, well, you already did the hard part. Twice. That is to compute it. It is the eigenvector associated with the highest eigenvalue.
With pca object no need to search for it, they are already sorted. So it is simply pca.components_[0]. And since you want to plot in the space D,E,F, you simply need to draw vector pca.components_[0][3:].
With correct scaling.
You can do that with plot providing just 2 points (first and last)
Here is my version (which, by the way, shows also what a minimal reproducible example is)
import numpy as np
np.set_printoptions (suppress=True, precision=5, linewidth=150)
import pandas as pd
from sklearn.decomposition import PCA
from sklearn.preprocessing import LabelEncoder
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
# Generation of random data along a given vector
vec=np.array([1, -1, 0.5, -0.5, 0.75, 0.75]).reshape(-1,1)
# 10000 random data, that are U[0,10]×vec + gaussian noise std=1
X=(vec*np.random.rand(10000)*10 + np.random.normal(0,1,(6,10000))).T
(A,B,C,D,E,F)=X.T
input_data = pd.DataFrame({'A':A,'B':B,'C':C,'D':D,'E':E, 'F':F, 'Grade':np.random.randint(1,5, (10000,))})
ncompo=6
pca = PCA (n_components = ncompo)
pca.fit(X)
# Redundant
cov_mat = np.cov(X, rowvar=0)
eig_vals, eig_vecs = np.linalg.eig(cov_mat)
# See
print("Eigen values")
print(eig_vals)
print(pca.explained_variance_)
print("Eigen vec")
print(eig_vecs)
print(pca.components_)
# Note, compare first components to
print("Main component")
print(vec/np.linalg.norm(vec))
print(pca.components_[0])
#3d Graph
le = LabelEncoder()
le.fit(input_data.Grade)
number = le.transform(input_data.Grade)
fig = plt.figure()
colormap = np.array(['green', 'blue', 'red', 'yellow'])
ax = fig.add_subplot(111, projection='3d')
ax.scatter(D, E, F, c=colormap[number])
U=pca.components_[0]
sc1=max(D)/U[3]
sc2=min(D)/U[3]
# Draw the 1st principal component as a blue line
ax.plot([sc1*U[3],sc2*U[3]], [sc1*U[4], sc2*U[4]], [sc1*U[5], sc2*U[5]], linewidth=3)
ax.set_xlabel('D')
ax.set_ylabel('E')
ax.set_zlabel('F')
plt.title('PCA')
plt.show()
My example is not that minimal, because I took advantage of it to illustrate my first remark, and also computed PCA twice, to compare both result.
So, here I print, eigenvalues
Eigen values
[30.88941 1.01334 0.99512 0.96493 0.97692 0.98101]
[30.88941 1.01334 0.99512 0.98101 0.97692 0.96493]
(1st being your computation by diagonalisation of covariance matrix, 2nd pca.explained_variance_)
As you can see, they are the same, except sorting for the 1st one
Like wise,
Eigen vec
[[-0.52251 -0.27292 0.40863 -0.06321 0.26699 0.6405 ]
[ 0.52521 0.07577 -0.34211 0.27583 -0.04161 0.72357]
[-0.26266 -0.41332 -0.60091 0.38027 0.47573 -0.16779]
[ 0.26354 -0.52548 0.47284 0.59159 -0.24029 -0.15204]
[-0.39493 0.63946 0.07496 0.64966 -0.08619 0.00252]
[-0.3959 -0.25276 -0.35452 -0.0572 -0.79718 0.12217]]
[[ 0.52251 -0.52521 0.26266 -0.26354 0.39493 0.3959 ]
[-0.27292 0.07577 -0.41332 -0.52548 0.63946 -0.25276]
[-0.40863 0.34211 0.60091 -0.47284 -0.07496 0.35452]
[-0.6405 -0.72357 0.16779 0.15204 -0.00252 -0.12217]
[-0.26699 0.04161 -0.47573 0.24029 0.08619 0.79718]
[-0.06321 0.27583 0.38027 0.59159 0.64966 -0.0572 ]]
Also the same, but for sorting and transpose.
Eigen vectors are presented column wise when you diagonalize a matrix.
Where as for pca.components_ each line is an eigen vector.
But you can see that in the 1st matrix, the eigen vector associated to the biggest eigen value, that is, since biggest eigen value was the 1st one, the 1st column (-0.52, 0.52, etc.)
is also the same as the first line of pca.components_.
Like wise, the 4th biggest eigen value in your diagonalisation was the last one.
And if you look at the last column of your eigen vectors (0.64, 0.72, -0.76...), it is the same as the 4th line of pca.components_ (with a irrelevant ×-1 factor)
So, long story short, you already have eigenvals in pca.explained_variance_ sorted from the biggest to the smallest. And eigen vectors in pca_components_, in the same order.
Last thing I print here, is comparison between the first component (pca.components_[0]) and the vector I used to generate the data in the first place (my data are all colinear to a vector vec, + a gaussian noise).
Main component
[[ 0.52523]
[-0.52523]
[ 0.26261]
[-0.26261]
[ 0.39392]
[ 0.39392]]
[ 0.52251 -0.52521 0.26266 -0.26354 0.39493 0.3959 ]
As expected, PCA did find correctly that main axis.
So, that was just side comments.
What is really what you were looking for is
ax.plot([sc1*U[3],sc2*U[3]], [sc1*U[4], sc2*U[4]], [sc1*U[5], sc2*U[5]], linewidth=3)
sc1 and sc2 being just scaling factors (here I choose it so that it scales approx like the data. Another way would have been to set ax.set_xlim, ax.set_ylim, ax.set_zlim from D.min(), D.max(), E.min(), E.max(), etc.
And then just use big values for sc1 and sc2, like
sc1=1000
sc2=-1000

Understanding OpenCV homography at a minimum of points

I am quite intrigued by the idea of a homography and try to get it to work at a minimal example with python and OpenCV. Yet, my tests do not pass and I am not quite sure why. I pass in a set of corresponding points into the findHomography function according to This
and then multiply the homography matrix to receive my new point.
so the idea behind it is to find the planar coordinate transformation and then transform the points with
X' = H#X
where X' are the new coordinates and X are the coordinates in the new coordinate frame.
Here is some minimal code example:
import cv2
import numpy as np
import matplotlib.pyplot as plt
points = np.array([
[675, 585],
[675, 1722],
[3155, 580],
[3162, 1722],
])
t_points = np.array([
[0,0],
[0, 8.23],
[23.77, 0],
[23.77, 8.23]
])
pt = np.array([675, 580+(1722-580)/2, 0])
pt_test = np.array([0,8.23/2, 0])
def get_h_matrix(src_list, dst_list):
src_pts = np.array(src_list).reshape(-1,1,2)
dst_pts = np.array(dst_list).reshape(-1,1,2)
H, mask = cv2.findHomography(src_pts, dst_pts)
return H
H = get_h_matrix(points, t_points)
transformed = H#pt
plt.scatter(t_points[:,0], t_points[:,1], color = 'blue')
plt.scatter(transformed[0], transformed[1], color = 'orange')
plt.scatter(pt_test[0], pt_test[1], color = 'green')
plt.show()
plt.scatter(points[:,0], points[:,1], color = 'blue')
plt.scatter(pt[0],pt[1], color = 'orange')
plt.show()
where the output corresponds to the following plot
Plot of the coordinate Transformation. We can see that the green point, where the transformed point actually should be, is not even close to the orange point, where the homography transformed the point to.
Maybe somebody can see the error in my train of thoughts.
Your help is kindly appreciated.
EDIT: I swaped the points array a few times, because I thought I made a mistake, but still the wrong transformation.

As Micka mentioned in the comments, the problem is the representation of the test points.
pt = [x,y,1]
instead of
pt = [x,y,0]
after the transformation, the homogeneous coordinates get transformed back by
pt' = pt'/pt'[2]
I appreciate the help.

relative rotation in 3D

I have an object (XYZ-coordinate system where Z is up) that rotates from t0 to t1 with the corresponding rotation matrices:
import numpy as np
from scipy.spatial.transform import Rotation as R
r_0 = np.array([[-0.02659679, -0.00281247, 0.99964229],
[ 0.76308514, -0.64603356, 0.01848528],
[ 0.64575048, 0.76330382, 0.01932857]])
r_1 = np.array([[ 0.05114056, -0.03815443, 0.99796237],
[-0.30594799, 0.95062582, 0.05202294],
[-0.95067369, -0.30798506, 0.03694226]])
# Calculate the relative rotation matrix from t0 to t1
rot_mat_rel = np.matmul(np.transpose(r_0), r_1)
r = R.from_maxtrix(rot_mat_rel)
# Obtain angles
print(r.as_euler('xyz', degrees=True)
# Result
array([ -1.52028392, -1.55242217, -148.10677483])
The problem is, that the relative angles look wrong to me but I can't find my mistake. What I wanted to know is how much the object rotated along x, y and z.
Edit: Code to for plots: https://codeshare.io/GA9zK8

You can use matrix_from_euler_xyz from this tutorial to check your results.
(You might need to run pip3 install pytransform3d in your terminal where you are running your python code from, or !pip3 install pytransform3d from Jupyter Notebook if you are using that.)
Preparing the data:
import numpy as np
from scipy.spatial.transform import Rotation as R
r_0 = np.array([[-0.02659679, -0.00281247, 0.99964229],
[ 0.76308514, -0.64603356, 0.01848528],
[ 0.64575048, 0.76330382, 0.01932857]])
r_1 = np.array([[ 0.05114056, -0.03815443, 0.99796237],
[-0.30594799, 0.95062582, 0.05202294],
[-0.95067369, -0.30798506, 0.03694226]])
# Calculate the relative rotation matrix from t0 to t1
rot_mat_rel = np.matmul(np.transpose(r_0), r_1)
r = R.from_matrix(rot_mat_rel)
Let's plot what the rotation r means in practice:
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from pytransform3d.rotations import *
ax = plot_basis(R=np.eye(3), ax_s=1)
p = np.array([0, 0, 0])
R = matrix_from_euler_xyz(r.as_euler('xyz'))
plot_basis(ax, R, p, alpha = 0.5)
plt.show()
We obtain this plot:
You can check if this is what you expected or not.
Check the rotation matrix which the pytransform3d module calculated from Euler angles r:
matrix_from_euler_xyz(r.as_euler('xyz'))
Giving ouput:
array([[-0.84872253, -0.52814402, 0.02709157],
[ 0.52754172, -0.84911505, -0.02652111],
[ 0.03701082, -0.00821713, 0.99928108]])
which is exactly the traspose of np.matmul(np.transpose(r_0), r_1):
array([[-0.84872253, 0.52754172, 0.03701082],
[-0.52814402, -0.84911505, -0.00821714],
[ 0.02709157, -0.02652111, 0.99928109]])
Which seems a good sign & may be a good starting point for checking your math.
As I don't see what you would expect to get, I suggest you experiment with plotting your results with the tools outlined here, and check step by step that what you have is what you have expected to have.

I'm probably a bit late and zabop's answer already points to the right direction. I just want to clarify two things.
There are several ambiguities when we work with transformations that can make things more confusing. The two things that might make the code here a bit confusing are:
active vs. passive rotation
intrinsic vs. extrinsic rotation
I'm starting from your example above:
import numpy as np
r_0 = np.array([[-0.02659679, -0.00281247, 0.99964229],
[ 0.76308514, -0.64603356, 0.01848528],
[ 0.64575048, 0.76330382, 0.01932857]])
r_1 = np.array([[ 0.05114056, -0.03815443, 0.99796237],
[-0.30594799, 0.95062582, 0.05202294],
[-0.95067369, -0.30798506, 0.03694226]])
The way I would calculate a rotation matrix that rotates r_0 to r_1 is the following (different from your code!):
r0_to_r1 = r_1.dot(r_0.T)
r0_to_r1
Result:
array([[ 0.99635252, 0.08212126, 0.0231898 ],
[ 0.05746796, -0.84663889, 0.52905579],
[ 0.06308011, -0.52579339, -0.84827012]])
I use the extrinsic convention for concatenation of rotation matrices, that is, r_1 is applied after r_0.T. (If r_0 and r_1 were real numbers, we would write r_1 - r_0 to obtain a number that transforms r_0 to r_1.)
You can verify that r0_to_r1 rotates from r_0 to r_1:
from numpy.testing import assert_array_almost_equal
# verify correctness: apply r0_to_r1 after r_0
assert_array_almost_equal(r_1, r0_to_r1.dot(r_0))
# would raise an error if test fails
Anyway, the intrinsic convention would also work:
r0_to_r1_intrinsic = r_0.T.dot(r_1)
assert_array_almost_equal(r_1, r_0.dot(r0_to_r1_intrinsic))
Since zabop introduced pytransform3d, I would also like to clarify that scipy uses active rotation matrices and the rotation matrix that pytransform3d.rotations.euler_xyz_from_matrix produces is a passive rotation matrix! This wasn't documented so clearly in previous versions. You can transform an active rotation matrix to a passive rotation matrix and vice versa with the matrix transpose. Both pytransform3d's function and scipy's Rotation.to_euler("xyz", ...) use the intrinsic concatenation convention.
from scipy.spatial.transform import Rotation as R
r = R.from_matrix(r0_to_r1)
euler_xyz_intrinsic_active_degrees = r.as_euler('xyz', degrees=True)
euler_xyz_intrinsic_active_degrees
Result: array([-148.20762964, -3.6166255 , 3.30106818])
You can obtain the same result with pytransform3d (note that we obtain the passive rotation matrix by .T):
import pytransform3d.rotations as pr
euler_xyz_intrinsic_active_radians = pr.euler_xyz_from_matrix(r0_to_r1.T)
np.rad2deg(euler_xyz_intrinsic_active_radians)
Result: array([-148.20762951, -3.61662542, 3.30106799])
You can also obtain the rotation matrix from euler angles with pytransform3d (note that we obtain the active rotation matrix by .T):
r0_to_r1_from_euler = pr.matrix_from_euler_xyz(euler_xyz_intrinsic_active_radians).T
r0_to_r1_from_euler
Result:
array([[ 0.99635251, 0.08212125, 0.0231898 ],
[ 0.05746796, -0.84663889, 0.52905579],
[ 0.06308011, -0.52579339, -0.84827013]])

contourf() plots white space over finite data

I'm attempting to plot a 3D chart using matplotlib.pyplot.contourf() with the following program:
import numpy as np
import matplotlib.pyplot as plt
import scipy
# calculates Fast Fourier transforms for each value in the 1D array "Altitude"
# and stacks them vertically to form a 2D array of fft values called "Fourier"
Fourier = np.array([])
for i in range(len(Altitude)):
Ne_fft = Ne_lowpass[i,:]/np.average(Ne_lowpass[i,:])
Ne_fft = Ne_fft - Ne_fft.mean()
W = scipy.fftpack.fftfreq(10*Ne_fft.size, d=(Time[-1]-Time[0])/len(Ne_fft))
P = 1/abs(W)
FFT = abs(scipy.fftpack.fft(Ne_fft, n=10*len(Ne_fft)))
FFT = FFT**2
if len(Fourier) == 0:
Fourier = FFT
else:
Fourier = np.vstack((Fourier,FFT))
# plots the 2D contourf plot of "Fourier", with respect to "Altitude" and period "P"
plt.figure(5)
C = plt.contourf(P,Altitude,Fourier,100,cmap='jet')
plt.xscale('log')
plt.xlim([1,P[np.argmax(P)+1]])
plt.ylim([59,687])
plt.ylabel("Altitude")
plt.xlabel("Period")
plt.title("Power spectrum of Ne")
cbar = plt.colorbar(C)
cbar.set_label("Power", fontsize = 16)
For the most part it is working fine; however, in some places useless white space is plotted. the plot produced can be found here (sorry, I don't have enough reputation points to attach images directly)
The purpose of this program is to calculate a series of Fast Fourier Transforms across 1 axis of a 2 dimensional numpy array, and stack them up to display a contour plot depicting which periodicities are most prominent in the data.
I checked the parts of the plotted quantity that appear white, and finite values are still present, although much smaller than noticable quantities elsewhere in the plot:
print(Fourier[100:,14000:])
[[ 2.41147887e-03 1.50783490e-02 4.82620482e-02 ..., 1.49769976e+03
5.88859945e+02 1.31930217e+02]
[ 2.12684922e-03 1.44076962e-02 4.65881565e-02 ..., 1.54719976e+03
6.14086374e+02 1.38727145e+02]
[ 1.84414615e-03 1.38162140e-02 4.51940720e-02 ..., 1.56478339e+03
6.23619105e+02 1.41367042e+02]
...,
[ 3.51539440e-03 3.20182148e-03 2.38117665e-03 ..., 2.43824864e+03
1.18676851e+03 3.13067945e+02]
[ 3.51256439e-03 3.19924000e-03 2.37923875e-03 ..., 2.43805298e+03
1.18667139e+03 3.13042038e+02]
[ 3.50985146e-03 3.19677302e-03 2.37741084e-03 ..., 2.43790243e+03
1.18659640e+03 3.13021994e+02]]
print(np.isfinite(Fourier.all()))
True
print(np.isnan(Fourier.any()))
False
Is the white space present because the values are so small compared to the rest of the plot? I'm not sure at all how to fix this.

You can fix this problem by adding option extend='both'.
Example:
C = plt.contourf(P,Altitude, Fourier,100, cmap='jet', extend='both')
Ref: https://matplotlib.org/examples/pylab_examples/contourf_demo.html

In the line plt.contourf(P,Altitude,Fourier,100,cmap='jet') you are taking 100 automatically chosen levels for the contour plot. "Automatic" in this case does not guarantee that those levels include all data.
If you want to make sure they all data is included you may define you own levels to use
plt.contourf(x, y, Z, np.linspace(Z.min(), Z.max(), 100))

How to produce the following images (gabor patches)

I am trying to create four gabor patches, very similar to those below.
I don't need them to be identical to the pictures below, but similar.
Despite a bit of tinkering, I have been unable to reproduce these images...
I believe they were created in MATLAB originally. I don't have access to the original MATLAB code.
I have the following code in python (2.7.10):
import numpy as np
from scipy.misc import toimage # One can also use matplotlib*
data = gabor_fn(sigma = ???, theta = 0, Lambda = ???, psi = ???, gamma = ???)
toimage(data).show()
*graphing a numpy array with matplotlib
gabor_fn, from here, is defined below:
def gabor_fn(sigma,theta,Lambda,psi,gamma):
sigma_x = sigma;
sigma_y = float(sigma)/gamma;
# Bounding box
nstds = 3;
xmax = max(abs(nstds*sigma_x*numpy.cos(theta)),abs(nstds*sigma_y*numpy.sin(theta)));
xmax = numpy.ceil(max(1,xmax));
ymax = max(abs(nstds*sigma_x*numpy.sin(theta)),abs(nstds*sigma_y*numpy.cos(theta)));
ymax = numpy.ceil(max(1,ymax));
xmin = -xmax; ymin = -ymax;
(x,y) = numpy.meshgrid(numpy.arange(xmin,xmax+1),numpy.arange(ymin,ymax+1 ));
(y,x) = numpy.meshgrid(numpy.arange(ymin,ymax+1),numpy.arange(xmin,xmax+1 ));
# Rotation
x_theta=x*numpy.cos(theta)+y*numpy.sin(theta);
y_theta=-x*numpy.sin(theta)+y*numpy.cos(theta);
gb= numpy.exp(-.5*(x_theta**2/sigma_x**2+y_theta**2/sigma_y**2))*numpy.cos(2*numpy.pi/Lambda*x_theta+psi);
return gb
As you may be able to tell, the only difference (I believe) between the images is contrast. So, gabor_fn would likely needed to be altered to do allow for this (unless I misunderstand one of the params)...I'm just not sure how.
UPDATE:
from math import pi
from matplotlib import pyplot as plt
data = gabor_fn(sigma=5.,theta=pi/2.,Lambda=12.5,psi=90,gamma=1.)
unit = #From left to right, unit was set to 1, 3, 7 and 9.
bound = 0.0009/unit
fig = plt.imshow(
data
,cmap = 'gray'
,interpolation='none'
,vmin = -bound
,vmax = bound
)
plt.axis('off')

The problem you are having is a visualization problem (although, I think you are chossing too large parameters).
By default matplotlib, and scipy's (toimage) use bilinear (or trilinear) interpolation, depending on your matplotlib's configuration script. That's why your image looks so smooth. It is because your pixels values are being interpolated, and you are not displaying the raw kernel you have just calculated.
Try using matplotlib with no interpolation:
from matplotlib import pyplot as plt
plt.imshow(data, 'gray', interpolation='none')
plt.show()
For the following parameters:
data = gabor_fn(sigma=5.,theta=pi/2.,Lambda=25.,psi=90,gamma=1.)
You get this output:
If you reduce lamda to 15, you get something like this:
Additionally, the sigma you choose changes the strength of the smoothing, adding parameters vmin=-1 and vmax=1 to imshow (similar to what #kazemakase) suggested, will give you the desired contrast.
Check this guide for sensible values (and ways to use) gabor kernels:
http://scikit-image.org/docs/dev/auto_examples/plot_gabor.html

It seems like toimage scales the input data so that the min/max values are mapped to black/white.
I do not know what amplitudes to reasonably expect from gabor patches, but you should try something like this:
toimage(data, cmin=-1, cmax=1).show()
This tells toimage what range your data is in. You can try to play around with cmin and cmax, but make sure they are symmetric (i.e. cmin=-x, cmax=x) so that a value of 0 maps to grey.