Module 6: Dimensionality Reduction
Lecture 30: Principal Component Analysis (PCA)
Properties
Also known as
Karhunen-Loeve transform (KLT)
Works for
distances only as others are not invariant to rotation
Mean-centering
Easier way to compute covariance:
is covariance matrix
Allows use of
SVD
to compute PCA
Can be done using
SVD
Eigenvector matrix
of
is really the
SVD
transform matrix
for
Different from
SVD
of
though
How many dimensions to retain?
Based on energy (similar to SVD)
Total energy is sum of eigenvalues
Retain
dimensions such that
of the energy is retained
In the above example,
= 1
retains
of the energy
distances only as others are not invariant to rotation
is covariance matrix
of 
is really the 
though
dimensions such that 
of the energy is retained
of the energy