Module 6: Dimensionality Reduction
  Lecture 30: Principal Component Analysis (PCA)
 

                                            

 

 

Properties
  • Also known as Karhunen-Loeve transform (KLT)
  • Works for distances only as others are not invariant to rotation
  • Mean-centering
 
  • Easier way to compute covariance: is covariance matrix
 
  • Allows use of SVD to compute PCA
  • Can be done using SVD
 
  • Eigenvector matrix of is really the SVD transform matrix for
 
  • Different from SVD of though
  • How many dimensions to retain?
 
  • Based on energy (similar to SVD)
 
  • Total energy is sum of eigenvalues
 
  • Retain dimensions such that of the energy is retained
 
  • In the above example, = 1 retains of the energy
  • Running time: for of size