used in data visualization, data compression and Feature Extraction
Aim
- Find a linear orthonormal projection to maximize data Variance in the new vector space (linear transformation)
- find a new basis of maximum variance; basis named principal components (PCs) and their importance decided by variance
- A new low-dimensional representation achieved by projecting data points onto few (< original ) PCs
Math
PCA math
As any real-valued covariance matrix (leading to PCA) is semi-definite positive and symmetric, its eigenvectors (PCs) are always orthogonal each other.PCA math
⚠ Switch to EXCALIDRAW VIEW in the MORE OPTIONS menu of this document. ⚠
mean vector, Variance, Correlation & Covariance, SVD
Text Elements
Embedded files
8b6394ddcd3a6aa1eb089c7d2db4b77cf484481f: mean vector 9ed74a76804101ebc57a442a8d6e343fbd06cb51: Variance af80ce2d21812822dcd6ce79ccb3ccf2b19730f9: Kronecker Delta
Link to original
PCA can only provides only up to d′ PCs (d′ ≤ d) where d′ is the rank of a covariance matrix used for PCA.
Dual PCA
Dual PCA
⚠ Switch to EXCALIDRAW VIEW in the MORE OPTIONS menu of this document. ⚠
Text Elements
Embedded files
c5995f9d728872603a09961eafa72e56a76e0eda: SVD
Link to original
Proportion of Variance
Proportion of Variance
⚠ Switch to EXCALIDRAW VIEW in the MORE OPTIONS menu of this document. ⚠
Text Elements
Link to original
- can be used to find optimum intrinsic dimensionality
- also works for the dual PCA although there are only up to N non-zero eigenvalues and at least d − N zero values.
Limitations
- non-orthogonality
- Data distribution may not satisfy the PCA assumption: orthogonal dimensions,
- Independent Component Analysis: captures the maximum variance in non-orthogonal dimensions
- inconsistency to discrimination
- in classification, axis of maximum variance in PCA may be inconsistent with that of the largest discrimination
- Linear Discriminative Analysis (LDA) : Finds out the axis of the largest discrimination by considering the label information
- non-linearity
- non linear manifolds