In PCA, what are the principal components?

In PCA, what are the principal components?

• A. Nonlinear transformations
• B. Orthogonal directions that capture maximum variance
• C. Random projections
• D. Discrete clusters

Answer: (B)

Principal components are the orthogonal directions in the feature space along which the data varies the most when you project onto them. The first principal component points along the direction of maximum variance; the second is the next independent direction (perpendicular to the first) with the next highest variance, and so on. These directions are the eigenvectors of the centered data's covariance matrix (or, equivalently, the left singular vectors in the SVD of the centered data). Each component has an associated eigenvalue that quantifies how much variance it explains, which lets you decide how many to keep for dimensionality reduction. They are linear combinations of the original variables, uncorrelated due to orthogonality, and they are not nonlinear transforms, random projections, or discrete clusters.

Principal components are the orthogonal directions in the feature space along which the data varies the most when you project onto them. The first principal component points along the direction of maximum variance; the second is the next independent direction (perpendicular to the first) with the next highest variance, and so on. These directions are the eigenvectors of the centered data's covariance matrix (or, equivalently, the left singular vectors in the SVD of the centered data). Each component has an associated eigenvalue that quantifies how much variance it explains, which lets you decide how many to keep for dimensionality reduction. They are linear combinations of the original variables, uncorrelated due to orthogonality, and they are not nonlinear transforms, random projections, or discrete clusters.