Which dimensionality reduction technique is linear, mathematical, and assumes data lie on a linear subspace?

Which dimensionality reduction technique is linear, mathematical, and assumes data lie on a linear subspace?

• A. PCA
• B. Autoencoders
• C. Clustering
• D. K-means

Answer: (A)

The key idea is reducing dimensionality by projecting data onto a linear subspace. Principal Component Analysis does exactly this in a purely linear, algebraic way. It starts by centering the data and computing the covariance matrix, then finds the eigenvectors corresponding to the largest eigenvalues. These eigenvectors form an orthogonal basis for the best-fitting low-dimensional linear subspace. By projecting the data onto the span of the top eigenvectors, you get a lower-dimensional representation that preserves as much variance as possible under a linear transformation. This makes PCA a linear, mathematical approach that explicitly assumes the data can be well approximated by a linear subspace. If the data truly lie on a nonlinear manifold, PCA won’t capture that structure as effectively; nonlinear methods like autoencoders can model more complex shapes. Clustering methods and K-means, on the other hand, don’t aim to produce a linear subspace for dimensionality reduction—they’re about grouping data (K-means) or general unsupervised organization (clustering) rather than projecting onto a linear subspace.

The key idea is reducing dimensionality by projecting data onto a linear subspace. Principal Component Analysis does exactly this in a purely linear, algebraic way. It starts by centering the data and computing the covariance matrix, then finds the eigenvectors corresponding to the largest eigenvalues. These eigenvectors form an orthogonal basis for the best-fitting low-dimensional linear subspace. By projecting the data onto the span of the top eigenvectors, you get a lower-dimensional representation that preserves as much variance as possible under a linear transformation. This makes PCA a linear, mathematical approach that explicitly assumes the data can be well approximated by a linear subspace.

If the data truly lie on a nonlinear manifold, PCA won’t capture that structure as effectively; nonlinear methods like autoencoders can model more complex shapes. Clustering methods and K-means, on the other hand, don’t aim to produce a linear subspace for dimensionality reduction—they’re about grouping data (K-means) or general unsupervised organization (clustering) rather than projecting onto a linear subspace.