Matrix Decomposition in Statistics and Recommenders
Matrix decomposition became a modeling language because many high-dimensional datasets have lower-dimensional structure. The same idea appears as principal components in statistics, truncated SVD in information retrieval, and latent user-item factors in recommender systems.
Verified chronology
| Year | Milestone | Why it followed |
|---|---|---|
| 1901 | Karl Pearson published “On Lines and Planes of Closest Fit to Systems of Points in Space,” an early PCA formulation. | Multivariate data needed low-dimensional summaries of variation. |
| 1936 | Eckart and Young published the low-rank matrix approximation theorem. | It gave a precise optimality result: truncated decompositions are best approximations under common norms. |
| 1990 | Deerwester, Dumais, Furnas, Landauer, and Harshman published latent semantic analysis. | Term-document matrices were sparse and lexical; truncated SVD could reveal latent associations beyond exact word overlap. |
| 2006-2009 | Netflix Prize-era recommenders popularized low-rank user-item factor models; Koren, Bell, and Volinsky summarized matrix factorization in 2009. | Sparse ratings could be explained by shared latent taste and item dimensions rather than only neighbor similarity. |
Historical mechanism
Singular value decomposition factorizes a matrix into orthogonal directions and singular values. Low-rank approximation keeps only the largest components, trading detail for structure. In PCA, the retained directions summarize variance. In latent semantic analysis, they smooth term-document associations. In classical SVD, the matrix is treated as a numerical object to decompose.
Recommender matrix factorization changed the statistical meaning. The user-item matrix is mostly missing, not fully observed with noise. Practical factor models therefore optimize only observed interactions plus regularization, often with biases and confidence weights. That is why SVD versus matrix factorization is a real distinction in recommenders: the algebraic decomposition and the predictive objective are related but not identical.
The historical lesson is that low rank is a useful assumption, not a law. It works when hidden factors explain enough structure; it fails when missingness, exposure, popularity, or nonlinear preference patterns dominate.
References
- Pearson, 1901, On Lines and Planes of Closest Fit to Systems of Points in Space
- Eckart and Young, 1936, The approximation of one matrix by another of lower rank
- Deerwester et al., 1990, Indexing by latent semantic analysis
- Koren, Bell, and Volinsky, 2009, Matrix factorization techniques for recommender systems
Nav