Unsupervised Learning

Unsupervised learning fits structure from $X$ without observed target labels. The output may be clusters, components, density estimates, embeddings, or anomaly scores. Because there is no direct $y$, validation depends more heavily on assumptions and downstream utility than in supervised learning.

Structure without labels

Without labels, the method’s definition of “structure” is the whole game. Variance, Euclidean distance, density, and neighborhood preservation can all disagree, and dimensionality reduction that helps visualization may discard information a later classifier needs. The main families differ in what they output and what they optimize:

Task	Output	Typical objective
Clustering	group assignments	within-cluster distance (k-means)
Dimensionality reduction	low-dimensional coordinates	retained variance (PCA)
Density estimation	a probability model	data likelihood
Anomaly detection	outlier scores	a score threshold

Common objectives

Many unsupervised methods optimize a reconstruction, partition, or likelihood objective. Clustering with k-means partitions the points into clusters $C_1,\dots ,C_K$ to minimize the within-cluster squared distance,

$$\underset{C_1,\dots ,C_K}{\min }\sum _k\sum _{x_i\in C_k}\Vert x_i-{\mu }_k{\Vert }_2^2,$$

where $x_i$ is a data point and ${\mu }_k$ is the centroid (mean) of the points assigned to cluster $C_k$. PCA solves ${\max }_{W^{\top }W=I}\mathrm{tr}(W^{\top }{X}^{\top }XW)$ for an orthonormal projection $W$, and anomaly methods learn a score $s(x)$ and flag points whose score crosses a threshold.

When reference labels are available after clustering, the adjusted Rand index compares how pairs of points are grouped while correcting for chance:

$$\mathrm{ARI}=\frac{RI-\mathbb{E}[RI]}{\max (RI)-\mathbb{E}[RI]},$$

where $RI$ is the Rand index — the fraction of point pairs that are either together in both the clustering and the reference, or apart in both — and $\mathbb{E}[RI]$ is its expected value under random labeling.

It is an external validation score, not an unsupervised objective.

Worked example

This snippet clusters standardized Iris measurements without labels, then compares internal silhouette quality with adjusted Rand agreement against the hidden species labels.

from sklearn.cluster import KMeans
from sklearn.datasets import load_iris
from sklearn.metrics import adjusted_rand_score, silhouette_score
from sklearn.preprocessing import StandardScaler
 
X, y = load_iris(return_X_y=True)
X2 = StandardScaler().fit_transform(X)
labels = KMeans(n_clusters=3, random_state=19, n_init=10).fit_predict(X2)
print("silhouette", round(silhouette_score(X2, labels), 3))
print("adjusted_rand_vs_species", round(adjusted_rand_score(y, labels), 3))

Observed output:

silhouette 0.46
adjusted_rand_vs_species 0.62

The silhouette score uses only geometry. Adjusted Rand uses species labels after the fact, showing that geometric clusters partially but imperfectly align with botanical classes.

Caveats

Scaling can dominate unsupervised results. Internal metrics can reward artificial structure even when clusters are not actionable. If labels are later used to choose the unsupervised representation, that choice becomes supervised model selection.

References

• scikit-learn User Guide: Unsupervised learning
• scikit-learn User Guide: Clustering

Nav

Section — Classical Machine Learning

← Interpretability Dimensionality Reduction →