Anomaly Detection

Anomaly detection ranks or flags observations that look unusual relative to a reference distribution. It is often unsupervised learning, but evaluation usually becomes supervised once analysts label true incidents. Compared with clustering, the goal is not to assign every point to a group; compared with class imbalance, the rare class may not be labeled at training time.

What counts as an anomaly

An anomaly is not just a rare point. It is rare under the model of normal behavior and relevant to the operational question. A new legitimate customer segment can look anomalous, while a common failure pattern may stop looking unusual after enough incidents accumulate. Methods differ in how they define “unusual”:

Family	Idea	Example method
Density	flag points in low-probability regions	Gaussian model, kernel density
Distance / neighborhood	flag points far from their neighbors	kNN distance, local outlier factor
Isolation	anomalies are easy to separate off	Isolation Forest
Reconstruction	flag points the model rebuilds poorly	PCA or autoencoder error

Scoring and flagging

Many methods learn an anomaly score $s(x)$ where larger (or smaller) means more anomalous, then flag a point when the score crosses a threshold $\tau$: $\hat{y}=1\{s(x)\ge \tau \}$, the indicator $1\{\cdot \}$ being $1$ when the rule fires. Density methods instead flag points of low estimated density $\hat{p}(x)$, using $\hat{y}=1\{\hat{p}(x)<\tau \}$. Isolation Forest isolates points by random partitioning; anomalies tend to have shorter average path lengths.

Worked example

This example creates a normal two-dimensional cloud plus a small separated outlier cluster, then checks whether Isolation Forest flags the separated points at the requested contamination rate.

import numpy as np
from sklearn.ensemble import IsolationForest
 
rng = np.random.default_rng(22)
normal = rng.normal(0, 1, size=(120, 2))
outliers = rng.normal(6, .5, size=(6, 2))
X = np.vstack([normal, outliers])
iso = IsolationForest(contamination=6/126, random_state=22).fit(X)
pred = iso.predict(X)
print("flagged_total", int((pred == -1).sum()))
print("outliers_flagged", int((pred[-6:] == -1).sum()), "of", len(outliers))
print("normal_flagged", int((pred[:-6] == -1).sum()), "of", len(normal))

Observed output:

flagged_total 6
outliers_flagged 6 of 6
normal_flagged 0 of 120

The synthetic outliers are far from the normal cloud, so the fitted contamination threshold flags exactly those six points.

Caveats

The contamination parameter bakes in an expected alert rate. High-dimensional distance concentration can make “far away” less meaningful. Concept drift changes normality, so anomaly systems need monitoring and analyst feedback loops.

References

• scikit-learn User Guide: Novelty and Outlier Detection
• scikit-learn User Guide: Isolation Forest

Nav

Section — Classical Machine Learning

← Clustering Evaluation Metrics →