Offline Versus Online Evaluation

Offline evaluation replays or splits historical data. Online evaluation measures behavior under actual exposure, usually through experiments. Recommenders need both because recommendations change what users see and therefore change future labels.

Replay and off-policy estimation

For randomized logged bandit data, replay keeps only events where the new policy matches the logged action:

$$\hat{V}(\pi )=\frac{1}{|M|}\sum _{t\in M}{r}_t,M=\{t:\pi (x_t)=a_t\}.$$

With known propensities $p_t$, inverse propensity scoring uses

$${\hat{V}}_{\text{IPS}}(\pi )=\frac{1}{T}\sum _t\frac{1\{\pi (x_t)=a_t\}{r}_t}{p_t}.$$

This is different from ordinary evaluation of recommenders on held-out items.

Worked example

Replay evaluation keeps only rows where the candidate policy chose the same action as the logged policy:

Event	Logged action	Reward	Candidate action	Replay?
1	0	1	0	yes
2	1	0	0	no
3	0	0	0	yes
4	2	1	2	yes
5	1	1	2	no
6	2	0	2	yes

Only four events can be replayed because rewards for unshown actions are missing. Among the matched rows, rewards are $1,0,1,0$, so the replay CTR is $2/4=0.5$. Bandit algorithms require this partial-feedback discipline.

Caveats

Offline ranking metrics are cheap and reproducible but can overfit historical exposure. Online tests measure real behavior but are slower, riskier, and sensitive to interference. Use offline gates to reject bad candidates, then confirm important changes with online experiments.

References

• Li et al., 2010, Unbiased Offline Evaluation of Contextual-bandit-based News Article Recommendation Algorithms
• Herlocker et al., 2004, Evaluating Collaborative Filtering Recommender Systems

Nav

Section — Recommendation Systems and Personalization

← Evaluation of Recommenders Diversity, Novelty, Coverage, and Serendipity →