Definition
The distributional distance between the incident population a classifier was trained on and the population it is currently inferring over.
The Measurement Gap
There is no metric that tells you the divergence between the incident population a classifier was trained on and the population it is currently inferring over, which means model currency degrades between manual retraining events with no trigger.
Formula
TPD = sum over combinations c of |p_infer(c) - p_train(c)| / 2
Proposed formula, pending validation pilots.
At a Glance
Decision Supported
When to retrain, and whether the cause is a shifting operation or a training filter that was never representative.