Definition
The correlation between release-gate evaluation scores and realized production outcome quality for promoted models, measured over a rolling promotion window.
The Measurement Gap
There is no metric that tells you how release-gate eval scores relate to realized production outcomes, which means promotion decisions optimize a proxy whose divergence is undetected until user harm surfaces.
Formula
EODI = corr_rank(eval_score_m, outcome_quality_m) over promoted models m in window W
Proposed formula, pending validation pilots.
At a Glance
Decision Supported
Whether the eval gate can be trusted as a promotion criterion, and when the eval suite must be rebuilt rather than the models retuned.