Eval-Outcome Divergence Index (EODI)

GMI-MLOPS-001MLOps / AI Platform OperationsIntegration-RequiredPattern O-30: Proxy-Objective Divergence

Definition

The correlation between release-gate evaluation scores and realized production outcome quality for promoted models, measured over a rolling promotion window.

The Measurement Gap

There is no metric that tells you how release-gate eval scores relate to realized production outcomes, which means promotion decisions optimize a proxy whose divergence is undetected until user harm surfaces.

Formula

EODI = corr_rank(eval_score_m, outcome_quality_m) over promoted models m in window W

Proposed formula, pending validation pilots.

At a Glance

Category
Evaluation Integrity

Decision Supported

Whether the eval gate can be trusted as a promotion criterion, and when the eval suite must be rebuilt rather than the models retuned.

Eval Task Coverage Ratio (ETCR) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment