Eval Task Coverage Ratio (ETCR)

GMI-MLOPS-003MLOps / AI Platform OperationsIntegration-RequiredPattern O-7: Inspection Coverage

Definition

The share of the production task distribution, by traffic weight, that the release eval suite exercises at representative difficulty.

The Measurement Gap

There is no metric that tells you what fraction of the production task distribution the eval suite covers, which means passing models are untested on a growing share of real traffic.

Formula

ETCR = sum(w_t for covered t) / sum(w_t for all t)

Proposed formula, pending validation pilots.

At a Glance

Category
Evaluation Integrity

Decision Supported

Whether a passing eval means anything for the traffic actually served, and where eval investment goes next.

← Eval-Outcome Divergence Index (EODI)LLM Judge Calibration Index (LJCI) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment