Definition
A composite measure of whether AI explanations are sufficient for technicians to act correctly without additional diagnostic investigation, multiplied by the ratio of baseline resolution time to AI-explained resolution time for the same asset class and failure type.
The Measurement Gap
Explainable AI added factor-importance displays and contributing-factor explanations to predictive maintenance interfaces. Vendors declared victory at the point of showing an explanation, not at the point of the explanation being useful. Precision, recall, and F1 measure whether the prediction was right; they say nothing about whether the explanation guided action. MTTR cannot distinguish a 45-minute resolution where the AI led the technician straight to the cause from a 45-minute resolution where the technician ignored the AI and diagnosed independently. A true explanation that does not narrow the diagnostic search is decoration. Cognitive Load per Diagnostic Resolution (CLDR) will tell you technicians are working hard: it will not tell you whether the explanation is the failing element.
Decision Supported:
Whether to invest in explanation redesign, additional contextual content, or alternative explanation formats, and which asset classes or failure modes need the most attention.
Formula
Plain language: Two parts. First, what fraction of AI-explained work orders met three sufficiency tests (first action matched the verified root cause, no diagnostic escalation occurred, diagnostic phase was at or below the median for that failure type). Second, how much faster are AI-explained work orders than matched non-AI baselines? Multiply the two.
Notation:
ESRE = ESR × RER
ESR = N_sufficient / N_ai_explained
RER = T_median_baseline / T_median_ai_explained
Variables:
| Variable | Definition | Unit |
|---|---|---|
| ESR | Explanation Sufficiency Rate: fraction of AI-explained WOs meeting all three sufficiency criteria | Ratio (0–1) |
| N_sufficient | AI-explained WOs where first action matched verified root cause, no escalation, diagnostic time ≤ median | Count |
| N_ai_explained | Total AI-explained work orders in the period | Count |
| RER | Resolution Efficiency Ratio: baseline median resolution time ÷ AI-explained median resolution time | Ratio |
| T_median_baseline | Median resolution time for non-AI WOs in same asset class and failure type | Time |
| T_median_ai_explained | Median resolution time for AI-explained WOs in same asset class and failure type | Time |
Proposed formula, pending validation pilots.