Critical Spare Tail Risk Calibration (CSTRC)

GMI-MROSC-011MRO Supply ChainInfrastructure-DependentPattern O-1: Confidence Calibration

Definition

The agreement between the AI-generated critical-spare risk score and the actual realized production-impact distribution (probability times dollars), measured against the tail (P95, P99) rather than the mean.

The Measurement Gap

The AI gives you a risk score of 7.2 out of 10 for a critical spare. That single number drives safety stock level, reorder priority, and sourcing strategy. But the reality is not a single number: it is a probability distribution of dollar impact. A P50 outcome might be $40,000 in production loss. A P95 outcome might be $2.4 million. A P99 outcome (a simultaneous failure during a planned outage) might be $8 million. The risk score of 7.2 captures none of this distribution. If the score is calibrated to the mean, it systematically under-weights the tail events that drive the most catastrophic outcomes. If it is calibrated to the P95, it over-stocks average-risk parts. The legacy metrics are no help. Carrying cost percentage penalizes high stock regardless of risk. Service level rewards hits regardless of consequence. Stockout rate counts events but not dollars. ABC classification sorts by movement, not by impact. The score-driven stocking decision ignores the shape of the production-impact distribution, and the shape is where the $8 million surprise lives. A critical spare risk score that matches P50 reality but misses P95 reality is a tool that works until it doesn't.

Decision Supported:

Whether the critical-spare risk score can be trusted to set stocking policy for high-impact, low-frequency stockouts, and whether tail-risk-aware stocking rules are needed for parts where the P95 impact exceeds a defined threshold.

Formula

Plain language: The AI gives you one risk score. Reality is a probability distribution of dollar impact. Measure how well the score matches the distribution, especially the tail. If the P95 predicted impact matches P95 realized impact, the calibration is good where it matters most.

Notation:

CSTRC = 1 - |realized_P95_impact - predicted_P95_impact| / realized_P95_impact

Computed across critical SKUs. Report with a confidence grade.

Variables:

Variable Definition Unit
realized_P95_impact 95th percentile production-impact dollars from realized stockout events or simulation $
predicted_P95_impact 95th percentile production-impact dollars from the AI risk model $

Confidence Grading (required alongside every CSTRC value):

Grade Basis Interpretation
A: Empirical 5+ critical-spare stockout events in the measurement window provide direct tail observation Measurement is grounded in observed outcomes; highest confidence
B: Hybrid 1-4 observed events plus Monte Carlo simulation with Bayesian updating from those events Measurement blends observation and simulation; moderate confidence
C: Simulated Zero observed events; CSTRC derived entirely from Monte Carlo simulation of impact distributions Measurement is modeled, not observed; treat as indicative, not validated

Organizations must report the confidence grade alongside the CSTRC score. A CSTRC of 0.85 Grade A is a validated risk model. A CSTRC of 0.85 Grade C is a simulation that agrees with itself: it tells you the model is internally consistent but not that it matches reality. Grade C scores should trigger proactive stress-testing against historical near-miss events and cross-asset analogues rather than acceptance as calibrated.

Proposed formula, pending validation pilots.

At a Glance

Unit
Percentage
Trend Direction
Higher
Reporting Frequency
Quarterly (with simulation updates)
Category
D - Critical Parts & Substitution Integrity
← Criticality-Optimization Divergence Index (CODI)Repair Recommendation Outcome Index (RROI) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment