Definition
The SKU-weighted gap between the demand forecast model's stated confidence and its measured accuracy, computed separately for each part class so that systematic over-confidence on rare critical parts is not hidden inside an overall fill rate average.
The Measurement Gap
Your demand forecasting platform tells you it is 90% confident in a bearing kit forecast, and the planner sets safety stock accordingly. But on a rare-critical hydraulic seal with only six prior data points, the same platform claims 92% confidence, and the forecast is a coin flip sold as near-certainty. Fill rate is an average over the SKU universe; it tells you 96% of demand events were satisfied and nothing about whether the 4% that missed were concentrated on the parts that matter most. Forecast accuracy (MAPE, WAPE) measures mean error magnitude but does not tell you whether the model's stated confidence at each forecast event matches its actual hit rate at that confidence band. Inventory turns shows the cost of wrong safety stock decisions but only in dollars and only after the cash is committed, never pointing back to the source. The decision the AI produces ("trust this forecast at 90% confidence") is made thousands of times daily across the SKU universe, and it is structurally invisible to every standard MRO supply chain metric. Models are systematically over-confident on rare-critical parts where they have the least data, which are exactly the parts where a forecast miss is most expensive.
Decision Supported:
How much trust to place in the demand forecast at the part-class level when setting safety stock, scheduling replenishment, or making rationalization decisions, and which part classes require recalibration or class-specific retraining.
Formula
Plain language: For every confidence band the model produces, measure how often the forecast was actually right within that band, separately for each part class. The weighted gap between stated confidence and measured accuracy is the calibration error.
Notation:
DFCEC(c) = Σ_b |accuracy(b, c) − confidence(b)| × n(b, c) / Σ_b n(b, c)
Variables:
| Variable | Definition | Unit |
|---|---|---|
| c | Part class (high-runner, intermittent, slow-mover, rare-critical) | Category |
| b | Forecast confidence band (10-percentage-point bands) | Category |
| accuracy(b, c) | Empirical hit rate of forecasts in band b for class c | % |
| confidence(b) | Stated confidence midpoint of band b | % |
| n(b, c) | Count of forecasts in band b for class c | Count |
Proposed formula, pending validation pilots.