Demand Forecast Calibration Error by Class (DFCEC)

GMI-MROSC-001MRO Supply ChainPilot-ReadyPattern O-1: Confidence Calibration

Definition

The SKU-weighted gap between the demand forecast model's stated confidence and its measured accuracy, computed separately for each part class so that systematic over-confidence on rare critical parts is not hidden inside an overall fill rate average.

The Measurement Gap

Your demand forecasting platform tells you it is 90% confident in a bearing kit forecast, and the planner sets safety stock accordingly. But on a rare-critical hydraulic seal with only six prior data points, the same platform claims 92% confidence, and the forecast is a coin flip sold as near-certainty. Fill rate is an average over the SKU universe; it tells you 96% of demand events were satisfied and nothing about whether the 4% that missed were concentrated on the parts that matter most. Forecast accuracy (MAPE, WAPE) measures mean error magnitude but does not tell you whether the model's stated confidence at each forecast event matches its actual hit rate at that confidence band. Inventory turns shows the cost of wrong safety stock decisions but only in dollars and only after the cash is committed, never pointing back to the source. The decision the AI produces ("trust this forecast at 90% confidence") is made thousands of times daily across the SKU universe, and it is structurally invisible to every standard MRO supply chain metric. Models are systematically over-confident on rare-critical parts where they have the least data, which are exactly the parts where a forecast miss is most expensive.

Decision Supported:

How much trust to place in the demand forecast at the part-class level when setting safety stock, scheduling replenishment, or making rationalization decisions, and which part classes require recalibration or class-specific retraining.

Formula

Plain language: For every confidence band the model produces, measure how often the forecast was actually right within that band, separately for each part class. The weighted gap between stated confidence and measured accuracy is the calibration error.

Notation:

DFCEC(c) = Σ_b |accuracy(b, c) − confidence(b)| × n(b, c) / Σ_b n(b, c)

Variables:

Variable Definition Unit
c Part class (high-runner, intermittent, slow-mover, rare-critical) Category
b Forecast confidence band (10-percentage-point bands) Category
accuracy(b, c) Empirical hit rate of forecasts in band b for class c %
confidence(b) Stated confidence midpoint of band b %
n(b, c) Count of forecasts in band b for class c Count

Proposed formula, pending validation pilots.

At a Glance

Unit
Percentage
Trend Direction
Lower
Reporting Frequency
Weekly (rolling 12-month window)
Category
A - Forecast & Model Integrity
Demand Model Drift Index (DMDI) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment