Demand Model Drift Index (DMDI)

GMI-MROSC-002MRO Supply ChainIntegration-RequiredPattern O-2: Model Drift / Currency

Definition

The statistical distance between the current MRO demand distribution and the distribution the forecast model was trained on, surfacing silent obsolescence of forecast models after disruption events.

The Measurement Gap

In February 2020, your demand forecasting model was performing well. By August 2020, the same model (same architecture, same training data) was producing recommendations nobody trusted. Demand patterns had shifted. Lockdowns changed PM cadence. Supply availability changed ordering behavior. The model's view of the world had silently become obsolete, but it continued issuing high-confidence forecasts on a distribution that no longer existed. Tariff shifts, sanctions, vendor bankruptcies, geopolitical disruptions, weather events, and technology refreshes all shift the demand distribution. Each time, every model trained on the prior distribution loses fidelity while the forecasts continue unchanged. Forecast accuracy (MAPE, WAPE) shows degradation only after 30 to 90 days of actuals accumulate: exactly the window during which wrong reorder decisions get committed. Inventory turns shifts even slower. Fill rate shows stockout consequences after they have already happened. What is needed is a leading indicator that detects input distribution shift before the actuals confirm the model is wrong. Drift measurement is well-established in MLOps but is almost never instrumented in supply chain models because platform vendors do not surface it and operations teams do not know to ask.

Decision Supported:

When to retrain the demand model, when to widen confidence intervals, and when to require human review of automated reorder recommendations during periods of confirmed distributional shift.

Formula

Plain language: Compare the shape of recent demand to the shape the model learned on. The bigger the difference, the staler the model. A large distance means the model is making predictions about a world that no longer exists.

Notation:

DMDI = PSI(D_current, D_training)

PSI = Σ_i (P_current(i) − P_training(i)) × ln(P_current(i) / P_training(i))

Variables:

Variable Definition Unit
PSI Population Stability Index (or KS / Wasserstein distance) Index (unitless)
D_current Current demand distribution over rolling 90-day window Distribution
D_training Demand distribution from model training period Distribution
P_current(i) Proportion of demand events in bucket i, current window %
P_training(i) Proportion of demand events in bucket i, training window %
i Demand distribution bucket (deciles of training period) Category

Proposed formula, pending validation pilots.

At a Glance

Unit
Index
Trend Direction
Lower; approaching 0
Reporting Frequency
Weekly (rolling 90-day window)
Category
A - Forecast & Model Integrity
← Demand Forecast Calibration Error by Class (DFCEC)Supply Network Twin Fidelity (SNTF) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment