Definition
The statistical distance between the current MRO demand distribution and the distribution the forecast model was trained on, surfacing silent obsolescence of forecast models after disruption events.
The Measurement Gap
In February 2020, your demand forecasting model was performing well. By August 2020, the same model (same architecture, same training data) was producing recommendations nobody trusted. Demand patterns had shifted. Lockdowns changed PM cadence. Supply availability changed ordering behavior. The model's view of the world had silently become obsolete, but it continued issuing high-confidence forecasts on a distribution that no longer existed. Tariff shifts, sanctions, vendor bankruptcies, geopolitical disruptions, weather events, and technology refreshes all shift the demand distribution. Each time, every model trained on the prior distribution loses fidelity while the forecasts continue unchanged. Forecast accuracy (MAPE, WAPE) shows degradation only after 30 to 90 days of actuals accumulate: exactly the window during which wrong reorder decisions get committed. Inventory turns shifts even slower. Fill rate shows stockout consequences after they have already happened. What is needed is a leading indicator that detects input distribution shift before the actuals confirm the model is wrong. Drift measurement is well-established in MLOps but is almost never instrumented in supply chain models because platform vendors do not surface it and operations teams do not know to ask.
Decision Supported:
When to retrain the demand model, when to widen confidence intervals, and when to require human review of automated reorder recommendations during periods of confirmed distributional shift.
Formula
Plain language: Compare the shape of recent demand to the shape the model learned on. The bigger the difference, the staler the model. A large distance means the model is making predictions about a world that no longer exists.
Notation:
DMDI = PSI(D_current, D_training)
PSI = Σ_i (P_current(i) − P_training(i)) × ln(P_current(i) / P_training(i))
Variables:
| Variable | Definition | Unit |
|---|---|---|
| PSI | Population Stability Index (or KS / Wasserstein distance) | Index (unitless) |
| D_current | Current demand distribution over rolling 90-day window | Distribution |
| D_training | Demand distribution from model training period | Distribution |
| P_current(i) | Proportion of demand events in bucket i, current window | % |
| P_training(i) | Proportion of demand events in bucket i, training window | % |
| i | Demand distribution bucket (deciles of training period) | Category |
Proposed formula, pending validation pilots.