Definition
A composite score measuring how current a predictive model is relative to the operational context it serves, based on training data recency, operational condition drift, and prediction accuracy trend.
The Measurement Gap
A predictive model trained on 2022-2024 data is still running in 2026, but the fleet composition has changed (new locomotive acquisitions, retirements), operating routes have shifted, and loading patterns are different. The model's accuracy has degraded but no metric tracks model performance over time relative to operational context changes. Availability and MDBF measure outcomes, not the quality of the decision-making tool. A model can degrade for months before outcome metrics move, and by then the damage is done.
Decision Supported:
Is this model still valid for today's fleet, routes, and operating conditions, or has the world changed enough that the model's predictions are based on stale assumptions?
Formula
Plain language: Score three components on a 0-to-1 scale. Training data age: how old is the most recent data the model was trained on? Operating condition drift: how much have key operational parameters changed since training? Accuracy trend: is prediction accuracy stable, improving, or degrading? Average the three.
Notation:
MCI = (S_age + S_drift + S_trend) / 3
Where:
S_age = max(0, 1 - (T_now - T_train) / T_max)
S_drift = 1 - (D_current / D_max)
S_trend = PCI_current / PCI_baseline
Variables:
| Variable | Description | Unit |
|---|---|---|
| T_now | Current date | Date |
| T_train | Date of most recent training data included in model | Date |
| T_max | Maximum acceptable training data age (proposed: 24 months fleet, 36 months infrastructure) | Months |
| D_current | Composite operating condition drift: percent change in fleet composition + route-miles + GTM since training | Percentage |
| D_max | Maximum acceptable drift before mandatory retraining (proposed: 25%) | Percentage |
| PCI_current | Current PCI score | Dimensionless |
| PCI_baseline | PCI score at model deployment or last retraining | Dimensionless |
Sign Convention: Higher is better. 1.0 = perfectly current model. Below 0.5 = stale, retraining recommended.
Proposed formula, pending validation pilots.