Definition
PDAI measures the change in schedule quality produced by planners during AI system outages or manual-override periods compared against a baseline of planner-built schedules from the pre-AI period or early post-deployment. This metric determines whether AI dependency has degraded the planning capability that the organization needs as a fallback during system failures, model retraining, or situations that require human judgment beyond the AI's training data.
The Measurement Gap
The Measurement Gap
Maintenance planners who relied on the AI scheduler for two years may have lost the ability to build an adequate schedule manually. They no longer remember the constraint interactions (weather, certifications, travel, parts availability, curtailment windows) that the AI handles automatically. They no longer practice the judgment calls (when to bundle versus separate tasks, how to balance priority against geography, when to defer versus force-schedule) that they made daily before the AI.
This capability loss is invisible during normal operations because the AI handles scheduling. It becomes visible only during AI system outages, model retraining periods, or novel situations that fall outside the AI's training data. At that point, the planner must produce a manual schedule, and the quality of that schedule reveals how much capability has atrophied.
Without PDAI, organizations discover atrophy through performance degradation during outages. The schedule falls apart, tasks are sequenced poorly, constraints are violated, and the reliability engineering team scrambles to fix scheduling errors that the AI would have surfaced. This discovery comes at the worst possible time because it coincides with the system failure that made manual scheduling necessary.
The Decision This Metric Supports
PDAI answers whether the planners can build an adequate schedule manually if the AI system goes down. When PDAI is between minus 10% and 0%, the capability drift is within acceptable limits. When PDAI drops below minus 20%, the organization has a resilience risk: it cannot function without the AI scheduler, and any extended system outage will produce scheduling failures with downstream maintenance and reliability consequences.
Formula
Primary Formula:
PDAI = (Schedule_Quality_Manual_Current minus Schedule_Quality_Manual_Baseline) / Schedule_Quality_Manual_Baseline x 100
Where:
| Variable | Definition |
|---|---|
| Schedule_Quality_Manual_Current | composite schedule quality score during current manual-planning periods |
| Schedule_Quality_Manual_Baseline | composite schedule quality score from the baseline period (pre-AI or early post-deployment) |
Measurement frequency: Semi-annually, supplemented by quarterly structured manual planning exercises. AI system outages provide natural measurement opportunities, but they are too infrequent and unpredictable to serve as the primary data source. Structured exercises (analogous to disaster recovery drills) ensure regular measurement.
Edge cases requiring special handling:
- AI outages are rare and short, producing tiny sample sizes. Supplement with structured quarterly manual planning exercises where planners build a week's schedule without AI assistance, reviewed for quality and used as training.
- New planners who never built schedules manually have no personal baseline. Use the organizational baseline for comparison, but recognize that new planners may never achieve the historical standard.
- Manual schedules during outages may cover simpler operational situations because reduced operations during system failures may reduce scheduling complexity. Control for schedule complexity when comparing manual quality against baseline.
Confidence Grading (required alongside every PDAI value):
| Grade | Basis | Interpretation |
|---|---|---|
| A: Empirical | 5+ observed events in measurement window provide direct tail observation | Measurement grounded in observed outcomes; highest confidence |
| B: Hybrid | 1-4 observed events plus Monte Carlo simulation with Bayesian updating | Blends observation and simulation; moderate confidence |
| C: Simulated | Zero observed events; derived entirely from Monte Carlo simulation | Modeled, not observed; treat as indicative, not validated |
A PDAI of any value at Grade C is a simulation that agrees with itself. It tells you the model is internally consistent but not that it matches reality. Grade C scores should trigger proactive stress-testing against historical near-miss events and cross-asset analogues rather than acceptance as calibrated.
Proposed formula, pending validation pilots.