Definition
Time to diagnose and resolve events that escalated to human operators because the AI had no matching pattern, trended against a pre-AI or early-deployment baseline. A rising trend means the operators the organization depends on when the model fails are getting slower.
The Measurement Gap
The centralization of renewable fleet operations into AI-augmented Remote Operations Centers has delivered measurable improvements in routine event handling. Automated alarm classification, pattern-matched diagnostics, and playbook-driven response procedures mean that operators resolve known event types faster than ever. The measurement gap is that this efficiency gain may be masking a growing vulnerability: operator ability to diagnose novel events, situations the AI has never seen and for which no playbook exists.
When AI handles 95% of events automatically, operators practice independent diagnosis on only 5% of events. Over time, the diagnostic skills that operators exercised daily in pre-AI operations atrophy from disuse. The operators become dependent on the AI's pattern library for diagnosis, and when a genuinely novel event occurs, the event that is not in any playbook and does not match any pattern, the resolution takes longer than it would have taken before AI was deployed.
RODAI measures this effect by tracking the mean time to resolve novel events (those escalated to human operators because the AI had no matching pattern) over time. A rising trend indicates that operators are losing diagnostic proficiency. This is not a criticism of the ROC model; it is a recognition that automation-induced skill atrophy is a known human factors phenomenon that requires active countermeasures.
The decision RODAI supports is whether the organization is trading faster routine response for slower crisis response, and whether the ROC model needs structured exercises to maintain operator diagnostic proficiency for the events that matter most.
Formula
Measure the average time to diagnose and resolve events that escalated to human operators because the AI had no matching pattern. Compare across rolling quarters against a baseline period. A rising trend indicates that operators are losing diagnostic proficiency.
Formula (Notation):
RODAI = (MTTR_novel_current minus MTTR_novel_baseline) / MTTR_novel_baseline times 100
Variable Definitions:
| Variable | Definition |
|---|---|
| MTTR_novel_current | mean time to resolve novel (non-pattern-matched) events in the current measurement period, in hours |
| MTTR_novel_baseline | mean time to resolve novel events in the baseline period (first 12 months of ROC operation, or pre-ROC if data exists), in hours |
Edge Cases and Measurement Boundaries:
Novel event volume is typically low at well-run sites, creating small sample sizes that produce noisy RODAI values. Use rolling 6-month windows to accumulate sufficient data. System complexity may have genuinely increased over time (new asset types, new markets), which means rising MTTR may reflect harder problems rather than weaker operators. Control for event complexity if possible. Staff turnover introduces operators who never had the baseline skills; segment RODAI by operator tenure to distinguish atrophy from onboarding gaps.
Confidence Grading (required alongside every RODAI value):
| Grade | Basis | Interpretation |
|---|---|---|
| A: Empirical | 5+ observed events in measurement window provide direct tail observation | Measurement grounded in observed outcomes; highest confidence |
| B: Hybrid | 1-4 observed events plus Monte Carlo simulation with Bayesian updating | Blends observation and simulation; moderate confidence |
| C: Simulated | Zero observed events; derived entirely from Monte Carlo simulation | Modeled, not observed; treat as indicative, not validated |
A RODAI of any value at Grade C is a simulation that agrees with itself. It tells you the model is internally consistent but not that it matches reality. Grade C scores should trigger proactive stress-testing against historical near-miss events and cross-asset analogues rather than acceptance as calibrated.
Proposed formula, pending validation pilots.