Maintenance Rule Prediction Calibration Index (MRPCI)

GMI-NUC-029Nuclear OperationsIntegration-Required

Definition

Degree to which AI-predicted Maintenance Rule system performance trajectories match actual system performance over time, measuring whether the AI's predictions of (a)(2) criteria compliance are calibrated against reality.

The Measurement Gap

AI systems predicting Maintenance Rule system performance trajectories enable plants to anticipate when a system might approach (a)(2) performance criteria and proactively address degradation before an exceedance triggers (a)(1) goal-setting. But no metric measures whether these predictions are calibrated against actual system performance outcomes. Maintenance Rule (a)(2) criteria track whether systems meet performance goals. The Maintenance Rule periodic assessment reviews system performance retrospectively. Neither measures whether the AI's forward-looking predictions of system performance are accurate. The invisible failure mode is that the AI predicts a system will remain within (a)(2) criteria, the plant delays proactive action based on that prediction, and the system actually exceeds criteria, triggering a Maintenance Rule (a)(1) transition that earlier intervention would have opened an action window against. The prediction created a false sense of security that no metric detected until the exceedance occurred.

Decision Supported

Can the plant rely on AI predictions that a system will remain within Maintenance Rule (a)(2) performance criteria, or are the predictions systematically overconfident, enabling deferred (a)(1) goal-setting that should have started earlier?

Formula

Plain language: For each system where the AI predicted the performance trajectory (unavailability, reliability, or both), compare the AI's predicted value at each future time point to the actual measured value. Compute the prediction error profile. Specifically measure whether the AI predicted "within criteria" and the actual result was "exceeded criteria". This is the dangerous miss.

Notation:
MRPCI = 1 − (1/S) × Σ_s |Predicted_MR_s(t) − Actual_MR_s(t)| / Criteria_s

Missed exceedance rate:
MER = Systems_predicted_OK_actually_exceeded / Systems_predicted_OK

Variables:

  • Predicted_MR_s(t) = AI-predicted Maintenance Rule performance value for system s at time t (unavailability hours or failure count, as applicable)
  • Actual_MR_s(t) = actual measured performance value for system s at time t (same units)
  • Criteria_s = the (a)(2) performance criteria threshold for system s (same units: the reference scale for error magnitude)
  • S = number of systems with AI performance predictions (count)
  • MER = the rate at which the AI predicted systems would be OK but they actually exceeded criteria (dimensionless ratio: the missed call rate)

Sign convention: MRPCI = 1.0 means perfect prediction accuracy. Lower is worse. MER should be as close to 0.0 as possible: any missed exceedance means the plant was late responding to a Maintenance Rule performance problem.

Aggregation period: Quarterly (aligned with Maintenance Rule performance reviews).

Edge Cases

  • System performance changes due to a single event (one unexpected failure changes the reliability metric dramatically): evaluate whether the event was foreseeable. If it was a random component failure, the prediction miss is acceptable. If it was a degradation trend the AI should have detected, it's a calibration failure.
  • Systems with very few demands (standby safety systems tested quarterly): small sample sizes make prediction validation noisy. Use rolling 2-year windows for low-demand systems
  • MR criteria changes (tightened or relaxed): re-baseline predictions against the new criteria
  • Minimum: all MR (a)(2) systems with AI performance predictions (census)

Proposed formula, pending validation pilots.

At a Glance

Unit
Ratio
Trend Direction
Toward 1.0
Reporting Frequency
Quarterly
Category
Supply Chain and Qualification Integrity
← Schedule Dose Alignment Factor (SDAF)Vendor Qualification Currency Rate (VQCR) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment