RUL Confidence Calibration Index (RULCI)

GMI-AVI-007AviationIntegration-Required

Definition

Degree to which an engine or component remaining useful life (RUL) model's stated prediction confidence corresponds to its actual prediction accuracy: measured by comparing stated confidence bands to observed accuracy rates across all components tracked by an engine health monitoring platform.

The Measurement Gap

Legacy engine health monitoring relied on exceedance monitoring and fixed inspection intervals: the question was whether a parameter exceeded its limit, not how confident the system was in its remaining life estimate. Traditional metrics like shop visit rate and unscheduled removal rate measure outcomes after the failure mode has already materialized. They provide no visibility into whether the predictive model's confidence statements are trustworthy before a decision is made.

EHM platforms now issue RUL predictions with explicit confidence scores. Maintenance planning teams use these confidence scores to make engine-on-wing decisions, shop visit slot bookings, and lease-return timing: decisions carrying $2–5M+ cost implications per event. But whether those confidence scores are calibrated (whether "90% confident" means the prediction is correct 90% of the time) is not measured by any standard aviation maintenance metric.

Decision supported:

Should the maintenance planning team trust the RUL model's stated confidence when scheduling shop visits and on-wing extensions, or is the model overconfident or underconfident in ways that are creating systematic cost or safety risk? RCCI gives maintenance engineering the ability to answer this question quantitatively by confidence band, so that if a specific range is systematically miscalibrated, planning decisions can be adjusted accordingly (conservatively reducing the effective confidence used for planning) until the model is retrained on updated operational data.

Formula

Plain language: Group all RUL predictions by stated confidence band (70–80%, 80–90%, 90–95%, 95%+). Within each band, measure the percentage of predictions where the component actually survived to the predicted RUL within ±10% tolerance. Compare stated confidence to actual accuracy within each band. Perfect calibration means actual accuracy equals stated confidence in every band.

Notation:

RCCI(b) = Actual_accuracy(b) / Stated_confidence(b)
Overall_RCCI = (1/B) × Σ |1 − RCCI(b)|

Variables:

Variable Definition Source Unit
b Confidence band (e.g., 80–90%) Defined by analyst Index
Actual_accuracy(b) Percentage of predictions in band b where component reached predicted RUL (±10% tolerance) EHM outcome matching Percentage
Stated_confidence(b) Midpoint of confidence band b RUL model output Percentage
B Number of confidence bands Defined by analyst Count
Overall_RCCI Mean absolute calibration error across all bands Derived Dimensionless

Overall_RCCI = 0 is perfect calibration. Track direction separately: RCCI(b) < 1.0 means actual accuracy is lower than stated confidence (overconfidence. The more dangerous direction in aviation, as it supports extended on-wing time with underestimated failure probability). RCCI(b) > 1.0 means actual accuracy exceeds stated confidence (underconfidence: costly in premature removals but not a safety exposure).

Proposed formula, pending validation pilots.

At a Glance

Reporting Frequency
Monthly or quarterly; longer windows required due to low frequency of engine removal events
Category
Predictive Maintenance Calibration
← Alert Economic Timing Index (AETI)Cargo-Passenger Payload Coherence Score (CPPCS) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment