RUL Calibration Score (RULCS)

GMI-MIN-018MiningIntegration-Required

Definition

Degree to which a predictive maintenance model RUL estimates correspond to actual component life achieved, measured by comparing predicted RUL at time of alert to actual hours remaining at time of removal, binned by confidence level.

The Measurement Gap

The Measurement Gap

Predictive maintenance platforms generate remaining useful life estimates for heavy mobile equipment components. Maintenance teams use these RUL numbers to decide whether to pull a component now, schedule it for the next shutdown, or continue monitoring. The critical question is whether those numbers are accurate, and specifically whether the accuracy varies by the model's own stated confidence level. Legacy metrics like MTBF and PM Compliance measure whether maintenance happens on schedule. They cannot measure whether the predictions driving that schedule are trustworthy. Physical Availability tells you whether the asset is running, not whether the prediction that kept it running was correct. A model that consistently overestimates remaining life (RULCS below 1.0) creates unexpected failures that legacy metrics attribute to "random breakdowns" rather than to systematic prediction error. A model that consistently underestimates remaining life (RULCS above 1.0) drives premature replacements that inflate Cost per Ton without any metric identifying the prediction as the cause. The consequence compounds across fleets of 50 to 200 haul trucks where each component decision carries a $50K to $500K price tag.

Decision Supported: Can the maintenance team trust the RUL number when deciding whether to pull a component now, schedule it for next shutdown, or continue monitoring, and is the model more reliable for some component types or operating conditions than others?

Formula

Plain language: For each component removal, look back at the most recent RUL prediction issued before the removal event. Compare the predicted remaining hours to the actual remaining hours (time from prediction to removal). Group by the model's confidence level and compute the ratio. A perfectly calibrated model produces a ratio of 1.0 at every confidence level.

Notation:

RULCS(b) = Mean(Actual_RUL) / Mean(Predicted_RUL) for predictions in confidence band b

Overall: RULCS_weighted = Sum(n_b x RULCS(b)) / Sum(n_b)

Variable Definition Unit
Predicted_RUL Model's stated remaining useful life in hours at time of alert Hours
Actual_RUL Actual hours of operation between the prediction timestamp and the component removal event Hours
b Confidence band (e.g., [60-70%), [70-80%), [80-90%), [90-100%]) Band
n_b Number of predictions in band b Count

Proposed formula, pending validation pilots.

At a Glance

Unit
Ratio
Trend Direction
Toward 1.0
Reporting Frequency
Monthly
Category
Predictive Model Integrity & Calibration
← Premature Replacement Rate (PRR-MIN)Severity-Urgency Alignment Score (SUAS) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment