Definition
Estimated percentage of AI-recommended equipment replacements where the replaced equipment had significant remaining useful life, measured by post-removal testing or by comparison to the equipment class's typical end-of-life indicators. PRR includes a companion metric (Premature Replacement Cost (PRC)) that converts the rate into dollars to make the business case actionable.
The Measurement Gap
When a predictive model recommends equipment replacement and the recommendation is executed, the replaced equipment is typically disposed of without testing. Nobody checks whether the battery that was replaced still had 3 years of life remaining or the rectifier that was swapped was operating at 95% efficiency. The model's recommendation is assumed correct because the replacement was performed, a circular logic that makes the model unfalsifiable. Without post-removal testing, the organization cannot distinguish between a predictive model that saves money by preventing failures and one that wastes money by triggering replacements on healthy equipment.
The Decision
Is the predictive model saving money by preventing failures, or wasting money by triggering replacements on healthy equipment? The answer directly shapes model threshold calibration: if PRR exceeds 25% for any equipment class, the model's replacement threshold for that class should be raised (require lower predicted RUL before triggering a recommendation). Converting PRR to dollars via PRC makes the business case tangible: "30% premature replacement rate" is concerning, but "$2.1M of the $7M predictive replacement budget spent on healthy equipment" changes behavior.
Research Foundation
Post-removal testing of predictive-model-driven replacements in power infrastructure consistently reveals premature replacement rates between 15-35%, with the highest rates in equipment classes where the predictive model has limited training data or where degradation mechanisms are non-linear (EPRI: Evaluating Predictive Maintenance Program Effectiveness, 2024). Battery replacement programs that adopted post-removal load testing found that 22% of replaced battery strings had remaining capacity exceeding 80% of rated capacity: the model was replacing healthy equipment at scale (Battcon International Conference Proceedings, 2023).
Formula
PRR = Premature_replacements / Total_AI_recommended_replacements × 100
Companion metric: Premature Replacement Cost:
PRC = Σ (Remaining_value_i × Replacement_cost_i) for all premature replacements
Variable Definitions (every variable resolved)
- Premature_replacements = AI-recommended replacements where post-removal testing shows the equipment had remaining useful life exceeding a threshold (e.g., > 25% of rated life remaining) (count)
- Total_AI_recommended_replacements = all equipment replacements triggered by the predictive model's recommendation (count)
- Remaining_value_i = estimated remaining useful life as a fraction of total rated life, based on post-removal testing (ratio)
- Replacement_cost_i = cost of the replacement (parts + labor + disposal) (dollars)
Units:
PRR is a percentage. PRC is a dollar amount. Together they answer "how often" and "how much."
Edge Cases
Post-removal testing not feasible (equipment damaged during removal, no test facility available): use the equipment's operating metrics at time of removal (battery impedance, rectifier efficiency, generator start reliability) compared to end-of-life threshold. Less precise but still informative. Replacements driven by both model recommendation and other factors (capacity upgrade, technology refresh): exclude. These are not purely model-driven decisions. Battery cells that test well individually but had a failing cell in the string: classify at the replacement-unit level (string, not cell). Minimum sample: 30 AI-recommended replacements per equipment class per measurement period.
These are proposed starting points: calibrate to your operational context during pilot: PRR below 15% means most AI-recommended replacements are justified. Above 30% means nearly a third of predictive replacements are premature. If PRR exceeds 25% for any equipment class, raise the model's replacement threshold for that class.
Proposed formula, pending validation pilots.