Post-Intervention Model Relearning Index (PMRI)

GMI-AR-012Asset Reliability

Definition

Post-Intervention Model Relearning Index (PMRI) measures the number of days after a major maintenance intervention before the AI model's false alert rate on the affected asset returns to its pre-intervention baseline, and whether the relearned baseline is accurate.

The Measurement Gap

The paper mill had spent fourteen months building trust in its vibration analytics platform. The reliability engineer had calibrated the models, validated the alerts, and trained the operators to take the system seriously. By month fourteen, the team was acting on about 80% of the platform's recommendations, a healthy adoption rate. Then they did a major overhaul on Paper Machine 3's main drive.

New bearings. Rebalanced rotor. Replaced coupling. New seals throughout. The overhaul went well: the machine came back online running smoother than it had in two years. The vibration signature was fundamentally different. Lower amplitude across most frequency bands. Different harmonic pattern from the new coupling. Shifted resonance frequencies from the rebalanced rotor.

The analytics platform noticed immediately. It had spent fourteen months learning what Paper Machine 3's drive sounded like. Now it sounded completely different. Within 72 hours of restart, the platform generated nineteen alerts on the drive: elevated vibration deviation, abnormal spectral pattern, bearing frequency anomaly, coupling signature mismatch. Every alert was technically correct: the current signature deviated significantly from the learned baseline. Every alert was operationally wrong: the deviations were the new normal after a successful overhaul.

The shift supervisor, who had spent those same fourteen months learning to trust the system, did what trust produces: he mobilized a technician to investigate. The technician found nothing wrong. Then five more alerts came in overnight. Another investigation. Nothing wrong. By day four, the supervisor had stopped responding to alerts on that drive entirely. "The system is confused after the overhaul. Ignore it until it settles down."

On day eleven, a coupling alignment error from the overhaul (a genuine post-maintenance defect) produced a vibration alert that the supervisor ignored along with all the others. The misalignment progressed to coupling damage over the following week. The drive came down for emergency repair eighteen days after the overhaul. The analytics platform had detected the real problem on day eleven. Nobody was listening.

This is the post-intervention alert storm, and it plays out the same way everywhere I have seen major maintenance performed on assets with active AI monitoring. The AI model is trained on the pre-intervention signature. The intervention changes the signature. The model floods the operator with alerts that are mathematically valid and operationally meaningless. The operator suppresses alerts: either formally through a system suppression window or informally by ignoring them. Real post-maintenance defects get buried in the noise. Trust that took months to build evaporates in days.

No existing metric captures any part of this failure chain.

Asset Availability

records the emergency repair on day twenty-nine but cannot trace it back to an alert that was generated and ignored on day eleven.

MTBF

resets after the overhaul and starts counting from zero: it has no mechanism for measuring whether the model recovered from the intervention.

PdM Hit Rate

counts the nineteen false alerts as misses, which makes the model look bad in aggregate, but does not distinguish between normal false positives and post-intervention noise that every model will produce after every overhaul. The platform's own accuracy dashboard takes a hit that it should not be penalized for: the model did not degrade. The asset changed.

There is no metric that tells you how long after a major maintenance intervention the AI model takes to relearn the asset's new baseline, or whether the relearned baseline is correct.

That gap killed fourteen months of trust-building at the paper mill in less than two weeks.

Formula

Post-Intervention Model Relearning Index (PMRI) measures the number of days after a major maintenance intervention before the AI model's false alert rate on the affected asset returns to its pre-intervention baseline, and whether the relearned baseline is accurate.

In plain language:

After a major repair or overhaul, the AI model's alert accuracy drops because the asset's signature changed. Count the days until the false alert rate comes back down to where it was before the intervention. That number is PMRI. If PMRI is 7 days, the model recovers quickly. If PMRI is 45 days, you have six weeks of degraded monitoring on an asset that just had major work done: the exact window when post-maintenance defects are most likely to surface.

The formula:

PMRI = T_stabilize (days)

Where T_stabilize is the first day t after the intervention at which both conditions are met:

  1. FAR_t ≤ FAR_baseline × 1.2 (the false alert rate has returned to within 20% of the pre-intervention baseline)
  2. The condition in (1) holds for 7 consecutive days (the recovery is sustained, not a one-day dip)

The variables:

  • FAR_t = false alert rate on the affected asset at day t post-intervention. Computed as false alerts divided by total alerts on that asset on day t. A false alert is one where investigation or subsequent outcome confirms no actionable condition existed. This requires outcome determination for every alert during the relearning window, which is exactly the period when most organizations stop investigating because "the system is adjusting."
  • FAR_baseline = average false alert rate on the same asset in the 90 days prior to the intervention. This is your pre-intervention reference. If the asset had a FAR of 0.15 (15% of alerts were false) before the overhaul, the model has "relearned" when post-intervention FAR returns to 0.18 or below (within the 20% tolerance band).

The companion metric:

Relearning Accuracy Score (RAS):

RAS = 1 − |FAR_post_stable − FAR_baseline| / FAR_baseline

Where FAR_post_stable is the sustained false alert rate after PMRI is reached. RAS of 1.0 means the model returned to exactly its pre-intervention accuracy. RAS below 0.80 means the model stabilized at a different accuracy level than before: it learned something, but what it learned may not be the right baseline.

Worked example:

Paper Machine 3 main drive. Pre-overhaul FAR_baseline = 0.12 (12% of alerts were false positives over the prior 90 days, a well-performing model). Major overhaul completed on Day 0.

Days 1–3: 19 alerts generated, 18 are false (FAR = 0.95). The model is reacting to the new signature.
Days 4–7: 11 alerts, 9 false (FAR = 0.82). Alert volume is declining but accuracy is still terrible.
Days 8–14: 6 alerts, 4 false (FAR = 0.67). The model is starting to learn but is still far from baseline.
Days 15–21: 4 alerts, 2 false (FAR = 0.50). Improving but still four times the baseline rate.
Days 22–28: 3 alerts, 1 false (FAR = 0.33). Getting closer.
Days 29–35: 2 alerts, 0 false (FAR = 0.00). Suspiciously clean: sample size is too small for a stable estimate.
Days 36–42: 3 alerts, 0 false (FAR = 0.00). Still clean. But 0.00 for two weeks suggests the model may have over-learned: it has become too conservative and is now under-alerting.
Days 43–49: 4 alerts, 1 false (FAR = 0.25). Back above zero. Still above the 0.144 threshold (FAR_baseline × 1.2).
Days 50–56: 3 alerts, 0 false (FAR = 0.00). Fluctuating.
Days 57–67: Stabilizes with FAR averaging 0.11 across an 11-day window. Below the 0.144 threshold for 7+ consecutive days.

PMRI = 57 days.

That is almost two months of degraded AI monitoring on an asset that just had major work done. During those 57 days, the shift supervisor's informal suppression window was 7 days: he started ignoring alerts on day 4 and resumed paying attention around day 11. The real problem (coupling misalignment) appeared on day 11, during the gap between when the supervisor stopped listening and when the model recovered. PMRI of 57 days with a management suppression window of 7 days means there were 50 days of unmanaged model degradation.

RAS = 1 − |0.11 − 0.12| / 0.12 = 1 − 0.083 = 0.917

The model relearned accurately: its post-stabilization accuracy is close to its pre-intervention accuracy. The problem was not learning quality. It was learning speed.

Edge cases:

Model never stabilizes within 90 days: report as PMRI > 90 and escalate. The model architecture may not support post-intervention baseline shifts for this asset class: some models require manual retraining rather than autonomous adaptation.

Like-for-like component replacement (identical part, no signature change expected): PMRI should be near zero. If PMRI is elevated after a like-for-like replacement, the model is overfitting to individual component characteristics rather than asset-class patterns, that is a model architecture finding, not an intervention finding.

Multiple interventions on the same asset within the PMRI window: restart the clock from the most recent intervention. You cannot attribute alert behavior to one intervention when a second one has occurred.

Very low alert volume post-intervention (fewer than 2 alerts per week): FAR becomes statistically unreliable. Extend the stabilization window or aggregate across a cohort of recently-overhauled assets of the same class to establish a population-level PMRI.

Proposed formula, pending validation pilots.

At a Glance

Category
AI Data and Source Trustworthiness
← OEM Recommendation Accuracy Rate (ORAR)Prediction Confidence Calibration Index (PCCI-AR) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment