Definition
The Override Feedback Degradation Rate, or OFDR, measures the statistical relationship between how much your system's recommendation accuracy has changed and how high your cumulative override rate has climbed over the same period.
The Measurement Gap
A large airport maintenance team deployed a predictive system across 23 kilometers of baggage handling conveyors. The system read motor current signatures, vibration sensors, and historical failure records, and it came out of the gate performing well: 79% recommendation accuracy in year one. The team followed about 70% of those recommendations. The 30% they overrode were not arbitrary: sometimes the system flagged a belt drive replacement during peak travel season when taking equipment offline was operationally impossible. Sometimes a recommendation conflicted with a planned shutdown already on the calendar. The overrides were defensible in isolation. Nobody flagged them as a systemic problem.
Eighteen months later, accuracy had dropped to 68%. The data science team retrained the model and recovered to 73%. Three months after that it was back at 69%. They retrained again. The same pattern repeated. The team was spending more time retraining and getting less performance back each cycle.
The deeper investigation revealed what was actually happening. The system was designed to learn from outcomes. When the team acted on a recommendation and the part either held or failed, the system logged that result and updated its model. When the team overrode a recommendation, the outcome was either handled by other means, deferred, or occurred without any link back to the original prediction. The system saw nothing. Those 30% of overridden cases simply disappeared from the learning loop.
That matters because the overridden cases were not random. They were disproportionately the predictions where the system was uncertain, where operational context made the recommendation less applicable, or where the underlying condition was ambiguous. In other words, the team was overriding exactly the cases the system most needed to learn from. The model kept training on the 70% of cases it could observe (the easier, cleaner predictions) and became increasingly calibrated to those. Its ability to handle the harder, context-dependent cases deteriorated. The model was not drifting because of bad data coming in. It was drifting because good data was being systematically withheld.
Here is where the existing metrics fail. PdM Hit Rate showed declining accuracy but could not tell you why. Model drift dashboards flagged the degradation and labeled it "data shift": technically accurate but pointing nowhere useful. Retraining frequency told you how often you were hitting the reset button, not whether the resets were becoming less effective. Override rate told you what percentage of recommendations the team was rejecting, but it said nothing about whether that rejection pattern was corrupting the learning loop. You could watch all four of those metrics every week and still miss what was happening.
The airport's data science team eventually reconstructed the causal chain through a retrospective analysis that took weeks. They had no real-time signal. They had no threshold that would have triggered an alert at month six when the pattern was still correctable. By the time they understood the mechanism, they were already in a degradation spiral: lower accuracy produced more operationally difficult recommendations, which produced more overrides, which further starved the model of training data, which pushed accuracy lower still.
That spiral is the failure mode OFDR is built to detect. The measurement gap is precise: there is no metric that tells you whether repeated overrides of a prescriptive system are causing the system's future recommendations to degrade. Override rate and model accuracy both exist as standalone numbers. The relationship between them over time (whether one is driving the other) is not measured anywhere in a standard AI-enabled maintenance stack. That gap is what Override Feedback Degradation Rate closes.
Formula
The Override Feedback Degradation Rate, or OFDR, measures the statistical relationship between how much your system's recommendation accuracy has changed and how high your cumulative override rate has climbed over the same period.
The Formula
OFDR = Corr(ΔAcc_t, CumOR_t)
Where ΔAcc_t is the change in system recommendation accuracy at time t relative to the baseline accuracy at deployment, CumOR_t is the cumulative override rate up to time t (calculated as total overrides divided by total recommendations since deployment) and Corr is a Pearson or Spearman correlation coefficient computed across your time series of monthly observations. The sign convention is the key to reading OFDR correctly. A result near zero means overrides and accuracy are moving independently of each other: the system is either not learning from outcomes at all (in which case overrides are irrelevant to performance) or the feedback loop is healthy enough that override patterns are not distorting the training data. A positive correlation is the desired state: it means accuracy is improving as overrides accumulate, which is the signature of a well-functioning human-in-the-loop retraining mechanism where operator judgment is making the model better. A negative OFDR is the problem signal. OFDR below −0.3 indicates material degradation: overrides are measurably hurting recommendation quality. OFDR below −0.6 indicates severe feedback loop damage and requires immediate structural intervention.
Worked Example
The baggage handling system from Section 1 illustrates how to compute this in practice. Tracking monthly accuracy and cumulative override rate across 18 months produces the following series:Month 1: accuracy 0.79, cumulative override rate 0.28, ΔAcc 0.00
Month 3: accuracy 0.77, cumulative override rate 0.29, ΔAcc −0.02
Month 6: accuracy 0.75, cumulative override rate 0.30, ΔAcc −0.04
Month 9: accuracy 0.72, cumulative override rate 0.31, ΔAcc −0.07
Month 12: accuracy 0.69, cumulative override rate 0.32, ΔAcc −0.10
Month 15: accuracy 0.68, cumulative override rate 0.30, ΔAcc −0.11
Month 18: accuracy 0.67, cumulative override rate 0.31, ΔAcc −0.12
Computing the Pearson correlation between the ΔAcc column and the CumOR column yields OFDR = −0.74. That number tells you that as the cumulative override rate persisted in the 29–32% range, system accuracy declined in lockstep. The correlation is strong enough to rule out coincidence. The override pattern is the mechanism.
Edge Cases
Three edge cases require specific handling. First, low override rate. If your team overrides fewer than 5% of recommendations, the cumulative override rate will barely move over time. There is not enough variation in CumOR_t to produce a meaningful correlation. Report this state as "insufficient override variance" rather than computing a near-zero OFDR that looks healthy but is simply uninformative. The measurement prerequisite is a sustained override rate above roughly 15% to generate the variation the correlation needs. Second, systems designed to learn from overrides. Some human-in-the-loop architectures are explicitly built to incorporate override events into retraining: the operator's rejection of a recommendation is itself a training signal. In those systems, you expect OFDR near zero or positive. If you compute a negative OFDR on a system with explicit override-to-retraining integration, that is not a healthy result with a confusing sign: it means the retraining mechanism is not working. The integration is broken. Negative OFDR on a learning-from-overrides system is a system architecture failure, not an operator behavior problem. Third, confounding data drift. If your override rate spiked during a period when your equipment fleet was also changing significantly (new assets introduced, major process changes, sensor replacements) then data drift and override-induced degradation may be occurring simultaneously. OFDR cannot distinguish between them from the aggregate numbers alone. Segment your analysis into known drift periods and stable periods. If OFDR is strongly negative in stable periods where no major fleet or process changes occurred, the override mechanism is the driver. If OFDR is negative only during drift periods, the accuracy decline may be primarily environmental.
Cross-References
OFDR sits at a specific point in the causal chain of AI-enabled maintenance measurement. Upstream, the Override Outcome Index (OOI, GMI-AR-020) captures whether individual overrides are producing better outcomes than the system recommended, that override behavior directly shapes the feedback loop OFDR measures. The Operator Trust Calibration Score (OTCS, GMI-AR-021) captures whether operator trust is calibrated to actual system accuracy; when trust is miscalibrated low, override rates climb, and OFDR begins tracking whether that elevated override rate is damaging learning. Downstream, the Model Drift Detection Rate (MDDR, GMI-AR-018) captures model drift as it manifests: MDDR shows you that the model is changing; OFDR explains why. The Autonomous WO Signal-to-Noise Ratio (AWSNR, GMI-AR-005) captures the downstream operational consequence of degraded recommendations appearing as noisier, less reliable work orders. The death spiral OFDR is designed to detect runs through all of these metrics. Lower accuracy produces recommendations that operators find less credible or less operationally viable, which produces more overrides, which starves the model of training data in precisely the areas where it is already struggling, which pushes accuracy lower. Each cycle of the spiral is harder to break than the last. OFDR detects the spiral by measuring whether accuracy and override rate are moving together in the wrong direction: before the relationship becomes irreversible. This is why I classify OFDR as a terminal learning-loop metric. It is the last signal before the feedback mechanism fails permanently.
Proposed formula, pending validation pilots.
At a Glance
Decision Supported
Is our pattern of overriding the AI system causing it to give worse recommendations over time? Are we unknowingly sabotaging our own tool?