Strategy Override Safety Index (SOSI)

GMI-AR-008Asset ReliabilityResearch-Forward

Definition

The Strategy Override Safety Index answers a specific question: when an algorithm recommends changing a maintenance strategy, what percentage of the failure modes the previous strategy addressed does the new strategy still cover?The

The Measurement Gap

The copper mine's reliability team was doing everything right. They had completed a full RCM analysis on their haul truck fleet (47 trucks running 24 hours in extreme ambient temperatures) and the engine cooling system had received particular attention. The previous strategy was thorough: 14 tasks, developed through documented failure mode analysis, covering everything from weekly coolant sampling to a triennial thermostat replacement. Each task traced back to a specific failure mode. The strategy had been built by engineers who understood that some failures do not announce themselves with rising trend lines. Some failures simply wait.

Then the AI-assisted strategy recommendation came in. The algorithm had ingested 8 years of work order history, maintenance cost records, and condition monitoring data. Its recommendation: eliminate 5 tasks as "non-value-adding." The analysis was coherent on its face. Two eliminated tasks were genuinely redundant: vibration analysis and oil sampling were together covering a failure mode that either one could catch. One task had never found a defect in its entire history. Two tasks addressed failure modes with zero recorded occurrences across the entire 8-year fleet history.

The reliability manager reviewed the recommendation. The elimination rationale was documented. Cost modeling showed savings. The manager approved the change.

PM Compliance improved. Fewer PMs in the schedule means compliance rates go up, that is arithmetic, not performance. Total maintenance cost decreased. The RCM completion rate showed 100% complete for the updated strategy. Every dashboard metric moved in the right direction. The reliability team looked like it was executing well.

Fourteen months later, a thermostat housing failure on Truck 31 caused thermal runaway. The engine reached temperatures it was never designed to sustain. The damage was catastrophic: $380,000 for the engine alone, plus 19 days of lost production on a high-value unit. The investigation traced directly back to the eliminated triennial thermostat replacement task.

Here is what the algorithm had seen: zero thermostat housing failures in 8 years. Zero defects found on any thermostat inspection. The task was expensive, required a planned shutdown, and had never produced evidence of value. From a historical failure data perspective, it looked exactly like waste.

What the algorithm could not see: the task had produced zero failures precisely because it existed. The triennial replacement was the protective barrier against a failure mode that, when it occurs, occurs catastrophically and without warning. The thermostat housing does not degrade gradually in ways that condition monitoring can detect. It fails when it fails. The time-based replacement task was not addressing a gradual failure pattern: it was addressing a failure mode that is invisible until the moment it is not.

PM Compliance cannot tell you this. PM Compliance measures whether scheduled tasks are being completed on time. It has no mechanism for tracking whether the scheduled tasks still cover the failure modes the previous strategy was designed to address. When you eliminate tasks to improve compliance rates, you are measuring a smaller denominator. The metric improves. The protection disappears.

RCM completion rate measures whether an RCM analysis was performed and documented. The mine had a 100% RCM completion rate. The analysis was complete. The strategy change was documented. The coverage gap was invisible in every metric the organization tracked.

Strategy approval rate (whether proposed changes go through a formal review process) does not evaluate the technical adequacy of those changes. The approval happened. The documentation was in order. The process was followed. The failure mode lost its protection.

There is no metric that tells you whether an algorithmically recommended strategy change has removed protection against rare catastrophic failure modes.

That gap is what the Strategy Override Safety Index addresses. When an algorithm recommends changing from time-based PM to condition-based monitoring, or eliminating tasks based on historical failure data, there must be a systematic check: did the new strategy explicitly cover every failure mode the old strategy addressed? Because the most dangerous failure modes are precisely the ones with no historical record: the ones that were being prevented.

Formula

The Strategy Override Safety Index answers a specific question: when an algorithm recommends changing a maintenance strategy, what percentage of the failure modes the previous strategy addressed does the new strategy still cover?

The Formula

SOSI = FM_x / FM_t × 100FM_t is the total number of failure modes addressed by the previous maintenance strategy, including every low-frequency catastrophic mode documented in your FMEA or RCM records. FM_x is the number of those failure modes that are explicitly or equivalently covered by the new recommended strategy, verified through a formal crosswalk. Multiply by 100 and you have a coverage percentage. The thresholds are not negotiable on critical assets. SOSI = 100% is required before implementing any algorithmically recommended strategy change on a critical asset. For non-critical assets, ≥95% is acceptable with documented risk acceptance, meaning someone with authority has signed off on the specific failure modes in the gap, understands the consequence if those modes occur, and has made a conscious decision to accept that exposure. Below 90%, you have significant coverage gaps that require strategy redesign before implementation, regardless of asset criticality. SOSI is a terminal safety metric. It has no downstream KPI. Its output is a binary go/no-go decision on strategy changes. You run the crosswalk, you get a number, and that number either clears the change or stops it.

Worked Example

Return to the haul truck engine cooling system. The previous strategy addressed 11 distinct failure modes through 14 tasks. Some tasks covered the same failure mode with different detection methods, that is intentional redundancy, not waste. The algorithm's new recommended strategy retains 9 tasks and eliminates 5.The FMEA crosswalk requires going through all 11 failure modes from the previous strategy and determining whether the new strategy covers each one. Ten of the eleven check out. The algorithm correctly identified that two tasks were redundant for a single failure mode, and the retained CM coverage is sufficient. Several other failure modes are covered by tasks that remain in the new strategy unchanged. Failure mode 11 (thermostat housing failure leading to thermal runaway) has no coverage in the new strategy. The only task that addressed it was the triennial thermostat replacement, and that task was eliminated. No condition monitoring task in the new strategy is capable of detecting thermostat housing distress within the P-F interval for this failure mode. SOSI = 10/11 × 100 = 90.9%.On a non-critical asset, 90.9% might pass with documented risk acceptance. On a haul truck engine operating in extreme ambient temperatures, where thermostat housing failure produces immediate catastrophic damage with no warning, 90.9% does not meet the threshold for critical assets. The strategy change cannot be implemented as recommended. The reliability team needs to either restore the triennial replacement task or demonstrate (with P-F interval data) that condition monitoring can detect thermostat housing distress in time to intervene. That last requirement is where most arguments fail under examination. "We can cover that with thermal imaging" is not an equivalent coverage claim until you have verified that thermal imaging detects thermostat housing distress (not just thermostat valve position drift) within the P-F interval. The coverage claim must be specific to the failure mode, not generic to the monitoring technology.

Edge Cases

Failure modes with no recorded occurrence must still be counted in FM_t. This is the most important edge case, and it is counterintuitive. If a failure mode appears in your FMEA or RCM records, it counts: regardless of whether it has ever occurred in your fleet history. Zero historical occurrences means either the failure mode has been successfully prevented, or you have not operated long enough to encounter it, or your failure codes are too coarse to capture it. None of those are reasons to exclude it from the crosswalk. In fact, the failure modes with no recorded occurrence are the most dangerous ones to lose coverage on, because you have no experience managing the consequences when they occur. Multiple sequential strategy changes must be evaluated cumulatively. If an algorithm recommended change A eighteen months ago, and now recommends change B, you need to crosswalk both changes against the original baseline strategy. Coverage gaps compound. A failure mode that was 80% covered after change A may be 0% covered after change B. Evaluating each change in isolation misses the cumulative drift. No FMEA or RCM records means SOSI cannot be computed. Full stop. If you do not have a documented failure mode inventory for the previous strategy, you cannot verify that the new strategy covers the same failure modes, because you do not know what those failure modes were. In this situation, the correct response is to flag the proposed change as an "unvalidated strategy change" and require completion of an FMEA before any algorithm-recommended changes are implemented. Running SOSI without an FMEA is not a degraded version of the metric: it is not the metric at all.

Cross-References

SOSI depends on two upstream metrics. FCFR (the Failure Code Fidelity Rate from GMI-AR-001) measures whether failure codes in your EAM accurately reflect actual failure modes. Corrupted failure codes invalidate the FMEA crosswalk because the algorithm's recommendations are trained on work order history, and if that history mislabels failure modes, the algorithm cannot correctly identify which failure modes its recommended changes affect. SOSI at 100% means nothing if the underlying failure codes do not map to real failure modes. DPVR (the Degradation Projection Validity Rate from GMI-AR-007) measures whether condition-based coverage claims are reliable. When a strategy change substitutes CM for time-based PM, the coverage equivalence argument depends on the CM system's ability to detect the failure mode within the P-F interval. If DPVR is low, those projection claims are unreliable, and what looks like equivalent coverage in the crosswalk may not be equivalent in practice. A failed crosswalk from bad projections is still a failed crosswalk.

Proposed formula, pending validation pilots.

At a Glance

Category
Autonomous Action and Strategy Integrity

Decision Supported

Did the algorithm's recommended strategy change remove a protective task without knowing it? Am I trading visible PM compliance for invisible catastrophic risk?

← Risk-Weighted Inspection Coverage Index (RWICI-AR)Autonomous WO Signal-to-Noise Ratio (AWSNR) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment