Definition
OEM Recommendation Accuracy Rate (ORAR) measures what percentage of OEM-generated maintenance recommendations correspond to actual asset conditions confirmed by inspection or failure: measured on your fleet, not on the OEM's reference fleet.
The Measurement Gap
The fleet manager at a regional transit authority ran 340 diesel buses across seven depots. Three years ago, the authority signed a connected-fleet agreement with the engine manufacturer. The deal sounded like a no-brainer: the OEM's telematics platform would monitor engine health continuously, send maintenance recommendations directly into the CMMS, and the authority would follow those recommendations under a performance guarantee. Fewer unplanned failures. Lower lifecycle cost. Data-driven maintenance from the people who built the engine.
By year three, the fleet manager had a problem he could feel but could not prove. His parts spend on engines was up 22% despite a fleet that was three years younger on average than when the program started. His technicians were replacing turbochargers at intervals the OEM recommended but that his own shop data didn't support: the pulled turbos were consistently within acceptable tolerances. Injector replacements were running at roughly double the rate his pre-connected-fleet failure history suggested was necessary. When he challenged the OEM's regional service rep, the response was reasonable-sounding: "our models are trained on a global fleet of 180,000 vehicles, and we're seeing failure patterns your local data doesn't capture yet."
That answer is hard to argue with, and it might even be true. But the fleet manager had no way to test it. He had no metric that compared the OEM's recommendation rate against his own fleet's actual failure rate for the same components. He could see that parts spend was up. He could see that pulled components looked healthy. He could not connect those observations to a systematic measurement of whether the OEM's recommendations were accurate for his fleet or systematically conservative.
This is not a story about a bad OEM. The manufacturer's model may genuinely detect failure patterns that a 340-bus fleet doesn't generate enough data to see on its own. Or the model may be calibrated to a global fleet that includes buses running in conditions (extreme heat, altitude, fuel quality variation) that don't apply in the mid-Atlantic. Or the replacement intervals may include a commercial margin that serves the OEM's parts revenue alongside the operator's reliability. The fleet manager cannot distinguish between these explanations because he has no metric for it.
Here is why the existing metrics miss this entirely.
PM Compliance
tracks whether the authority followed the recommendations, and they did, faithfully, which is why compliance looks excellent and parts spend is climbing.
MTBF
on engines actually improved slightly, which the OEM points to as evidence the program is working. But MTBF would also improve if you replaced every component at half its useful life: you'd prevent failures by throwing away perfectly good parts. MTBF cannot distinguish between failures prevented by timely intervention and failures prevented by premature replacement.
Total Maintenance Cost
shows the spend increase, but finance attributes it to "inflation and supply chain conditions" because nobody has isolated the OEM recommendation accuracy as a variable.
Asset Availability
looks healthy: the buses are running. But availability doesn't tell you whether the maintenance that kept them running was necessary or whether you're buying availability with unnecessary parts replacements that a less conservative approach would not have required.
The measurement gap is precise: there is no metric that tells you whether OEM or vendor AI recommendations match your fleet's actual failure rates for the same components, which means you cannot distinguish between recommendations that protect your assets and recommendations that protect the OEM's revenue stream.
I want to be careful here. I am not saying OEMs are deliberately over-recommending maintenance. Some are conservative because liability exposure makes under-recommendation more costly than over-recommendation. Some are calibrated to fleets operating in harsher conditions than yours. Some have genuinely superior failure data. The point is not that OEMs are wrong: it is that you have no way to know, and "trust the OEM" is not a measurement strategy.
Formula
OEM Recommendation Accuracy Rate (ORAR) measures what percentage of OEM-generated maintenance recommendations correspond to actual asset conditions confirmed by inspection or failure: measured on your fleet, not on the OEM's reference fleet.
In plain language:
Count the OEM's recommendations. For each one, check whether the recommended action was actually warranted: did the component actually have the condition the OEM flagged? Divide confirmed-warranted by total recommendations. The result tells you how often the OEM is right about your assets specifically.
The formula:
ORAR = R_confirmed / R_total × 100
Where:
- R_confirmed = count of OEM recommendations where inspection findings, condition data, or subsequent failure confirmed the flagged condition existed within the recommended action window. "Confirmed" means a qualified technician or inspector verified that the component was degraded, damaged, or trending toward failure at the time the recommendation was received. A component pulled and found within acceptable tolerances is NOT confirmed, that is an unnecessary replacement.
- R_total = total OEM recommendations received in the evaluation period, excluding administrative or scheduling-only communications.
The companion metric matters as much as the primary:
OEM Conservatism Ratio (OCR):
OCR = OEM_interval / Operator_interval
Where OEM_interval is the average recommended replacement or service interval from the OEM for a specific component, and Operator_interval is the average actual run-to-replacement interval in the operator's own failure history for the same component. OCR above 1.0 means the OEM recommends more frequent replacement than your data supports. OCR of 1.5 means the OEM says replace at 50% more frequently than your failure data warrants.
Worked example:
The transit authority audits OEM turbocharger recommendations over 12 months. The OEM platform generated 47 turbocharger replacement recommendations across the 340-bus fleet. Each recommendation included a health score, a predicted remaining useful life, and a recommended action window.
The authority's reliability engineer reviewed the outcomes:
- 28 recommendations were followed. Of those, 11 showed confirmed degradation on teardown: scored, cracked turbine blades, measurable bearing wear beyond tolerance. 17 showed components within acceptable tolerances: the turbo was functional and would have continued operating.
- 19 recommendations were not followed (deferred or rejected by the fleet manager based on recent inspection history). Of those 19, 3 resulted in turbocharger failures within the OEM's predicted window: the OEM was right and the fleet manager was wrong. 16 experienced no turbocharger issues within the window or beyond: the fleet manager was right to defer.
R_confirmed = 11 (confirmed from followed) + 3 (confirmed from unfollowed that failed) = 14
R_total = 47
ORAR = 14 / 47 × 100 = 29.8%
The OEM's turbocharger recommendations are accurate less than 30% of the time on this fleet. Seven out of ten recommendations result in either unnecessary replacement or unnecessary investigation.
Now the conservatism ratio: the OEM recommends turbocharger service at an average interval of 18,000 engine hours. The authority's own failure data shows average turbocharger run-to-failure at 31,400 engine hours.
OCR = 18,000 / 31,400 = 0.57
The OEM is recommending replacement at 57% of the interval the authority's own fleet data supports. That conservatism (whether driven by liability, different fleet conditions in the training data, or commercial interest) is costing the authority 17 unnecessary turbocharger replacements per year at roughly $4,200 each. That is $71,400 per year on one component family from one OEM recommendation stream.
Edge cases:
When a recommendation was followed and no failure occurred, the determination hinges on inspection findings at time of action. A component pulled and found degraded beyond tolerance is R_confirmed. A component pulled and found within tolerance is not confirmed: even if it might have eventually failed. We are measuring whether the recommendation was warranted at the time it was made, not whether the component would have eventually needed attention.
When a recommendation was not followed and the asset did not fail within the window, it counts against ORAR: the OEM recommended action that was not needed. Track this population carefully, because these deferred recommendations are the counterfactual evidence the OEM cannot produce from its own data.
Minimum sample: 30 OEM recommendations with confirmed outcomes per component family before computing ORAR for that family. Component-level segmentation is essential: the OEM may be accurate on engines and inaccurate on turbochargers. A blended ORAR hides the actionable variation.
Proposed formula, pending validation pilots.