Definition
The Autonomous Work Order Signal-to-Noise Ratio measures what your predictive maintenance system is actually producing: useful work or noise. The formula is straightforward.
The Measurement Gap
The maintenance planner at a pulp and paper mill arrived Monday morning to 47 new work orders in her queue. Twenty-three of them had been generated overnight by the plant's predictive maintenance system: flagged conditions on pumps, drives, and conveyor bearings across three production lines. She started triaging. By 10 AM she had cancelled nine as duplicates of work already sitting in the backlog. Four more targeted assets scheduled for decommissioning in the next planned shutdown. Two called for inspections on equipment her crew had physically checked the previous Friday with no findings. Three recommended interventions during a production run that couldn't be interrupted for another six weeks. That left five work orders that warranted real action.
She spent three hours on a Monday morning sorting machine-generated noise from actual work. The machine gave her 23 work orders. Five of them were worth doing.
Here's what your current metrics tell you about that morning: nothing useful.
PM Compliance rewards her if she executes all 23. The metric doesn't distinguish between a technician who closes a work order on a bearing that was genuinely failing and one who closes a work order on equipment that was already decommissioned. Compliance is compliance. A planner who dutifully executes every machine-generated work order (no matter how redundant or impractical) scores better on PM Compliance than a planner who reads the situation, cancels 18 intelligently, and sends her crew after the five that actually matter. The metric punishes judgment.
Schedule Compliance is equally blind. It measures whether you executed what you planned, not whether what you planned was worth executing. If the schedule included 23 machine-generated work orders and your crew touched all 23 (even the ones for assets being torn down in eight weeks) you hit your number.
Backlog size captures a symptom. When machine-generated work orders flood the queue, backlog grows. But backlog size can't tell you whether that growth represents real deferred work or accumulated noise. A backlog of 200 with a 70% signal rate is a fundamentally different problem than a backlog of 200 with a 20% signal rate, and backlog size treats them identically.
Work Order Completion Rate has the same blind spot. Completing a no-fault-found work order and completing a work order that prevented a bearing failure both count as completions. The metric can't see the difference.
The problem isn't that AI-driven maintenance systems generate work orders. That's the point. The problem is what happens to planner judgment, technician trust, and maintenance effectiveness when the ratio of noise to signal climbs without anyone measuring it.
Alarm fatigue is a well-documented failure mode in high-stakes industrial and clinical environments. When the proportion of false alerts climbs high enough, trained professionals start systematically discounting alerts: including the ones that matter. A predictive maintenance system that generates work orders at a 20% signal rate isn't a productivity tool. It's a noise machine slowly teaching your planners to ignore the queue.
There's no organizational defense against that erosion as long as you can't measure it. You can't tune a system you can't see. You can't hold a model accountable for the quality of its outputs if the only thing you're tracking is how many outputs it produced.
There is no metric that tells you what proportion of machine-generated work orders are value-adding vs. noise.
That's the gap this chapter addresses.
Formula
The Autonomous Work Order Signal-to-Noise Ratio measures what your predictive maintenance system is actually producing: useful work or noise. The formula is straightforward. AWSNR = WO_va / WO_total × 100WO_va is the count of machine-generated work orders that result in a value-adding maintenance action during the evaluation period. WO_total is the count of all machine-generated work orders in the same period, excluding any still open at evaluation time. The result is a percentage. Higher is better. A score of 70% means seven of every ten machine-generated work orders produced real maintenance value. A score of 22% means your planners are spending roughly four hours on noise for every hour they spend on work that matters.
What Counts as Value-Adding
A machine-generated work order is value-adding if it results in one of three outcomes: a confirmed condition found (the problem the model predicted was actually there), a defect corrected (a real defect was found and fixed, even if not exactly what was predicted), or a failure prevented (the work stopped a failure that would otherwise have occurred). The trigger does not have to be precisely right. The work has to produce a real result. A work order is noise if it is cancelled as a duplicate of existing backlog, closed as found-no-fault, voided because the asset is operationally impractical to work on in the planning horizon, or withdrawn because the asset is decommissioned or removed from service. These are all forms of the system generating a recommendation that the organization cannot or should not act on.
Worked Example
Consider a 90-day evaluation period at a chemical plant with an active AI work order generation system. The system generated 312 work orders. At evaluation time, 11 are still open: active jobs not yet completed. Those 11 are excluded from the calculation and reported separately as pending outcome. That leaves 301 classified work orders. Breaking them down: 147 resulted in value-adding action (confirmed conditions found, defects corrected, or failures prevented). 58 were closed as found-no-fault: the technician inspected the asset and found nothing. 41 were duplicates of work already in backlog or already completed. 33 were classified as operationally impractical: real work, wrong timing, no access window in the planning horizon. 22 had overlapping scope with other active work orders. AWSNR = 147 / 301 × 100 = 48.8%
That is a failing score. At 48.8%, the system is generating more noise than signal. The thresholds are: 70% or above is acceptable performance; 50–70% means the model or generation rules need tuning; below 50% means the system is generating more noise than signal and autonomous work order generation should be suspended until the root cause is identified and corrected. At 48.8%, this plant is one review cycle away from needing to shut the autonomous generation off entirely.
Edge Cases
Work orders still open at evaluation time: exclude them from both numerator and denominator. Report the pending count separately. Including them would distort the ratio, an open work order has not produced an outcome yet, so classifying it either way is a guess. Partially value-adding work orders: classify as value-adding. If a model predicted a bearing failure and the technician found a seal leak instead (different problem, different fix) that work order still produced value. The trigger was imperfect; the outcome was real. Track trigger accuracy separately if you want to improve model precision, but do not penalize the work order for being approximately right in a useful direction. Human-modified work orders: classify based on final outcome, not original machine-generated content. If a planner rewrites a machine-generated work order significantly before issuing it, the outcome still counts toward AWSNR, but track modification rate separately. High modification rates tell you the model is generating work orders that require significant human correction before they are actionable, which is a quality signal worth watching even when the eventual outcome is value-adding. Zero machine-generated work orders: AWSNR is not computable. If your system generated no machine-originated work orders in the evaluation period (because the system was offline, the program is just starting, or generation was suspended) report the metric as not applicable, not as zero. Zero and not-computable are different organizational conditions.
Cross-References
The upstream drivers of AWSNR are the quality of the models generating work orders and the quality of the data feeding those models. Prediction Confidence Calibration Index (GMI-AR-017) measures model calibration: miscalibrated models generate noisy work orders at predictable rates, and a declining PCCI will show up in AWSNR before planners start complaining about it. Model Drift Detection Rate (GMI-AR-018) measures how fast a model's prediction accuracy is changing over time, a model with a negative MDDR is producing increasingly unreliable predictions, which translates directly into increased noise in the work order queue. Failure Code Fidelity Rate (GMI-AR-001) affects this because miscoded historical failures generate misguided work orders, a model trained on bad outcome data learns the wrong patterns. Inspection Detection Accuracy Rate (GMI-AR-002) matters because inaccurate inspection records trigger false work orders when the model interprets inspection noise as a real condition change. Autonomy Boundary Calibration Score (GMI-AR-006) is the final upstream link. If the boundary between human and AI authority is too permissive, the system generates autonomous work orders for conditions it shouldn't be acting on independently, directly degrading the signal-to-noise ratio. The downstream consequences of a low AWSNR are measurable. Operator Trust Calibration Score (GMI-AR-021) captures what happens when planners and operators lose confidence in a system that consistently generates noise, that erosion of trust is real and it affects how the organization responds to future alerts, including genuine ones. Prediction Threshold Value Ratio (GMI-AR-004) makes the financial cost explicit: every noise work order in your denominator is a direct false positive cost, and PTVR translates that into dollars your organization can actually act on.
Proposed formula, pending validation pilots.
At a Glance
Decision Supported
Is the autonomous work order system generating useful work, or is it flooding my planners with noise that buries real needs?