Definition
FMCA measures the percentage of auto-classified failure modes on condition-generated work orders that match the actual failure mode identified by the technician during repair. This metric validates whether the automated front door of the work management process is sending the right information downstream for planning, parts staging, and technician assignment.
The Measurement Gap
The Measurement Gap
Condition monitoring systems in renewable energy operations generate work orders automatically when sensor data indicates a developing fault. The system assigns a failure mode classification based on the sensor signature: vibration patterns suggest bearing degradation, temperature anomalies suggest electrical connection issues, power curve deviations suggest blade damage. This classification drives everything downstream: the planner assigns a technician with the right skill set, stages the predicted parts, and estimates the job duration based on the classified failure mode.
If the classification is wrong, the entire downstream chain is compromised. A work order classified as a bearing issue dispatches a mechanical technician with bearing replacement parts. The technician arrives, diagnoses the problem as an electrical connection fault, and must either return for the correct parts and tools or attempt a repair with inadequate preparation. The first-visit fix fails, a second trip is required, and the asset remains down longer than necessary.
Without FMCA, the organization trusts the classification without validation. Work orders are generated, dispatched, and completed. If the technician finds a different failure mode than classified, that information may be recorded in the work order completion notes but is not systematically compared against the original classification. The mismatch is treated as a normal part of maintenance rather than a failure of the classification system.
The Decision This Metric Supports
FMCA answers whether the planner can trust the AI's failure mode classification to assign the right technician, stage the right parts, and estimate the right duration. When FMCA is above 75%, the classification is useful for planning purposes. When FMCA drops below 60%, every auto-classified work order should be reviewed by the planner before execution planning begins.
Formula
Primary metric (miss rate on safety-critical failure modes):
MR_critical = Safety_critical_misclassified / Total_safety_critical
Secondary metric (overall weighted accuracy):
FMCA = (Full_Match + 0.5 x Partial_Match) / Total_Classified x 100
Why miss rate on safety-critical failure modes is primary: Misclassifying a non-critical failure mode as critical triggers conservative maintenance action but no safety gap. Misclassifying a safety-critical failure mode (structural fatigue, blade delamination, generator bearing degradation) as non-critical delays intervention on conditions that can cause catastrophic failure, turbine collapse, or blade throw. Set MR_critical below 5%; overall FMCA above 75%.
| Variable | Definition |
|---|---|
| Full_Match | Work orders where AI-classified failure mode exactly matched technician-confirmed actual failure mode |
| Partial_Match | Work orders where AI classification was in the right system/subsystem but wrong specific component (e.g., "gearbox bearing" when actual was "gearbox seal") |
| Total_Classified | All auto-classified work orders with confirmed actual failure modes in the period |
| Safety_critical_misclassified | Safety-critical failures the AI classified as non-critical categories |
| Total_safety_critical | All confirmed safety-critical failures in the period |
Measurement frequency: Monthly, with quarterly trend analysis.
Segmentation: Track by failure mode category. Electrical faults often achieve FMCA above 85% due to clear SCADA signatures. Mechanical faults may achieve only 45%.
Edge cases:
- No actual failure mode recorded by technician: track separately as data quality metric
- Multiple failure modes found: credit AI if its classification was among actual modes
- "No problem found" outcomes: count as mismatch (false positive)
- If "no actual recorded" rate exceeds 30%, FMCA itself becomes unreliable
Proposed formula, pending validation pilots.