Definition
Accuracy of ML condition monitoring severity classifications segmented by equipment family.
The Measurement Gap
What This Metric Measures
CMEFC measures the accuracy of ML condition monitoring severity classifications when segmented by equipment family: 2-stroke versus 4-stroke engines, water-tube versus fire-tube boilers, axial versus radial turbochargers. It computes the percentage of severity classifications that correctly matched the subsequent outcome within a defined running-hour window, revealing whether the diagnostic model applies correct baselines across the fleet's equipment diversity.
The Decision It Supports
Is the condition monitoring system correctly diagnosing severity for all equipment types in the fleet, or is it applying thresholds from one equipment family to another: causing false assurance on some equipment and false alarms on others? A vibration level that is normal for a 2-stroke slow-speed engine is a warning sign for a 4-stroke medium-speed engine. If the ML system applies the 2-stroke baseline to both, the chief engineer receives a normal classification for a 4-stroke bearing that is actually deteriorating.
Why Legacy Metrics Cannot See This
Condition monitoring accuracy is reported as an aggregate across all equipment. Vibration analysis trending shows the trend but not whether the severity threshold is appropriate for the specific equipment family. The monitoring system can show stable trends while applying the wrong baseline: the bearing looks healthy by 2-stroke standards while failing by 4-stroke standards. CMEFC decomposes accuracy by equipment family and reveals which families have calibrated diagnostics and which are receiving mismatched severity classifications.
Formula
Primary Formula
CMEFC(f) = Correct_classifications(f) / Total_classifications(f)
Variable Definitions
- Correct_classifications(f) = ML severity classifications for equipment family f where the outcome within N running hours matched the severity level (normal = no failure in 720 hours, warning = degradation in 720-2000 hours, critical = failure or emergency action in under 720 hours)
- Total_classifications(f) = all ML severity classifications for equipment family f in the measurement period
- f = equipment family (e.g., 2-stroke main engine, 4-stroke auxiliary engine, water-tube boiler, axial turbocharger, centrifugal purifier)
Units:
Percentage per equipment family. Compare across families to identify which have accurate diagnostics and which do not. The variance across families is as important as the absolute values.
Edge Cases
Outcome windows per severity level require clear definition: proposed: normal means no event within 720 running hours, warning means event between 720 and 2,000 hours, critical means event within 720 hours. Equipment families with few classified events should be aggregated over longer periods or combined with related sub-families. Minimum sample of 20 classifications per equipment family per measurement period. The most dangerous error type is classified-as-normal but failed-within-720-hours: track this specific false-negative rate separately.
Proposed formula, pending validation pilots.