Definition
Predictive model accuracy per vessel segment identifying which segments get reliable predictions.
The Measurement Gap
What This Metric Measures
FMAS measures predictive maintenance model accuracy per vessel segment (by vessel type, age, engine type, operating profile, and similarity to the training population), identifying which segments receive accurate predictions and which receive unreliable ones. It computes both precision and recall per segment, revealing where the model works well and where it is effectively blind.
The Decision It Supports
Which vessels in the fleet should trust the predictive maintenance model's recommendations, and which should discount them because the model was not trained on their type? A PdM vendor may report 85% fleet-wide accuracy, but if that average masks 95% accuracy on the 5-year-old bulk carriers that dominated the training data and 40% accuracy on the 20-year-old tankers the vendor has never seen, the fleet technical director is making deferral decisions on those tankers based on a model that is essentially guessing.
Why Legacy Metrics Cannot See This
Traditional PdM accuracy is reported as a fleet-wide average. MTBF and failure rate metrics are computed across the fleet or by equipment type but never segmented by vessel characteristics that determine model applicability. A fleet-wide accuracy of 80% tells the fleet technical director nothing about whether the model works on their specific vessel types. FMAS replaces the misleading fleet average with a segment-by-segment accuracy map that reveals exactly where the model is reliable and where it is not.
Formula
Primary Formula
FMAS(s) = (Precision(s) + Recall(s)) / 2
Where:
Precision(s) = true_positive_predictions(s) / total_positive_predictions(s) for segment s
Recall(s) = true_positive_predictions(s) / total_actual_failures(s) for segment s
Variable Definitions
- true_positive_predictions(s) = failures correctly predicted by the model for vessels in segment s (count)
- total_positive_predictions(s) = all failure predictions issued by the model for segment s, including false alarms (count)
- total_actual_failures(s) = all actual equipment failures that occurred in segment s, whether predicted or not (count)
- s = vessel segment defined by vessel type, age band, engine family, or trade route pattern
Units: 0 to 1 per segment. A value of 1.0 means perfect prediction accuracy for that segment. Compare across segments to identify where the model works and where it fails.
Edge Cases
Segments with few failure events create statistical noise: aggregate over 12+ months or combine with similar segments to achieve minimum sample sizes. If the model never predicts failures for a segment (precision undefined), the model is effectively blind to that segment and should be flagged as a critical gap. Minimum sample of 10 predicted events per segment per measurement period.
Proposed formula, pending validation pilots.