Fleet Model Accuracy Segmentation (FMAS)

GMI-MAR-018Maritime IndustriesIntegration-Required

Definition

Predictive model accuracy per vessel segment identifying which segments get reliable predictions.

The Measurement Gap

What This Metric Measures

FMAS measures predictive maintenance model accuracy per vessel segment (by vessel type, age, engine type, operating profile, and similarity to the training population), identifying which segments receive accurate predictions and which receive unreliable ones. It computes both precision and recall per segment, revealing where the model works well and where it is effectively blind.

The Decision It Supports

Which vessels in the fleet should trust the predictive maintenance model's recommendations, and which should discount them because the model was not trained on their type? A PdM vendor may report 85% fleet-wide accuracy, but if that average masks 95% accuracy on the 5-year-old bulk carriers that dominated the training data and 40% accuracy on the 20-year-old tankers the vendor has never seen, the fleet technical director is making deferral decisions on those tankers based on a model that is essentially guessing.

Why Legacy Metrics Cannot See This

Traditional PdM accuracy is reported as a fleet-wide average. MTBF and failure rate metrics are computed across the fleet or by equipment type but never segmented by vessel characteristics that determine model applicability. A fleet-wide accuracy of 80% tells the fleet technical director nothing about whether the model works on their specific vessel types. FMAS replaces the misleading fleet average with a segment-by-segment accuracy map that reveals exactly where the model is reliable and where it is not.

Formula

Primary Formula

FMAS(s) = (Precision(s) + Recall(s)) / 2

Where:
Precision(s) = true_positive_predictions(s) / total_positive_predictions(s) for segment s
Recall(s) = true_positive_predictions(s) / total_actual_failures(s) for segment s

Variable Definitions

  • true_positive_predictions(s) = failures correctly predicted by the model for vessels in segment s (count)
  • total_positive_predictions(s) = all failure predictions issued by the model for segment s, including false alarms (count)
  • total_actual_failures(s) = all actual equipment failures that occurred in segment s, whether predicted or not (count)
  • s = vessel segment defined by vessel type, age band, engine family, or trade route pattern

Units: 0 to 1 per segment. A value of 1.0 means perfect prediction accuracy for that segment. Compare across segments to identify where the model works and where it fails.

Edge Cases

Segments with few failure events create statistical noise: aggregate over 12+ months or combine with similar segments to achieve minimum sample sizes. If the model never predicts failures for a segment (precision undefined), the model is effectively blind to that segment and should be flagged as a critical gap. Minimum sample of 10 predicted events per segment per measurement period.

Proposed formula, pending validation pilots.

At a Glance

Unit
Score (0 to 1)
Trend Direction
Higher
Reporting Frequency
Quarterly per fleet segment
Category
Predictive Maintenance and Deferral Risk
← Deferral Concentration Index (DCI)Operating Condition-Adjusted RUL Accuracy (OCRA) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment