Definition
Degree to which the voyage optimizer stated confidence corresponds to actual route outcome quality.
The Measurement Gap
What This Metric Measures:
Degree to which the voyage optimizer's stated confidence level for a route recommendation corresponds to the actual outcome quality at that confidence level. VOCC groups completed voyages by the optimizer's stated confidence band (80-85%, 85-90%, 90-95%, 95%+) and measures whether the actual percentage of successful outcomes within each band matches the stated confidence. Perfect calibration means that when the optimizer says it is 90% confident, 90% of voyages in that band achieve outcomes within acceptable deviation.
The Decision It Supports:
Can the master trust the optimizer's confidence score when deciding whether to follow, modify, or override a route recommendation, or is the system systematically overconfident? If the optimizer reports 95% confidence but only 70% of those voyages achieve acceptable outcomes, the master is making decisions on unreliable confidence information.
Why Legacy Metrics Cannot See This:
Voyage optimization platforms report average fuel savings and recommendation acceptance rates. They do not report whether their confidence scores are calibrated: whether the stated probability of a good outcome matches the observed frequency. A system that is 90% confident on every recommendation and achieves 70% success has a calibration gap of 0.78, meaning the master should mentally discount every confidence score by 22%. Legacy reporting shows high average confidence and reasonable success rates without revealing the systematic overconfidence that distorts the master's decision-making.
Formula
Primary Formula:
VOCC(b) = Actual_success_rate(b) / Stated_confidence_midpoint(b)
Variable Definitions:
b: confidence band (for example, 80-85%, 85-90%, 90-95%, 95%+).
Actual_success_rate(b): percentage of voyages in band b where the route outcome was within acceptable deviation of optimal, defined as within 5% of total voyage cost versus the best-available alternative.
Stated_confidence_midpoint(b): midpoint of the confidence band (for example, 0.925 for the 90-95% band).
Units:
Dimensionless ratio per confidence band. Perfect calibration equals 1.0 at every band. Above 1.0 means underconfident (conservative). Below 1.0 means overconfident (dangerous for decision-making).
Edge Cases:
Proposed formula, pending validation pilots.