Voyage Optimizer Confidence Calibration (VOCC)

GMI-MAR-009Maritime IndustriesIntegration-Required

Definition

Degree to which the voyage optimizer stated confidence corresponds to actual route outcome quality.

The Measurement Gap

What This Metric Measures:
Degree to which the voyage optimizer's stated confidence level for a route recommendation corresponds to the actual outcome quality at that confidence level. VOCC groups completed voyages by the optimizer's stated confidence band (80-85%, 85-90%, 90-95%, 95%+) and measures whether the actual percentage of successful outcomes within each band matches the stated confidence. Perfect calibration means that when the optimizer says it is 90% confident, 90% of voyages in that band achieve outcomes within acceptable deviation.

The Decision It Supports:

Can the master trust the optimizer's confidence score when deciding whether to follow, modify, or override a route recommendation, or is the system systematically overconfident? If the optimizer reports 95% confidence but only 70% of those voyages achieve acceptable outcomes, the master is making decisions on unreliable confidence information.

Why Legacy Metrics Cannot See This:

Voyage optimization platforms report average fuel savings and recommendation acceptance rates. They do not report whether their confidence scores are calibrated: whether the stated probability of a good outcome matches the observed frequency. A system that is 90% confident on every recommendation and achieves 70% success has a calibration gap of 0.78, meaning the master should mentally discount every confidence score by 22%. Legacy reporting shows high average confidence and reasonable success rates without revealing the systematic overconfidence that distorts the master's decision-making.

Formula

Primary Formula:

VOCC(b) = Actual_success_rate(b) / Stated_confidence_midpoint(b)

Variable Definitions:

b: confidence band (for example, 80-85%, 85-90%, 90-95%, 95%+).

Actual_success_rate(b): percentage of voyages in band b where the route outcome was within acceptable deviation of optimal, defined as within 5% of total voyage cost versus the best-available alternative.

Stated_confidence_midpoint(b): midpoint of the confidence band (for example, 0.925 for the 90-95% band).

Units:

Dimensionless ratio per confidence band. Perfect calibration equals 1.0 at every band. Above 1.0 means underconfident (conservative). Below 1.0 means overconfident (dangerous for decision-making).

Edge Cases:

Proposed formula, pending validation pilots.

At a Glance

Unit
Ratio
Trend Direction
Toward 1.0
Reporting Frequency
Rolling semi-annually
Category
Voyage and Fleet Commercial Decision Quality
← Route Optimization Net Economic Value (RONEV)

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment