Definition
Percentage of NLP-classified PSC deficiency severities that match actual regulatory severity under current concentrated inspection campaign priorities.
The Measurement Gap
Port State Control inspections generate deficiency findings that range from minor observations to detainable deficiencies, and the severity classification determines everything: corrective action urgency, resource allocation, and whether the vessel can sail. NLP systems now classify these deficiencies automatically, but their classification accuracy depends on understanding the current enforcement environment, including which concentrated inspection campaigns (CICs) are active and how Paris MoU or Tokyo MoU detention guidelines interpret specific deficiency codes. Legacy metrics like PSC detention rate measure the outcome after the damage is done, a vessel is already detained. PSC deficiency count tracks volume but not severity accuracy. PMS completion rate tells you whether maintenance was done, not whether the resulting condition was correctly classified by an AI system. LTIFR measures crew safety incidents, not regulatory classification errors. None of these metrics can detect the specific failure mode where an NLP system under-classifies a detainable deficiency as a minor observation, allowing the vessel to sail with an unresolved condition that the next port state inspector will catch: resulting in detention, commercial disruption, and flag state scrutiny.
Decision Supported:
Can the organization trust NLP severity classification of PSC findings to determine corrective action urgency, or must every classification be manually reviewed against current enforcement priorities?
Formula
Plain language: Sample NLP-classified PSC deficiencies and compare each NLP-assigned severity (minor observation, deficiency, detainable deficiency) against the actual regulatory severity determined by the Paris MoU/Tokyo MoU deficiency code table, current CIC priorities, and flag-state-specific detention thresholds.
Primary metric (under-classification rate on detainable deficiencies):
UCR_detainable = Under_classified_detainable / Total_actually_detainable
Secondary metric (overall accuracy):
PDCA = Correctly_classified / Total_sampled
Why under-classification rate on detainable deficiencies is primary: Over-classifying a minor observation as detainable creates unnecessary operational disruption but no safety gap. Under-classifying a detainable deficiency as minor means the vessel sails with an unresolved condition that warranted detention. The consequence is unaddressed safety risk, potential flag-state sanctions, and port-state enforcement escalation on the next inspection. Set UCR_detainable below 2%; overall PDCA above 95%.
Variables:
| Variable | Definition | Unit |
|---|---|---|
| Correctly_classified | NLP-assigned severity matches regulatory severity | Count |
| Under_classified_detainable | Detainable deficiencies NLP assigned lower severity | Count |
| Total_actually_detainable | Deficiencies confirmed as detainable by expert review | Count |
| Over_classified | NLP assigned higher severity than warranted | Count |
| Total_sampled | PSC deficiency classifications evaluated in the period | Count |
Proposed formula, pending validation pilots.