Definition
Percentage of maintenance discrepancy narratives classified as "routine" by the NLP system that a qualified inspector or A&P mechanic would classify as airworthiness-relevant: measuring the rate at which AI classification misses safety-critical maintenance findings.
The Measurement Gap
Maintenance discrepancy reporting systems capture tens of thousands of technician narratives annually. AI classifiers that automatically route discrepancies to routine maintenance work queues vs. airworthiness review queues are increasingly common as carriers seek to manage documentation volume. The quality of this routing decision determines whether airworthiness-relevant findings receive the regulatory and engineering attention they require.
Traditional maintenance quality metrics measure discrepancy closure rates, repeat defect rates, and MEL/CDL item counts: they don't measure whether the AI routing is correctly identifying which items require airworthiness attention in the first place. An MRO operation can have excellent closure metrics and still have a systematic problem with the AI classifier routing structural integrity, flight control, or powerplant discrepancies into routine queues because the narrative language was informal, technical terminology was abbreviated, or the item description was ambiguous.
Decision supported:
Is the AI maintenance discrepancy classifier correctly identifying items that require airworthiness attention, or are airworthiness-relevant items being buried in the routine work queue. Invisible to regulatory, engineering, and safety oversight? AMCR provides the measurement signal that tells maintenance leadership how often the classifier is making the wrong call in the dangerous direction. A 3% AMCR means 3 in every 100 routine-classified items should have received airworthiness review, which at high volume can mean dozens of missed items per quarter. This measurement enables a risk-based decision about whether a human review gate is needed for specific high-consequence discrepancy types.
Formula
Plain language: Of the discrepancy narratives the NLP classified as "routine," how many does a qualified A&P mechanic or inspector identify as actually airworthiness-relevant? That leakage rate is the primary metric. Overall classification accuracy is reported alongside it for model health, but the false-negative rate on airworthiness events is the headline number.
Primary metric: Airworthiness False-Negative Rate:
AMCR_FNR = Misclassified_airworthiness_items / Sampled_routine_classifications
Secondary metric: Overall Classification Accuracy:
AMCR_OA = correctly_classified / sampled_events
Variables:
| Variable | Definition | Unit |
|---|---|---|
| Misclassified_airworthiness_items | Narratives classified as routine by AI but assessed as airworthiness-relevant by expert review (false negatives) | Count |
| Sampled_routine_classifications | Total narratives sampled from the AI's "routine" classification bin | Count |
| correctly_classified | All events (airworthiness + routine) where AI matches expert assessment | Count |
| sampled_events | Audit sample drawn from both bins, stratified by aircraft type and discrepancy type | Count |
Units: Percentage. 0% = no airworthiness items misclassified as routine. Higher values indicate more missed airworthiness items.
Minimum sample: 100 routine-classified narratives per quarter, stratified by aircraft type and discrepancy type to ensure coverage of high-consequence categories (structural, flight control, powerplant, landing gear).
Why false-negative rate is primary:
In aviation maintenance, classifying a routine discrepancy as airworthiness-relevant creates additional review labor. Classifying an airworthiness-relevant discrepancy as routine means the item may not receive the engineering evaluation, mandatory inspection, or airworthiness directive compliance check it requires, creating undetected findings and potential FAA enforcement exposure. The asymmetry is a consequence structure, not a preference. A 95% overall accuracy that misses 5% of airworthiness events is a compliance liability, not a success.
Proposed formula, pending validation pilots.