Definition
The percentage of AI-certified "complete" work packages that are free of semantic errors when reviewed by a trained human auditor. Semantic errors are logical inconsistencies in technical records (part number variant mismatches, specification values outside allowable range, findings-task mismatches, and missing traceability links) that a syntactic completeness check would not catch.
The Measurement Gap
AI-assisted work package review systems can verify that required fields are populated, that sign-offs are present, and that the document structure is correct. What they cannot yet reliably do is verify that the content of those fields is logically consistent with the engineering context of the task. A technician records a part number that is a valid part number for a different aircraft variant. An AI reviewer confirms the field is populated and the format is correct. The work package closes. The wrong part is on the aircraft.
This is the semantic gap that TRSAR targets. Legacy work package audits rely on human reviewers sampling closed packages against technical libraries, a process that is accurate but expensive and low-frequency. AI-certified work packages are being closed at volumes that make comprehensive human review impractical. TRSAR provides a structured, sampled measurement of the error rate that AI certification is missing, establishing whether the AI verification system is operating at acceptable accuracy or whether systematic error types are slipping through. Without this metric, organizations accepting AI certification of work packages are operating without a feedback loop on the accuracy of that certification.
Formula
Plain language: Subtract the proportion of sampled AI-verified work packages containing semantic errors from one, expressed as a percentage.
Notation:
TRSAR = (1 − (P_semantic_errors / P_sampled)) × 100
Variables:
| Variable | Definition | Source |
|---|---|---|
| P_semantic_errors | Work packages in the sample containing one or more semantic errors | Human auditor review |
| P_sampled | Total work packages drawn from the AI-certified closed population | Work package management system |
Semantic error taxonomy (four types):
- Part number variant mismatch: valid part number recorded but incorrect for the aircraft variant, serial number range, or modification status
- Specification value out of range: torque value, gap measurement, or other parameter recorded outside allowable range for that task
- Findings-task mismatch: findings section documents a defect type inconsistent with the corrective action recorded
- Missing traceability: required reference to a previous finding, engineering order, or airworthiness directive is absent
Proposed formula, pending validation pilots.