Definition
The percentage of actionable alerts, trends, or recommendations generated by NLP, text-mining, or pattern-recognition intelligence systems that, upon investigation, turn out to be artifacts rather than genuine operational findings.
The Measurement Gap
The NLP system detects a "trend" in wheel flat mentions across maintenance logs. Investigation reveals a new crew supervisor uses "wheel flat" where predecessors wrote "tread damage." The trend is linguistic, not mechanical. Resources are diverted to investigate a phantom problem. No legacy metric distinguishes NLP-detected trends from linguistic pattern changes. Maintenance backlog shows the wasted effort but does not attribute it to intelligence source quality.
Decision Supported:
When the NLP or pattern recognition system flags a trend, how often is it a genuine operational finding versus an artifact? Should we investigate every alert or apply a quality filter first?
Formula
Plain language: Over an evaluation period, count investigations triggered by intelligence system outputs. Of those, count how many confirmed genuine operational findings versus how many were artifacts (linguistic changes, data entry pattern shifts, seasonal noise). Divide artifacts by total.
Notation:
ISFDR = A_false / (A_false + A_true)
Variables:
| Variable | Description | Unit |
|---|---|---|
| A_false | Investigations triggered by intelligence output closed as artifact / no operational finding | Count |
| A_true | Investigations that confirmed a genuine finding and resulted in corrective action | Count |
Sign Convention: Lower is better. 0 = every intelligence output was genuine. 1.0 = every output was an artifact.
Proposed formula, pending validation pilots.