Definition
Difference in radiologist miss rate between studies where the AI marked findings as absent ("AI-normal") versus studies read without AI assistance, measuring whether the AI's "all clear" signal reduces diagnostic vigilance.
The Measurement Gap
The Decision That Legacy Metrics Cannot Support
Is the AI's "normal" label making radiologists less likely to independently identify subtle findings, and if so, how much diagnostic accuracy is being traded for workflow speed?
What Goes Unmeasured Today
Automation bias (the tendency for humans to defer to automated recommendations even when independent evidence contradicts them) has been extensively studied in aviation, manufacturing, and military contexts (Parasuraman & Manzey, 2010). In diagnostic radiology, the risk takes a specific form: when an AI system labels a study as "normal" or "no significant findings," the radiologist's independent search pattern may shorten, their attention to subtle abnormalities may decrease, and findings they would have caught without AI assistance go unreported.
Healthcare organizations track radiologist accuracy through peer review programs, random case audits, and discordance analysis. These legacy quality metrics measure overall miss rates without distinguishing between studies where the radiologist worked independently and studies where an AI "all clear" signal was present. The critical question (whether the AI's normal label itself causes radiologists to miss more) requires comparing miss rates between AI-assisted and non-AI conditions, a comparison that standard QA programs do not perform.
The measurement gap is particularly dangerous because automation bias is invisible in aggregate statistics. If the AI correctly identifies 98% of abnormal studies and radiologists miss 2% more on AI-normal studies, the net accuracy may still appear acceptable. But the 2% increase represents findings that would have been caught without AI, a direct patient safety regression caused by the tool that was supposed to improve safety.
The Invisible Failure Mode
The AI's "normal" label creates a cognitive anchor. Radiologists who see the AI's assessment before beginning their independent review may unconsciously shorten their search or lower their suspicion threshold. This anchoring effect is well-documented in cognitive psychology (Parasuraman & Manzey, 2010) but is not captured by any legacy radiology quality metric. The failure mode is a net reduction in diagnostic vigilance masked by the AI's overall accuracy improvement.
Formula
Primary Formula
ABD = Miss_Rate_AI_Normal − Miss_Rate_No_AI
Variable Definitions
| Variable | Definition | Unit |
|---|---|---|
| Miss_Rate_AI_Normal | Missed findings / total findings present on studies where AI reported no abnormality | Percentage |
| Miss_Rate_No_AI | Missed findings / total findings present on studies read without AI assistance (control or pre-AI baseline) | Percentage |
| Missed finding | Clinically significant finding confirmed by subsequent imaging, pathology, or clinical course that was not reported in the initial read | Binary per finding |
Data Requirements
| Data Element | Source System | Classification |
|---|---|---|
| AI finding output per study (including "no finding" / "normal") | AI diagnostic platform | NATIVE |
| Initial radiologist interpretation per study | PACS/RIS | NATIVE |
| Subsequent imaging, pathology, or clinical confirmation | EHR: longitudinal patient record | INTEGRATION |
| Missed-finding identification | Derived: discordance between initial read and subsequent confirmation | DERIVED |
| Pre-AI baseline miss rate or randomized control group | Historical radiology QA data | NATIVE (if QA existed) / NOT CAPTURED (if not) |
Edge Cases and Exclusions
Establishing ground truth for "missed finding" requires clinical follow-up data, which may have a 6-12 month lag; track ABD as a lagging indicator. If AI is deployed on all studies, a randomized A/B design is needed to establish the control miss rate: ethically complex in clinical settings. Use pre-AI baseline data if available, with controls for population changes. Segment by radiologist experience level, as junior radiologists may show higher ABD than senior radiologists. Minimum sample: 200 AI-normal studies with clinical follow-up to establish statistical power.
Proposed formula, pending validation pilots.