Automation Bias Differential (ABD)

GMI-HC-003HealthcareIntegration-RequiredPattern O-5: Override Value / Trust

Definition

Difference in radiologist miss rate between studies where the AI marked findings as absent ("AI-normal") versus studies read without AI assistance, measuring whether the AI's "all clear" signal reduces diagnostic vigilance.

The Measurement Gap

The Decision That Legacy Metrics Cannot Support

Is the AI's "normal" label making radiologists less likely to independently identify subtle findings, and if so, how much diagnostic accuracy is being traded for workflow speed?

What Goes Unmeasured Today

Automation bias (the tendency for humans to defer to automated recommendations even when independent evidence contradicts them) has been extensively studied in aviation, manufacturing, and military contexts (Parasuraman & Manzey, 2010). In diagnostic radiology, the risk takes a specific form: when an AI system labels a study as "normal" or "no significant findings," the radiologist's independent search pattern may shorten, their attention to subtle abnormalities may decrease, and findings they would have caught without AI assistance go unreported.

Healthcare organizations track radiologist accuracy through peer review programs, random case audits, and discordance analysis. These legacy quality metrics measure overall miss rates without distinguishing between studies where the radiologist worked independently and studies where an AI "all clear" signal was present. The critical question (whether the AI's normal label itself causes radiologists to miss more) requires comparing miss rates between AI-assisted and non-AI conditions, a comparison that standard QA programs do not perform.

The measurement gap is particularly dangerous because automation bias is invisible in aggregate statistics. If the AI correctly identifies 98% of abnormal studies and radiologists miss 2% more on AI-normal studies, the net accuracy may still appear acceptable. But the 2% increase represents findings that would have been caught without AI, a direct patient safety regression caused by the tool that was supposed to improve safety.

The Invisible Failure Mode

The AI's "normal" label creates a cognitive anchor. Radiologists who see the AI's assessment before beginning their independent review may unconsciously shorten their search or lower their suspicion threshold. This anchoring effect is well-documented in cognitive psychology (Parasuraman & Manzey, 2010) but is not captured by any legacy radiology quality metric. The failure mode is a net reduction in diagnostic vigilance masked by the AI's overall accuracy improvement.

Formula

Primary Formula

ABD = Miss_Rate_AI_Normal − Miss_Rate_No_AI

Variable Definitions

Variable Definition Unit
Miss_Rate_AI_Normal Missed findings / total findings present on studies where AI reported no abnormality Percentage
Miss_Rate_No_AI Missed findings / total findings present on studies read without AI assistance (control or pre-AI baseline) Percentage
Missed finding Clinically significant finding confirmed by subsequent imaging, pathology, or clinical course that was not reported in the initial read Binary per finding

Data Requirements

Data Element Source System Classification
AI finding output per study (including "no finding" / "normal") AI diagnostic platform NATIVE
Initial radiologist interpretation per study PACS/RIS NATIVE
Subsequent imaging, pathology, or clinical confirmation EHR: longitudinal patient record INTEGRATION
Missed-finding identification Derived: discordance between initial read and subsequent confirmation DERIVED
Pre-AI baseline miss rate or randomized control group Historical radiology QA data NATIVE (if QA existed) / NOT CAPTURED (if not)

Edge Cases and Exclusions

Establishing ground truth for "missed finding" requires clinical follow-up data, which may have a 6-12 month lag; track ABD as a lagging indicator. If AI is deployed on all studies, a randomized A/B design is needed to establish the control miss rate: ethically complex in clinical settings. Use pre-AI baseline data if available, with controls for population changes. Segment by radiologist experience level, as junior radiologists may show higher ABD than senior radiologists. Minimum sample: 200 AI-normal studies with clinical follow-up to establish statistical power.

Proposed formula, pending validation pilots.

At a Glance

Category
Diagnostic AI Calibration and Clinical Trust
← AI Worklist Prioritization Equity Score (WPES)Diagnostic Confidence Calibration Index (DCCI) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment