Definition
Ratio of read-time delay for AI-deprioritized studies versus AI-prioritized studies, segmented by clinical urgency, measuring whether AI worklist reordering creates harmful read-time disparities for non-flagged study types.
The Measurement Gap
The Decision That Legacy Metrics Cannot Support
Is the AI worklist pushing certain modalities or finding types to the back of the queue in ways that delay clinically time-sensitive reads?
What Goes Unmeasured Today
AI-powered worklist prioritization tools reorder radiology reading queues based on detected urgency: stroke findings get pushed to the top, routine chest X-rays sink to the bottom. Healthcare organizations measure whether the AI correctly identifies urgent findings (sensitivity) and whether overall turnaround times improve (mean TAT). But almost none measure the distributional impact: which studies wait longer because other studies were pushed ahead of them, and whether that delay creates clinically meaningful harm for specific modalities or finding types.
The problem is not that prioritization exists: manual triage has always prioritized some studies over others. The problem is that AI-driven prioritization operates at scale, consistently, and invisibly. A human triage radiologist applies judgment case by case; an AI algorithm applies a fixed prioritization logic to every study, every shift. If that logic systematically deprioritizes a particular modality (abdominal ultrasound, for example) studies in that modality accumulate disproportionate delays that never surface in aggregate TAT reporting.
Legacy turnaround time metrics report averages. A mean TAT of 45 minutes may mask the fact that AI-flagged chest CTs are being read in 12 minutes while unflagged abdominal ultrasounds wait 3 hours. The average looks acceptable; the disparity is clinically dangerous if time-sensitive pathology exists in the deprioritized modality. Without modality-segmented delay ratios, organizations cannot detect whether AI worklist optimization is creating new access inequities in diagnostic services.
The Invisible Failure Mode
AI worklist reordering creates a zero-sum queue. Every study pushed up pushes other studies down. If the prioritization algorithm has a modality blind spot (if it was trained primarily on chest CT and head CT pathology) studies from other modalities may be systematically delayed without anyone recognizing the pattern. The delay is invisible because no one compares flagged versus unflagged turnaround times by modality.
Formula
Primary Formula
WPES(m) = Median_TAT_unflagged(m) / Median_TAT_flagged(m)
Disparity Measure:
WPES_disparity = max(WPES across modalities) − min(WPES across modalities)
Variable Definitions
| Variable | Definition | Unit |
|---|---|---|
| m | Modality or study type (chest CT, abdominal US, brain MRI, etc.) | Category |
| Median_TAT_flagged(m) | Median order-to-final-read time for AI-flagged studies in modality m | Minutes |
| Median_TAT_unflagged(m) | Median order-to-final-read time for non-flagged studies in modality m | Minutes |
| WPES_disparity | Maximum WPES minus minimum WPES across all modalities | Ratio difference |
Data Requirements
| Data Element | Source System | Classification |
|---|---|---|
| Study order timestamp | RIS / EHR CPOE | NATIVE |
| Final read completion timestamp | PACS / RIS | NATIVE |
| AI urgency flag (yes/no, with score if available) | AI diagnostic platform | NATIVE |
| Study modality and body part | RIS | NATIVE |
| Order priority (STAT, urgent, routine) | RIS / EHR CPOE | NATIVE |
Edge Cases and Exclusions
Modalities where AI is not deployed should be excluded: they have their own TAT profile unaffected by AI prioritization. After-hours and daytime periods should be segmented separately due to different worklist dynamics. STAT orders should be excluded since these are prioritized by clinical order, not by AI. Minimum sample size is 50 studies per modality per measurement period.
Proposed formula, pending validation pilots.