Definition
Mean delay between a clinical alert firing and the clinical team initiating a responsive action, correlated with the alert system's historical false positive rate for that alert type, measuring whether alert credibility erosion is driving clinically dangerous response delays.
The Measurement Gap
The Decision That Legacy Metrics Cannot Support
Is the sepsis alert that fired at 14:00 being acted on at 14:05 (the team trusts the alert) or at 17:00 (the team does not trust the alert and waits for clinical confirmation), and does the delay correlate with the alert's track record of false positives?
What Goes Unmeasured Today
Healthcare organizations measure whether clinicians respond to alerts (override versus accept) but rarely measure how long they take to respond, and almost never correlate response time with the alert system's historical reliability. This temporal dimension of trust is critical because a clinician who acknowledges a sepsis alert but delays treatment initiation by two hours is exhibiting a fundamentally different behavior than one who acts within minutes, even though both appear as "alert acknowledged" in the CDS log.
The response lag reveals something that override rates cannot: whether clinicians have learned to distrust specific alert types based on experience. An alert type with a 70% false positive rate will generate slower responses than one with a 20% false positive rate, not because clinicians consciously calculate false positive rates, but because repeated false alarms calibrate their urgency response downward. This learned distrust is invisible in binary accept/override metrics.
Legacy time-to-treatment metrics (time-to-antibiotics for sepsis, door-to-balloon for STEMI) capture downstream delays but cannot attribute the delay to alert credibility. A 90-minute time-to-antibiotics may reflect ED crowding, order-entry workflow, or pharmacy turnaround, or it may reflect a clinical team that has learned to ignore the sepsis alert because it fires on every febrile patient. Without measuring the alert-to-action interval and correlating it with alert type reliability, the root cause remains hidden.
The Invisible Failure Mode
Alert credibility erosion is a gradual, per-alert-type phenomenon. A sepsis alert that was initially trusted may accumulate enough false positives over months that clinical teams develop an unconscious delay pattern: they wait for additional clinical signs before acting on the alert. The delay grows slowly and is distributed across shifts and providers, making it invisible in aggregate time-to-treatment reporting.
Formula
Primary Formula
ACRL(a) = Median(T_action − T_alert) for alert type a
Correlation:
r(ACRL, FPR) across alert types a
Variable Definitions
| Variable | Definition | Unit |
|---|---|---|
| a | Alert type (sepsis screen, drug interaction, deterioration, etc.) | Category |
| T_alert | Timestamp when alert fires | Datetime |
| T_action | Timestamp of first responsive clinical action (medication order, assessment, escalation) | Datetime |
| FPR(a) | False positive rate for alert type a over trailing 90 days | Ratio (0–1) |
| r(ACRL, FPR) | Pearson correlation between ACRL and FPR across alert types | Correlation coefficient (-1 to 1) |
Data Requirements
| Data Element | Source System | Classification |
|---|---|---|
| Alert fire timestamp and type | EHR CDS engine | NATIVE |
| Alert acknowledgment timestamp | EHR CDS log | NATIVE |
| Clinical action timestamps (orders, assessments) | EHR CPOE / flowsheet | NATIVE |
| Alert-to-action linkage | Derived: temporal proximity + clinical relevance matching | DERIVED |
| Historical false positive rate by alert type | Derived: alert outcomes over trailing 90 days | DERIVED |
Edge Cases and Exclusions
Defining "responsive action" requires clinical informatics judgment to distinguish alert-driven actions from routine care: use timestamp proximity plus order type matching. Separate acknowledgment time from action time, as acknowledging and ignoring is a different behavior than delayed action. Night shift delays should be segmented or normalized by shift. Minimum sample: 50 instances per alert type with confirmed responsive action.
Proposed formula, pending validation pilots.