CDS Cognitive Yield Ratio (CDSYR)

GMI-HC-006HealthcareInfrastructure-DependentPattern O-4: Alert Quality / Signal-to-Noise Ratio

Definition

Ratio of CDS-driven care plan changes (treatment modifications, prevented adverse events) to CDS-driven clinician actions (alert reviews, override documentation, acknowledgment clicks), measuring whether the CDS system is returning clinical value proportionate to the cognitive burden it imposes.

The Measurement Gap

The Decision That Legacy Metrics Cannot Support

Is the CDS system worth the cognitive cost? For every hour of clinician attention consumed by alerts, how many patient outcomes improve?

What Goes Unmeasured Today

Healthcare organizations track how many CDS alerts fire, how many are overridden, and sometimes how many result in order modifications. What they almost never measure is the ratio of clinically meaningful outcomes (care plan changes that improved or protected patient safety) to the total cognitive burden imposed (every alert review, acknowledgment click, and override justification the clinician had to process). This yield ratio is the fundamental question for CDS governance: is the system a net contributor to care quality or a net drain on clinician attention?

The measurement gap exists because the numerator (care plan changes attributable to CDS) is difficult to define and track. When a clinician sees a drug interaction alert and changes the medication order, was that change caused by the alert or would the clinician have made the same decision independently? When a sepsis alert fires and antibiotics are ordered, was the alert the trigger or was the clinical team already planning treatment? Attribution requires either structured alert-response fields (which most organizations have not configured) or clinical review of temporal proximity between alerts and order changes.

The denominator is also underestimated. Legacy metrics count alert presentations, but the real cognitive burden includes the mental context-switching cost of each interruption, the time spent documenting override reasons, and the cumulative fatigue effect across a shift. A single alert may take only 10 seconds to dismiss, but 200 alerts across a 12-hour shift represent a substantial attention tax.

The Invisible Failure Mode

A CDS system where 1 in 100 interactions changes care (CCYR = 0.01) is consuming clinician attention at a rate that may actively harm care quality through fatigue and distraction, even if the 1-in-100 changes are individually valuable. The system's net contribution cannot be assessed without measuring the yield ratio.

Formula

Primary Formula

CCYR = Care_Plan_Changes_CDS / Total_CDS_Interactions

Variable Definitions

Variable Definition Unit
Care_Plan_Changes_CDS Instances where a CDS alert directly led to a treatment modification, order change, dose adjustment, or safety intervention Count
Total_CDS_Interactions Total alert presentations + required acknowledgment actions + override documentation actions Count
Attribution window Time window within which a care plan change is linked to a preceding alert (recommended: 30 minutes) Minutes

Data Requirements

Data Element Source System Classification
CDS alert presentations (all types, with metadata) EHR CDS engine NATIVE
Clinician actions on alerts (accept, override, acknowledge) EHR CDS log NATIVE
Post-alert order changes within time window EHR CPOE NATIVE
Clinical relevance assessment (alert content → order change) Derived or manual clinical review MANUAL / DERIVED
Prevented adverse event estimation Pharmacy review, clinical safety reports MANUAL
Clinician time-per-alert estimate Time-motion study or EHR session analytics NOT CAPTURED (requires instrumentation)

Edge Cases and Exclusions

Defining "care plan change driven by CDS" requires clinical judgment. Use structured alert-response fields if available; otherwise use temporal proximity plus order type matching. Passive alerts (informational display, no action required) should be included in the denominator but should not be expected to drive care changes: segment CCYR for interruptive versus passive alerts. Prevented adverse events are counterfactual and should be estimated using established methods. Minimum sample: 1,000 CDS interactions per measurement period for a stable denominator.

Proposed formula, pending validation pilots.

At a Glance

Category
Clinical Decision Support Signal Quality
← Alert Credibility Response Lag (ACRL)Critical Alert Override Discrimination (CAOD) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment