Definition
Percentage of AI-recommended therapeutic substitutions that are clinically appropriate at the individual patient level: accounting for patient-specific contraindications, interactions, and intolerances that the formulary optimizer's population-level equivalence logic does not capture.
The Measurement Gap
The Decision That Legacy Metrics Cannot Support
Is the AI formulary optimizer producing substitutions that are pharmacologically equivalent but clinically inappropriate for specific patients, and at what rate?
What Goes Unmeasured Today
AI-driven formulary optimization tools recommend therapeutic substitutions to reduce pharmacy costs, improve formulary compliance, or address drug shortages. These tools operate at the population level: they determine that Drug A is therapeutically equivalent to Drug B based on pharmacological class, mechanism of action, and clinical trial data. What they often do not assess is whether the substitution is appropriate for a specific patient whose medication list, allergy profile, kidney function, or comorbidities make the "equivalent" drug inappropriate.
Healthcare organizations track formulary compliance rates, substitution volumes, and pharmacy cost savings. These legacy metrics measure whether substitutions are happening and whether they save money, but they do not measure whether the substitutions are clinically safe at the individual patient level. A formulary optimizer that achieves 98% substitution acceptance may be generating clinically inappropriate recommendations that pharmacists catch, or worse, that pass through to the patient.
The measurement gap exists because clinical appropriateness requires patient-level assessment that formulary optimization tools were not designed to perform. The tool operates on drug equivalence rules; clinical appropriateness requires evaluating the drug in the context of the patient's full clinical picture. When the pharmacist catches an inappropriate substitution, that catch is rarely documented in a structured way that enables systematic measurement of the AI's patient-level error rate.
The Invisible Failure Mode
AI formulary substitutions that are population-level equivalent but patient-level inappropriate create a safety gap that is invisible in aggregate substitution metrics. The pharmacist safety net catches most errors, but the catch rate is not measured, the pass-through rate is not measured, and the total error rate is not known. Without SCAR, the organization trusts an algorithm without measuring whether that trust is warranted at the individual patient level.
Formula
Primary Formula
SCAR = Appropriate_substitutions / Total_sampled_substitutions
Error Decomposition:
SCAR_caught = Inappropriate_caught / Total_sampled
SCAR_passed = Inappropriate_passed / Total_sampled
Variable Definitions
| Variable | Definition | Unit |
|---|---|---|
| Appropriate_substitutions | Substitutions clinically sound for the specific patient | Count |
| Total_sampled_substitutions | Total substitutions in the audit sample | Count |
| Inappropriate_caught | Substitutions that were clinically inappropriate and pharmacist overrode | Count |
| Inappropriate_passed | Substitutions that were clinically inappropriate and reached the patient | Count |
| SCAR_caught | Error rate caught by pharmacist verification | Percentage |
| SCAR_passed | Error rate that reached the patient | Percentage |
Data Requirements
| Data Element | Source System | Classification |
|---|---|---|
| AI-recommended substitutions (original, alternative, rationale) | Formulary optimization AI / pharmacy system | NATIVE |
| Pharmacist action (approved, overrode, modified) | Pharmacy dispensing system | NATIVE |
| Patient medication list, allergies, diagnoses, labs | EHR | NATIVE |
| Clinical appropriateness assessment | Pharmacist review (structured scoring) | MANUAL |
Edge Cases and Exclusions
Clinical appropriateness requires pharmacist clinical judgment, not just formulary equivalence. This is inherently manual for the validation step. Assess whether inappropriateness was due to AI algorithm error or data gap (incomplete allergy documentation). Minimum sample: 100 substitutions per quarter, stratified by drug class.
Proposed formula, pending validation pilots.