Definition
Proportion of AI-recommended OOS retest strategies that are scientifically justified, meaning the recommendation identifies a specific assignable cause and the retest design addresses that cause. Distinguishes better retests from retests merely likelier to pass.
The Measurement Gap
After implementing an AI OOS recommendation system, investigation closure rate improved from 67% to 84% within 15 days, and OOS retest pass rate increased from 71% to 83%. But nobody asked: is the higher pass rate because the AI recommends better-designed retests that correctly identify laboratory error, or because the AI recommends strategies statistically designed to produce a passing result?
The FDA's 2006 OOS Guidance is explicit: retesting is justified only when there is a plausible assignable cause. "Retest until you get a passing result" is not acceptable. But an AI optimized to "resolve" investigations can recommend strategies that amount to statistical dilution: large enough sample sizes that a single genuine OOS is overwhelmed by passing retests from a marginally out-of-specification batch.
Decision enabled:
When ORSSVS drops below 80%, the organization implements mandatory cause-linkage documentation for all AI retest recommendations. Below 50%, the AI recommendation system requires redesign to prioritize cause identification over resolution probability.
Formula
ORSSVS = scientifically_justified_retests / total_AI_recommended_retests
A retest is scientifically justified if the AI recommendation identifies a specific assignable cause AND the retest design addresses that cause.
Worked Example:
Six-month review, 47 AI-recommended retest strategies reviewed by the site quality director:
Scientifically justified (29): Examples include "analyst technique variability suspected (retest with different analyst using documented preparation timer" and "HPLC system suitability marginal) retest on alternative qualified system." Each identifies a specific cause with specific remediation.
Not scientifically justified (18): Examples include "increase sample size to n=6 and apply USP averaging criteria" (no cause identified) and "perform retest. Historical pass rate for this product is 97.3%" (recommendation based on outcome probability, not cause).
ORSSVS = 29 / 47 = 61.7%
Nearly 40% of AI retests lack scientific basis. Of the 18 unjustified retests, 15 produced passing results that closed the investigation. But in 4 of those 15 cases, subsequent batch analysis indicated the original OOS was likely genuine: batches released with potentially real quality issues.
Sign Convention:
Higher is better. Above 80% means mostly scientifically grounded. Below 50% means predominantly dilution strategies.
Proposed formula, pending validation pilots.