OOS Retest Strategy Scientific Validity Score (ORSSVS)

GMI-PH-032Pharmaceutical OperationsIntegration-Required

Definition

Proportion of AI-recommended OOS retest strategies that are scientifically justified, meaning the recommendation identifies a specific assignable cause and the retest design addresses that cause. Distinguishes better retests from retests merely likelier to pass.

The Measurement Gap

After implementing an AI OOS recommendation system, investigation closure rate improved from 67% to 84% within 15 days, and OOS retest pass rate increased from 71% to 83%. But nobody asked: is the higher pass rate because the AI recommends better-designed retests that correctly identify laboratory error, or because the AI recommends strategies statistically designed to produce a passing result?

The FDA's 2006 OOS Guidance is explicit: retesting is justified only when there is a plausible assignable cause. "Retest until you get a passing result" is not acceptable. But an AI optimized to "resolve" investigations can recommend strategies that amount to statistical dilution: large enough sample sizes that a single genuine OOS is overwhelmed by passing retests from a marginally out-of-specification batch.

Decision enabled:

When ORSSVS drops below 80%, the organization implements mandatory cause-linkage documentation for all AI retest recommendations. Below 50%, the AI recommendation system requires redesign to prioritize cause identification over resolution probability.

Formula

ORSSVS = scientifically_justified_retests / total_AI_recommended_retests

A retest is scientifically justified if the AI recommendation identifies a specific assignable cause AND the retest design addresses that cause.

Worked Example:

Six-month review, 47 AI-recommended retest strategies reviewed by the site quality director:

Scientifically justified (29): Examples include "analyst technique variability suspected (retest with different analyst using documented preparation timer" and "HPLC system suitability marginal) retest on alternative qualified system." Each identifies a specific cause with specific remediation.

Not scientifically justified (18): Examples include "increase sample size to n=6 and apply USP averaging criteria" (no cause identified) and "perform retest. Historical pass rate for this product is 97.3%" (recommendation based on outcome probability, not cause).

ORSSVS = 29 / 47 = 61.7%

Nearly 40% of AI retests lack scientific basis. Of the 18 unjustified retests, 15 produced passing results that closed the investigation. But in 4 of those 15 cases, subsequent batch analysis indicated the original OOS was likely genuine: batches released with potentially real quality issues.

Sign Convention:

Higher is better. Above 80% means mostly scientifically grounded. Below 50% means predominantly dilution strategies.

Proposed formula, pending validation pilots.

At a Glance

Unit
Percentage
Trend Direction
Higher
Reporting Frequency
Monthly; per OOS retest cycle
Category
Quality System & Investigation Effectiveness
← Cross-Site Deviation Pattern Detection Rate (CSDPDR)Root Cause Investigation Independence Score (RCIIS) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment