Stability Information Value Index (SIVI)

GMI-PH-022Pharmaceutical OperationsResearch-Forward

Definition

Percentage of AI-eliminated stability time points that would have detected a trajectory deviation earlier, among eliminations where subsequent data makes that judgment possible. Puts a value on the information the model chose not to collect.

The Measurement Gap

When the stability prediction model says "I am confident enough that you can skip the 15-month time point," the organization saves testing cost. But nobody tracks whether the skipped time point would have revealed a degradation pathway the model didn't predict. For one product, the eliminated 15-month point turns out to be exactly where a secondary degradation impurity becomes detectable, an impurity the model's training data never included. The $340K in testing savings is quantified and celebrated. The information lost from the eliminated time points is assumed to be zero based on model confidence. No metric measures the gap between what the model thinks it knows and what the eliminated data would have revealed. The cost savings are visible; the information loss is invisible until a supplementary regulatory filing or a product quality event forces the organization to confront what it chose not to measure.

Decision Supported:

Did eliminating those stability testing time points actually save money net of information value, or did we trade measurable cost savings for unmeasured information loss?

Formula

Notation:

SIVI = (Σ Informative_Eliminations / Σ Total_Assessable_Eliminations) × 100

Variables:

  • Informative_Eliminations = eliminated time points where subsequent data showed a trajectory deviation that the eliminated time point would have detected earlier
  • Total_Assessable_Eliminations = eliminated time points for which sufficient subsequent data now exists for retrospective assessment

Worked Example: 23 time points eliminated over 18 months. Retrospective analysis: 18 confirmed the model's predicted trajectory (correctly eliminated). 3 showed trajectory deviations the eliminated time point would have caught earlier. 2 have insufficient subsequent data (too recent). SIVI = 3/21 = 14.3%. Above the 10% action threshold: one in seven eliminated time points would have provided useful information.

Edge Cases:

  • Exclude eliminations where insufficient subsequent data exists (too recent) from both numerator and denominator
  • "Informative" means the eliminated time point would have detected a deviation or trend at least one time point earlier than actual detection
  • SIVI inherently requires patience: 12–18 months before assessable data accumulates

Sign convention: Lower SIVI is better. Zero means all eliminations were correctly made.

Proposed formula, pending validation pilots.

At a Glance

Unit
Percentage
Trend Direction
Higher
Reporting Frequency
Per skipped time point; annual value retrospective
Category
Resource Allocation & Lab Governance
← Non-Release Testing Schedule Compliance (NRTSC)Cumulative Thermal Exposure Index (CTEI) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment