Signal Sensitivity Parity Index (SSPI)

GMI-PH-041Pharmaceutical OperationsInfrastructure-Dependent

Definition

Ratio of AI safety signal detection sensitivity for novel modality products to sensitivity for established small molecules. Exposes a signal detection platform trained on whichever modalities happened to have data volume.

The Measurement Gap

A pharmaceutical company markets 34 products: 26 small molecules, 4 monoclonal antibodies, 2 gene therapies, 1 cell therapy, and 1 mRNA therapeutic. The AI signal detection platform was trained primarily on small molecule data because that is what existed in volume. It detects a hepatotoxicity signal for a mature cardiovascular product 3 months before traditional analysis. The PV director cites 40% improvement in signal detection timeliness across the portfolio.

But the gene therapies produce delayed-onset immune responses appearing 6-12 months post-treatment. The cell therapy generates cytokine storms with unusual temporal profiles. The mRNA therapeutic has reactogenicity driven by the lipid nanoparticle delivery system. The AI's temporal association windows, disproportionality thresholds, and expected reporting rates were all derived from small molecule pharmacology. It misses or delays novel modality signals because their patterns do not match the training distribution.

Legacy metrics report signal detection as a portfolio-wide aggregate. The 40% improvement is entirely driven by the 26 small molecules. No current metric compares detection sensitivity across product modalities to reveal where the safety net has gaps.

Formula

SSPI = sensitivity_novel / sensitivity_established

Where:

  • sensitivity_novel = AI detection rate for validated safety signals in novel modality products
  • sensitivity_established = AI detection rate for validated safety signals in established small-molecule products
  • Validated safety signals = signals ultimately confirmed by expert review, regardless of detection method

Worked Example:

Annual signal detection analysis:

Established products (26 small molecules): 41 confirmed signals. AI detected 34 before or concurrently with manual detection. Sensitivity_established = 34/41 = 82.9%.

Novel products (8 advanced therapies): 13 confirmed signals. AI detected 5 before or concurrently with manual detection. Sensitivity_novel = 5/13 = 38.5%.

SSPI = 38.5% / 82.9% = 0.464.

The AI is roughly half as sensitive to novel modality signals. Of 8 missed signals: 3 were delayed-onset events outside the algorithm's 90-day temporal window. 2 were adverse events coded to MedDRA terms the model associated only with disease progression. 2 were low-frequency signals where disproportionality could not reach statistical significance. 1 was gradual efficacy loss the algorithm did not recognize as a safety signal.

Proposed formula, pending validation pilots.

At a Glance

Unit
Percentage
Trend Direction
Higher
Reporting Frequency
Quarterly; stratified by product modality
Category
Pharmacovigilance
← Causality Assessment Novelty Bias Index (CANBI)Batch Release Independence Score (BRIS) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment