Adaptive Trial Generalizability Index (ATGI)

GMI-PH-048Pharmaceutical OperationsResearch-Forward

Definition

Similarity between the enrolled adaptive trial population and the target treatment population, computed as one minus the Jensen-Shannon divergence across patient characteristics. Response-adaptive randomization can meet an endpoint on a population the label will not describe.

The Measurement Gap

A biopharmaceutical company completes a Phase III adaptive trial for a type 2 diabetes therapeutic. The AI used Bayesian response-adaptive randomization, adjusting enrollment ratios based on interim efficacy signals and prioritizing patients predicted to have the strongest treatment response. The trial met its primary endpoint. Marketing projects $2.8 billion peak-year revenue based on a broad label.

The regulatory science team runs the generalizability analysis. Trial population: mean age 54.2, 62% male, 14% with cardiovascular comorbidities, mean diabetes duration 6.2 years. Target population from NHANES and ADA registry: mean age 61.7, 48% male, 42% with cardiovascular comorbidities, mean diabetes duration 11.3 years.

The AI optimized for treatment effect signal by preferentially enrolling younger patients, shorter disease duration, and fewer comorbidities. It produced a clean, efficient trial with a clear treatment effect by selecting the population that responds best. But the typical real-world patient is 61, has cardiovascular disease, and has 11 years of diabetes. The trial provides limited evidence for this patient.

The regulatory consequence: a narrower label than expected, a post-marketing requirement, or a complete response letter. Any outcome destroys the revenue projection.

Formula

ATGI = 1 - D_JS(P_enrolled || P_target)

Where D_JS = Jensen-Shannon divergence (symmetric, bounded 0 to 1)

Worked Example:

Type 2 diabetes trial, 3,847 enrolled patients versus target population of 8.4 million:

Marginal JS divergence by characteristic:

  • Age: trial concentrated 45-65 vs. target broadly distributed 35-80. JS = 0.187
  • Sex: trial 62% male vs. target 48% male. JS = 0.043
  • Cardiovascular comorbidity: trial 14% vs. target 42%. JS = 0.234
  • Diabetes duration: trial mean 6.2y vs. target mean 11.3y. JS = 0.211

Composite joint distribution JS divergence: D_JS = 0.312.

ATGI = 1 - 0.312 = 0.688.

Comparing to a traditionally designed trial in the same area: ATGI of 0.81. The AI optimization reduced generalizability by 15 percentage points while gaining 7 months of enrollment speed. If the label is restricted to patients under 65 with diabetes duration under 8 years and no CV comorbidities, the addressable market shrinks from 8.4M to 2.1M patients, and peak revenue drops from $2.8B to $830M.

Proposed formula, pending validation pilots.

At a Glance

Unit
Percentage
Trend Direction
Higher
Reporting Frequency
Per completed trial; retrospective
Category
Clinical Trials
← PAT Adjustment Directional Bias (PADB)Dose Escalation Uncertainty Incorporation Score (DEUIS) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment