Regulatory AI Currency Index (RACI)

GMI-PH-005Pharmaceutical OperationsResearch-Forward

Definition

Percentage of currently active regulatory requirements that the AI submission tool was confirmed to check correctly at its most recent validation. Measures how far the tool's validated scope has fallen behind an evolving regulatory landscape.

The Measurement Gap

The Gap: There is no metric that tells you whether AI-driven submission completeness assessments are degrading in accuracy as regulatory requirements evolve. Submission cycle time measures speed. First-round approval rate is a lagging indicator that only reveals the problem after rejection. Regulatory commitment tracking measures whether the team meets its own deadlines, not whether the AI's reference knowledge is current.

The Decision:

RACI drives the AI tool revalidation schedule. Instead of revalidating on a fixed calendar, revalidate when RACI indicates the knowledge gap has reached a threshold. When RACI drops below 90% for any region, trigger revalidation. This transforms regulatory AI maintenance from time-based to risk-based: you revalidate when you need to, not when the calendar says to.

Formula

Notation:

RACI = (Σ AI_Validated_Requirements / Σ Active_Regulatory_Requirements) × 100

Variables:

  • AI_Validated_Requirements = regulatory requirements the AI tool has been confirmed to correctly check, from its most recent validation exercise
  • Active_Regulatory_Requirements = total current regulatory requirements as defined by the regulatory intelligence function

Worked Example: For US FDA oral solid dosage NDA submissions: 347 active requirements. AI last validated 11 months ago against 338 requirements. Since then, 14 new or revised requirements identified, none validated. 5 originally validated requirements now superseded. RACI = (338 - 5) / 347 × 100 = 96.0%. The trend matters: at 1.3 new requirements per month, RACI drops roughly 0.4 points per month without revalidation.

Edge Cases:

  • Report RACI per region: different regions change requirements at different rates, and a global average obscures regional risk
  • "Validated against" means the AI was tested with a submission that correctly flagged presence or absence of the requirement; simply appearing in the reference database is not validation
  • Conditional requirements (only required for certain product/submission types) are harder to validate and most likely to be missed

Sign convention: Higher RACI is better. 100% = validated against every active requirement. Declining RACI means the regulatory landscape is evolving faster than the AI is being revalidated.

Proposed formula, pending validation pilots.

At a Glance

Unit
Percentage
Trend Direction
Higher
Reporting Frequency
Quarterly; triggered by regulatory guidance changes
Category
AI Model Fidelity & Drift
← PAT Model Concordance Rate (PMCR)Stability Prediction Calibration Score (SPCS) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment