Definition
Percentage of currently active regulatory requirements that the AI submission tool was confirmed to check correctly at its most recent validation. Measures how far the tool's validated scope has fallen behind an evolving regulatory landscape.
The Measurement Gap
The Gap: There is no metric that tells you whether AI-driven submission completeness assessments are degrading in accuracy as regulatory requirements evolve. Submission cycle time measures speed. First-round approval rate is a lagging indicator that only reveals the problem after rejection. Regulatory commitment tracking measures whether the team meets its own deadlines, not whether the AI's reference knowledge is current.
The Decision:
RACI drives the AI tool revalidation schedule. Instead of revalidating on a fixed calendar, revalidate when RACI indicates the knowledge gap has reached a threshold. When RACI drops below 90% for any region, trigger revalidation. This transforms regulatory AI maintenance from time-based to risk-based: you revalidate when you need to, not when the calendar says to.
Formula
Notation:
RACI = (Σ AI_Validated_Requirements / Σ Active_Regulatory_Requirements) × 100
Variables:
- AI_Validated_Requirements = regulatory requirements the AI tool has been confirmed to correctly check, from its most recent validation exercise
- Active_Regulatory_Requirements = total current regulatory requirements as defined by the regulatory intelligence function
Worked Example: For US FDA oral solid dosage NDA submissions: 347 active requirements. AI last validated 11 months ago against 338 requirements. Since then, 14 new or revised requirements identified, none validated. 5 originally validated requirements now superseded. RACI = (338 - 5) / 347 × 100 = 96.0%. The trend matters: at 1.3 new requirements per month, RACI drops roughly 0.4 points per month without revalidation.
Edge Cases:
- Report RACI per region: different regions change requirements at different rates, and a global average obscures regional risk
- "Validated against" means the AI was tested with a submission that correctly flagged presence or absence of the requirement; simply appearing in the reference database is not validation
- Conditional requirements (only required for certain product/submission types) are harder to validate and most likely to be missed
Sign convention: Higher RACI is better. 100% = validated against every active requirement. Declining RACI means the regulatory landscape is evolving faster than the AI is being revalidated.
Proposed formula, pending validation pilots.