Regulatory Fluency Atrophy Index (RFAI)

GMI-NUC-013Nuclear OperationsInfrastructure-Dependent

Definition

Change in licensed operators' ability to independently interpret Technical Specification requirements and make conservative operability determinations without NLP assistance, measured through structured assessment exercises.

The Measurement Gap

Licensed operators using NLP tools for Technical Specification interpretation develop reliance on the tool over time. When the NLP system is unavailable during a cybersecurity event, a novel condition outside the NLP training set, or a time-critical decision where NLP latency is unacceptable, operators must interpret regulatory requirements independently. No legacy metric measures whether this independent capability is degrading. Human performance error rate tracks execution errors during actual operations. Licensed operator examination pass rates measure initial qualification, not ongoing fluency. Requalification programs test against a fixed scenario bank, not against the specific skill of independent regulatory interpretation without AI assistance. The invisible failure mode is that operators gradually lose the regulatory interpretation fluency they developed before NLP deployment, and the degradation is invisible because they never need to demonstrate the skill independently as long as the NLP is available.

Decision Supported

Are operators maintaining the regulatory interpretation skills they will need when the NLP system is unavailable, or is reliance on the tool degrading the independent capability the plant will need during exactly the conditions when the tool is least available?

Formula

Plain language: Administer a structured Tech Spec interpretation exercise to licensed operators. Present a plant configuration scenario, require them to identify applicable LCOs, completion times, and required actions without NLP assistance. Score accuracy and response time. Compare against the baseline from before NLP deployment (or from initial NLP deployment period when operators still had practiced fluency).

Notation:
RFAI = (Score_current − Score_baseline) / Score_baseline

Where:
Score = Accuracy × (T_allowed / T_actual)

Variables:

  • Accuracy = fraction of correct Technical Specification interpretations in the assessment (dimensionless, 0 to 1)
  • T_allowed = allowed response time for the assessment scenario (minutes: representing the time available for the decision in real operations)
  • T_actual = actual response time to produce the interpretation (minutes)
  • Score penalizes both inaccuracy and slowness. A correct-but-slow interpretation is operationally dangerous if the decision is time-critical.
  • Score_current = average Score across operators in the current assessment period
  • Score_baseline = average Score from baseline period (pre-NLP deployment or first 6 months post-deployment)

Sign convention: RFAI = 0.0 means no change from baseline. Negative values indicate atrophy (worse accuracy, slower response, or both). More negative is worse.

Aggregation period: Semi-annually (aligned with licensed operator requalification cycle).

Edge Cases

  • Operator turnover since baseline: new operators who were trained with NLP from the start have no pre-NLP baseline. Measure their absolute performance and compare to the legacy baseline as a cohort: the concern is whether the training pipeline produces operators who can function without NLP, not just whether individual operators have declined.
  • Assessment difficulty must be calibrated consistently across periods: use the same scenario bank or statistically equated scenario sets
  • Minimum sample: All licensed operators (the population is small enough for census-level measurement)

Confidence Grading (required alongside every RFAI value):

Grade Basis Interpretation
A: Empirical 5+ observed events in measurement window provide direct tail observation Measurement grounded in observed outcomes; highest confidence
B: Hybrid 1-4 observed events plus Monte Carlo simulation with Bayesian updating Blends observation and simulation; moderate confidence
C: Simulated Zero observed events; derived entirely from Monte Carlo simulation Modeled, not observed; treat as indicative, not validated

A RFAI of any value at Grade C is a simulation that agrees with itself. It tells you the model is internally consistent but not that it matches reality. Grade C scores should trigger proactive stress-testing against historical near-miss events and cross-asset analogues rather than acceptance as calibrated.

Proposed formula, pending validation pilots.

At a Glance

Unit
Ratio
Trend Direction
Higher
Reporting Frequency
Annually
Category
Radiation Protection and Human Capital
← ALARA Dose Equity Index (ADEI)Condition Report Significance Classification Accuracy (CRSCA) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment