Definition
Percentage of AI-generated or AI-revised maintenance procedures that are fully aligned with the current as-built plant configuration, with no steps referencing components, flow paths, or configurations that have been modified since the design basis drawings the AI used.
The Measurement Gap
AI systems generating or revising maintenance procedures must work from current plant configuration data. When the AI uses design-basis drawings that do not reflect completed modifications, temporary changes, or in-progress design changes, the resulting procedures may contain steps that reference components, flow paths, or configurations that no longer exist in the as-built plant. No legacy metric measures procedure-to-configuration alignment at the point of AI generation. Procedure revision metrics track the number of procedures revised and the timeliness of revision cycles. Human performance error rates capture execution errors, some of which are caused by procedure-plant mismatches. Neither measures the AI's accuracy in producing procedures that match the actual plant. The invisible failure mode is that the AI generates procedures that are technically correct for the design-basis configuration but wrong for the as-modified plant, and the mismatch is discovered only when a field worker encounters a step that does not match the physical plant.
Decision Supported
Can AI-generated procedures be issued to the field after standard technical review, or do they require a configuration-specific walkdown to catch mismatches between the procedure and the actual plant?
Formula
Plain language: Sample AI-generated or AI-revised procedures before field issuance. For each, verify every procedural step that references a physical component, flow path, valve lineup, or instrument location against the current plant configuration (including temp mods, design changes, and as-built deviations). Score each procedure as: Aligned (all steps match current configuration), Minor Mismatch (steps reference components that exist but in different locations or with different identifiers than the AI used (cosmetic, correctable in the field), or Major Mismatch (steps reference a flow path, isolation point, or component arrangement that doesn't match current configuration) could cause execution error or plant event).
Notation:
PCAR = (Aligned + 0.75 × Minor_Mismatch) / N_procedures
Variables:
- Aligned = procedures where every step matches current as-built configuration (count)
- Minor_Mismatch = procedures with cosmetic mismatches (component ID changes, minor location differences) that wouldn't cause execution errors (count)
- Major_Mismatch = procedures with flow path, isolation, or arrangement mismatches that could cause execution errors (count)
- N_procedures = total AI-generated/revised procedures evaluated (count)
Sign convention: 1.0 = perfect alignment. Lower is worse. Major Mismatches are the safety-critical failure mode.
Aggregation period: Monthly rolling, stratified by procedure type (operating, maintenance, surveillance, emergency).
Edge Cases
- Procedure step references "as applicable" or conditional steps: evaluate whether the AI correctly identified which conditions apply to the current configuration
- Procedure for a system that has had no modifications: trivially aligned. Include in baseline but focus trend analysis on systems with active modification history
- Minimum sample: 20 procedures per month (prioritize procedures for systems with recent modifications)
Proposed formula, pending validation pilots.