Pipe Condition Grade Validation Rate (PCGVR)

GMI-WT-023Water TreatmentInfrastructure-DependentPattern O-7: Inspection Coverage

Definition

PCGVR measures what fraction of AI-assigned pipe condition grades have been validated against physical evidence from excavations, inspections, or failure analysis, and how accurately those grades matched observed conditions.

The Measurement Gap

Measurement Gap and Decision Supported

The Invisible Failure Mode: A large water utility uses a machine learning model to assign condition grades (1–5) to its 2,800 miles of distribution pipe. The model uses pipe material, age, soil type, break history, and operating pressure to predict condition. The utility bases its 10-year capital improvement program on these grades, prioritizing Grade 4 and 5 pipes for replacement. But the model has been physically validated against only 3.1% of the network: the segments where emergency repairs or planned projects happened to expose pipe. In those exposed segments, the model is 81% accurate for cast iron but only 54% accurate for ductile iron installed between 1975 and 1990. The capital program allocates $47M to ductile iron replacement based on condition grades that are wrong nearly half the time for that specific cohort.

What Existing Metrics Miss:

Model accuracy metrics reported at commissioning use whatever validation data existed at that time and are rarely updated. Break rate tracking tells you where pipes are failing but not whether the model predicted those failures correctly. Without PCGVR, utilities cannot distinguish between condition grades backed by physical evidence and grades that are purely algorithmic projections with no ground truth.

The Decision PCGVR Enables:

For which pipe cohorts (material, vintage, diameter, soil type) does the AI condition model have sufficient validated accuracy to drive capital decisions, and for which cohorts should grades be treated as unvalidated estimates requiring supplemental investigation before committing capital? PCGVR converts a single model accuracy number into a cohort-specific validation map.

Cross-References:

PCGVR connects directly to Inspection Classification Equity Index (ICEI, GMI-WT-008), which measures whether inspection-based classification models exhibit systematic bias across asset categories. Both address the same fundamental problem (AI classification accuracy varies by cohort) but ICEI focuses on inspection classification while PCGVR focuses on condition grading. PCGVR also connects to LDCC (GMI-WT-022) because detection dead zones and condition validation gaps often overlap in the same network segments. Capital Allocation Fidelity Index (CAFI, GMI-WT-013) is directly affected by PCGVR: unvalidated condition grades that drive capital recommendations create CAFI misalignment.

Formula

Plain language: Two components. First, the validation rate: what fraction of AI-graded pipes have been field-verified? Second, the agreement rate: of those verified, what fraction had AI grades within one grade of the field-observed condition?

Notation:
PCGVR_coverage = Validated_segments / AI_graded_segments
PCGVR_accuracy = Correct_within_one_grade / Validated_segments

Where:

  • AI_graded_segments = total pipe segments with AI condition grades
  • Validated_segments = segments where field observation (excavation, failure forensics, coupon testing) provides ground truth
  • Correct_within_one_grade = validated segments where AI grade was within plus or minus 1 of field-observed grade on a 5-point scale

Units: Two percentages. Report both. High accuracy on a tiny sample (coverage = 2%) is not trustworthy. Low accuracy on a large sample is a different kind of problem.

Proposed formula, pending validation pilots.

At a Glance

Category
Distribution System Intelligence
← Leak Detection Coverage Confidence (LDCC)Twin Calibration Divergence Rate (TCDR) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment