Definition
CDCAR measures the agreement rate between AI-generated PACP (Pipeline Assessment Certification Program) condition codes and expert human reviewers examining the same CCTV inspection footage.
The Measurement Gap
The Measurement Gap
What Legacy Metrics See:
Traditional CCTV inspection programs track footage reviewed per day, segments inspected per year, and defect counts by PACP grade. AI-assisted inspection platforms report processing speed (segments per hour), defect detection counts, and sometimes an overall accuracy percentage measured against the platform's own training data. Vendor accuracy claims typically cite 90%+ detection rates measured under controlled conditions.
What Legacy Metrics Miss:
No standard metric measures concordance between AI and expert human reviewers on the same footage under field conditions. Vendor accuracy claims are measured against curated datasets with clear defect examples, not against the ambiguous, poorly lit, partially obstructed footage that constitutes real-world CCTV inspection. The gap matters because rehabilitation decisions worth millions of dollars flow directly from PACP condition codes. If AI assigns a grade 4 fracture where a human expert sees a grade 2 crack, the utility may schedule emergency rehabilitation on a pipe that needs only monitoring, or vice versa. Without field-condition concordance measurement, the utility has no basis for trusting AI codes at any specific confidence level.
The Decision This KPI Enables:
CDCAR enables the utility to set evidence-based policies for AI inspection: which defect types can be AI-classified without human review, which require human confirmation, and which should be human-classified with AI assist only. When CDCAR exceeds 75% for a specific defect category, the utility can confidently reduce human review for that category. When CDCAR falls below 60%, the utility knows AI classification for that category is not reliable enough for autonomous use. This category-level policy is far more useful than a blanket "trust AI" or "don't trust AI" approach.
Relevant Research:
NASSCO studies on inter-rater reliability for PACP coding document agreement rates of 65–82% between experienced human coders, depending on defect type and severity. Research on AI-assisted pipe inspection shows that AI performance varies dramatically by pipe material, diameter, and defect type: with excellent performance on structural defects in concrete pipe but significantly lower accuracy for corrosion in metallic pipe.
Formula
Plain language: Sample AI-coded CCTV segments. Have a certified PACP reviewer independently code the same footage. Compare at the defect level: for each defect identified by either the AI or the human, did the other agree on presence, code, and grade?
Notation:
CDCAR = Agreed_defects / (Agreed + AI_only + Human_only)
Where:
- Agreed_defects = defects where both AI and human identified the defect AND assigned the same PACP code AND grade within plus or minus 1
- AI_only = defects identified by AI but not by human reviewer (false positives)
- Human_only = defects identified by human but missed by AI (false negatives)
Units: Percentage. Track sub-components: AI false positive rate, AI false negative rate, and grade agreement rate on agreed defects. False negatives (missed defects) are worse than false positives (extra defects) for rehab prioritization.
Proposed formula, pending validation pilots.