Definition
CPAQ measures how accurately an AI complaint aggregation system identifies real system events from clusters of customer complaints in space and time.
The Measurement Gap
The Measurement Gap
What Legacy Metrics See:
Customer service systems track complaint volumes, response times, and resolution rates. GIS-linked complaint systems can map complaints geographically. Advanced utilities may have heat maps showing complaint density by area and time period. SCADA systems independently track pressure, flow, and water quality at fixed monitoring points.
What Legacy Metrics Miss:
No standard metric evaluates whether the utility's complaint aggregation system accurately translates individual complaints into system event detection. A utility receiving 15 taste-and-odor complaints from a single pressure zone over 4 hours may be experiencing a treatment upset, a main break with sediment disturbance, or simply a coincidence of unrelated individual issues. The aggregation system's ability to distinguish real events from noise determines whether operations staff receive a meaningful early warning or a steady stream of false alarms that they eventually learn to ignore. Without CPAQ, the utility has no way to measure whether its complaint intelligence system is actually providing useful operational intelligence or just generating noise.
The Decision This KPI Enables:
CPAQ enables the utility to calibrate its complaint aggregation thresholds: how many complaints, in how small an area, over how short a time, trigger an event investigation. When CPAQ shows high false event rates, the utility should raise the aggregation threshold (require more complaints or tighter spatial-temporal clustering before declaring an event). When CPAQ shows low detection rates, the threshold is too high and real events are being missed. The calibration is system-specific: a dense urban system with high complaint volume needs different thresholds than a rural system where any complaint cluster is significant.
Relevant Research:
Research on spatial-temporal anomaly detection in urban water systems demonstrates that customer complaints can provide early detection of contamination events 2–6 hours before fixed monitoring stations detect changes, but only when aggregation algorithms are properly calibrated. False alarm rates in complaint-based detection systems range from 15% to 45% depending on aggregation parameters and complaint classification quality.
Formula
Plain language: For known system events (confirmed main breaks, boil water advisories, pressure zone failures), check whether the complaint aggregation system correctly identified the event from the complaint pattern. For periods with no system events, check whether the system generated false event alerts from coincidental complaint clusters.
Notation:
CPAQ = (Event_detection_rate x 0.6) + ((1 - False_event_rate) x 0.4)
Where:
- Event_detection_rate = confirmed system events correctly identified by complaint aggregation / total confirmed system events with 3 or more associated complaints
- False_event_rate = false event alerts / total event alerts generated
- Weight event detection higher (0.6) because missing a real event is worse than a false alert
Units: 0 to 1. 1.0 = perfect detection with no false alerts. The two sub-components tell different stories: high detection with high false alerts is a noisy system; low detection with low false alerts is a blind system.
Proposed formula, pending validation pilots.