Definition
PMSG measures the weighted fraction of regulatory or operationally critical water quality parameters where the utility relies on AI predictions rather than direct measurements for decision-making.
The Measurement Gap
Measurement Gap and Decision Supported
The Invisible Failure Mode: A medium-sized drinking water utility deploys AI-based water quality prediction models for turbidity, chlorine residual, and THM formation potential. The models perform well during commissioning, and over the following 18 months, operational managers gradually reduce manual sampling frequency for parameters where predictions consistently match lab results. Turbidity grab samples drop from 8 per day to 3. THM formation potential tests drop from weekly to monthly. The operational budget improves, and nobody notices a problem: until a source water contamination event introduces a compound the model was never trained to detect. The reduced sampling frequency means the contamination goes unnoticed for 6 hours longer than it would have under the original sampling regime. The prediction model was performing exactly as designed; the failure was in the substitution decision, not the model.
What Existing Metrics Miss:
Model accuracy metrics tell you whether the model is currently right. They do not tell you how much measurement coverage you have lost by relying on the model. A utility can have excellent WQPC and favorable DPMCI while simultaneously operating with dangerous measurement blind spots. No standard water quality metric tracks the progressive substitution of predictions for measurements. PMSG fills this governance gap.
The Decision PMSG Enables:
For which water quality parameters has prediction substitution exceeded the risk-appropriate threshold, and where must direct measurement frequency be restored or maintained? PMSG provides a weighted inventory of substitution risk, enabling utilities to set parameter-specific ceilings on acceptable prediction reliance.
Cross-References:
PMSG connects directly to WQPC (GMI-WT-019) and DPMCI (GMI-WT-020). Substitution is only defensible when WQPC confirms calibration quality and DPMCI confirms model currency. If either degrades, PMSG thresholds should tighten. PMSG also connects to Regulatory Event Classification Accuracy (RECA, GMI-WT-029): over-reliance on predictions for regulatory parameters may affect the accuracy of regulatory event detection and classification.
Formula
Plain language: For each water quality parameter at each monitoring location, calculate the fraction of time covered by direct measurement (grab samples or online analyzers) vs. the fraction covered by model predictions. Weight by the public health significance of the parameter.
Notation:
PMSG = Sum(w_p x (1 - Coverage_measured_p)) / Sum(w_p)
Where:
- Coverage_measured_p = fraction of time parameter p is directly measured at the decision point (online analyzer uptime, or sample frequency x representativeness window)
- w_p = public health significance weight (acute health effects [e.g., microbial, nitrate]: w=3.0; chronic health effects [e.g., lead, DBPs]: w=2.0; aesthetic [e.g., taste, odor, color]: w=1.0)
- Summed across all monitored parameters and locations
Units: 0 to 1. 0 = everything is directly measured at all times. 1 = everything relies on model prediction. Most utilities will be 0.3-0.7.
Proposed formula, pending validation pilots.