Disinfection Byproduct Precursor Model Currency Index (DPMCI)

GMI-WT-020Water TreatmentInfrastructure-DependentPattern O-2: Model Drift / Currency

Definition

DPMCI measures whether a disinfection byproduct precursor prediction model's accuracy has degraded relative to its performance at the time of last calibration.

The Measurement Gap

Measurement Gap and Decision Supported

The Invisible Failure Mode: A regional water utility calibrates a formation potential model using 18 months of paired raw water TOC, UV254, bromide, and finished water THM/HAA5 data. At calibration, the model predicts formation potential within ±8% of actual lab results. Over the next three years, the utility relies on the model to optimize coagulation for precursor removal and to project DBP compliance margins. Nobody rechecks calibration accuracy because the model continues to produce plausible-looking numbers. But source water TOC character has shifted (the ratio of humic to fulvic acids has changed due to upstream wetland restoration) and the model's predictions are now ±19% off actual values. The utility's projected compliance margin is an illusion built on a stale model.

What Existing Metrics Miss:

Standard DBP compliance monitoring catches the problem only after formation potential exceeds regulatory limits. Model accuracy metrics computed at deployment (R², RMSE against training data) become historical artifacts that do not reflect current performance. Without DPMCI, there is no systematic mechanism to detect the gap between a model's calibration-era performance and its current-era performance. The degradation is gradual and invisible because the model continues to produce outputs that look reasonable.

The Decision DPMCI Enables:

Is this DBP precursor model still current enough to drive treatment optimization decisions, or has source water change degraded its accuracy to the point where recalibration or retraining is required? DPMCI provides a quantitative trigger for recalibration rather than relying on operator intuition or compliance failures.

Cross-References:

DPMCI connects upstream to source water characterization and model training data vintage. Downstream, DPMCI degradation affects treatment optimization quality, a stale model drives suboptimal coagulant dosing, which manifests in Chemical Dosing Lag and Impact Score (CLDIS, GMI-WT-016). DPMCI also connects to WQPC (GMI-WT-019): WQPC measures confidence calibration at a point in time, while DPMCI measures accuracy drift over time. A model can be well-calibrated (good WQPC) but degrading (rising DPMCI), or poorly calibrated (poor WQPC) but stable (flat DPMCI). Both metrics are needed for complete model health assessment.

Formula

Plain language: Periodically run a formation potential test (THMFP or HAAFP) on raw water under standardized conditions. Compare the model's predicted formation under those same conditions to the lab result. The ratio of prediction error to baseline error (from the model's original calibration period) indicates whether the model is losing currency.

Notation:
DPMCI = |Predicted_FP - Actual_FP|_current / |Predicted_FP - Actual_FP|_calibration

Where:

  • Predicted_FP = model-predicted formation potential under standardized test conditions (ug/L)
  • Actual_FP = lab-measured formation potential under the same conditions (ug/L)
  • current = most recent formation potential test
  • calibration = average prediction error during the model's original calibration period

Units: Ratio. 1.0 = model accuracy unchanged since calibration. >1.0 = model accuracy degrading (precursor assumptions drifting). <1.0 = model accuracy improving. Track trend quarterly.

Proposed formula, pending validation pilots.

At a Glance

Category
Water Quality Prediction Reliability
← Regulatory Event Classification Accuracy (RECA-WT)Prediction-Measurement Substitution Gap (PMSG) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment