Congestion Forecast Calibration Score (CFCS)

GMI-EU-026Energy & UtilitiesPilot-Ready

Definition

Measures whether the AI congestion forecast's predicted congestion severity at each node corresponds to actual congestion outcomes in settlement, segmented by confidence level.

The Measurement Gap

Traditional transmission and generation planning relies on deterministic contingency analysis and static congestion studies. These approaches produce congestion assessments but do not evaluate whether the AI forecast model's confidence tiers correspond to actual congestion outcomes, and they certainly do not identify which nodes are systematically under- or over-predicted at each confidence level.

The specific fall-short: a congestion forecast that is accurate at the 50th percentile across the entire network can still be badly wrong (and dangerously wrong) at specific high-consequence nodes. Aggregate accuracy statistics obscure node-level bias. FTR managers and generation schedulers need to know which nodes they can trust the forecast at, not just whether the network-level average was close.

The exact operational decision CFCS enables:

which nodes and confidence bands can support generation pre-positioning and FTR hedging, and which require additional uncertainty margins or manual override.

What goes wrong without this information:

the utility pre-positions generation based on a congestion forecast that appears accurate in aggregate but is systematically biased at specific nodes. FTR positions are valued on congestion paths where the model consistently under-forecasts congestion severity. Losses accumulate in individual nodes and positions that never triggered a portfolio-level alert.

How AI/ML makes this measurement possible:

probabilistic congestion forecasting tools now produce node-level confidence scores at each market interval. Settlement systems produce realized congestion costs at the same nodal resolution. The calibration computation (comparing predicted confidence-banded costs to realized costs by node) is a natural extension of the data that AI-era forecasting already generates. Without the AI's probabilistic outputs, there would be no confidence bands to evaluate.

Formula

CFCS(b) = Actual_congestion_cost(b) / Predicted_congestion_cost(b)

Variables:

  • b: confidence band of the congestion forecast (expressed as probability tiers (0–25%, 25–50%, 50–75%, 75–100% congestion probability)) Source: congestion forecasting engine output

Actual_congestion_cost(b): realized congestion costs at nodes where the AI predicted congestion in band b ($/MWh). Source: ISO/RTO LMP decomposition from settlement system

Predicted_congestion_cost(b): model-forecasted congestion costs for those same nodes and intervals ($/MWh). Source: congestion forecasting engine

Calculation Frequency: Computed per node per quarter. Requires a minimum of 20 congestion events per node per quarter for statistical stability.

Edge Cases:

Nodes with zero predicted congestion that experience actual congestion are classified as "blind spots" and tracked separately as a miss rate rather than as a CFCS score. When transmission topology changes mid-period, segment CFCS into pre- and post-topology windows to avoid conflating regime changes with model error.

Output:

Ratio at each confidence band. Perfect calibration: CFCS = 1.0 at every band. Values above 1.0 indicate underestimation of congestion severity: the more dangerous direction because it leads to under-hedging. Values below 1.0 indicate overestimation: costly but conservative. Workable range: 0.85–1.15 across all bands. Calibration failure threshold: any band exceeding 1.3 or falling below 0.7 triggers model investigation.

Proposed formula, pending validation pilots.

At a Glance

Category
Grid Operations, Dispatch, and Edge Autonomy

Decision Supported

Can we trust the congestion forecast for generation pre-positioning and FTR management, and at which nodes is the model systematically biased?

← Restoration Routing Deviation Attribution (RRDA)Dispatch Model Regime Alignment Index (DMRAI) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment