Definition
Economic outcome of operator overrides compared against the estimated outcome had the AI recommendation been followed. Establishes whether overrides are adding value through field knowledge or destroying it through habit.
The Measurement Gap
AI recommendations in renewable operations span curtailment response, maintenance scheduling, dispatch routing, and storage dispatch. In every case, operators retain the ability to override the AI's recommendation. The question that no one systematically answers is whether those overrides are adding or destroying value.
Some overrides are clearly valuable. An operator who overrides a curtailment recommendation because they know the grid operator is about to revise the order is using local knowledge the AI does not have. A maintenance planner who overrides an AI schedule because they know a crane is already mobilized for a nearby site is exploiting an opportunity the optimizer did not model. These overrides improve outcomes.
Other overrides are destructive. An operator who overrides AI-optimized storage dispatch because "we always charge during off-peak" is imposing time-based habits on a system designed to optimize against real-time market signals. A planner who reverts AI-optimized maintenance sequences to traditional calendar-based schedules is negating the predictive model's value. These overrides degrade outcomes.
The measurement gap is that organizations do not measure the net economic impact of overrides, so they cannot distinguish between the valuable and the destructive. Without OVS, the default organizational response to overrides is political: some managers want to constrain overrides to protect AI investment, others want to preserve operator autonomy. OVS transforms this from a political debate into an economic analysis.
The decision OVS supports is whether the organization should encourage or constrain operator overrides, and specifically for which decision types human intervention is adding the most or least value.
Formula
For each override, estimate the economic outcome of the operator's decision and compare it against the estimated outcome if the AI's recommendation had been followed. A positive result means the override was valuable. A negative result means the override was costly. Average across all overrides, segmented by decision type.
Formula (Notation):
OVS(d) = (1/N_d) times the sum of (Outcome_Override_i minus Outcome_AI_Counterfactual_i)
Variable Definitions:
| Variable | Definition |
|---|---|
| d | decision type (curtailment, maintenance scheduling, dispatch, storage) |
| Outcome_Override_i | estimated economic outcome in dollars of the operator's override decision |
| Outcome_AI_Counterfactual_i | estimated economic outcome in dollars of the AI's original recommendation (requires counterfactual estimation) |
| N_d | number of overrides of decision type d |
Edge Cases and Measurement Boundaries:
Counterfactual estimation is inherently uncertain. Use conservative estimates and wide confidence intervals. Overrides during extreme events may have different characteristics than routine overrides; segment by operating condition. Safety-motivated overrides should be excluded from OVS or tracked separately as a "safety override" category, since their value cannot be measured in economic terms alone.
Proposed formula, pending validation pilots.