Definition
Measures whether human overrides of autonomous treatment systems improve or degrade process outcomes compared to what the AI system recommended, reported as the fraction of overrides that produced equal or better outcomes.
The Measurement Gap
When operators override autonomous dosing, aeration, or pump scheduling systems, no metric distinguishes value-adding overrides (operator correctly catches an AI mistake during unusual conditions) from value-destroying overrides (operator negates valid optimization based on habit, distrust, or outdated experience). PM completion rate, compliance rate, and process quality metrics do not differentiate between AI-guided operations and operator-overridden operations. A plant could have a 30% override rate where most overrides are destroying value (consuming chemicals unnecessarily, reducing energy savings, or introducing process variability) and nobody knows because no metric tracks what the AI would have done versus what the operator did instead.
Decision Supported:
Are operators adding value when they override the AI, or are they systematically negating valid optimization, and which operators or conditions produce value-adding versus value-destroying overrides?
Formula
Plain language: Every time an operator overrides the autonomous system, record what the AI recommended and what the operator did instead. Then look at the process outcome over the next relevant period. If the outcome was at least as good as what the AI predicted its recommendation would achieve, score the override as value-adding. Divide value-adding overrides by total overrides.
Notation:
OVI = Σ Vᵢ / N
Where Vᵢ = 1 if Oᵢ ≥ Pᵢ (outcome met or exceeded AI-predicted outcome), Vᵢ = 0 if Oᵢ < Pᵢ (outcome worse than predicted).
Variables:
| Variable | Definition | Unit |
|---|---|---|
| Oᵢ | Actual process outcome during override period i, measured as composite of quality compliance and cost efficiency | Dimensionless score |
| Pᵢ | AI-predicted outcome had its recommendation been followed for event i | Dimensionless score |
| N | Total override events in reporting period | Count |
| Vᵢ | Binary value indicator: 1 if override improved or maintained outcome, 0 if it degraded outcome | Binary |
Proposed formula, pending validation pilots.