Definition
The joint measure of how often accepted category suggestions proved wrong and how often overridden suggestions proved right.
The Measurement Gap
There is no metric that tells you how often an accepted category suggestion later proved wrong compared with how often an override proved correct, which means suggestion quality and human judgment quality are conflated in a single acceptance rate.
Formula
SAQ = (A_correct / A_total, O_correct / O_total), reported as an ordered pair
Proposed formula, pending validation pilots.
At a Glance
Decision Supported
Whether an acceptance rate is evidence of model quality or of user deference, and whether to invest in the model or in reviewer judgment.