Definition
The share of the outcome observations a detection or alert model learns from that were censored by its own prior decisions: estrus outcomes observed only on cows the system flagged and the herd bred, disease outcomes observed only where the alert triggered a spray, weed outcomes observed only where the classifier chose to spray.
The Measurement Gap
There is no metric that tells you what share of the outcomes a detection model learns from were censored by its own prior decisions (bred because flagged, sprayed because alerted), which means the model is scored correct against a ground truth it shaped.
Formula
SCOS = N_outcomes_under_own_constraint / N_outcomes_in_training, per model; companion: N_random_unflagged_observations per period
Proposed formula, pending validation pilots.
At a Glance
Decision Supported
Whether a model's reported accuracy is measured against a ground truth the model shaped, and how large a randomized unflagged sample is needed to restore an honest signal.