Definition
The correlation between orchestration chain depth (number of sequential agent steps before human checkpoint) and the rate of quality deviations (checkpoint rejections, post-deployment defects, rework requests), indicating whether deeper orchestration chains degrade outcome quality.
The Measurement Gap
There is no metric that tells you whether increasing orchestration chain depth is degrading human review effectiveness, which means the governance checkpoint structure can exist in form while eroding in substance.
Formula
Plain language: Group all orchestrated tasks by their chain depth (how many agent steps occurred before the first human checkpoint). For each depth level, calculate the quality deviation rate. Plot depth against deviation rate. The ODII is the slope of this relationship, a positive slope means deeper chains produce more quality problems.
Notation:
ODII = r(depth, deviation_rate)
Where r is the Pearson correlation coefficient between orchestration depth and quality deviation rate across all depth levels with sufficient sample size.
deviation_rate_d = (rejections_d + defects_d + rework_d) / T_d
Variables:
- depth = number of sequential agent steps before human checkpoint (integer, NATIVE from orchestration log)
- deviation_rate_d = quality deviation rate at depth level d (proportion, DERIVED)
- rejections_d = checkpoint rejections for tasks at depth d (count, NATIVE)
- defects_d = defects traced to artifacts produced at depth d within 30 days (count, INTEGRATION)
- rework_d = rework requests on artifacts produced at depth d (count, INTEGRATION)
- T_d = total tasks at depth d (count, NATIVE)
Proposed formula, pending validation pilots.
At a Glance
Decision Supported
As we chain more agent steps before human review, is quality holding steady or degrading? Should we set a maximum orchestration depth for different task types?