Orchestration Depth Impact Index (ODII)

GMI-SWD-025Software DeliveryPilot-Ready

Definition

The correlation between orchestration chain depth (number of sequential agent steps before human checkpoint) and the rate of quality deviations (checkpoint rejections, post-deployment defects, rework requests), indicating whether deeper orchestration chains degrade outcome quality.

The Measurement Gap

There is no metric that tells you whether increasing orchestration chain depth is degrading human review effectiveness, which means the governance checkpoint structure can exist in form while eroding in substance.

Formula

Plain language: Group all orchestrated tasks by their chain depth (how many agent steps occurred before the first human checkpoint). For each depth level, calculate the quality deviation rate. Plot depth against deviation rate. The ODII is the slope of this relationship, a positive slope means deeper chains produce more quality problems.

Notation:

ODII = r(depth, deviation_rate)

Where r is the Pearson correlation coefficient between orchestration depth and quality deviation rate across all depth levels with sufficient sample size.

deviation_rate_d = (rejections_d + defects_d + rework_d) / T_d

Variables:

  • depth = number of sequential agent steps before human checkpoint (integer, NATIVE from orchestration log)
  • deviation_rate_d = quality deviation rate at depth level d (proportion, DERIVED)
  • rejections_d = checkpoint rejections for tasks at depth d (count, NATIVE)
  • defects_d = defects traced to artifacts produced at depth d within 30 days (count, INTEGRATION)
  • rework_d = rework requests on artifacts produced at depth d (count, INTEGRATION)
  • T_d = total tasks at depth d (count, NATIVE)

Proposed formula, pending validation pilots.

At a Glance

Category
Multi-Agent Coordination Integrity

Decision Supported

As we chain more agent steps before human review, is quality holding steady or degrading? Should we set a maximum orchestration depth for different task types?

← Cross-Agent Coherence Score (XACS)Remediation Learning Rate (RLR) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment