Review Attention Displacement Index (RADI)

GMI-SWD-004Software DeliveryInfrastructure-DependentPattern O-24: Attention Displacement

Definition

Degree to which visible AI review feedback displaces human reviewer attention away from the design, domain, and integration concerns the AI cannot evaluate.

The Measurement Gap

Measurement Gap and Decision

The Gap: AI code review tools produce visible, actionable feedback on surface-level issues -- style, null safety, common bug patterns. Human reviewers see this feedback, implicitly conclude the thorough checking has been done, and shift their limited attention toward the same dimensions the AI already covers. Meanwhile, dimensions the AI cannot assess -- architectural fitness, business logic correctness, operational readiness, cross-service interaction safety -- receive less human attention than before the AI tool existed.

This is attention displacement. DORA metrics do not detect it because they measure throughput, not review quality distribution. Comment counts do not detect it because total comments increase while the distribution across quality dimensions shifts. Review time does not detect it because reviews may take the same time but cover a narrower range of concerns.

The Decision:

RADI measures the degree to which human reviewers concentrate feedback on dimensions AI tools also cover while reducing feedback on dimensions AI tools do not cover. When RADI shows negative displacement on critical dimensions (architectural fitness, business logic), the intervention is structural: required review checklists for uncovered dimensions, not removing AI review.

Formula

Formula:

RADI(dim_d) = Proportion_human_d_post_AI - Proportion_human_d_pre_AI

Where:

  • Proportion_human_d_post_AI = (human comments in dimension d during post-AI period) / (total human comments during post-AI period)
  • Proportion_human_d_pre_AI = (human comments in dimension d during pre-AI period) / (total human comments during pre-AI period)

Review dimensions (8 categories): bug_detection, architectural_fit, business_logic, operational_readiness, style, performance, security, testability.

Worked Example:

Pre-AI baseline (Q4-Q1): 842 human comments. Post-AI (Q2-Q3): 1,106 human comments.

RADI per dimension:

  • bug_detection: 26.0% - 20.0% = +0.06 (shifted toward -- AI covers this, humans followed)
  • architectural_fit: 10.0% - 17.0% = -0.07 (shifted away -- AI weak here, humans withdrew)
  • business_logic: 9.0% - 14.0% = -0.05 (shifted away -- AI cannot assess this)
  • operational_readiness: 5.0% - 9.0% = -0.04 (shifted away)
  • style: 28.0% - 22.0% = +0.06 (shifted toward -- AI covers this extensively)
  • performance: 11.0% - 7.0% = +0.04 (shifted toward)
  • security: 7.0% - 6.0% = +0.01 (minimal shift)
  • testability: 4.0% - 5.0% = -0.01 (minimal shift)

Critical finding: RADI(architectural_fit) = -0.07 and RADI(business_logic) = -0.05. Human attention is moving away from the two dimensions where AI coverage is weakest.

Proposed formula, pending validation pilots.

At a Glance

Unit
Index
Trend Direction
Lower
Reporting Frequency
Quarterly
Category
Review and Verification Integrity
← Effective Review Depth Index (ERDI)Test Specification Alignment Score (TSAS) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment