Definition
Combined verification depth actually achieved by AI and human review gates on each change, measuring whether two approvals verify complementary concerns or the same shallow ones twice.
The Measurement Gap
Measurement Gap and Decision
The Gap: Two approval gates on every PR -- one AI, one human -- looks like doubled review capacity. But the dashboard does not show what those two reviews actually verify. The AI checks style, null dereferences, unused imports, and idiomatic patterns at depth level 2-3 on those dimensions. The human reviewer, seeing the AI has covered the surface, scans the PR, confirms the AI suggestions look reasonable, and approves. The human review becomes depth level 1 across most dimensions.
Before AI review, a human reviewer spent 25 minutes on a complex PR producing comments across 4-5 dimensions at depth level 2. Now the same reviewer spends 8 minutes with fewer substantive comments. Two approvals may represent less total verification depth than one thorough human approval pre-AI. No metric counts review substance rather than review occurrences.
The Decision:
ERDI estimates the net verification depth when AI and human review combine. It produces a single score answering: how deeply is this PR actually being scrutinized across all dimensions that matter? When ERDI declines from baseline, the intervention is structured checklists restoring depth on degraded dimensions.
Formula
Formula:
ERDI = (1/N) x SUM_PR [ (1/D) x SUM_d min(depth_human_d + depth_AI_d, 3) ]
Where:
- depth_human_d = depth score (0-3) of human review on dimension d
- depth_AI_d = depth score (0-3) of AI review on dimension d
- D = 8 (number of dimensions)
- N = number of PRs in the measurement period
- min(..., 3) caps combined depth at maximum meaningful level
Depth anchors: 0 = not addressed; 1 = mentioned superficially; 2 = specific observations with justification; 3 = thoroughly investigated with alternatives considered.
Worked Example:
Payment retry service PR. AI review depths: bug=2, arch=0, business=0, ops=1, style=3, perf=1, security=2, test=0. Human review depths: bug=1, arch=1, business=0, ops=0, style=1, perf=0, security=0, test=1.
Combined (capped at 3): bug=3, arch=1, business=0, ops=1, style=3, perf=1, security=2, test=1.
ERDI = (1/8) x (3+1+0+1+3+1+2+1) = 1.50
Pre-AI comparison for same reviewer on similar PR: bug=2, arch=2, business=2, ops=1, style=2, perf=1, security=1, test=1. Pre-AI ERDI = 1.50.
Same total score but different distribution. Pre-AI had more even coverage. Post-AI has peaks (bug, style) and valleys (business logic = 0). Track per-dimension depth alongside the composite.
Proposed formula, pending validation pilots.