Checkpoint Review Depth Index (CRDI)

GMI-SWD-026Software DeliveryPilot-Ready

Definition

A behavioral composite score measuring whether human governance checkpoints are receiving substantive review or being rubber-stamped, based on review duration, modification rate, and comment density relative to artifact complexity.

The Measurement Gap

There is no metric that tells you whether human governance checkpoints are receiving substantive review or being rubber-stamped, which means the human-in-the-loop architecture may provide compliance theater rather than actual oversight.

Formula

Plain language: For each checkpoint review, score three dimensions: (1) Did the reviewer spend a reasonable amount of time relative to the artifact's complexity? (2) Did the reviewer make any modifications or add comments? (3) Is the approval pattern consistent with substantive review (not 100% approve, not instant approval)? Combine these into a single depth score.

Notation:

CRDI = (w_time × S_time + w_mod × S_mod + w_pattern × S_pattern) × 100

Where weights sum to 1.0: w_time = 0.4, w_mod = 0.35, w_pattern = 0.25.

Sub-scores (each normalized 0–1):

S_time = min(1.0, review_duration / expected_duration)
where expected_duration = artifact_LOC × 0.5 minutes (baseline: 30 seconds per line of changed code)

S_mod = modifications_or_comments / max(1, expected_modifications)
where expected_modifications = max(1, artifact_LOC / 50): expect at least 1 comment per 50 lines

S_pattern = 1 - |approval_rate_reviewer_30d - 0.85|
where 0.85 is the expected approval rate for a reviewer doing substantive review
(100% approval → S_pattern = 0.15; 85% approval → S_pattern = 1.0; 50% approval → S_pattern = 0.65)

Variables:

  • review_duration = time between checkpoint presented and checkpoint approved/rejected (minutes, NATIVE from platform governance log)
  • artifact_LOC = lines of code changed in the artifact under review (count, NATIVE from source control diff)
  • modifications_or_comments = count of changes made or comments added during review (count, NATIVE from review log)
  • approval_rate_reviewer_30d = the individual reviewer's checkpoint approval rate over the past 30 days (proportion, DERIVED)

Proposed formula, pending validation pilots.

At a Glance

Category
Governance Checkpoint Effectiveness

Decision Supported

Is the human-in-the-loop architecture providing genuine oversight, or has it degraded into compliance theater where developers approve AI output without meaningful review?

← Audit Trail Actionability Ratio (ATAR)AI Orchestration Net Value Index (AONVI) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment