Definition
A behavioral composite score measuring whether human governance checkpoints are receiving substantive review or being rubber-stamped, based on review duration, modification rate, and comment density relative to artifact complexity.
The Measurement Gap
There is no metric that tells you whether human governance checkpoints are receiving substantive review or being rubber-stamped, which means the human-in-the-loop architecture may provide compliance theater rather than actual oversight.
Formula
Plain language: For each checkpoint review, score three dimensions: (1) Did the reviewer spend a reasonable amount of time relative to the artifact's complexity? (2) Did the reviewer make any modifications or add comments? (3) Is the approval pattern consistent with substantive review (not 100% approve, not instant approval)? Combine these into a single depth score.
Notation:
CRDI = (w_time × S_time + w_mod × S_mod + w_pattern × S_pattern) × 100
Where weights sum to 1.0: w_time = 0.4, w_mod = 0.35, w_pattern = 0.25.
Sub-scores (each normalized 0–1):
S_time = min(1.0, review_duration / expected_duration)
where expected_duration = artifact_LOC × 0.5 minutes (baseline: 30 seconds per line of changed code)
S_mod = modifications_or_comments / max(1, expected_modifications)
where expected_modifications = max(1, artifact_LOC / 50): expect at least 1 comment per 50 lines
S_pattern = 1 - |approval_rate_reviewer_30d - 0.85|
where 0.85 is the expected approval rate for a reviewer doing substantive review
(100% approval → S_pattern = 0.15; 85% approval → S_pattern = 1.0; 50% approval → S_pattern = 0.65)
Variables:
- review_duration = time between checkpoint presented and checkpoint approved/rejected (minutes, NATIVE from platform governance log)
- artifact_LOC = lines of code changed in the artifact under review (count, NATIVE from source control diff)
- modifications_or_comments = count of changes made or comments added during review (count, NATIVE from review log)
- approval_rate_reviewer_30d = the individual reviewer's checkpoint approval rate over the past 30 days (proportion, DERIVED)
Proposed formula, pending validation pilots.
At a Glance
Decision Supported
Is the human-in-the-loop architecture providing genuine oversight, or has it degraded into compliance theater where developers approve AI output without meaningful review?