Definition
The degree to which a witness selection or investigator selection tracks the rank order the algorithm produced rather than independent judgment.
The Measurement Gap
There is no metric that tells you how strongly a witness or investigator selection tracks the rank order an algorithm produced, which means an identification procedure can confirm the algorithm while being recorded as testing it.
Formula
CRIF = ( P_observed(rank 1 selected) - 1/m ) / ( 1 - 1/m ).
Proposed formula, pending validation pilots.
At a Glance
Decision Supported
Whether an identification procedure built downstream of an algorithmic ranking is independent enough to be relied on or disclosed as influenced.