AI Assessment Anchoring Bias Index (AABI)

GMI-HCM-005Human Capital ManagementInfrastructure-DependentPattern O-21: Anchor Erosion

Definition

The degree to which human assessment scores are anchored by AI pre-scores, measured as the difference in AI-human correlation between conditions where humans see AI scores before evaluating versus conditions where humans evaluate blind.

The Measurement Gap

When a human evaluator sees an AI score before forming their own assessment, anchoring bias is virtually guaranteed. Decades of cognitive psychology research have established that exposure to any numerical anchor shifts subsequent judgment toward that anchor, even when the anchor is demonstrably random. In AI-assisted assessment, the anchoring effect is compounded by algorithm appreciation: the tendency to defer to algorithmic outputs as more objective, more data-driven, and more defensible than human intuition.

The result is that "human-in-the-loop" assessment becomes a performance rather than an independent evaluation. The human adjusts from the AI anchor rather than forming an independent judgment, and the organization loses the complementary signal that human evaluation was supposed to provide. Every metric that measures "human+AI combined accuracy" is compromised because the human score is no longer independent: it is a derivative of the AI score.

No legacy HR metric detects this corruption. Inter-rater reliability among human evaluators measures agreement but not anchoring. Assessment validity studies typically do not control for whether humans saw AI scores before evaluating. The organization assumes it has two independent assessment signals when it actually has one signal reflected through a slightly noisy mirror. The cost is invisible but real: human evaluation capacity is consumed without adding the independent predictive value it was designed to provide.

Decision Supported:

Are our human evaluators forming independent assessments, or are they anchored by AI pre-scores to the point where human evaluation adds no independent signal?

Formula

Plain language: Run an A/B protocol where some evaluators see AI scores before assessing (anchored condition) and others evaluate blind (unanchored condition). AABI is the difference in AI-human score correlation between the two conditions.

Notation:

AABI = r(human, AI | anchored) − r(human, AI | blind)

Variables:

Variable Definition Unit
r(human, AI | anchored) Pearson correlation between human assessment scores and AI scores in the condition where humans see AI scores before evaluating Correlation (-1 to +1)
r(human, AI | blind) Pearson correlation between human assessment scores and AI scores in the condition where humans evaluate without seeing AI scores Correlation (-1 to +1)

Proposed formula, pending validation pilots.

At a Glance

Unit
Difference (-1 to 1)
Trend Direction
Lower; approaching 0
Reporting Frequency
Quarterly (requires A/B protocol)
Category
Human-AI Assessment Calibration
← Screening Score Predictive Validity Index (SSPVI)Assessment Disagreement Resolution Index (ADRI) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment