Definition
Effort Estimation Anchoring Bias Index (EEABI) measures the systematic directional error in AI effort estimates compared to actual project outcomes.
In plain language: take a sample of completed projects.
The Measurement Gap
A platform engineering organization adopted an AI-driven effort estimator 14 months ago. The tool ingests Jira project history, learns the relationship between issue characteristics (description complexity, component, assignee, related issues) and actual effort, and produces effort estimates that the team uses for sprint planning and roadmap commitments. The team likes it: estimates appear quickly, with confidence intervals, and the dashboard reports estimation accuracy improving over time.
In a quarterly portfolio review, an engineering director notices a pattern. Over the past year, 73% of initiatives have come in over the AI estimate. The average overrun across all initiatives is 38%. The AI tool's "estimation accuracy" dashboard reports accuracy at 82%, but the calculation uses a different definition: a project counts as "accurate" if it completes within the 50% confidence interval, which is wide. By the team's actual experience, the estimates are systematically low.
Investigation reveals that the AI estimator was trained on historical Jira data: story points assigned by humans, then completed with actual time tracked. The historical data already encoded systematic underestimation: humans had been giving optimistic story point estimates for years, and projects had been overrunning consistently. The AI learned the underestimation pattern from the training data. Its "estimates" are essentially predictions of what humans would have estimated, not predictions of actual completion time. The team replaced human optimism with machine-learned optimism and called it data-driven.
The legacy portfolio metrics on the team's dashboard report a healthy estimation practice.
Estimation accuracy: 82% (per the AI tool's wide confidence interval).
Velocity: stable, trending up.
Sprint completion rate: 87%.
Estimation effort: down significantly (the AI does it faster than humans).
Estimate freshness: estimates are current at planning time.
Engineering satisfaction with estimation: tracked qualitatively, mostly positive.
None of these report on whether the estimates are systematically biased. The velocity metric absorbs the bias by treating consistent overruns as the new normal. The sprint completion rate uses scope completed against scope committed, where scope was committed based on the biased estimates. The accuracy metric uses a definition that hides directional bias.
These metrics were designed when humans produced estimates and humans assessed accuracy. When a human estimator was consistently optimistic, peers and managers noticed. With AI-generated estimates, the bias is harder to spot because the tool's outputs look authoritative. The historical underestimation pattern is now embedded in the tool and operates at scale.
There is no metric that tells you whether AI effort estimates are anchored to historical underestimation patterns rather than actual outcomes, which means "data-driven" estimates reproduce the same biases as the human estimates they replaced.
Formula
Effort Estimation Anchoring Bias Index (EEABI) measures the systematic directional error in AI effort estimates compared to actual project outcomes.
In plain language: take a sample of completed projects. For each, compute the percent difference between the AI's estimate and the actual effort consumed. Average those percent differences (preserving sign). The result is the directional bias. A value near 0 means the estimates are unbiased. A negative value means the estimates systematically underestimate. A positive value means they systematically overestimate.
The formula:
EEABI = (1/N) × Σ (Effort_estimated_i − Effort_actual_i) / Effort_actual_i
Where:
- Effort_estimated_i = the AI-generated effort estimate for project i (person-hours or story points)
- Effort_actual_i = the actual effort consumed by project i
- N = the count of completed projects in the measurement period
Units are a signed percentage. 0 means no systematic bias. Negative means underestimation. Positive means overestimation.
A worked example. The platform engineering organization runs EEABI on 25 completed initiatives from the trailing four quarters.
Sample of the data (person-hours):
| Project | Estimated | Actual | (E − A) / A |
|---|---|---|---|
| Auth refactor | 480 | 720 | −0.333 |
| Migration phase 1 | 1,200 | 1,580 | −0.241 |
| Dashboard v2 | 320 | 290 | +0.103 |
| API gateway | 880 | 1,340 | −0.343 |
| Notification service | 600 | 920 | −0.348 |
| ...19 more projects... |
Average across 25 projects:
EEABI = (1/25) × Σ (E_i − A_i) / A_i = −0.184 = −18.4%
Per the catalog guidance (between −10% and +10% reasonably unbiased, below −20% systematic underestimation) this EEABI of −18.4% is approaching but not yet crossing the action threshold. The trend matters: if EEABI was −12% the prior quarter and is −18.4% this quarter, the bias is growing and intervention is warranted. The team should retrain the estimator on actual outcomes rather than on historical estimates, accepting that the new estimates will likely look "higher" than current ones.
Three edge cases need explicit treatment. First, control for scope changes: projects whose scope expanded during execution should be excluded from EEABI calculation or analyzed separately, because the actual effort reflects expanded scope while the estimate reflected original scope. Second, exclude cancelled projects entirely; cancelled projects have indeterminate actual effort. Third, the minimum sample is 15 completed projects per measurement period; below that the average is too noisy.
Cross-references. Downstream of EEABI is Portfolio Value Prediction Accuracy (PVPA). If effort estimates are biased, the cost side of value-cost calculations used in portfolio decisions is also biased.
Proposed formula, pending validation pilots.