Damage Estimation Segment Bias (DESB)

GMI-INS-015InsuranceIntegration-Required

Definition

Measures the systematic bias in AI computer vision damage estimates by property type, construction class, or damage category, revealing segments where the model overestimates or underestimates relative to expert assessment.

The Measurement Gap

Computer vision models are increasingly used to estimate property damage from photographs and aerial imagery. These models can process claims at a speed human adjusters cannot match, enabling straight-through processing for routine damage estimates. But speed is not accuracy, and aggregate accuracy is not segment accuracy.

A model can achieve 95% accuracy on average while systematically overestimating hail damage on wood frame structures by 18% and underestimating fire damage on manufactured housing by 22%. The average hides the bias. Segments where the model overestimates create claims leakage: the carrier pays more than the damage warrants. Segments where the model underestimates create customer harm and errors-and-omissions litigation risk: the carrier pays less than the damage warrants.

Legacy metrics for AI damage estimation report aggregate accuracy against expert assessment. They do not decompose performance by property segment to reveal systematic directional bias. A claims director reviewing aggregate model accuracy sees a number that looks acceptable while specific property types receive materially incorrect estimates every time.

DESB closes this gap by computing the mean signed error by property segment, revealing not just where the model is inaccurate but in which direction it is wrong. Overestimation and underestimation have different business consequences and require different remediation strategies.

Decision Supported:

For which property types and damage categories is the computer vision model accurate enough to rely on, and for which should the carrier require human inspection to supplement or override the AI estimate?

Formula

Plain Language: For a sample of AI-estimated claims, compare the AI damage estimate against expert assessment. Compute the mean signed error (bias) by property segment. Segments with consistent positive bias indicate overestimation; segments with consistent negative bias indicate underestimation.

Formula:

DESB(s) = (1/N_s) x Sum of (Estimate_AI_i - Estimate_expert_i) / Estimate_expert_i x 100

Variables:

  • s: property segment (construction class multiplied by damage type, e.g., "wood frame / hail," "commercial flat roof / wind," "manufactured housing / fire")
  • Estimate_AI_i: AI computer vision damage estimate for claim i in segment s (dollars)
  • Estimate_expert_i: expert assessor's damage estimate for the same claim (dollars)
  • N_s: number of sampled claims in segment s
  • Positive DESB(s) indicates overestimation in segment s. Negative indicates underestimation.

Units:

Signed percentage. Report both segment bias and absolute magnitude.

Data Requirements:

Data Element Source System Classification
AI damage estimates with property/damage classification Computer vision estimation platform (Tractable, Verisk, Cape Analytics) NATIVE
Expert field assessments for sampled claims Structured audit program / independent adjusting firms MANUAL
Property characteristics (construction, occupancy, age) Underwriting data / property databases NATIVE
Damage type classification Claims management system NATIVE

Proposed formula, pending validation pilots.

At a Glance

Unit
Percentage
Trend Direction
Lower; approaching 0
Reporting Frequency
Quarterly
Category
AI Assessment Accuracy -- Vision and IoT
← Emerging Liability Blind Spot Index (ELBSI)IoT Signal Degradation Index (IOTDI) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment