Agent Claim Calibration Error (ACCE)

GMI-SOLO-007Solo AI OperationsPilot-ReadyPattern O-1: Confidence Calibration

Definition

Gap between how confidently agents present factual claims and how often those claims survive spot-checking in the operator's own domain.

The Measurement Gap

There is no metric for whether agent-stated confidence matches spot-checked accuracy in the operator's specific domain. Trust gets calibrated by the agent's tone rather than by its track record.

Formula

ACCE = sum over b of ( w_b * | acc_b - conf_b | )

Proposed formula, pending validation pilots.

At a Glance

Unit
Statistic (0 to 1)
Trend Direction
Lower
Reporting Frequency
Weekly spot-check, reported monthly
Category
Verification and Quality

Decision Supported

Whether to accept a claim or verify its sourcing, and what trust level to assign to a codified asset that relies on agent-produced facts.

← Platform Spend Yield (PSY)Operator Edit Value (OEV) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment