Representment Evidence Quality Index (REQI)

GMI-PAY-008Payments & Transaction ProcessingIntegration-Required

Definition

Win-rate differential between AI-assembled and human-assembled chargeback representment packages on comparable dispute types.

The Measurement Gap

The Measurement Gap

No metric tells you whether AI-assembled chargeback evidence is as effective as the human-assembled evidence the representment model was trained on. Chargeback representment (the process of contesting a cardholder dispute with supporting evidence) is increasingly handled by AI systems that select which chargebacks to fight and assemble the evidence packages submitted to the card network. The selection decision and the evidence quality are different problems, but they are conflated in the only metric most organizations track: win rate.

Win rate on contested chargebacks may decline for two entirely different reasons. The representment model may be selecting the wrong disputes to fight, choosing cases with low win probability. Or the model may be selecting the right disputes but assembling weaker evidence packages than human analysts would have produced for the same cases. The distinction matters because the remediation is completely different: one requires retraining the selection model while the other requires improving the evidence assembly pipeline.

Chargeback rate measures dispute volume. Win rate measures representment outcomes. Neither measures evidence quality: the specific contribution of the AI evidence assembly step to the representment outcome. Without a controlled comparison between AI-assembled and human-assembled evidence on comparable dispute types, there is no way to determine whether AI representment is performing at, below, or above human capability. The gap is compounded by the fact that as AI handles an increasing share of representments, the human comparison baseline shrinks, making future measurement progressively harder unless a control group is maintained now.

Decision Supported

Is the AI evidence assembly pipeline producing representment packages that win at the same rate as human-assembled packages for comparable dispute types?

Formula

Plain Language

For each dispute type and amount range, divide the win rate of AI-assembled representment packages by the win rate of human-assembled packages. A ratio of 1.0 means AI evidence is equally effective. Below 1.0 means AI evidence is weaker. Aggregate across dispute types weighted by volume.

Notation

Per-type: REQI = WinRate_AI(d,a) / WinRate_human(d,a)

Aggregate: REQI_agg = Σ(w_da × REQI_da) / Σ w_da

Variables

Variable Definition Unit
WinRate_AI(d,a) Representment win rate for AI-assembled evidence, by dispute type d and amount range a %
WinRate_human(d,a) Representment win rate for human-assembled evidence, by dispute type d and amount range a %
w_da Volume weight for dispute type d and amount range a Count
d Dispute type category (e.g., Visa reason code, Mastercard reason code) Category
a Transaction amount range $ range

Proposed formula, pending validation pilots.

At a Glance

Unit
Ratio
Trend Direction
Toward 1.0; Approaching 1.0 or above
Reporting Frequency
Quarterly (with 6-month lookback)
Category
Dispute & Chargeback Decision Quality
← Dispute Auto-Resolution Fairness Index (DARFI)Authentication Friction Yield (AFY) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment