Definition
Measures the degree to which the SIU referral queue's composition reflects the current fraud threat landscape versus the model's historical training patterns, detecting whether emerging fraud schemes are being underscored and underinvestigated.
The Measurement Gap
Fraud detection models are trained on historical confirmed fraud. They learn the patterns that investigators have already identified, documented, and resolved. The model then scores incoming claims against those historical patterns and refers the highest-scoring claims to the SIU for investigation.
The problem is that fraud evolves. Organized fraud operations adapt their methods. New scheme types emerge that look nothing like the confirmed fraud the model learned from. The model continues to fill the SIU queue with claims that match historical patterns while emerging schemes receive lower scores and slip past the detection threshold.
Legacy fraud metrics report referral volumes, investigation outcomes, and recovery rates. They do not measure whether the composition of the referral queue matches the actual composition of fraud occurring in the current environment. A fraud team can report a 92% conviction rate on investigated referrals while missing entire categories of emerging fraud that never made it into the queue.
SQCI closes this gap by comparing the distribution of fraud scheme types in the SIU referral queue against the distribution of confirmed fraud outcomes from completed investigations. When the queue distribution diverges from the confirmed fraud distribution, the model is drifting, and investigators are chasing the wrong targets.
Decision Supported:
Is the fraud model sending investigators after today's fraud or yesterday's, and should the model be retrained or supplemented with rules targeting emerging scheme types the model has not learned yet?
Formula
Plain Language: Compare the distribution of fraud scheme types in the SIU referral queue against the distribution of confirmed fraud scheme types from completed investigations. If the queue is heavy on traditional schemes and light on emerging ones relative to confirmed outcomes, the model is drifting.
Formula:
SQCI = 1 - JSD(Q || C)
Where JSD(Q || C) = Jensen-Shannon Divergence between queue distribution and confirmed distribution.
Variables:
- Q: probability distribution of fraud scheme types in the current SIU referral queue (percentage of referrals by scheme type)
- C: probability distribution of fraud scheme types in confirmed fraud outcomes from the prior 12 months (percentage of confirmed cases by scheme type)
- JSD: Jensen-Shannon Divergence (0 = identical distributions, 1 = maximally different)
Units:
0 to 1. SQCI near 1.0 means the queue closely matches the confirmed fraud landscape. SQCI declining toward 0 means the queue composition is diverging from actual fraud patterns and the model is drifting.
Data Requirements:
| Data Element | Source System | Classification |
|---|---|---|
| SIU referral records with scheme type classification | SIU case management / fraud analytics platform | NATIVE |
| Referral source (AI model vs. human) | Fraud analytics platform | NATIVE |
| Confirmed fraud outcomes with scheme type | SIU case resolution records | NATIVE |
| Fraud scheme taxonomy | SIU operational definitions (map to NICB/CAIF categories or internal taxonomy) | MANUAL (define once) |
Proposed formula, pending validation pilots.