Definition
The degree to which flight-risk probability scores produced by an attrition prediction model match actual departure rates within each probability band.
The Measurement Gap
Attrition prediction models are now deployed across organizations of every size, producing flight-risk scores that drive retention interventions, succession planning, and management attention. These models output probability scores ("this employee has a 75% chance of leaving in the next 12 months") and organizations act on those probabilities as if they were calibrated. But calibration (the property that a stated 75% probability actually corresponds to a 75% departure rate) is almost never validated.
Legacy turnover metrics measure aggregate departure rates but cannot evaluate model accuracy at the individual level. Overall attrition rate tells you how many people left but not whether the model correctly identified who would leave. Even model accuracy metrics like AUC-ROC (if computed at all) measure ranking ability, not calibration, a model can perfectly rank employees by risk and still assign probability scores that are wildly miscalibrated. An organization using a model with excellent ranking but poor calibration will correctly prioritize which employees to intervene with but will dramatically over- or under-estimate the expected return on those interventions.
The cost of miscalibration cascades through every decision that depends on the probability magnitude: retention budgets sized to uncalibrated risk scores, intervention intensity calibrated to fictional probabilities, and workforce planning models that treat model outputs as actuarial truth. Without APCS, organizations cannot distinguish between a model that says "75% risk" and means it, and one that says "75% risk" when the true departure rate for that band is 30%.
Decision Supported:
Do the probability scores from our attrition model mean what they say, or are we making retention and workforce planning decisions based on miscalibrated risk estimates?
Formula
Plain language: Group employees into probability bands based on the model's flight-risk score. For each band, compare the model's stated probability with the actual departure rate. APCS measures how well the stated probabilities match reality, averaged across all bands.
Notation:
APCS = 1 − (1/B) × Σ |actual_rate(b) − stated_midpoint(b)|
Variables:
| Variable | Definition | Unit |
|---|---|---|
| B | Number of probability bands (e.g., deciles: 0–10%, 10–20%, ..., 90–100%) | Count |
| actual_rate(b) | Actual departure rate among employees in probability band b during the measurement period | Percentage |
| stated_midpoint(b) | Midpoint of the probability range for band b (e.g., 15% for the 10–20% band) | Percentage |
Proposed formula, pending validation pilots.