Definition
RASAR measures the change in recruiter candidate-evaluation accuracy on non-AI-screened assessments over time.
The Measurement Gap
Measurement Gap and Decision Impact
By periodically having recruiters evaluate standardized candidate profiles without AI assistance and comparing their accuracy against a baseline, RASAR detects whether recruiters are losing the ability to independently assess candidates as they become dependent on AI screening outputs.
The Decision It Supports:
Is the recruiting function maintaining human evaluation capability for non-standard candidates and system failures? When the AI screening system encounters edge cases it cannot handle, when it fails entirely, or when candidates bypass automated screening through executive referrals: are recruiters still capable of making accurate assessment decisions independently?
Why It Requires a New Metric:
Legacy recruiting metrics measured recruiter productivity (requisitions filled, time-to-fill, pipeline velocity) and quality (quality of hire composites, hiring manager satisfaction). No legacy metric measured recruiter skill maintenance because AI-driven screening did not exist to create the dependency that makes atrophy possible. The concern is not new to HCM (skill atrophy in human operators who rely on automated systems is well-documented in aviation, manufacturing, and healthcare) but it is new to recruiting.
The measurement gap RASAR addresses is the absence of any metric that detects whether human evaluation capability is degrading as AI takes over candidate screening, a failure mode that only becomes visible when the AI system fails or encounters candidates it cannot assess.
Skill atrophy in automated environments is one of the most well-documented phenomena in human factors research. When operators rely on automated systems for routine decisions, their ability to perform those decisions independently degrades over time. The mechanism is straightforward: skills that are not exercised atrophy. Recruiters who spend two years reviewing AI-ranked candidate slates rather than reading resumes and making independent assessments will gradually lose the ability to evaluate candidates effectively without AI support.
This matters for three reasons. First, AI screening systems encounter edge cases they cannot handle: candidates with non-traditional backgrounds, career changers, applicants from industries the model has not seen, and candidates whose qualifications do not fit the structured fields the AI expects. These edge cases require human evaluation capability that atrophy undermines. Second, AI systems fail entirely: outages, model retraining periods, data pipeline breaks. During failures, the recruiting function must operate manually, and atrophied assessment skills mean degraded hiring quality during exactly the periods when human judgment is most needed. Third, executive referrals, diversity pipeline candidates, and strategic hires often bypass automated screening intentionally. Recruiter assessment accuracy for these candidates depends on skills the recruiters may no longer be exercising.
Formula
Formula:
RASAR = (Accuracy_current − Accuracy_baseline) / Accuracy_baseline × 100
Variable Definitions:
| Variable | Definition | Unit | Source |
|---|---|---|---|
| Accuracy_current | Recruiter accuracy on the most recent blind standardized candidate assessment exercise (percentage of candidates correctly evaluated against validated outcomes) | Percentage | Calibration exercise results |
| Accuracy_baseline | Recruiter accuracy on the same type of exercise from the baseline measurement period (typically pre-AI or early-AI adoption) | Percentage | Historical calibration data |
Units: Percentage change from baseline. Negative values indicate atrophy. Positive values indicate improvement.
All thresholds are proposed starting points for organizational calibration, not industry standards.
Edge Cases and Exclusions:
- Refresh the standardized assessment periodically to prevent memorization effects while maintaining difficulty equivalence across versions
- Segment by recruiter tenure and AI exposure level: recruiters with longer AI dependence are expected to show more atrophy
- If the organization is fully committed to AI screening with no intention of reverting, RASAR may be deprioritized, but capability preservation still matters for edge cases and system failures
Data Requirements:
| Data Element | Source System | Classification |
|---|---|---|
| Standardized blind assessment scores (current) | Custom calibration exercise | MANUAL |
| Baseline assessment scores (historical) | Historical calibration data | NOT CAPTURED (requires establishing baseline) |
| Recruiter tenure and AI screening exposure duration | HRIS | NATIVE |
| Assessment validation set with known correct evaluations | Assessment design team | MANUAL |
Confidence Grading (required alongside every RASAR value):
| Grade | Basis | Interpretation |
|---|---|---|
| A: Empirical | 5+ observed events in measurement window provide direct tail observation | Measurement grounded in observed outcomes; highest confidence |
| B: Hybrid | 1-4 observed events plus Monte Carlo simulation with Bayesian updating | Blends observation and simulation; moderate confidence |
| C: Simulated | Zero observed events; derived entirely from Monte Carlo simulation | Modeled, not observed; treat as indicative, not validated |
A RASAR of any value at Grade C is a simulation that agrees with itself. It tells you the model is internally consistent but not that it matches reality. Grade C scores should trigger proactive stress-testing against historical near-miss events and cross-asset analogues rather than acceptance as calibrated.
Proposed formula, pending validation pilots.