Definition
Degree to which a driver's ML-based risk score reflects their current operating exposure versus the historical profile the model was trained on, measuring staleness of the risk model's input distribution when drivers change routes, regions, or cargo types.
The Measurement Gap
Driver risk scoring systems produce a number that looks precise but rarely reveal whether that number reflects current reality. The blind metrics include:
Current risk score: the score itself tells you nothing about whether the inputs driving it are current. A driver reassigned from regional flatbed to long-haul hazmat may carry the same score for months because the model has not ingested the change in operating characteristics. The score looks authoritative; the inputs are stale.
Insurance premium tier: derived from risk scores, premium tiers inherit whatever staleness exists in the underlying model. A fleet paying premiums based on outdated risk profiles is either overpaying for low-risk drivers or underpaying for high-risk ones, and cannot tell which without measuring input currency.
- Accident/incident rate, a lagging outcome metric that only reveals risk model failure after the event. By the time incident rates spike for a cohort of mis-scored drivers, the damage is done.
Without RSCI, fleets have no way to distinguish a risk score that reflects a driver's current operating exposure from one that reflects a profile the driver no longer matches. Safety interventions get targeted at the wrong drivers, insurance premiums drift from reality, and the risk model degrades silently.
Decision Supported:
Is the risk model scoring drivers based on what they are doing NOW, or based on a historical profile that no longer matches their operating exposure?
Formula
Plain language: Build two distributions of a driver's operating characteristics: one from the last 90 days (what the driver is actually doing) and one from the risk model's training window (what the model thinks the driver does). Measure how different these distributions are using Jensen-Shannon divergence, then subtract from 1 so the result reads as a currency score. Close to 1.0 means the model's view matches current reality; close to 0.0 means the model is working from a stale picture.
Notation:
$$RSCI = 1 - JSD(P_{current} | P_{historical})$$
Where JSD is the Jensen-Shannon divergence, bounded between 0 and 1.
Variables:
| Variable | Description | Source |
|---|---|---|
| P_current | Distribution of driver operating characteristics over last 90 days: percentage interstate vs. urban, percentage hazmat vs. general freight, average daily miles, top 3 operating states | TMS / ELD / dispatch |
| P_historical | Same operating characteristics over the risk model's training window | Risk scoring platform / insurance carrier |
| JSD | Jensen-Shannon divergence between P_current and P_historical; 0 = identical distributions, 1 = completely different | Calculated |
| RSCI | Currency index; 1.0 = model inputs fully current, 0.0 = model inputs completely stale | Calculated |
The operating characteristics used for P_current and P_historical should match the features the risk model actually consumes. If the model uses additional features (hours-of-service patterns, violation history), include those in the distribution comparison. Use driver ID (never driver name) as the linking key.
Proposed formula, pending validation pilots.