Definition
Degree to which the AI vendor scorer's recency weighting causes ranking inversions where long-term superior vendors rank below long-term inferior vendors with better recent scores.
The Measurement Gap
Measurement Gap and Decision
The Measurement Gap
Traditional vendor scorecards use simple averages or weighted averages with static weights chosen by procurement policy. AI-powered vendor scoring introduces dynamic time-weighting that adapts based on data patterns, but the weighting logic is often opaque. Without VSTC, procurement has no way to know whether the AI is appropriately weighting recent performance or has calibrated its recency bias so aggressively that one bad quarter erases three years of excellent service.
The consequence of excessive recency weighting is vendor relationship instability. Reliable vendors get deprioritized after a single rough period. New vendors with brief good track records leapfrog established performers. Contract decisions made on these rankings reallocate work to less proven vendors while penalizing partners who have demonstrated sustained quality.
The Decision It Supports
VSTC tells procurement leadership how much the AI's time-weighting changes vendor rankings compared to equal-weighted history. Some inversion is healthy: it means the system responds to genuine performance changes. Too much inversion means recency dominates, and the organization risks destabilizing vendor relationships based on short-term fluctuations. The metric enables procurement to set a policy threshold for acceptable inversion rates before AI rankings drive contract decisions.
Formula
Plain Language
For each vendor pair in the same service category with at least 12 months of data, compute both the recency-weighted ranking (from the AI scorer) and the equal-weighted ranking. Count pairs where the rankings are inverted. Divide by total comparable pairs.
Formal Notation
$$VSTC = \frac{Inverted_pairs}{Total_comparable_pairs} \times 100$$
Variable Definitions
| Variable | Definition | Unit |
|---|---|---|
| Inverted_pairs | Vendor pairs where the recency-weighted ranking reverses the equal-weighted ranking | Count |
| Total_comparable_pairs | All vendor pairs in the same category with 12+ months of performance data | Count |
Units: Percentage. A value of 0% means recency weighting changes no rankings relative to equal weighting. Some inversion is expected and desirable. The question is where the threshold falls between "responsive to change" and "overreacting to noise."
Proposed formula, pending validation pilots.
At a Glance
Decision Supported
Is the AI producing vendor rankings that reflect sustained capability, or overreacting to recent events?