Definition
Capacity Prediction Accuracy (CPA) measures the mean absolute percentage error between AI-generated demand forecasts and actual resource consumption, evaluated at the specific lead time at which provisioning decisions are made.
The Measurement Gap
The auto-scaler fires at 6 AM. Your capacity planning model (AWS Predictive Scaling, Azure Predictive Autoscale, or a third-party optimizer like Spot.io or Densify) has been analyzing the last 14 days of traffic patterns and predicted a 38% demand spike starting at 8 AM for your e-commerce checkout tier. Fourteen additional EC2 instances spin up. Your on-call engineer sees the pre-scaling event in the dashboard, notes that it looks consistent with Tuesday morning historical patterns, and starts their day.
By 10 AM, actual demand is running 16% above baseline. Not 38%. Eight of those fourteen instances are at sub-20% CPU utilization, provisioned for a surge that showed up at less than half its predicted magnitude. By 2 PM, the auto-scaler scales back down. Eight instances ran for eight hours. At roughly $0.40 per instance-hour, that's $25.60 for one scaling event that over-predicted by more than 100% of the actual excess.
That $25 doesn't appear on anyone's radar. It shows up in the cloud bill under "EC2 compute" alongside the rest of your infrastructure spend. The prediction error that caused it is invisible to every metric your team tracks. CPU utilization was healthy because excess capacity absorbed the load. Uptime was 99.99% because the service was never under-provisioned. Auto-scaling event count logged as normal. Cloud spend came in at budget because the over-provisioning was baked into the model's systematic bias toward excess.
Now run that math across 400 comparable scaling decisions per quarter: different services, different prediction errors, all individually invisible, all collectively significant.
The existing capacity metrics were designed to answer "do we have enough?" not "was the prediction right?" Utilization measures consumption after provisioning has already happened. By the time you're looking at CPU percentages, the provisioning decision was made hours or days earlier based on a forecast that nobody reviewed. Cloud spend reports tell you what you spent, not whether the spend was justified by prediction accuracy. Auto-scaling event counts tell you how often the system reacted, not whether the reactions were proportionate.
These metrics also have no mechanism to surface systematic over-prediction. If the model is biased toward forecasting 30% higher than actual demand, every individual metric looks fine: utilization stays healthy, uptime stays high, spend stays within budget. The bias is invisible because the consequence is excess capacity, and excess capacity doesn't trigger alerts.
The organizational incentive structure compounds the problem. Under-provisioning gets you paged at 2 AM. Over-provisioning gets you nothing, not a notification, not a review, not a flag. When the AI's forecasting model is wrong in the direction of excess, there's no mechanism to catch it, because the org's historical pain memory comes entirely from being short on capacity, not from wasting it.
The measurement gap is specific: there is no metric that tells you whether AI capacity predictions match actual demand at the decision-relevant horizon, which means the model driving your provisioning decisions has never been scored against outcomes. Capacity Prediction Accuracy (CPA) is the score that doesn't exist yet.
Formula
Capacity Prediction Accuracy (CPA) measures the mean absolute percentage error between AI-generated demand forecasts and actual resource consumption, evaluated at the specific lead time at which provisioning decisions are made.
In Plain Language
for each auto-scaling event triggered by an AI demand forecast, compare what the model predicted demand would be to what demand actually materialized. Compute the percentage error on each prediction. Average those errors across all events in the measurement period. Subtract from 100. The result tells you how close your forecasting model is to the actual demand it's provisioning for.
Formula
CPA = 1 − (1/N) × Σ |Demand_predicted_i − Demand_actual_i| / Demand_actual_i
Where:
- Demand_predicted_i = the AI-generated demand forecast for resource type r at time t+h, produced at decision point t: where h is your provisioning lead time (the horizon at which scaling decisions commit)
- Demand_actual_i = actual resource consumption for resource type r at time t+h
- N = number of forecast-to-actual comparison points in the measurement periodSign Convention CPA is expressed as a percentage on a 0–100 scale. 100% = perfect forecast accuracy; every prediction matched actual demand exactly. Higher is better. CPA can go negative if the mean absolute percentage error exceeds 100%, which would mean the model's errors are larger than the actual demand values being predicted. If you see a negative CPA, disable the ML forecast and use a rules-based buffer until the model is retrained. Companion metric: Over-Provisioning Waste (OPW):OPW = Σ (Provisioned_i − Demand_actual_i) × Unit_cost_i [where Provisioned_i > Demand_actual_i]OPW translates prediction error into dollars. CPA tells you the model is 75% accurate. OPW tells you that 75% accuracy costs $42,000 per month in idle compute. These two numbers together give you both the diagnosis and the business case. CPA alone is interesting. CPA plus OPW is actionable.
Worked Example
A platform engineering team runs predictive auto-scaling on AWS for a multi-tier SaaS application. Over one quarter, the predictive scaler made 240 scaling decisions across three service tiers. Reviewing forecast-to-actual pairs:
- Average absolute percentage error across 240 decisions: 24%
- CPA = 1 − 0.24 = 0.76, or 76%
- OPW for the quarter (decisions where provisioned capacity exceeded actual demand): $51,200
The 76% CPA is close to the workable threshold but reveals meaningful error. Breaking it down by tier, two tiers are at 88% and 84% CPA, but a third (the batch processing tier) is at 54% CPA because the ML model struggles with its irregular demand patterns. That single tier accounts for 71% of total OPW. The fix is targeted: retrain the model specifically on batch workload patterns, or bypass ML forecasting for that tier and use a manual scheduled scaling rule.
Edge Cases
When actual demand approaches zero during maintenance windows or overnight lulls, the denominator shrinks and percentage errors spike, a 5-instance error on a baseline of 10 appears worse than a 50-instance error on a baseline of 500. Exclude forecast windows where actual demand falls below 5% of that service's daily peak, or apply a minimum denominator floor. If the auto-scaler overrides the ML forecast based on a manual rule or operator intervention, exclude that event from CPA: you're measuring the model's accuracy, not the human's. Minimum sample: 30 forecast-to-actual comparisons per service tier per measurement period. Cross-references: CPA is a foundational KPI with no upstream dependencies in this book. It feeds several downstream metrics. Commitment Recommendation Accuracy (CRA, GMI-IT-013) is directly downstream: the AI's cloud commitment purchasing recommendations are based on demand forecasts, so when CPA is poor, CRA almost certainly follows. Procurement Forecast Variance (PFV, GMI-IT-026) measures the same forecasting accuracy problem applied to hardware procurement cycles. Model Return on Inference Investment (MRII, GMI-IT-023) depends on accurate compute cost accounting, which CPA errors compound.
Proposed formula, pending validation pilots.