Definition
Blast Radius Prediction Accuracy (BRPA) measures the ratio of predicted to actual affected systems for changes that produce observable impact.
The Measurement Gap
A large e-commerce platform uses an AI-driven change risk assessment tool that predicts the blast radius of proposed changes, which systems will be affected, how, and with what likely severity. The tool integrates with the CI/CD pipeline and surfaces predictions to the change approver. The approver uses the prediction to size the change review (small radius gets fast-track approval, large radius gets architecture review). The team has been running this for over a year. Approval lead time is down 60% from the pre-AI baseline. Change failure rate is reported as stable.
Then a configuration change to a backend caching layer is approved through fast-track because the AI predicted affected systems = 3 (the caching layer itself, the immediate consumer service, and a metrics collector). The change deploys. Within 20 minutes, 12 services are showing degraded behavior. By 40 minutes, the order processing pipeline is failing. The incident takes 90 minutes to resolve through rollback and recovery. Total blast: 12 services affected, $1.4M in lost orders.
The AI predicted 3 affected systems. The actual blast was 12. The dependency model the AI was operating from did not include the implicit dependencies: services that connected to the caching layer through paths the dependency mapper had not discovered. The prediction was wrong because the data was wrong, and the team did not have a metric that would have flagged the prediction's unreliability before the change was approved.
The legacy change metrics report a healthy change pipeline.
Change failure rate: 4.2% (within industry benchmark).
Mean time to recover: 23 minutes average.
Approval lead time: down 60%.
Deployment frequency: up 40%.
Change approval pass rate: 92%.
Post-change incidents: tracked, mostly small.
None of these report on whether the blast radius predictions are accurate. Change failure rate measures whether changes succeed or fail, not whether the impact was correctly predicted when they fail. MTTR measures recovery speed, not prediction quality. Approval lead time improved because predictions were trusted as authoritative for scoping the review. The trust was unaudited.
These metrics were designed when blast radius assessment was made by human change reviewers who walked the dependency map manually. The walk surfaced implicit dependencies the documented map missed because the reviewer asked questions and chased the connections. With AI-driven blast radius prediction, the walk is automated against the documented dependency map, and dependencies not in the map are invisible to the prediction.
There is no metric that tells you whether AI blast radius predictions account for implicit dependencies not in the dependency model, which means changes are approved based on scope assessments that systematically undercount.
Formula
Blast Radius Prediction Accuracy (BRPA) measures the ratio of predicted to actual affected systems for changes that produce observable impact. In plain language: every time a change produces observable downstream impact, compare the AI's pre-change prediction of affected systems against the actual count of systems impacted. BRPA is the prediction-to-actual ratio. A value of 1.0 means perfect prediction. A value below 1.0 means systematic underestimation.
The formula:
BRPA = Predicted_affected_systems / Actual_affected_systems
Where:
- Predicted_affected_systems = the count of systems the AI predicted would be affected by the change
- Actual_affected_systems = the count of systems actually impacted (observed via incident, degradation, or measurable behavior change)
Units are a ratio. 1.0 means perfect prediction. Below 1.0 means underestimation. Above 1.0 means overestimation. A worked example. The e-commerce platform runs BRPA on the trailing quarter:15 changes with observable downstream impact during the quarter. For each:| Change | Predicted | Actual | BRPA ||--------|-----------|--------|------|| Caching config | 3 | 12 | 0.25 || Auth library update | 4 | 4 | 1.00 || Database parameter | 2 | 8 | 0.25 || Service mesh policy | 6 | 7 | 0.86 || Feature flag toggle | 1 | 3 | 0.33 || ...10 more changes... | | | |Mean BRPA across 15 changes: 0.58.Per the catalog guidance (above 0.80 captures most scope, below 0.50 missing more than half) this BRPA of 0.58 is below the action threshold. The team needs to augment the dependency model with implicit dependencies before trusting the predictor for scope-based approval routing. The fast-track approval for predicted-small-radius changes should be paused until BRPA improves. Three edge cases need explicit treatment. First, BRPA is only measurable for changes with observable impact. Changes that produced no impact contribute no data because the prediction cannot be evaluated. Track the count of changes with predictions separately from changes with measured BRPA. Second, include indirect impact (performance degradation, error rate increase) in the actual count, not just hard failures. Third, the minimum is 15 changes with observable impact per quarter for stable BRPA computation. Cross-references. Upstream of BRPA is Dependency Discovery Coverage Rate (DDCR). If the dependency model is incomplete (low DDCR), BRPA will be poor regardless of how good the predictor's reasoning is. Downstream is
Change Risk Scoring Accuracy (CRSA, GMI-IT-012): both metrics measure aspects of change prediction quality; BRPA focuses on scope, CRSA on risk magnitude. The misinterpretation to watch for: a team reports BRPA = 0.85 (above the threshold) and concludes the predictor is reliable. They miss that the 0.85 mean is dominated by changes where prediction = 1 and actual = 1 (small, well-understood changes), while the high-impact tail of changes shows BRPA = 0.3 (large changes that systematically blast wider than predicted). Report BRPA both overall and for the high-impact tail; the tail behavior is what matters operationally.
Proposed formula, pending validation pilots.