PORTFOLIO VALUE PREDICTION ACCURACY (PVPA)

GMI-IT-061IT OperationsInfrastructure-Dependent

Definition

Portfolio Value Prediction Accuracy (PVPA) measures the correlation between AI-predicted business value at funding time and actual measured business value 6-12 months post-launch.

The Measurement Gap

A fintech company adopted an AI-driven portfolio optimizer 18 months ago. The tool ranks proposed initiatives by predicted business value, recommends funding allocations, and the portfolio governance team uses the recommendations as input to quarterly investment decisions. The optimizer has produced impressive-looking analytics: predicted value distributions, sensitivity analysis, scenario comparison. The portfolio committee has been making funding decisions with the tool's recommendations for six funding cycles.

Then a board member asks a question during the annual portfolio review. "We've funded a lot of initiatives based on this AI's predictions. How many of them have delivered the value we predicted?" The team has not measured this. They go back through the funding decisions from 18-24 months ago: initiatives that have had time to launch and run for at least six months post-launch. The list is 22 initiatives. They attempt to measure actual business value for each:

  • 4 initiatives have measurable post-launch business value that can be compared to prediction
  • 9 initiatives have ambiguous post-launch measurement: value was claimed but the measurement methodology shifted post-launch
  • 5 initiatives never had a post-launch value measurement set up; the team moved on after delivery
  • 4 initiatives were cancelled, pivoted, or merged into other initiatives before post-launch measurement was possible

Of the 4 measurable initiatives, the comparison is uncomfortable: 1 delivered close to prediction, 1 delivered substantially less, 2 delivered substantially more. With a sample of 4, the team cannot say whether the AI optimizer is predicting accurately or just producing impressive-looking numbers.

The legacy portfolio metrics report a healthy portfolio practice.

Funding decisions made: 22 in the trailing 24 months.

Initiatives launched: 18 of 22 (4 cancelled).

On-time launch rate: 78%.

Budget variance: tracked and within tolerance.

Initiative completion rate: high.

Roadmap alignment: high.

None of these report on whether the predictions driving funding decisions were accurate. Funding decisions were made. Initiatives launched. Budgets were managed. The question of whether the predicted value materialized is not asked because the measurement infrastructure was never set up.

These metrics were designed when portfolio decisions were made on judgment with predicted value as one input among many. The judgment was checked against outcomes informally: the executive team noticed when an initiative delivered more or less than expected. With AI-driven portfolio optimization, the predictions take a more central role in decisions, and the absence of systematic post-launch validation becomes more consequential.

There is no metric that tells you whether AI portfolio value predictions correlate with actual post-launch value realization, which means investment allocation is optimized against predictions that are never validated.

Formula

Portfolio Value Prediction Accuracy (PVPA) measures the correlation between AI-predicted business value at funding time and actual measured business value 6-12 months post-launch.

In plain language: for each funded initiative, record the predicted value at funding. Six to twelve months after launch, measure the actual business value. Compute the correlation across the portfolio. A value above 0.7 means the predictions are operationally useful. A value below 0.3 means the predictions are noise.

The formula:

PVPA = Corr(Value_predicted, Value_actual)

Where:

  • Value_predicted = the AI-estimated business value at the funding decision (dollars or other consistent unit)
  • Value_actual = the measured business value 6-12 months post-launch

Units are a correlation coefficient from −1.0 to 1.0. Above 0.6 means useful signal. Below 0.3 means not better than random.

A worked example. The fintech company, after the board's question, invests in setting up post-launch value measurement for new initiatives. After a full year of new initiatives reaching the 6-month post-launch mark, they have 12 initiatives with measurable predicted-vs-actual value:

Sample of the data ($M projected business value, post-launch):

Initiative Predicted Actual
Mobile checkout 4.8 6.2
Lending model 12.0 3.4
Fraud platform 8.5 11.0
Customer service AI 3.2 1.8
Pricing optimization 6.7 4.1
...7 more initiatives...

Computing the correlation across 12 initiatives:

PVPA = 0.52

Per the catalog guidance (above 0.6 useful, below 0.3 not better than random) this PVPA of 0.52 is in the marginal range. The correlation exists but is weaker than would justify treating predictions as authoritative. The portfolio committee should use predictions as one input among several rather than as the primary basis for ranking.

Three edge cases need explicit treatment. First, "business value" must be defined consistently at prediction time and at measurement time. If the prediction was based on revenue and the measurement is based on customer satisfaction, the comparison is incoherent. Document the value definition per initiative. Second, exclude cancelled and pivoted initiatives, but track those counts separately because high cancellation rates may indicate that the predictions are too speculative. Third, the minimum sample is 10 funded initiatives with post-launch measurement; below that the correlation is too noisy.

Cross-references. Upstream of PVPA is

Effort Estimation Anchoring Bias Index (EEABI): predicted value and predicted cost are both inputs to portfolio decisions; biased cost estimates and biased value predictions compound.

Proposed formula, pending validation pilots.

At a Glance

Category
AI-Era IT Operations
← Policy Enforcement False Positive Rate (PEFPR)Prediction Coverage Breadth (PCB) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment