Definition
MRII measures the ratio of estimated business value generated by a model to its total inference cost over a measurement period: capturing whether each model in the portfolio earns more than it costs to serve.
The Measurement Gap
At the end of the fiscal quarter, the CTO asks a simple question: which of our 34 deployed models is generating the most value per dollar of compute? Nobody has an answer.
The GPU infrastructure team can tell you total inference cost by service. The data science team can tell you model performance metrics for each model. The product team can tell you revenue and engagement numbers. But the specific question (which model is generating business value that exceeds its inference cost, and which isn't) requires connecting three datasets that have never been connected for this purpose. It takes three weeks and a special project to produce an approximate answer, and by the time it's ready, the quarterly planning decisions have already been made on intuition.
This is the structural failure that MRII addresses. Organizations have gotten good at measuring what models do technically (accuracy, latency, throughput) and what they cost operationally (GPU-hours, infrastructure spend). They have not gotten good at measuring what models return economically: the ratio of business value generated to inference dollars consumed.
The result is that GPU budget allocation across a model portfolio becomes a political exercise rather than an economic one. The team with the most compelling story, the most recent production incident, or the most senior champion gets the compute budget increase. The model that generates 15× its inference cost in fraud prevention quietly keeps running at its current allocation. The model that costs more to serve than it returns in business value doesn't get cut because nobody knows it's in that position.
Legacy metrics fail at the portfolio level because they were designed to measure individual model health. Accuracy tells you whether the model is performing. Latency tells you whether it's fast enough. Throughput tells you whether it can handle the load. None of these metrics tell you whether the model is economically earning its place in the portfolio. A model can be perfectly accurate, perfectly fast, and thoroughly not worth running, because the business decision it's supporting isn't valuable enough relative to its compute cost.
Inference cost is tracked at the infrastructure level, usually as a total line item per service or per GPU cluster. It's not attributed back to individual models in a way that supports economic comparison. Business value is tracked in the business intelligence layer (revenue, cost savings, risk mitigation) but not connected to the specific models generating it.
The measurement gap: there is no standard metric that connects model inference cost to the business value each model delivers. GPU budget allocation across a model portfolio has no ROI basis. That's what Model Return on Inference Investment is designed to fill.
Formula
Model Return on Inference Investment (MRII) is proposed. No model serving platform reports it. No ML observability tool computes it. It's a portfolio-level economic metric that requires connecting cost data and business value data in a way that most organizations haven't operationalized. The concept isn't novel: ROI analysis is foundational to capital allocation. Its application to individual models in a production ML portfolio is.
Definition:
MRII measures the ratio of estimated business value generated by a model to its total inference cost over a measurement period. Capturing whether each model in the portfolio earns more than it costs to serve.
The Formula
Plain language: For each model, estimate the business value it generated during the period. Divide by the total cost of serving that model during the same period. A ratio above 1.0 means the model earns its keep. Below 1.0 means you're paying more to run it than it returns.
Notation:MRII = V_business / C_inference
Where:
- V_business = estimated business value attributable to model predictions during the measurement period
- C_inference = (GPU_hours_inference × GPU_rate) + C_serving_infra
- GPU_hours_inference = compute consumed for inference during the period (training costs excluded: MRII covers serving cost only)
- GPU_rate = cost per GPU-hour for serving workloads
- C_serving_infra = allocated share of serving platform overhead (load balancers, feature store queries, monitoring infrastructure) proportional to inference volume share
Sign convention: MRII above 1.0 is positive return on inference investment. MRII below 1.0 is negative: the model costs more than it returns. MRII at exactly 1.0 is breakeven. Compute it as a period metric, at minimum quarterly.
Units: Ratio. Always state the period: "MRII of 8.4 for Q1" is interpretable; "MRII of 8.4" without context is not.
Worked Example
A fraud detection model at a mid-size fintech company. Measurement period: Q1, 13 weeks. Inference cost: The model runs on a 2-GPU inference cluster serving 150,000 predictions per day. Average cluster cost: $180 per day. Over 91 days: $16,380 in direct compute. Feature store queries add approximately $1,200 in infrastructure overhead. Total C_inference = $17,580.Business value: During Q1, the model flagged 2,840 transactions that were confirmed as fraudulent by the fraud review team. Average fraud loss per confirmed transaction: $340. Total fraud prevented: $965,600. The model also reduced manual review volume by 18% compared to the rule-based predecessor, saving approximately $8,400 in analyst labor. Total V_business = $974,000.MRII = $974,000 / $17,580 = 55.4. The model returns $55 for every $1 spent on inference. That's a clear signal for increased investment, not cost optimization. Now contrast with a recommendation model at the same company. Inference cost: $12,000 per quarter. Business value: The team estimates $9,500 in incremental revenue based on A/B test results from eight months ago. Never updated since. MRII = $9,500 / $12,000 = 0.79. The model costs more to run than it returns. This model is a candidate for compression, consolidation, or retirement, but nobody knew until MRII was computed.
Edge Cases
Business value attribution is the hard part. For most models, you need a proxy rather than a direct measurement. Establish the proxy per model with the relevant business team, a one-time calibration reviewed annually. Conservative estimates are acceptable; the goal is portfolio ranking, not audited financial statements. Models with shared infrastructure: allocate infrastructure cost proportionally by inference volume. If two models share a GPU cluster and one consumes 70% of capacity, attribute 70% of cluster cost to that model. Defensive models (compliance, safety): business value = cost of the non-compliance or safety incident avoided. Estimate conservatively. These models often have very high MRII (avoiding a $2M regulatory fine with a $20K inference cost gives MRII of 100) but the value is lumpy and counterfactual. Flag as "defensive value estimate" to distinguish from directly measurable business returns. Models serving multiple purposes: attribute value across use cases, or pick the primary one and note the simplification. Better to be approximately right about the primary value than precisely wrong about a weighted average of uncertain values.
Cross-References
MRII is downstream of REY (GMI-IT-019): a model that retrains frequently with low REY accumulates operational cost that degrades MRII over time. It's downstream of CQRD (GMI-IT-022): if compression reduces inference cost but also reduces business value by degrading quality on high-value subpopulations, the MRII calculation must account for both the cost reduction and the value reduction. MRII is the terminal metric in this catalog: it's the portfolio-level economic signal that drives investment decisions, with no other KPI depending on it as an input.
MRII requires infrastructure most organizations don't have: per-model inference cost attribution (billing is usually aggregated at the service or cluster level) and per-model business value estimation (requiring a connection between prediction activity and business metrics). Both require deliberate organizational investment. The math is simple. The organizational alignment to produce the inputs is the hard part.
Proposed formula, pending validation pilots.