Retention Offer Cannibalization Rate (ROCR)

GMI-TEL-009TelecommunicationsIntegration-Required

Definition

Percentage of AI-triggered retention offers delivered to subscribers who would have remained without them, measuring discount spend consumed by non-causal targeting.

The Measurement Gap

Measurement Gap and Decision

What ROCR Measures: ROCR estimates the percentage of AI-triggered retention offers that were delivered to subscribers who would have remained customers without the offer. It quantifies the revenue cannibalized by unnecessary discounts driven by false-positive churn predictions.

Decision Supported:

Is the churn prediction model's intervention threshold set correctly, or is it triggering expensive retention offers on subscribers who were not actually going to leave? Without ROCR, marketing reports "saves" without distinguishing genuine saves from subscribers who received an unnecessary discount.

Why This Is Unmeasured:

Churn prevention programs report offer volume, acceptance rate, and "retained subscribers." None of these answer the causal question: would these subscribers have left without the offer? The only way to measure this is with a control group, a random holdout of at-risk subscribers who do not receive the offer. Most telecom operators resist holdout testing because it feels like deliberately losing customers. But without it, the organization cannot distinguish between a churn model that catches genuine flight risks and one that hands out discounts to loyal subscribers. In practice, ROCR of 50–70% is common in telecom: the majority of "saves" are not saves (McKinsey, "The Churn Prediction Paradox in Telecommunications," 2023; GSMA Intelligence, "AI-Driven Retention: Measuring Causal Impact," 2024).

Formula

Plain Language: Establish a control group, a random sample of subscribers who score above the churn threshold but do NOT receive the retention offer. Compare their actual churn rate to the churn rate of the treated group over the same period. The difference is the "true saves." Everything else in the treated group that did not churn was going to stay anyway: those are cannibalized offers.

Notation:

ROCR = 1 − [(Churn_rate_control − Churn_rate_treated) / Churn_rate_control]

Variable Definitions:

  • Churn_rate_control = actual churn rate of the holdout group (subscribers scored at-risk but not offered retention) over the measurement period (percentage, 0–100)
  • Churn_rate_treated = actual churn rate of the treated group (subscribers scored at-risk and given retention offer) over the same measurement period (percentage, 0–100)
  • The numerator (Churn_rate_control − Churn_rate_treated) represents the true causal effect of the retention offer: the churn reduction attributable to the offer itself

Units: Percentage. 0% = every offer was causally necessary (all treated subscribers would have churned without it). 100% = the offer had zero causal effect (everyone would have stayed anyway).

Sign Convention:

Lower is better. Lower ROCR means a higher proportion of offers are genuinely preventing churn rather than discounting loyal subscribers.

Edge Cases:

  • Control group sizing: a 5–10% holdout of the triggered population provides sufficient statistical power while limiting customer exposure.
  • Contamination: control group subscribers may receive other retention touches (generic campaigns, proactive service improvements). Exclude cross-contaminated records or use intent-to-treat analysis.
  • Minimum sample: 200 or more subscribers per group per measurement period for statistically meaningful comparison.

Proposed formula, pending validation pilots.

At a Glance

Unit
Percentage
Trend Direction
Lower
Reporting Frequency
Quarterly
Category
AI-Driven Customer Value and Retention Integrity
← Churn Score Reflexivity Index (CSRI)Sentiment Classification Equity Score (SCES) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment