Definition
Difference in return rates between AI-recommended purchases and customer-initiated purchases, measuring whether the recommendation engine is optimizing purchase probability at the expense of post-purchase satisfaction.
The Measurement Gap
Recommendation engines optimize for purchase probability, not satisfaction probability. When customers find products through their own search or browse, they apply their own judgment about fit and suitability. When the algorithm recommends a product, it applies a statistical model trained on what similar customers bought, not on whether those customers were happy. The gap between AI-recommended return rates and self-discovered return rates is a direct measurement of that judgment quality differential. Traditional recommendation metrics (click-through rate, conversion rate, average order value) treat the purchase as the finish line. RRD treats the purchase as the midpoint and asks whether the customer kept the product. A recommendation engine with a high conversion rate and a high return differential is not performing well; it is performing expensively. Every returned recommendation carries reverse logistics cost, customer frustration, and the hidden cost of training the algorithm on a purchase signal that was actually a mistake.
Decision Supported:
Is the recommendation engine optimizing for conversion at the expense of customer satisfaction, and should return likelihood be incorporated into the recommendation scoring model?
Formula
Formula:
RRD = ReturnRate_AI_recommended - ReturnRate_customer_initiated
Variables:
- ReturnRate_AI_recommended = returns / purchases for items where the purchase was attributed to an AI recommendation click
- ReturnRate_customer_initiated = returns / purchases for items where the purchase came from search, browse, or direct navigation
- Attribution window: last-click within session
Units: Percentage points. RRD = 0 means no difference in return rates between sources. RRD > 0 means AI-recommended purchases are returned more often. An RRD of +5pp means AI-recommended items have a return rate five percentage points higher than self-discovered items.
Edge Cases and Exclusions:
Category mix must be controlled. AI recommendations may skew toward categories with inherently higher return rates such as apparel. Compute RRD within each category to isolate the recommendation effect from category effects. Similarly, control for price band since higher-priced items carry different return behaviors. Segment return reasons to distinguish recommendation quality issues ("not as described," "didn't fit") from source-independent reasons ("changed mind"). For sessions with multiple touchpoints, attribute to the strongest AI signal. Require a minimum of 200 returns per source segment per measurement period for statistical stability.
Proposed formula, pending validation pilots.