Definition
Proportion of virtual-assistant recommendations that match the customer's stated need when independently assessed, measuring whether the assistant is answering the question the customer actually asked.
The Measurement Gap
Virtual shopping assistants take a customer's stated need and return product recommendations, but no metric measures whether the recommended products match what the customer asked for. A customer who says "I need a waterproof hiking jacket under $150 in size medium" has expressed measurable requirements. If the AI recommends a water-resistant windbreaker at $180 in size large, it has missed on three of four requirements. If the customer buys it anyway because the assistant was persuasive, the purchase is a conversion success and a need-match failure. Most virtual assistant platforms measure task completion ("did the customer buy?") and post-interaction satisfaction surveys. Neither captures whether the recommendation addressed the stated need. A customer can rate an interaction highly because the assistant was polite while still receiving suggestions that missed the mark. NMAR isolates recommendation quality from interaction quality, and that distinction matters for system improvement.
Decision Supported:
Is the virtual shopping assistant recommending products that match stated customer requirements, or is it defaulting to high-conversion products regardless of expressed needs?
Formula
Formula:
NMAR = (Full_matches + 0.5 x Partial_matches) / Total_assisted_recommendations
Variables:
- Full_matches = recommendations where all expressed requirements are satisfied by the recommended product's attributes
- Partial_matches = recommendations where some but not all requirements are met
- Total_assisted_recommendations = all product recommendations made in assisted interactions where a need was expressed
Units: 0 to 1. NMAR = 1.0 means every recommendation matched the stated need perfectly. NMAR = 0.5 means half the recommendations missed the mark. Track by interaction channel and product category.
Edge Cases and Exclusions:
Vague customer requirements ("I need a nice shirt") should be excluded from NMAR. Only include interactions with at least one measurable attribute requirement (price, size, feature, use case). When multiple recommendations are made in a single interaction, score each individually. If the customer changes requirements mid-interaction, score against the most recent requirement statement. Automated scoring requires NLP extraction of requirements from conversation transcripts and attribute matching against the product catalog. Manual scoring on a sample is the recommended pilot approach before investing in automated extraction.
Proposed formula, pending validation pilots.