Tier Routing Economic Accuracy (TREA)

GMI-IT-033IT OperationsIntegration-Required

Definition

Tier Routing Economic Accuracy (TREA) is a proposed measure of what percentage of incidents are routed by the AI to the tier that ultimately resolves them: measuring whether the routing engine is directing work to the right-cost resource level.

The Measurement Gap

The ticket came in at 9:23 AM: "Outlook not opening, spinning indefinitely." The AI routing engine classified it as Tier-2. It shouldn't have. This was a straightforward application issue, most likely a corrupted Outlook profile or a cached credential problem, resolvable in fifteen minutes by any competent helpdesk technician. But the routing engine had seen similar symptom language in its training data associated with a deeper Exchange connectivity issue that had required senior engineering attention three months prior. It pattern-matched on the complaint, not on the actual incident characteristics, and sent the ticket up the stack.

The Tier-2 engineer picked it up at 9:41 AM, diagnosed a corrupted profile in four minutes, applied the standard fix in eight minutes, and closed the ticket. Total Tier-2 time: twelve minutes. The correct Tier-1 resolution would have taken the same twelve minutes. The economic difference: the senior engineer's loaded hourly rate was 2.4 times the Tier-1 rate. For a twelve-minute ticket, the misrouting cost roughly $18 in misallocated labor. Trivial in isolation.

On the same day, an incident involving a multi-factor authentication configuration failure for a client migrating to a new Azure tenant was classified as Tier-1. The Tier-1 technician spent 47 minutes on it, escalated, the second Tier-1 tech spent another 20 minutes, escalated again, and a Tier-2 engineer ultimately resolved it in 35 minutes. Total elapsed time: 1.7 hours across three technicians before resolution. If the AI had routed it to Tier-2 initially, the Tier-2 engineer would have resolved it in 35 minutes. The under-classification cost: 67 minutes of combined Tier-1 time that produced nothing, two escalation handoffs that disrupted queue flow, and a client who waited 1.7 hours instead of 35 minutes.

Neither routing decision appeared in the MSP's performance metrics as a routing error. Escalation rate tracked what percentage of tickets escalated, a volume metric, not an accuracy metric. A 15% escalation rate looks identical whether those escalations were appropriate (Tier-1 genuinely couldn't handle the incident) or failures (Tier-1 spent an hour before escalating what should have been sent to Tier-2 from the start). Resolution time was measured per ticket in total, not broken down by routing path quality. Technician utilization was high, but utilization measures time consumed, not whether that time was spent on work that should have arrived at that tier in the first place.

The structural gap is that tier routing has an economic cost function that escalation rate and utilization don't capture. Under-classification generates two costs: the wasted Tier-1 time before escalation and the disruption overhead of the escalation event itself. Over-classification generates a different cost: senior engineering capacity consumed by problems a junior technician could have handled. Both are real and both appear nowhere in standard MSP dashboards because those dashboards were built before AI was making routing decisions.

There is no metric that measures what percentage of incidents were routed to the tier that actually resolved them, or what the monthly economic cost of routing errors is. That gap is what Tier Routing Economic Accuracy is designed to quantify.

Formula

Tier Routing Economic Accuracy (TREA) is a proposed measure of what percentage of incidents are routed by the AI to the tier that ultimately resolves them: measuring whether the routing engine is directing work to the right-cost resource level. The "economic" in the name is intentional. This isn't just about routing accuracy: it's about whether misrouting is creating measurable labor waste. The economic framing is what makes TREA operationally compelling: an under-classification rate of 25% sounds abstract, but "this routing model wasted $14,000 of technician time last month" gets leadership attention.

The formula, in three layers:

Plain language: For each incident in the measurement period, compare the tier the AI initially assigned to the tier that actually resolved the incident. The resolving tier is the tier that closed the ticket: accounting for any escalations that occurred. If the assigned tier equals the resolving tier, the routing was correct. If the resolving tier was higher than assigned, the AI under-classified (routed too low, work escalated up). If the resolving tier was lower, the AI over-classified (routed too high, ticket reassigned down). Divide correct routings by total incidents.

Notation:TREA = I_correct_tier / I_total

Where:

  • I_correct_tier = incidents where AI-assigned tier = tier that resolved the incident
  • I_total = total incidents with AI tier assignment in the measurement periodAlso compute directional error rates separately:

  • Under_rate = I_escalated / I_total (AI routed too low: incident required escalation to a higher tier)

  • Over_rate = I_de-escalated / I_total (AI routed too high: ticket was reassigned to a lower tier before or during resolution)

Worked Example

In a month with 1,200 AI-routed incidents, 810 were resolved at the assigned tier: TREA = 67.5%. Of the 390 misrouted incidents, 280 were under-classified (23.3% under-rate) and 110 were over-classified (9.2% over-rate). Converting to economics: under-classification costs an average of 35 minutes of wasted Tier-1 time per incident at $65/hour = (280 × 35/60 × $65) = $10,558. Over-classification costs an average of 20 minutes of wasted Tier-2 time per incident at $110/hour = (110 × 20/60 × $110) = $4,033. Total monthly routing error cost: $14,591 in misallocated labor. Invisible in any standard MSP dashboard.

Sign convention:

Higher TREA is better. Track Under_rate and Over_rate separately because they indicate different failure modes in the routing model. A high Under_rate means the model is systematically underestimating incident complexity, likely from training data gaps or feature limitations in the classification model. A high Over_rate means the model is over-conservative: routing to senior tiers to hedge against under-classification errors, at the cost of wasting expensive engineering time on routine issues.

Edge Cases

Tier-0 automation routing: if the AI routes to automated remediation and it resolves successfully, count as correct. If routed to automation and automation fails (requiring Tier-1 pickup) count as under-classification. The automation tier is a valid tier for TREA purposes. Incidents with multiple escalations: use the final resolving tier. A ticket that traveled Tier-1 to Tier-2 to Tier-3 before being resolved was resolved at Tier-3. Initial assignment of Tier-1 is under-classification by two tiers. Measure the initial assignment against the final resolution, not against an intermediate escalation point. Client-specific routing rules: some clients have contractual agreements requiring minimum tier assignment. High-touch clients where everything goes to Tier-2, or clients with specific escalation paths for compliance reasons. The AI should incorporate these rules, and TREA for those clients should be calculated against the rule-adjusted expectation, not just against the final resolving tier. For low incident volumes (fewer than 100 incidents per month) TREA will be noisy. Use rolling 90-day windows to stabilize the estimate.

Cross-references:

Upstream, TREA depends on AIOps Correlation Accuracy (ACA, GMI-IT-004) from the existing IT KPI set. If alert correlation within a client's environment is inaccurate, the incident the AI sees is a distorted representation of the actual problem: symptoms without clear cause attribution. The routing engine classifies based on what the incident looks like; if what it sees is a symptom cluster rather than a root cause, the routing decision is based on misleading input. Improving ACA raises the ceiling for TREA improvement. Lateral connection to AEV: for clients where automation failure rates are high (low SR_c in the AEV framework), Tier-0 routing systematically under-classifies. The AI routes to automation, automation fails, Tier-1 picks up the ticket. This inflates Under_rate specifically for those clients and makes the TREA number for those clients look like a routing model problem when the actual issue is automation quality. Segment TREA by client when AEV is also measured: the clients with low SR_c will predictably have lower TREA from the Tier-0 under-classification effect. Downstream, TREA connects to technician utilization. Under-classification wastes Tier-1 time on problems they can't solve; over-classification wastes Tier-2 and Tier-3 time on problems Tier-1 could handle. Both reduce effective capacity utilization at the tier where the work actually belongs. TREA is classified as Integration-Required. PSA platforms log incident creation, tier assignment, and resolution, but the AI's initial tier recommendation is sometimes overwritten before the ticket reaches a queue. Preserving the AI's initial assignment in a read-only field requires a PSA workflow configuration change. That's the integration work.

Proposed formula, pending validation pilots.

At a Glance

Category
AI-Era IT Operations
← Threat Model Coverage Drift (TMCD)Training Experiment Efficiency (TEE) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment