ETA Promise Calibration Gap (EPCG)

GMI-TF-031Fleet & TransportationPilot-Ready

Definition

Mean signed difference between the ETA communicated to the customer at order confirmation and actual delivery time, measuring whether the AI ETA model systematically over-promises (positive EPCG) or under-promises (negative EPCG) relative to what the fleet actually delivers.

The Measurement Gap

Most fleet operations measure delivery timing through metrics that obscure the relationship between what was promised and what was delivered. The blind metrics include:

  • On-time delivery (OTD), a binary pass/fail metric that counts deliveries as "on-time" or "late" against a window (typically ±15 or ±30 minutes). OTD cannot distinguish between a system that promises tight ETAs and barely misses them from one that promises loose ETAs and easily hits them. A fleet with 95% OTD may be systematically padding ETAs by an hour to guarantee the metric: delivering on-time by promising late.

Average delivery time: measures how long deliveries take but never compares that duration to what was communicated to the customer. A fleet averaging 4.2 hours per delivery looks consistent, but if the AI promised 3.5 hours, every delivery is 42 minutes late from the customer's perspective.

  • Customer satisfaction scores, a lagging indicator that captures the accumulated effect of ETA miscalibration months after the pattern has established itself. By the time CSAT declines, the customer relationship damage is already done.

None of these metrics answer the directional question: is the ETA model systematically optimistic? EPCG provides the leading signal that OTD and CSAT can only reflect after the fact.

Decision Supported:

Is the ETA model building or eroding customer trust, and is the bias systematic enough to require recalibration of the promise algorithm?

Formula

Plain language: For each delivery, subtract the promised ETA from the actual arrival time. Average those differences. A positive result means deliveries are arriving later than promised on average (the model is optimistic). A negative result means deliveries arrive earlier than promised (the model is conservative). Zero means perfectly calibrated, not perfectly accurate, but unbiased on average.

Notation:

$$EPCG = \frac{1}{N} \sum_{i=1}^{N} (ActualArrival_i - PromisedETA_i)$$

Positive EPCG = average delivery LATER than promised (optimistic model). Negative EPCG = average delivery EARLIER than promised (conservative model).

Supplemental: Promise Reliability Rate (PRR):

$$PRR = \frac{\text{Deliveries within } \pm 15 \text{ min of promised}}{Total_Deliveries} \times 100$$

PRR complements EPCG by measuring dispersion around the promise. A fleet can have EPCG near zero (unbiased on average) but low PRR (high variance: some very early, some very late).

Supplemental: Average ETA Updates:

$$AvgETAUpdates = \frac{\text{Total ETA revisions across all deliveries}}{N}$$

Frequent ETA revisions indicate the initial promise was unreliable and had to be corrected in-flight.

Variables:

Variable Description Source
PromisedETA_i Initial ETA communicated to customer at order confirmation TMS / customer portal / order management
ActualArrival_i Actual delivery timestamp Telematics geofence / proof of delivery
N Number of deliveries in measurement period Calculated
ETA revisions Count of ETA updates per delivery TMS / customer communications

Proposed formula, pending validation pilots.

At a Glance

Unit
Time (minutes)
Trend Direction
Lower; approaching 0
Reporting Frequency
Monthly
Category
Compliance and Risk Calibration
← DQ File Classification Accuracy (DQCA)Risk Score Currency Index (RSCUI) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment