Definition
Mean absolute error between the estimated arrival time communicated to passengers via automated disruption notifications and the actual arrival time, measuring whether the information passengers use for connection and planning decisions is reliable.
The Measurement Gap
The Measurement Gap
Transit agencies and passenger railroads send automated disruption notifications with estimated arrival times. What they don't measure is whether those ETAs are accurate. An optimistic ETA causes passengers to wait at the station expecting a train that arrives much later, or worse, to miss connection windows they would have rebooked if told the truth. A pessimistic ETA causes passengers to abandon trips they would have completed.
This gap matters because disruption notifications have become a critical decision input for passengers. When the app says "your train is expected at 3:15," passengers make connection, pickup, and rebooking decisions based on that estimate. If the train actually arrives at 4:00, those decisions were wrong and the notification made things worse than no notification at all.
Decision Supported:
Are disruption notifications helping passengers make good decisions, or are optimistic ETAs causing them to miss connections they would have otherwise rebooked? DEAI provides the accuracy data to calibrate ETA models and determine whether point estimates should be replaced with uncertainty ranges.
Formula
Plain Language: For each automated disruption notification that included an estimated new arrival time, compare the ETA to the actual arrival time. Average the error.
Notation:
DEAI = 1 − (1/N) × Σ |ETA_communicated_i − Arrival_actual_i| / Acceptable_error_threshold
Variables:
| Variable | Definition | Unit | Source | Classification |
|---|---|---|---|---|
| ETA_communicated_i | Estimated arrival time sent to passengers in disruption notification i | Timestamp | Messaging system | NATIVE |
| Arrival_actual_i | Actual arrival time of the train referenced in notification i | Timestamp | Operations system | NATIVE |
| Acceptable_error_threshold | Maximum acceptable ETA error (recommended: 15 minutes for corridor services, 30 minutes for long-distance) | Minutes | Policy parameter | REFERENCE |
| N | Number of disruption notifications with ETAs | Count | Messaging system | DERIVED |
Sign Convention: 1.0 = all ETAs within acceptable error. Lower = less accurate. Negative = ETAs are on average farther off than the acceptable threshold.
Edge Cases:
- Multiple notifications for the same disruption: use the most recent ETA communicated before the passenger's decision point (connection time, rebooking window).
- Cancellations: if the train was ultimately cancelled, any ETA previously communicated was wrong by definition. Track cancellation-after-ETA as a separate category.
- Minimum sample: 30 disruption notifications with ETAs per measurement period.
Proposed formula, pending validation pilots.