Definition
BESAI measures the trend in manual troubleshooting performance on complex or novel failure events: those that fall outside automated diagnosis. It tracks resolution time, escalation frequency, and diagnostic accuracy for non-routine events over time. Rising BESAI indicates that hands-on diagnostic skills are atrophying as automation absorbs routine work, leaving the workforce less capable of handling the complex failures that automation cannot resolve (Parasuraman & Manzey, 2010).
The Measurement Gap
Measurement Gap and Decision
Decision Question: As AI and building automation handle more routine anomalies, are building engineers and BMETs losing the diagnostic skills needed to resolve complex failures that fall outside automated capabilities?
What Goes Unmeasured Today:
Building engineers who once diagnosed AHU problems by listening to bearings, feeling vibration, and reading gauge patterns now monitor dashboards. BMETs who once tested infusion pump flow accuracy with manual calibration equipment now rely on condition health scores. The routine diagnostic exercise that maintained hands-on skills has been absorbed by automation. This is not a complaint about automation: it is a recognition that skill maintenance requires practice, and automation removes the practice opportunities. When a novel failure occurs (a multi-system cascade, an equipment malfunction outside the sensor envelope, a building system interaction the BAS was not programmed to handle) the response requires diagnostic skills that have not been exercised. Resolution takes longer. Escalation to outside contractors increases. Misdiagnosis rates rise for non-routine events. But no metric tracks this trend because MTTR is calculated across all work orders, and the routine work that automation improved dominates the average.
Invisible Failure Mode:
Aggregate maintenance performance improves while the organization's capacity to handle novel failures silently degrades. The workforce becomes efficient at normal operations and fragile against the abnormal events that carry the highest consequence (Bainbridge, 1983).
Formula
Primary Formula:
BESAI = (MTTR_complex_current / MTTR_complex_baseline) × (Escalation_Rate_current / Escalation_Rate_baseline)
Variable Definitions:
| Variable | Definition | Source |
|---|---|---|
| MTTR_complex_current | Mean time to repair for non-routine, complex, or novel failure events in current measurement period | CMMS/EAM filtered by event classification |
| MTTR_complex_baseline | Mean time to repair for comparable events in baseline period (pre-automation or early-automation) | CMMS/EAM historical |
| Escalation_Rate_current | Percentage of complex events requiring external contractor or specialist escalation in current period | CMMS/EAM escalation records |
| Escalation_Rate_baseline | Escalation rate for comparable events in baseline period | CMMS/EAM historical |
| Complex/novel events | Work orders classified as non-routine, multi-system, or requiring manual diagnosis (excludes routine PMs and simple corrective work) | Work order classification |
Data Requirements:
Work order records classified by complexity level. Resolution time for complex events. Escalation records distinguishing internal resolution from external contractor involvement. Baseline period data (minimum 12 months) from before or early in automation adoption.
Edge Cases and Limitations:
Baseline period selection critically affects interpretation. Choose a period when the workforce was fully staffed and experienced. Workforce turnover independent of automation also affects complex-event performance; control for experience level. The definition of "complex/novel event" requires clinical judgment: establish classification criteria with engineering leadership.
Confidence Grading (required alongside every BESAI value):
| Grade | Basis | Interpretation |
|---|---|---|
| A: Empirical | 5+ observed events in measurement window provide direct tail observation | Measurement grounded in observed outcomes; highest confidence |
| B: Hybrid | 1-4 observed events plus Monte Carlo simulation with Bayesian updating | Blends observation and simulation; moderate confidence |
| C: Simulated | Zero observed events; derived entirely from Monte Carlo simulation | Modeled, not observed; treat as indicative, not validated |
A BESAI of any value at Grade C is a simulation that agrees with itself. It tells you the model is internally consistent but not that it matches reality. Grade C scores should trigger proactive stress-testing against historical near-miss events and cross-asset analogues rather than acceptance as calibrated.
Proposed formula, pending validation pilots.