System-Type Prediction Accuracy Variance (STPAV)

GMI-HC-024HealthcareIntegration-Required

Definition

STPAV measures the variance in prediction accuracy across building system categories: rotating equipment, electrical systems, HVAC controls, plumbing, and building envelope. A low STPAV indicates the predictive model performs consistently across all system types. A high STPAV reveals that aggregate prediction accuracy hides significant performance differences between system categories, meaning the model is reliable for some systems and unreliable for others.

The Measurement Gap

Measurement Gap and Decision

Decision Question: Does the predictive maintenance model perform equally well across all building system types, or does the aggregate accuracy conceal critical blind spots in specific system categories?

What Goes Unmeasured Today:

Predictive maintenance platforms report a single accuracy metric across all monitored equipment. That aggregate number might be 82%: acceptable enough to justify the program. But rotating equipment (chillers, pumps, fans) may drive 85% accuracy because vibration-based degradation signatures are well-understood, while electrical systems (switchgear, UPS, transfer switches) may sit at 45% accuracy because electrical failure modes have different degradation patterns that the model handles poorly. The aggregate metric hides this disparity. Facilities leadership sees one number and extends the predictive program to cover more equipment, including electrical systems where the model's performance is unreliable. Decisions that should require higher scrutiny (like deferring PM on an automatic transfer switch that serves surgical suites) get the same treatment as deferring PM on a secondary chilled water pump. The failure consequence is orders of magnitude different, and the prediction quality is orders of magnitude worse.

Invisible Failure Mode:

Aggregate prediction accuracy creates false confidence in system-specific reliability. Equipment categories with poor prediction performance receive the same trust as categories with strong performance, leading to high-risk deferrals in precisely the domains where the model is least accurate (Jardine et al., 2006).

Formula

Primary Formula:

STPAV = σ²(Accuracy_by_system_type)

Where σ² = variance of prediction accuracy rates across all defined building system categories.

Variable Definitions:

Variable Definition Source
Accuracy_by_system_type Prediction accuracy rate (predictions where actual outcome matched prediction / total predictions) for each building system category Predictive maintenance platform + CMMS/EAM
System categories Defined groupings: rotating equipment, electrical distribution, HVAC controls, plumbing/piping, building envelope, medical gas, fire/life safety Asset classification in CMMS/EAM
σ² Statistical variance across category-level accuracy rates Calculated

Data Requirements:

Prediction logs with predicted outcome (RUL, failure/no-failure) and actual outcome for each monitored asset. Asset classification by building system type in CMMS/EAM. Minimum 50 predictions per system type for statistical stability.

Edge Cases and Limitations:

System categories with fewer than 50 predictions should be flagged as statistically unreliable rather than excluded. Small samples with poor accuracy are a signal, not noise. Prediction accuracy definitions must be consistent across system types (same window, same failure criteria). New system types added to the predictive program will have insufficient history for meaningful accuracy calculation: track separately until volume reaches threshold.

Proposed formula, pending validation pilots.

At a Glance

Category
Predictive Building Maintenance Decision Quality
← Predictive Deferral Validation Rate (PDVR)Census Prediction Economic Accuracy (CPEA) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment