Definition
RECA measures the percentage of events with regulatory significance (refrigerant leaks, OSHA-reportable incidents, fire protection impairments, electrical safety events) that NLP classification systems correctly identify as requiring compliance action, with the companion Regulatory Miss Rate tracking the inverse failure mode.
The Measurement Gap
ITSM and CMMS systems generate work orders and incident records that are routed based on classifications assigned by technicians or AI systems. When those classifications are incorrect, the downstream routing is wrong, a refrigerant release classified as a routine HVAC service call never enters the EPA 608 compliance queue. Standard NLP performance metrics (overall accuracy, precision, recall) measure performance across all event types, which means rare but critical regulatory events are swamped by high-frequency routine events in the aggregate statistics. A system that correctly classifies 98% of all events may have a 30% miss rate specifically on refrigerant releases because they represent only 2% of the training data. This failure mode is invisible to any existing data center KPI.
Decision Supported:
RECA determines whether the NLP classification system can be relied upon as the detection layer for regulatory-significance events, or whether its miss rate on compliance-critical events requires parallel human review processes to ensure compliance events reach the appropriate queue.
Formula
Plain language: Of the events that compliance officers confirm had regulatory significance, what fraction did the NLP system miss entirely? That regulatory miss rate is the headline metric. Overall classification accuracy is reported alongside it for model health, but the false-negative rate on regulatory events is the number that matters for compliance risk.
Primary metric: Regulatory False-Negative Rate:
RECA_FNR = Missed_regulatory / Total_regulatory_events_in_sample
Secondary metric: Overall Regulatory Classification Accuracy:
RECA_OA = Correctly_classified_regulatory / Total_regulatory_events_in_sample
Variables:
| Variable | Definition | Unit |
|---|---|---|
| Missed_regulatory | Confirmed regulatory events that the NLP system classified as non-regulatory (false negatives) | Count |
| Total_regulatory_events_in_sample | Total confirmed regulatory events in the validation sample, per compliance officer review | Count |
| Correctly_classified_regulatory | Confirmed regulatory events the NLP system correctly flagged as requiring compliance action | Count |
Because regulatory events are rare, aggregate a minimum of 12 months of data before computing. Sample from all ITSM/CMMS events; have compliance officers classify ground truth. Multi-regulatory events (one incident triggering both OSHA and EPA obligations) count once per regulatory pathway.
Why false-negative rate is primary:
In data center operations, false positives (flagging routine events as regulatory) create investigation labor. False negatives (missing actual regulatory events) create unreported OSHA incidents, EPA violations, SOC 2 audit findings, and personal liability for facility managers. The asymmetry is a consequence structure, not a preference. A 95% overall accuracy that misses 8% of reportable environmental or safety events is a compliance liability, not a success.
Proposed formula, pending validation pilots.