Baseline Integrity Confidence (BIC)

GMI-WT-032Water TreatmentResearch-ForwardPattern O-2: Model Drift / Currency

Definition

BIC measures the confidence that an OT security anomaly detection model's training baseline represents uncompromised normal operations rather than a network state that already included attacker presence.

The Measurement Gap

Measurement Gap and Decision Supported

The Invisible Failure Mode: A water utility deploys OT security monitoring and allows the platform to learn network behavior over a 30-day training window. During that window, the utility's SCADA network is functioning normally by all observable measures. The platform establishes its baseline and begins monitoring. What the utility does not know is that a threat actor gained access to the network 8 months before deployment through a compromised vendor VPN connection and has been conducting low-level data exfiltration and maintaining persistent access. The anomaly detection platform learns this traffic as part of "normal": the attacker's periodic beaconing, lateral movement patterns, and data staging activities become part of the accepted baseline. The platform will never flag these activities as anomalous because it was trained to consider them normal.

What Existing Metrics Miss:

OT security platforms report detection rates, alert volumes, and false positive rates. All computed against the baseline. None of these metrics question whether the baseline itself is trustworthy. A platform can report excellent detection performance against its baseline while that baseline silently includes active compromise. Without BIC, there is no framework for evaluating baseline trustworthiness.

The Decision BIC Enables:

Should the utility trust the OT security platform's definition of normal, or does the uncertainty about baseline integrity require re-establishing the baseline during a confirmed-clean period with independent security validation?

Cross-References:

BIC is foundational to SAAA (GMI-WT-031). If the baseline is compromised, all anomaly attributions derived from it are unreliable. BIC also connects to CSARF (GMI-WT-033) through a different pathway: if low BIC triggers a baseline re-establishment, the resulting temporary increase in alerts during the transition period can accelerate analyst fatigue.

Formula

Plain language: Three components. (1) Was the training period vetted? Check whether the training data was collected during a period with an independent security assessment confirming no compromise. (2) Has the baseline been cross-validated? Compare the model's "normal" traffic patterns against known-clean reference patterns from vendor documentation or industry baselines. (3) Are there unexplained persistent anomalies? Track low-level alerts that persist at a constant rate, as constant background anomalies may indicate compromised traffic that the model has learned to accept.

Notation:
BIC = (Vetting_score x 0.4) + (Cross_validation_score x 0.3) + (Persistent_anomaly_score x 0.3)

Where:

  • Vetting_score = 1.0 if training period had independent security assessment confirming no compromise, 0.5 if assessment was internal-only, 0.0 if no assessment was performed
  • Cross_validation_score = fraction of traffic pattern categories where the model's "normal" matches industry reference patterns
  • Persistent_anomaly_score = 1.0 - (persistent_low_level_alerts / total_traffic_patterns). More persistent unexplained anomalies = lower score.

Units: 0 to 1. 1.0 = high confidence in baseline integrity. Below 0.5 = significant uncertainty about whether the baseline is clean.

Proposed formula, pending validation pilots.

At a Glance

Category
OT Cybersecurity Decision Quality
← Visual Assessment Scope Disclosure Score (VASDS)Cybersecurity Analyst Response Fatigue Index (CSARF) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment