Access Anomaly Calibration Index (AACI)

GMI-FAC-021Facilities ManagementIntegration-RequiredPattern O-4: Alert Quality / Signal-to-Noise Ratio

Definition

Ratio of confirmed-actionable access anomaly alerts to total anomaly alerts generated, tracked by alert confidence tier.

The Measurement Gap

Measurement Gap and Decision

The Measurement Gap

Traditional access control generates binary alerts: badge read at unusual time, access at unusual door. These alerts are high-volume and low-specificity. AI-enhanced systems add confidence scoring and pattern matching, promising to surface the alerts that matter while suppressing the noise. But without AACI, security management has no way to verify whether the confidence tiers are calibrated. A "high-confidence" alert that is actionable only 30% of the time is not high-confidence: it is misleading labeling that erodes trust faster than having no confidence scores at all.

The additional gap is uninvestigated alerts. When AACI drops and fatigue sets in, the uninvestigated rate rises. A rising uninvestigated rate is itself a leading indicator of security exposure, not because the system missed something, but because the humans stopped looking.

The Decision It Supports

AACI tells security leadership whether the anomaly detection system is trustworthy at each confidence tier. If the high-confidence tier shows AACI above 70%, security teams can be directed to investigate every high-confidence alert. If it shows 25%, the confidence model needs retraining before anyone should rely on it. AACI also reveals whether the system improves over time as it ingests more data, or whether it stagnates at initial calibration levels.

Formula

Plain Language

For each anomaly alert, track whether the investigation outcome was confirmed actionable, benign, or uninvestigated. Compute the actionable rate within each confidence tier separately.

Formal Notation

$$AACI(t) = \frac{Confirmed_actionable(t)}{Total_alerts(t)} \times 100$$

Variable Definitions

Variable Definition Unit
t Alert confidence tier (High ≥80%, Medium 50-80%, Low <50%) Tier
Confirmed_actionable(t) Alerts in tier t confirmed as genuine security concerns requiring response Count
Total_alerts(t) All alerts generated in tier t during the measurement period Count

Units: Percentage per tier. Report AACI for each tier independently, a composite obscures whether the confidence model is meaningful. Also track the uninvestigated rate per tier as a companion metric.

Proposed formula, pending validation pilots.

At a Glance

Category
Security Decision Quality & Fairness

Decision Supported

Is the access anomaly system generating alerts security teams can trust, or noise that trains the team to dismiss alerts?

← Facilities Alert Economic Yield (FAEY)Credential Deactivation Lag (CDL) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment