Authentication Friction Accuracy (AFA)

GMI-IT-029IT OperationsIntegration-Required

Definition

Authentication Friction Accuracy (AFA) is a proposed measure of how often the adaptive authentication system applies friction to genuinely risky login sessions versus incorrectly challenging legitimate users or failing to challenge compromised credentials. The concept is simple even though building it is not. Every authentication challenge the system issues is either well-placed (the session was genuinely risky) or misplaced (the session was normal and the challenge was unnecessary friction). Every authentication session the system lets through without challenge is either correctly cleared (low-risk session, correct call) or incorrectly cleared (a compromised credential that should have been stopped). AFA measures the percentage of those decisions the system got right. The formula, in three layers:

The Measurement Gap

The security engineer pulled up the authentication logs for a Monday morning. There were 340 MFA challenges issued between 8 AM and 9 AM: peak login hour. Eighty-two percent of them resolved normally: the user authenticated, the session proceeded. The remaining 18% looked like failures, but were they? Nobody was investigating. The engineer had no way to tell whether those 340 challenges were correctly placed: whether the 280 successes were on legitimately risky logins that the system correctly identified, or whether they were frustrated employees working from their usual desk, on their usual device, at their usual time, being challenged for no obvious reason.

Three things were happening in parallel. First, users who worked from home had learned to VPN into the corporate network before starting their workday. Not because their VPN usage policy required it, because being on a "known" IP address meant fewer MFA challenges. The security team had accidentally trained users to route their traffic through a bottleneck they couldn't monitor as closely. Second, a developer on the mobile application team had configured his personal laptop as a "trusted device" using an alternate authentication path, specifically to stop being challenged every time he switched between his home network and the coffee shop he worked from. He'd shared the method informally with three colleagues. Third, the security team's weekly report showed MFA challenge rate at 8.7% of all authentications: right in the target range. Every metric they tracked was green.

None of the three legacy metrics could see this failure. MFA challenge rate measured how often the system issued challenges, not whether the challenges were correctly targeted. MFA success rate measured whether users completed challenges successfully, not whether the sessions being challenged actually warranted them. MFA adoption rate measured how many users were enrolled, not how many were actively routing around friction. These metrics were designed for a simpler problem: are users enrolled in MFA? Is MFA being used? They answer those questions accurately. But they can't answer the question that matters when adaptive authentication is running: is the AI applying friction to the right sessions?

The structural reason for the gap is straightforward. Adaptive authentication was supposed to solve the binary friction problem. If you require MFA for everything, users find workarounds; if you require nothing, credential theft goes undetected. The adaptive model was supposed to thread that needle: high friction on genuinely risky logins, low friction on verified-normal sessions. But measuring whether the threading actually worked requires knowing which logins were genuinely risky, and that ground truth only becomes available after a security event, if at all. So teams default to measuring the simpler thing: how often friction was applied. Not whether it should have been.

Users are rational about friction. When adaptive authentication challenges them repeatedly on legitimate logins (from their home office, on their registered laptop, at the same time they log in every day) they don't conclude the system is protecting them. They conclude the system is wrong and find a way around it. The VPN pre-routing, the trusted device workaround, the authentication bypass app that the third-party contractor brought in. These are all predictable responses to misfired friction. And none of them show up in the MFA dashboard.

There is no metric that tells you whether adaptive authentication challenges were correctly targeted: whether the challenges landed on risky sessions or on employees just having a Monday morning. That gap is what Authentication Friction Accuracy is designed to close.

Formula

Authentication Friction Accuracy (AFA) is a proposed measure of how often the adaptive authentication system applies friction to genuinely risky login sessions versus incorrectly challenging legitimate users or failing to challenge compromised credentials. The concept is simple even though building it is not. Every authentication challenge the system issues is either well-placed (the session was genuinely risky) or misplaced (the session was normal and the challenge was unnecessary friction). Every authentication session the system lets through without challenge is either correctly cleared (low-risk session, correct call) or incorrectly cleared (a compromised credential that should have been stopped). AFA measures the percentage of those decisions the system got right.

The formula, in three layers:

Plain language: Compare every authentication challenge to its outcome. Classify each challenge as a true positive, the session was genuinely risky, confirmed by a subsequent security event, a failed re-authentication, or SOC investigation, or a false positive, the session was legitimate, the user completed authentication, and the session proceeded normally with no security events in the following week. Also classify non-challenged sessions as true negatives (low-risk, no security event followed within 7 days) or false negatives (appeared low-risk, but a security event occurred within 7 days). Compute overall classification accuracy.

Notation:AFA = (TP + TN) / (TP + TN + FP + FN)

Where:

  • TP = challenged sessions subsequently confirmed as genuinely risky (failed challenge followed by investigation, or succeeded challenge where session showed anomalous activity confirmed by SIEM)
  • TN = unchallenged sessions with no subsequent security events within 7 days
  • FP = challenged sessions that completed normally with no subsequent security events within 7 days (unnecessary friction on legitimate users)
  • FN = unchallenged sessions where a security event occurred within 7 days (risk was missed at the authentication decision point)

Worked Example

In a 30-day period with 100,000 authentication events, 8,500 challenges were issued. Of those challenges, 1,200 correlated with subsequent security events within 7 days (TP). The remaining 7,300 resolved normally with no security events (FP). Of the 91,500 unchallenged sessions, 91,100 had no subsequent security events (TN). 400 unchallenged sessions were followed by confirmed security events (FN). AFA = (1,200 + 91,100) / (1,200 + 91,100 + 7,300 + 400) = 92,300 / 99,000 = 93.2%. The number looks strong. But the false positive rate tells a more operationally important story: 7,300 / 8,500 = 85.9% of all challenges were fired at legitimate users. That's where the workarounds come from.

Sign convention:

Higher AFA is better. The more operationally relevant number is usually the false positive rate (FP / (TP + FP)) because that's what directly drives user behavior changes. Track both.

Edge Cases

The 7-day lookback window for classifying sessions as true positives or false negatives is a proposed starting point. Some credential compromises take longer to surface, a sophisticated attacker may sit on a session for weeks before acting. Teams should use the window their SOC team finds operationally meaningful and document the choice in the metric definition. Users who fail MFA challenges legitimately (forgot their phone, lost their hardware token) represent false positives from the friction standpoint. Whether to classify them as FP or TP depends on whether the risk score that triggered the challenge was independently elevated for security reasons. If the challenge was issued because the risk score was genuinely high (unknown device, unusual location), it's still a TP even if the user's failure was logistical. If the risk score was normal and the challenge fired without cause, it's a FP.For very low challenge volumes (fewer than 100 challenges per month) AFA will be highly sensitive to individual events. Use rolling 90-day windows and note the sample size alongside any reported figure.

Cross-references:

The quality of AFA depends upstream on the behavioral baseline model that generates risk scores. If that baseline is stale (built on last year's work patterns before a return-to-office policy change or a team restructuring) the risk scores degrade and AFA follows. That upstream dependency is tracked by Behavioral Baseline Currency, a deferred Research-Forward KPI in this framework. Downstream, AFA connects to the Analyst Trust Calibration Index (ATCI, GMI-IT-010) from the existing IT KPI set. If AFA is chronically low (challenges landing on the wrong sessions) security analysts lose confidence in the identity risk scores. Investigation queues fill with false alarms and genuinely risky sessions get less scrutiny. The same trust erosion dynamic ATCI measures in the SOC context plays out in the identity context through AFA.AFA is classified as Integration-Required. Authentication event data exists natively in identity platforms; the ground truth for whether a session was genuinely risky requires joining to SIEM events and SOC investigation records. That join is the integration work.

Proposed formula, pending validation pilots.

At a Glance

Category
AI-Era IT Operations
← ARCHITECTURE DEBT VISIBILITY RATIO (ADVR)Automation Effectiveness Variance (AEV) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment