Time to Calibrated Trust (TTCT)

GMI-MRO-010MROResearch-Forward

Definition

The elapsed time required for a technician's AI-recommendation acceptance behavior to converge on the model's actual empirical reliability, where convergence is sustained across consecutive observation windows within a defined tolerance.

The Measurement Gap

Predictive AI introduces a decision that did not exist before. Whether to act on a model's recommendation about an asset the technician has not personally inspected. Legacy adoption dashboards report acceptance rates without context: 70% acceptance is celebrated regardless of whether the model is right 60% or 85% of the time. Both under-trust (disuse) and over-trust (misuse) are invisible to these metrics. MTBF measures equipment behavior, not human behavior. OEE measures production output. PM Compliance measures whether scheduled work got done. Even model-side metrics (precision, recall, F1) evaluate the model in isolation, treating the human as irrelevant to outcome quality.

Decision Supported:

Whether AI deployment is producing calibrated technician judgment (trust matched to model reliability) or is producing automation bias or automation disuse, and where coaching or model improvement is needed.

Formula

Plain language: Track how often a technician accepts AI recommendations versus how often the model is actually right. TTCT is the time it takes for those two rates to come within a defined tolerance and stay there for several consecutive windows.

Notation:

TTCT = min(t) such that |A_t − R_t| ≤ ε for k consecutive windows

Variables:

Variable Definition Unit
A_t Technician acceptance rate in rolling window t (AI-triggered WOs completed ÷ total AI-triggered WOs created) Ratio (0–1)
R_t Model empirical correctness rate in window t (true positives ÷ (true positives + false positives), validated against failure-report outcomes) Ratio (0–1)
ε Convergence tolerance (recommended starting value: 0.10) Ratio
k Number of consecutive windows the gap must remain within ε (recommended: k = 3) Count
t Rolling window index (recommended: 30 days) Time

Direction of the gap matters: A_t > R_t indicates over-trust (automation bias); A_t < R_t indicates under-trust (automation disuse). The two require different interventions.

Proposed formula, pending validation pilots.

At a Glance

Unit
Time (months)
Trend Direction
Lower
Reporting Frequency
Rolling 30-day windows, quarterly review
Category
Human-AI Symbiosis & Workflow
← Skill Atrophy Trajectory (SAT)Work Order Feedback Quality and Model Learning Gain (WOFQ) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment