Intervention False Positive Rate (IFPR)

GMI-MFG-003ManufacturingPilot-Ready

Definition

IFPR measures the percentage of AI-triggered quality interventions (line stops, parameter adjustments, part segregation, rework orders) where post-intervention analysis confirms the flagged condition would not have resulted in a defect.

The Measurement Gap

The decision IFPR supports is whether the quality prediction model's intervention threshold is set correctly and whether the organization is spending more on unnecessary interventions than it would spend on the defects the model prevents. The measurement gap exists because plants count AI-triggered interventions but do not systematically verify whether each intervention was necessary. The model flags a condition, the line stops, the team investigates, and the event closes. Nobody asks: would this batch actually have produced defective parts if we had let it run? The invisible failure mode is economic waste from false prediction. Every unnecessary intervention has a real cost: line downtime, labor, scrapped material, and rework. If IFPR is 50%, half of all AI-triggered quality actions are waste. The companion metric, Intervention Net Value (INV), translates IFPR into dollars by comparing the cost of defects prevented by true positives against the cost of unnecessary interventions from false positives. INV is the business case metric. Even if IFPR is high, the model may still create net value if the defects it does prevent are expensive enough to justify the false alarm cost.

Formula

Count all interventions triggered by the quality prediction model. For each, determine whether the flagged condition would have actually caused a defect based on post-intervention analysis, comparison to similar unflagged parts, or controlled let-it-run experiments. The formula is: IFPR = Interventions_unnecessary / Interventions_total times 100. The companion metric is: INV = (Defects_prevented times Avg_defect_cost) minus (Interventions_unnecessary times Avg_intervention_cost).

Variables: Interventions_unnecessary is the count of AI-triggered interventions confirmed as non-defective by post-hoc analysis, per period. Interventions_total is all AI-triggered quality interventions in the period. Defects_prevented is the count of interventions where the flagged condition was confirmed as defective. Avg_defect_cost is the average cost in dollars of a defect reaching downstream, including scrap, rework, warranty, and recall exposure. Avg_intervention_cost is the average cost per intervention in dollars, including line downtime, labor, scrapped material, and rework.

Sign convention: IFPR of 0% means every intervention was necessary, 100% means every intervention was waste. For INV, positive means the prediction model creates net value, negative means it costs more than it saves.

Edge cases: confirmation is hard because you cannot always know whether an intervened part would have failed. Use controlled experiments during pilot. Minimum sample: 50 interventions per measurement period.

Proposed formula, pending validation pilots.

At a Glance

Unit
Percentage
Trend Direction
Lower
Reporting Frequency
Monthly
Category
AI Quality Prediction Calibration and Value
← Per-Defect-Type Detection Drift (PDTDD)Prediction Confidence Calibration Index (PCCI-MFG) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment