Inspection Detection Accuracy Rate (IDAR)

GMI-AR-002Asset ReliabilityInfrastructure-Dependent

Definition

IDAR is built from three components: recall, precision, and a composite score that balances them. Each one answers a different question about your AI inspection system's real-world performance.

The Measurement Gap

A Gulf Coast pipeline operator built what looked like a model AI inspection program. They deployed drone-based computer vision across their above-ground piping and vessel exteriors: CV trained to spot corrosion, coating degradation, flange leaks, and structural deformation. Two years in, they were covering four times the inspection footage of their old manual program at roughly the same cost. Every inspection record was clean: asset photographed, image processed, findings logged. The program looked like a success by every metric they were tracking.

Then a flange on a 12-inch high-pressure gas line failed.

Emergency shutdown. Environmental remediation. A regulatory report that nobody wanted to write. Total cost: over $1.4 million. When the incident team pulled the inspection history, they found that the drone had photographed that exact flange 11 weeks before the failure. The CV system had classified the image as "no defect detected." The inspection record showed "inspected, no findings": no asterisk, no confidence flag, no indication that anything was unusual about the result.

A human inspector reviewed that same image after the incident and saw it immediately: early-stage bolt corrosion, gasket degradation that had progressed beyond what you'd want to see on a high-pressure line. The flange sat in a shadow between two pipe racks. The drone captured it at an oblique angle. Under those conditions (poor lighting, constrained geometry) the CV model's detection accuracy had dropped dramatically. But the system didn't know to say so. It just said "no defect detected," and that answer went into the record with exactly the same weight as a clear-light, straight-on, ideal-conditions inspection.

The operations manager asked the question that should haunt every reliability engineer running an AI inspection program: "How do we know this isn't happening all over the plant?" Nobody could answer him.

That's the failure mode this chapter addresses. Not whether AI inspection happens: it clearly does. Not whether the drones fly and the cameras click and the images get processed: they do. The failure mode is invisible detection degradation: the CV system is completing inspections, but its ability to actually find defects is eroding under field conditions that nobody is measuring.

Here's the measurement problem you're sitting with right now. Inspection Completion Rate tells you what percentage of scheduled inspections were performed. It says nothing about whether those inspections found what they should have found. PM Compliance (one of the most commonly tracked maintenance KPIs) records whether a preventive maintenance task was completed on schedule. An AI-flagged inspection that runs on time and returns zero findings closes out the PM exactly the same way a thorough human inspection with three documented defects does. Both count as compliant. The metric cannot distinguish between "inspected and nothing was wrong" and "inspected and the system missed it."

Some programs track per-scan detection rate: the average number of defects found per inspection. That's better than nothing, but it conflates two very different signals: changes in actual asset health and changes in the AI system's detection capability. If detection rate drops, you don't know whether your assets got healthier or your model got worse under current field conditions.

There is no metric that tells you whether AI inspection systems maintain detection accuracy in real field conditions.

That gap is what IDAR (the Inspection Detection Accuracy Rate) is designed to close. The Gulf Coast operator couldn't answer their operations manager's question because they had no mechanism for benchmarking their AI system's real-world performance against a ground truth. They tracked completion. They tracked coverage. They tracked cost per inspection. They did not track whether the inspection, once completed, actually found what needed to be found. That distinction (between completing an inspection and accurately detecting defects during it) is the entire premise of this KPI.

Formula

IDAR is built from three components: recall, precision, and a composite score that balances them. Each one answers a different question about your AI inspection system's real-world performance. You need all three because recall and precision can move in opposite directions, and optimizing for one while ignoring the other will lead you to the wrong conclusion about whether your AI system is ready to operate with reduced human oversight.

IDAR_recall

answers the question: of the defects that actually exist, how many is the AI finding? The formula is IDAR_recall = D_both / D_human × 100, where D_both is the count of defects detected by both AI and human inspector during dual inspections, and D_human is the total count of defects detected by the human inspector. The human inspection is your ground truth benchmark. Recall tells you about false negatives: the things the AI missed that a qualified human would have caught. A low recall score is the Gulf Coast scenario: the AI is inspecting, completing, logging, and missing defects that are right there in the image.

IDAR_precision

answers the opposite question: when the AI says it found a defect, how often is it right? The formula is IDAR_precision = D_both / D_ai × 100, where D_ai is the total count of defects detected by the AI system. Precision tells you about false positives: cases where the AI flagged something that a qualified human reviewer determined was not actually a defect. High false positive rates are operationally expensive: they generate work orders that consume maintenance resources to investigate findings that aren't real. A low precision score doesn't create the safety risk that low recall does, but it destroys the economic case for AI inspection by flooding your work queue with noise.

IDAR_composite

combines recall and precision into a single number using the harmonic mean, which is the right tool here because it penalizes imbalanced scores harder than a simple average would. The formula is IDAR_composite = 2 × (IDAR_recall × IDAR_precision) / (IDAR_recall + IDAR_precision). The harmonic mean ensures that a system with 95% precision and 40% recall doesn't score well just because one number is high. Both components have to be strong for the composite to be strong. Use IDAR_composite ≥ 85% as your threshold for decisions about inspection frequency reduction. Below 75% means the AI cannot replace human inspection for that defect type or field condition: you need human redundancy, period. Here's how the math works in a real evaluation. A refinery runs dual inspections (drone CV system and qualified human inspector, independently) on 45 piping circuits over 30 days. Across all circuits: the human found 127 total defects; the AI found 108 total defects; 89 defects were found by both. IDAR_recall = 89/127 × 100 = 70.1%. IDAR_precision = 89/108 × 100 = 82.4%. IDAR_composite = 2 × (0.701 × 0.824) / (0.701 + 0.824) = 75.7%.Read those numbers carefully. The AI is missing about 30% of the defects a qualified human inspector catches: that's the recall gap, and it's not a rounding error. At the same time, when the AI does flag a defect, it's right 82% of the time: reasonable precision, meaning the system isn't generating massive false positive noise. The composite score of 75.7% sits just above the 75% floor. This AI system is not ready for independent inspection on these circuits. It needs human redundancy. The 19 defects the AI found that the human didn't (108 minus 89) deserve investigation rather than dismissal. In this evaluation, 14 turned out to be false positives: surface conditions the model interpreted as defects that weren't. But 5 were real defects the human missed. That's worth knowing. It means the AI genuinely adds value in certain detection scenarios, and your program should capture and analyze those cases systematically rather than treating them as anomalies. The picture gets much more actionable when you segment by field condition. Break those 45 circuits into categories based on the conditions at the time of inspection. Good lighting with clean surfaces: Recall 88.4%, Precision 91.2%, Composite 89.8%. That's a system performing above the 85% threshold, meaning it has earned the right to operate with reduced human oversight in those conditions. Poor lighting: Recall 54.3%, Precision 76.1%, Composite 63.4%. Well below the 75% floor, human inspection is mandatory. Wet or dirty surfaces: Recall 61.7%, Precision 68.9%, Composite 65.1%. Same conclusion, mandatory human redundancy. Obstructed angles: Recall 47.2%, Precision 72.3%, Composite 57.1%. The worst performance category, and the exact conditions that existed at the Gulf Coast flange. The segmented data is what makes IDAR actionable rather than just diagnostic. The overall composite of 75.7% tells you the system has a problem. The segmented breakdown tells you exactly where the problem lives: poor lighting, wet surfaces, and constrained angles account for almost all of the recall degradation. That's information you can act on: by changing drone flight protocols to improve capture angles, by scheduling inspections during better lighting conditions, by flagging assets in shadow zones for mandatory human redundancy. Without the segmentation, you know something is wrong. With it, you know what to fix. Two cross-references belong here as causal statements, not incidental mentions. ABCS (the Autonomy Boundary Calibration Score in GMI-AR-006) governs decisions about expanding AI inspection autonomy. If IDAR is low, expanding autonomy is dangerous. You cannot use ABCS to justify reducing human oversight if IDAR shows your detection accuracy is degrading under real field conditions; the autonomy score would be built on a foundation that doesn't hold. AWSNR (the Autonomous WO Signal-to-Noise Ratio in GMI-AR-005) measures the quality of inspection-triggered work orders. If IDAR is low, those work orders are suspect at the source: the inspection findings driving work generation cannot be trusted, so AWSNR will degrade as a downstream consequence. Fix IDAR first. Three edge cases need explicit handling. First, if the AI finds defects the human misses, those go into D_ai but not D_both: they improve precision numerically but don't affect recall. Don't dismiss them; investigate whether the AI is genuinely detecting a defect type or condition the human isn't trained to recognize. Second, track IDAR by condition category rather than just overall: environmental variation is the primary driver of AI performance degradation, and the overall number masks the most actionable information. Third, if D_human equals zero (the human inspector found no defects) recall is mathematically undefined. Report it as "no defects detected by benchmark" rather than claiming 100% recall. Require a minimum of 30 dual inspections per evaluation cycle, covering the full range of defect types and field conditions your assets actually present.

Proposed formula, pending validation pilots.

At a Glance

Category
AI Data and Source Trustworthiness

Decision Supported

Can I reduce manual inspection frequency based on AI inspection coverage, or is the AI missing defects that a human would catch?

← Failure Code Fidelity Rate (FCFR)Knowledge Retrieval Accuracy Score (KRAS) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment