Definition
Measures the change in an AI visual inspection system's detection rate for each defect type over time, weighted by severity and frequency.
The Measurement Gap
AI visual inspection systems in manufacturing are typically evaluated on aggregate metrics: overall detection rate, false positive rate, and throughput. These aggregate numbers are reported to management as proof the system is performing. The problem is that aggregate stability can mask category-level degradation.
Consider an inspection system monitoring five defect types. If detection improves on three common, low-severity types while declining on two rare, high-severity types, the aggregate detection rate stays flat or even improves (because the common types dominate the average). Meanwhile, the system is becoming less reliable at catching the defects that matter most.
This drift happens for predictable reasons. Product mix changes introduce new visual patterns. Lighting conditions shift with seasonal ambient changes. Camera degradation affects certain defect appearances more than others. Material changes alter the visual signature of specific defect types. Each of these factors affects some defect categories more than others, creating uneven drift that aggregate metrics cannot detect.
Without PDTDD, organizations discover category-level drift only when a specific defect type escapes to customers at an elevated rate. By then, the problem has been growing for months. PDTDD provides early warning by tracking detection performance at the granularity where drift actually occurs.
Formula
PDTDD is calculated as the weighted average change in detection recall across all defect types:
PDTDD = Sum of (w_t x Delta_Recall_t) / Sum of w_t
Where:
- w_t = weight of defect type t based on severity and frequency (dimensionless)
- Delta_Recall_t = change in AI detection recall for defect type t between current and previous measurement period (percentage points)
Negative PDTDD indicates overall detection is degrading on weighted-important defect types. Positive indicates improvement. Zero means stable performance.
Key considerations: PDTDD depends on ground-truth data from audit inspections (human or enhanced-method inspection on a sample of parts). Without an audit program, PDTDD is unmeasurable. The audit must be structured by defect type and large enough to produce statistically meaningful recall estimates per category.
For defect types with zero occurrences in a measurement period, carry forward the previous recall estimate and flag as "untested." This prevents artificial stability scores when certain defect types simply have not appeared. New defect types should be assigned recall of zero for the first period, creating a visible gap in PDTDD that triggers attention. The weighting scheme should reflect business impact: a 5-percentage-point decline in detection of high-severity, safety-critical defects matters more than the same decline for cosmetic defects.
Proposed formula, pending validation pilots.