Definition
PNBSR measures the percentage of quality defects that occurred on parts or batches where the prediction model reported nominal confidence, meaning the model did not see the defect coming because the defect type or condition fell outside its training distribution.
The Measurement Gap
The decision PNBSR supports is whether the quality prediction model is keeping up with new defect modes or optimizing for historical patterns while novel defects pass through undetected. The measurement gap exists because prediction model accuracy is typically evaluated as a single aggregate number. If overall accuracy is 92%, the model looks healthy. But aggregate accuracy masks category-level blindness: the model may be excellent at predicting dimensional defects (which dominate the training data) while being completely blind to surface contamination defects from a new material lot. The invisible failure mode is silent scope erosion. As production evolves through new materials, new processes, and new product geometries, the model's training distribution becomes progressively narrower relative to the production reality. Defects that fall outside the training distribution do not generate low-confidence predictions because the model does not know what it does not know. It reports nominal confidence on parts it has never seen before, and those parts fail in the field. A rising PNBSR over time is the clearest signal that the model's training data needs refreshing against current production conditions.
Formula
Count defects found by downstream inspection or field returns. Of those, count how many occurred on parts or batches where the quality prediction model did not flag elevated risk at the time of production. The formula is: PNBSR = Defects_unflagged / Defects_total times 100.
Variables: Defects_unflagged is the count of defects found on parts or batches where the prediction model's risk score was below the intervention threshold at production time, per measurement period. Defects_total is all defects identified by downstream inspection, final test, or field returns in the measurement period.
Sign convention: 0% means the model flagged every defect in advance (unrealistic). 100% means the model missed every defect. A rising trend is the danger signal indicating the model's blind spot is growing.
Edge cases: if Defects_total is zero, PNBSR is undefined and the measurement period should be extended. Defects found by the AI's own inline inspection should be excluded because PNBSR measures what the prediction model misses, not what the inspection model misses. Distinguish model-miss (the defect type is one the model should detect) from model-scope (the defect type was never in the model's training scope). Track both, but PNBSR rising for in-scope defect types is the urgent signal. Minimum sample: 20 defects per measurement period.
Proposed formula, pending validation pilots.