Definition
PCCI-Mfg measures the degree to which a quality prediction model's stated confidence levels correspond to its actual prediction accuracy at each confidence level.
The Measurement Gap
The decision PCCI-Mfg supports is whether inspection frequency or scope should be reduced for parts and batches the quality prediction model flags as high-confidence pass. If the model says a batch has a 95% probability of passing inspection, should the organization skip the inspection? That decision is only safe if the model's confidence is calibrated. The measurement gap exists because quality prediction platforms report confidence scores without validation infrastructure to verify those scores against actual outcomes. Plants deploy the model, observe that overall defect rates are acceptable, and assume the confidence scores are trustworthy. The invisible failure mode is overconfidence: the model reports 90% confidence but actual accuracy at that level is 75%. This creates a false sense of security that justifies inspection reductions the model's actual accuracy does not support. When PCCI-Mfg is not measured, the organization discovers the miscalibration only after defects escape to customers or downstream processes. A declining PCCI-Mfg over time indicates model degradation as production conditions drift from the training data.
Formula
For each confidence band (such as 80-85%, 85-90%, 90-95%, 95-100%), compare the model's actual accuracy to its stated confidence. The per-band calibration is: PCCI-Mfg(b) = Actual_accuracy(b) / Stated_confidence(b). The overall index is the weighted average: Overall PCCI-Mfg = Sum(n_b times PCCI-Mfg(b)) / Sum(n_b).
Variables: Actual_accuracy(b) is the percentage of predictions in confidence band b confirmed correct by downstream inspection or field outcome, expressed as a dimensionless ratio per measurement period. Stated_confidence(b) is the midpoint of confidence band b as reported by the prediction model, also a dimensionless ratio. n_b is the count of predictions in band b per measurement period.
Sign convention: 1.0 equals perfect calibration. Values above 1.0 indicate the model is underconfident (conservative, stating 90% when actual accuracy is 95%). Values below 1.0 indicate overconfidence (dangerous, stating 90% when actual accuracy is 75%). Below 1.0 is the risk direction.
Edge cases: if a confidence band has fewer than 30 predictions in the period, widen the band or extend the period because small samples make calibration assessment meaningless. If no downstream inspection exists to validate predictions, PCCI-Mfg is undefined because you cannot calibrate what you cannot verify. Models that do not produce confidence scores make PCCI-Mfg not applicable.
Proposed formula, pending validation pilots.