Definition
Measures whether a production-aware predictive maintenance model's confidence scores correspond to actual prediction accuracy when segmented by product type.
The Measurement Gap
Traditional PdM model validation computes a single aggregate accuracy or calibration score across all operating conditions. This approach masks a critical problem in multi-product manufacturing: the model may achieve 90% accuracy overall because it performs at 95% on the three high-volume products that dominate the training data and 60% on the two low-volume products that were underrepresented. A maintenance planner relying on that 90% aggregate accuracy will trust PdM alerts equally regardless of what product is running, and will make poor intervention decisions during low-volume product runs where the model is unreliable.
The decision PCCCI supports is whether PdM alert response criteria should vary by product type. If the model's confidence is well-calibrated across all products, a single alert threshold and response protocol is sufficient. If PCCCI reveals that the model is overconfident on certain products (stating 90% confidence when actual accuracy is 70%), those product types need either different alert thresholds, additional human validation before acting on alerts, or model retraining with product-specific features. Without per-product calibration measurement, the organization is blind to this risk and treats all PdM outputs as equally trustworthy regardless of production context.
Formula
PCCCI computes calibration at the intersection of product type and confidence band, then aggregates to a per-product score that reveals where the model's stated confidence diverges from reality.
PCCCI(p, b) = Actual_accuracy(p, b) / Stated_confidence(b)
Aggregate per product: PCCCI(p) = Sum over b of (n_pb times PCCCI(p,b)) / Sum over b of n_pb
Where Actual_accuracy(p, b) is the percentage of predictions in confidence band b on product type p that were confirmed correct (dimensionless ratio). Stated_confidence(b) is the midpoint of confidence band b as reported by the PdM model (dimensionless ratio). n_pb is the number of predictions in band b on product type p (count). p is the product type identifier. b represents confidence bands such as 70-80%, 80-90%, 90-95%, and 95-100%.
A PCCCI of 1.0 means perfect calibration for that product: the model's stated confidence matches its actual accuracy. Below 1.0 means overconfident for this product, which is the danger signal. The model claims 90% confidence but delivers only 70% accuracy on this product type. Above 1.0 means underconfident, which is inefficient but not dangerous. The model is more accurate than it claims, leading to potentially unnecessary conservatism in maintenance responses.
Edge cases require attention. Low production volume for a product type may yield insufficient samples in each confidence band. In that case, aggregate to a single per-product calibration instead of per-band, or extend the measurement period. Product types not in the model's training data will likely show severe miscalibration, which is an expected and informative result, not a data quality problem. Minimum sample requirements are 30 predictions per product type per measurement period, with 15 per band for band-level analysis.
Proposed formula, pending validation pilots.