Optimization Fragility Index (OFI)

GMI-MFG-019ManufacturingInfrastructure-DependentPattern O-11: Optimization Fragility

Definition

Measures the ratio of yield variance under an AI-optimized recipe to yield variance under the previous recipe, across the full range of normal production variation.

The Measurement Gap

Experienced process engineers and operators develop recipes over years of running production under real conditions. Their recipes account for material lot variation, seasonal temperature changes, equipment wear patterns, and the dozens of small factors that affect yield in practice. The resulting recipes may not produce the highest theoretical yield, but they produce consistent yield across the range of conditions the plant actually encounters.

AI optimization systems target a different objective. They analyze process data and identify parameter combinations that maximize yield in the data they can see. If the training data is dominated by production runs under favorable conditions (consistent material, stable temperatures, well-maintained equipment), the AI will optimize for those conditions. The result can be a recipe that performs excellently 70% of the time and poorly 30% of the time, producing a higher mean yield than the legacy recipe but with significantly more variance.

The gap is that recipe performance is typically reported as mean yield. A recipe that averages 94% yield looks better than one that averages 91% yield. But if the 94% recipe ranges from 85% to 99% while the 91% recipe ranges from 89% to 93%, the lower-mean recipe may be the better production choice because it never falls below the quality threshold. Without OFI, organizations cannot see this tradeoff, and they adopt AI-optimized recipes based on mean performance alone.

Formula

OFI is calculated as the ratio of yield variance under the AI recipe to yield variance under the legacy recipe:

OFI = variance_yield_AI / variance_yield_legacy

Companion metric: Robustness-Adjusted Yield (RAY) = mean_yield_AI minus k x stddev_yield_AI

Where:

  • variance_yield_AI = variance of yield under the AI-optimized recipe across production conditions (yield percentage squared, over the measurement period)
  • variance_yield_legacy = variance of yield under the previous recipe across comparable production conditions (yield percentage squared)
  • mean_yield_AI = mean yield under the AI recipe (percentage)
  • stddev_yield_AI = standard deviation of yield under the AI recipe (percentage)
  • k = risk factor, typically 1.5 to 2.0, representing how many standard deviations of downside the organization tolerates

OFI of 1.0 means equal robustness. Above 1.0 means the AI recipe is more fragile (higher variance). Below 1.0 means the AI recipe is more robust. For RAY, higher is better because it penalizes high mean yield that comes with high variance.

Key considerations: The legacy recipe may no longer be runnable if equipment or materials have changed. Use the most recent comparable period. Minimum 20 production runs for meaningful variance comparison. Control for confounders (operator variability, equipment wear, material lot variation) if possible. The cleanest test runs both recipes on the same material lots and equipment.

Proposed formula, pending validation pilots.

At a Glance

Unit
Index
Trend Direction
Lower
Reporting Frequency
Quarterly
Category
AI Formulation and Recipe Robustness
Temporal Property Validation Gap (TPVG) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment