Proxy Model Fidelity Decay Rate (PMFDR)

GMI-OG-007Oil and GasIntegration-Required

Definition

The Proxy Model Fidelity Decay Rate (PMFDR) measures the rate at which an AI reservoir proxy model is drifting away from the full-physics simulator over the months since the proxy was last calibrated.

Plain language.

The Measurement Gap

Aaron is the reservoir engineer at a tight-oil operator with about 240 wells across two contiguous sections of a Permian shale play. The asset team adopted an AI proxy reservoir model 22 months ago: the proxy was trained against the operator's full-physics ECLIPSE simulator using two years of production history and produces well-level EUR forecasts and injection-strategy responses in seconds rather than the hours a full-physics run requires. The model performed well at calibration: the proxy's predictions were within 4% of the simulator's for the 38 wells in the validation set.

In Q1 the asset team uses the proxy to evaluate an infill program: six wells at $7.5 million each, a $45 million capital decision. The proxy says the infill wells will deliver an average 312 MSTB EUR with reasonable spacing of the existing producers. The investment committee approves. The wells are drilled and completed over the next six months. By the time the third well has 90 days of production, Aaron can see the trajectory is well below the proxy forecast: the first three wells are tracking to 240–260 MSTB EUR, roughly 20% below the proxy prediction.

Aaron runs the full-physics simulator on the same well locations using the current field state. The simulator predicts 245 MSTB average EUR: close to what the wells are doing. The proxy was wrong by 22%. The proxy was wrong because the field's pressure, water cut, and inter-well communication had drifted materially in the 22 months since calibration. The proxy was trained on field conditions that no longer existed. The simulator, which incorporates the current well states and pressure measurements, captured the change. The proxy, by design, did not.

The investment committee did not make a bad decision against the information they had. They made a bad decision against the information they could have had if anyone had been measuring proxy fidelity. The $45 million was committed against numbers the proxy could not honestly produce anymore.

Legacy reservoir metrics cannot see this directly. Recovery factor is computed against ultimate recovery and is fundamentally a lagging measurement: it tells you what happened, not whether your forecasting tool is current. Decline rate analysis on producing wells reveals problems on each well as it produces, which is helpful for type-curve revisions but does nothing to validate the proxy used for capital-decision forecasts on wells not yet drilled. EUR variance (the spread between forecast EUR and actual EUR across a portfolio of wells) is a useful retrospective metric but is reported well-by-well after each well has produced enough for type-curve fitting, which is typically 6-12 months too late to inform a capital decision.

What is missing is a forward-looking measurement of how well the proxy still represents the reservoir state as of right now. That measurement requires comparing the proxy against the simulator on a defined cadence and tracking how the gap is growing.

The measurement gap is precise. AI proxy models drift relative to full-physics simulators as reservoir conditions change, and the drift accumulates between calibration events at a rate that depends on the velocity of field change. No current metric reports the drift rate before it is large enough to invalidate a capital decision. The question PMFDR proposes to answer is the one Aaron could not ask in March: are well placement and injection strategy decisions based on a proxy that accurately represents current conditions?

Formula

The Proxy Model Fidelity Decay Rate (PMFDR) measures the rate at which an AI reservoir proxy model is drifting away from the full-physics simulator over the months since the proxy was last calibrated.

Plain language. Take a set of output variables the proxy predicts: EUR, peak rate, water breakthrough timing, whatever the proxy is used to forecast. Run the same scenario through the full-physics simulator with current field conditions. Compute the average percentage gap between proxy and simulator across the output set, that is FidelityGap. Then track how FidelityGap is growing per month since the last calibration. That growth rate is PMFDR. If the proxy was 4% off the simulator at calibration and is now 14% off seven months later, PMFDR is roughly 1.4 percentage points per month. The metric is about velocity of drift, not absolute fidelity at any one moment.

Notation:

PMFDR = (FidelityGap_current − FidelityGap_calibration) ÷ Months_since_calibration

FidelityGap = (1/N) × Σ |Proxy_prediction_i − Simulator_prediction_i| ÷ Simulator_prediction_i

Where:

  • Proxy_prediction_i = the AI proxy's forecast for output variable i at a defined horizon (typically 1-year EUR)
  • Simulator_prediction_i = the full-physics simulator's forecast for the same variable at matched input conditions
  • N = the number of output variables being compared
  • FidelityGap_calibration = the FidelityGap measured at the most recent proxy calibration
  • Months_since_calibration = the elapsed time since that calibration in months
  • The metric is computed quarterly at minimum, monthly preferred

The absolute-value structure inside FidelityGap means PMFDR captures drift in either direction, a proxy that increasingly over-predicts EUR or increasingly under-predicts is both equally a fidelity problem. The PMFDR formula itself can be negative (FidelityGap improving) but the typical signal is positive (drift accumulating).

Edge cases. A stale simulator defeats the comparison. The simulator must be running on current field state (current pressures, current water cuts, current well status) for the comparison to be meaningful. If the simulator is itself out of date, both proxy and simulator are wrong and PMFDR measures the difference between two wrong answers. Quarterly simulator updates of the field state are the minimum for PMFDR to be diagnostic.

Recalibration resets PMFDR to near-zero. After a recalibration event, FidelityGap_current is back near FidelityGap_calibration (in fact, the new calibration baseline becomes the new FidelityGap_calibration). The PMFDR computation should reset the months-since-calibration clock at the new calibration date. The trend before and after the calibration is then trackable separately.

Quarterly comparisons are the minimum. The metric requires at least two FidelityGap measurements to compute, and the spacing should be no shorter than monthly (to give the proxy time to drift to a measurable degree) and no longer than quarterly (to catch drift before it invalidates capital decisions). Monthly is preferable when computational capacity allows.

Proposed formula, pending validation pilots.

At a Glance

Category
Subsurface Decision Quality
← Multi-Well Pad Interference Rate (MWPIR)Seismic Interpretation Independence Score (SIIS) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment