Specification Error Propagation Rate (SEPR)

GMI-SWD-019Software DeliveryIntegration-RequiredPattern O-23: Specification Fidelity

Definition

Rate at which specification-origin errors propagate through every downstream gate into production, measuring whether the pipeline can catch errors that were wrong on paper before any code existed.

The Measurement Gap

Measurement Gap and Decision

The Gap: Traditional defect classification asks "what broke?" SEPR asks the harder question: "Did the spec itself contain the error, and did every downstream gate fail to catch it because they were all validating against the same flawed source?" Most organizations classify defects by the code that failed, not the specification that authorized the wrong behavior. Post-mortems typically stop at "the code did X when it should have done Y" without asking "who told the code to do X, and was that instruction correct?"

The Decision: Is the SDLC vulnerable to specification-layer single points of failure where one wrong AI-generated document bypasses every quality gate? SEPR tells you whether your development process has independent validation at any point or whether all gates reference the same potentially-flawed source.

What Changes When You Have This Number:

Below 0.05, spec propagation is rare. Either AI specs are high quality or downstream gates catch errors independently. Between 0.05–0.15, some propagation is occurring and review processes need independent requirement validation. Above 0.15, AI specs are a material production defect source, a single point of failure in the SDLC. The diagnostic triad of SCG + TSAS + SEPR reveals whether the problem is too many spec errors being generated (high SCG) or gates that can't catch the errors (high TSAS).

Formula

Formula:

SEPR = Defects_spec_propagated / Defects_total

Where Defects_spec_propagated is production defects where root cause analysis identifies an AI-generated specification error not caught by any downstream quality gate (code review, testing, QA, staging), and Defects_total is total production defects in the period.

Sign Convention:

Lower is better. 0.0 = no spec propagation. Higher values indicate specification-layer vulnerability.

Worked Example:

Over Q3, 48 production defects underwent root-cause analysis:

  • 31 were implementation errors (developer misunderstood correct spec, introduced bug)
  • 9 were environment/configuration issues
  • 4 were correct implementations of correct specs that didn't handle unanticipated load patterns
  • 4 traced back to AI-generated specification errors that propagated:
  • Refund calculation using calendar days instead of enrollment days for partial-period accounts
  • Missing timezone handling in settlement cutoff: spec assumed UTC, production runs across timezones
  • API rate limit not specified: implementation had no throttling, partner system rejected excess calls
  • Audit trail format incompatible with compliance system: spec didn't reference compliance format requirements

For each of these 4: the code implemented the spec correctly (code review passed), tests validated against the spec (tests passed), QA verified acceptance criteria from the spec (QA passed). No gate had independent domain knowledge to question the spec itself.

SEPR = 4 / 48 = 0.083

This falls in the "some propagation" range. All four gaps are domain-specific requirements the AI couldn't derive from patterns: the same categories SCG predicts will be missing.

Minimum Volume:

20 defects with completed RCA per measurement period for statistical meaning.

Proposed formula, pending validation pilots.

At a Glance

Unit
Percentage
Trend Direction
Lower
Reporting Frequency
Quarterly
Category
Specification Quality
← Specification Completeness Gap (SCG)

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment