Specification Completeness Gap (SCG)

GMI-SWD-018Software DeliveryPilot-ReadyPattern O-23: Specification Fidelity

Definition

Share of domain-required requirements missing from AI-elaborated specifications, measuring systematic omission of the constraints the model's training data underrepresents.

The Measurement Gap

Measurement Gap and Decision

The Gap: No existing metric distinguishes between "specification exists" and "specification is complete for this domain." The missing requirements are systematically biased. AI misses domain-specific requirements, not random ones. The gap propagates perfectly through the SDLC: when the spec misses a requirement, the implementation won't handle it, tests won't verify it, QA won't check it. Nobody introduces the missing requirement at any gate because every gate trusts the specification as source of truth.

The Decision:

Can the team build against AI-generated specs, or do specs need domain expert review to catch requirements the AI missed because they're domain-specific, not pattern-common? This determines review process and resource allocation for specification augmentation.

What Changes When You Have This Number:

SCG below 0.15 means AI specs capture 85%+ of domain requirements. Adequate for most features with lightweight expert review. Between 0.15–0.35, AI specs are a good starting point but need systematic domain expert augmentation before implementation. Above 0.35, more than a third of requirements are missing and the spec is not safe to build against without thorough expert review. Gap categorization (regulatory, integration, historical, operational) reveals which domain knowledge types the AI consistently lacks.

Formula

Formula:

SCG = 1 − |Requirements_AI ∩ Requirements_expert| / |Requirements_expert|

Where Requirements_AI is the set of behavioral requirements in the AI-generated specification, Requirements_expert is the set identified by domain expert review (ground truth), and the intersection is semantically equivalent requirements present in both sets.

Sign Convention:

0.0 = no gap (AI captured everything the expert did). 1.0 = complete gap. Lower is better.

Worked Example:

AI generates spec for a refund processing workflow:

AI produces 23 requirements covering API endpoints, state machines, validation rules, error handling. Domain expert independently identifies 30 requirements: including promotional discount pro-rating, cross-border tax recalculation, partner system settlement timing, regulatory hold periods, and batch processing limits.

Matching: 20 of the expert's 30 requirements have semantic equivalents in the AI's 23. The AI also produced 3 requirements the expert didn't (valid pattern-based requirements the expert assumed as obvious).

Gap requirements (10): promotional discount handling, cross-border tax rules, partner settlement windows, regulatory holds, batch size limits, timezone handling for cutoff dates, partial-period proration, audit trail format requirements, rate limiting for third-party refund API, retry semantics for bank reversals.

SCG = 1 − 20/30 = 0.333

Semantic Matching:

Comparison requires semantic judgment, not string matching. Two differently-worded requirements describing the same behavior are a match.

Proposed formula, pending validation pilots.

At a Glance

Unit
Percentage
Trend Direction
Lower
Reporting Frequency
Quarterly
Category
Specification Quality
← Security Coverage Gap Index (SCGI)Specification Error Propagation Rate (SEPR) →

Assess Your Organization

The AI-GPS Assessment evaluates your AI governance maturity and recommends KPIs like this one for your implementation roadmap.

Take the Free AI-GPS Assessment