Definition
The percentage of multi-agent orchestrated workflows where all agent outputs (code, tests, documentation, pipeline configuration) are internally consistent with each other, measured by automated cross-artifact validation and post-deployment inconsistency detection.
The Measurement Gap
There is no metric that tells you whether outputs from multiple coordinated agents are internally consistent, which means code, tests, documentation, and pipeline configurations can contradict each other while each passes its individual quality check.
Formula
Plain language: For each orchestrated workflow that involved 2+ agents, check whether the outputs are consistent. Does the test agent's test suite actually test the code agent's implementation? Does the documentation agent's API doc match the code agent's interface? Does the pipeline agent's configuration deploy the right artifacts? Count workflows where all cross-artifact checks pass versus total multi-agent workflows.
Notation:
CACS = (W_coherent / W_multi_agent) × 100
Variables:
- W_coherent = multi-agent workflows where all cross-artifact consistency checks pass (count, DERIVED)
- W_multi_agent = total workflows involving 2+ agent types (count, NATIVE from orchestration log)
Cross-artifact consistency checks (each is pass/fail):
- Code-Test consistency: test assertions reference functions/interfaces that exist in the code output (automated, DERIVED via static analysis)
- Code-Doc consistency: API documentation describes the actual function signatures, parameters, and return types in the code output (automated, DERIVED via doc-code comparison tool)
- Code-Pipeline consistency: CI/CD configuration references the correct build targets, test suites, and deployment artifacts (automated, DERIVED via pipeline validation)
- Test-Doc consistency: documented behavior matches test expectations (automated where possible, MANUAL spot-check otherwise)
Proposed formula, pending validation pilots.
At a Glance
Decision Supported
When multiple agents work on the same change simultaneously, are their outputs aligned, or are we shipping internally inconsistent artifacts that pass individual quality checks but fail as an ensemble?