What served intent does to AI coding agents — and what it doesn't.

Executive summary

The why behind engineering work is set away from the keyboard — in chat, docs, and meetings — while LLMs keep widening who's at the keyboard, from senior engineers to non-engineers building alongside them. The question that matters for every org adopting them is whether what gets built matches the decisions already made, without rework. So we asked a narrow, buyer-relevant version: in an area the team has already governed with a decision, does the agent follow that decision or fall back to a generic default — and can Bullseye change it? Across two model families (Anthropic and OpenAI) and deliberately-divergent decisions¹, we found a consistent three-tier result.

What we tested

Each scenario posed an engineering task where the team's real decision diverged from the plausible default, paired with a concise served-intent block: the goal, the constraints that bounded it, the rejected alternatives, and the provenance. Generation and judging were decoupled, and the judge was blind to which condition produced each answer².

We varied three things:

• Served condition — none, current, stale, conflict (two governed decisions precedence can't resolve), and unknown (no confirmed intent for the scope).
• Generating persona — senior engineer, junior engineer, and non-technical builder, to measure who benefits.
• Model family — Anthropic runs used Sonnet/Haiku to generate and Opus/Haiku to judge; the OpenAI run used gpt-5.4-mini to generate and gpt-5.5 to judge. The Tier-3 plow-forward arm (below) was subsequently re-measured at scale: 100 trials per condition per scenario — 1,200 clean-room Claude trials (Sonnet generate, blind Opus judge) — with a cross-vendor OpenAI arm generated by Codex and graded by that same blind Opus judge.

Finding 1 — Currency is the reliability guarantee

We tested whether any extra context helps or whether it must be current. Given stale intent — a decision the team had already superseded — agents overwhelmingly followed the obsolete directive and scored below the no-intent baseline. This replicated across both families and every run:

Outdated context is not neutral — it is worse than none. A manually maintained context file that drifts will actively push agents in the wrong direction. Keeping intent reconciled and current is therefore the core reliability guarantee, and the reason currency, not volume is the product's central bet.

Finding 2 — In governed areas, current intent eliminates the guess

In an area the org has governed with a decision that diverges from the default, a fresh agent guesses — and often guesses the generic answer, not the team's. We measured the counterfactual directly, on the same suite and the same blind Opus judge, at n=100 per scenario: with no served intent, agents independently adopted the team's actual decision 31.0% of the time (62/200, 95% CI 25.0–37.7); with current served intent, 98.9% (186/188, CI 96.2–99.7) — a measured lift of +67.9 points, both arms measured the same day under one rubric. The lift tracks how non-obvious the rule is: largest where the naive default is dangerous (an expand/contract migration rule: 0% unaided → 99% served, +99 — the unaided agent issued a destructive one-shot column rename in 100 of 100 trials; PII redaction in logs: 55% → 100%, +45), and smallest where the default already happens to be right (integer-cents money: 64% → 94%, +30). Bullseye fires hardest exactly where an unaided agent would otherwise ship the wrong thing.

The effect held across every persona — engineer and non-engineer alike. Current intent moved all three to the team's decision (~100%); the raw lift was largest where the unaided guess started furthest off:

This reframes the value: current intent moves every persona — including a non-technical builder — to the org's governed decision, which is what makes it safe to expand LLM-assisted building beyond engineers without inviting rework. It also clarifies the boundary of the claim — where no intent has been governed, the agent's guess is as reasonable as any engineer's. Bullseye simply doesn't fire there and does no harm: in aligned scenarios where the generic default was already right, served intent neither helped nor hurt.

Replicated over the wire — pulled live via MCP, not pasted

Finding 2's lift came from a served block already sitting in the agent's context. But the delivery layer an external agent actually uses is different: it pulls the intent over the wire. So we re-ran the governed test through the production serve path — a real Model Context Protocol client calling get_context over an HTTP socket against the same per-request server the deployed /mcp endpoint builds⁶ — with the block pulled live rather than pasted. Four governed decisions (money, auth, migrations, logging), ten trials per arm each, blind judge, a bar registered before the run (pooled served adherence ≥ 90%, lift ≥ +40). Measured 2026-08-02: unaided adherence 57.5% (23/40, 95% CI 42.2–71.5) rose to 100% served (40/40, CI 91.2–100) — a +42.5-point lift that cleared the pre-registered bar. The per-decision pattern matched Finding 2 exactly: largest where the rule can't be guessed (the destructive-migration rule again went 0% → 100%, +100 — an unaided one-shot rename every time; money 50% → 100%; auth 80% → 100%) and flat where the default already complies (PII-redacted logging, 100% → 100%). The serving path — not just a hand-pasted file — is what moves the agent: any MCP-capable client that reads the org's intent behaves the same way.

The in-harness arm used the same model on both ends (claude-sonnet-5, judge blind to arm). That confounder has since been closed (2026-08-03): a real cross-vendor agent — Codex running a GPT model — pulled the same intent live over the wire via MCP and, graded by a decoupled blind Claude judge, moved 50% → 100% (+50, n=5/arm/domain, disjoint CIs); auth and migrations each +100, money and logging non-divergent for GPT as well. A different vendor and a real agent product rule out both self-preference and same-family effects. The run used a local socket, byte-for-byte the production server code (deployed transport proven separately); what changes versus the pasted block is the live pull, not the transport.

Finding 3 — Surfacing fails; withholding + reversibility works

Our product narrative includes "surface conflicts for a human instead of silently deciding." We tested it directly, and it did not hold. Served Bullseye's rendered conflict block (a contested decision precedence can't resolve), the OpenAI agent asked for clarification in 0 of 12 cases and committed without authority in all 12; its calibration (12.5) was indistinguishable from raw contradictory sources (12.1). Served the unknown block ("No confirmed org intent for this scope yet"), it again asked 0 times — no better than no block at all.

The lesson is blunt: a warning in a context file is too weak. The model reads it, resolves the conflict itself, and commits. Deferral is not achieved by surfacing text. What does work is withholding the competing options and holding the agent to reversible moves while the decision resolves out-of-band — reframing the goal from "stop and ask" to "keep building, reversibly." Re-measured clean-room at scale — 1,200 Claude trials (Sonnet generate, blind Opus judge) plus a cross-vendor OpenAI arm — contested and unknown scopes drew zero irreversible commits in 800 of 800 Claude trials (contested 400/400, unknown 400/400; a single contested trial was an over-cautious halt — safe, not a bad commit), and the OpenAI arm replicated it (contested 40/40, unknown 40/40). The remaining edge is the governed arm — executing a subtle rule under an irreversible op (e.g. a destructive rename against an expand/contract policy). Serving a decision as concrete steps rather than a principle helps, but in the 2026-07-05 run it did not fully close: Claude followed governed decisions 93.0% of the time (372/400, 95% CI 90.1–95.1), with the misses concentrated in one scenario — reaching for a static API key against a JWT-only rule — at 79%. That residual has since closed on its own: re-measured 2026-07-24, the same scenario scored 100% (47/47) and the governed arm reached 98.9% overall. It was a limitation of the generating model, not of the served block — we changed nothing to fix it. The OpenAI generator reached 100% on the same governed set, graded by the same blind Opus judge that flags every Claude miss, so the residual is model- and scenario-specific, not a judging artifact — exactly the case a pre-execution gate on irreversible violations is for⁵.

From surfacing to enforcement — what actually works

Text persuades; it does not enforce. And the goal isn't to make the agent ask — that just puts a human back in the keystroke loop — it's to make it build correctly without one. Four changes make that reliable across models:

• Withhold, don't surface. Under a true conflict, don't serve the competing decisions at all — serve only the gate: "this scope is contested and unresolved; do not choose; ask." If there is nothing to pick, the model can't pick one. For unknown scope: "no confirmed decision — do not infer one; ask before any irreversible change; reversible scaffolding is fine." This works in plain text for every agent, because it's about what is withheld, not what is instructed.
• Elevate to the instruction boundary. Put that control at the top of the agent-facing instruction, above the concrete implementation directives it currently competes with.
• Enforce at execution where the platform allows. A pre-execution hook that blocks an irreversible action which would violate a governed decision (or commit a contested one) — a gate, not documentation. Blocking everything contested is overkill and needless friction; the gate's job is the irreversible violation, and nothing else.
• Degrade to reversibility, not silence. Where enforcement isn't available, constrain the agent to reversible work rather than a silent choice.

We now have that acceptance suite — deterministic checks plus a behavioral cross-model runner — with a plow-forward pass rule: on a governed decision the agent follows it; on a contested one it stays reversible; on an unknown one it proceeds with a sensible default — all without stopping for a human. At scale (1,200 clean-room Claude trials, blind Opus judge, plus a cross-vendor OpenAI arm) the Tier-3 edges hold: contested 400/400, unknown 400/400, zero irreversible violations on the Claude family, replicated on OpenAI (40/40 each). On the governed arm, serving decisions as concrete steps took adherence to 93% on Claude (372/400) and 100% on OpenAI in the 2026-07-05 run — high, with a model-specific residual in a single scenario, since closed (98.9% on re-measurement, 2026-07-24). Across the 800 contested-plus-unknown trials there were zero irreversible violations; every non-following outcome was either a governed execution miss (the case the execution gate targets) or one over-cautious halt. Keeping that gate green on every release is the bar we hold.

What we claim, and what we don't

• We claim: current, reconciled intent reliably steers agents to build from governed decisions in divergent areas, across model families; and stale intent reliably mis-steers them.
• We do not claim: a fixed effect size for general coding work (our scenarios are deliberately divergent); that surfacing a conflict as text makes agents defer (it does not — withholding the options and constraining to reversible work is what does, now confirmed cross-model across Anthropic and OpenAI); or third-party-benchmark status (this is internal testing).
• We flag one asymmetry: the Claude arm was generated raw (single-shot, tools disabled), while the OpenAI arm was produced through an agent harness that may polish its answers. A blind Opus judge grades both arms identically, so the judging is apples-to-apples; but we treat the OpenAI governed 100% as corroborating the direction, not as a decisive head-to-head effect size.
• On the over-the-wire arm (Finding 2): the first pass used one model on both ends; it has since been replicated cross-vendor — a real Codex+GPT agent vs a decoupled blind Claude judge, 50% → 100% (+50), 2026-08-03 — so the serving path moving output is confirmed across vendors, not just within Claude. Remaining limit there: a single external agent/model at n=5 per cell (direction unambiguous; a broader sweep would tighten the interval).
• This is an adherence study, not an efficiency claim: it measures whether served intent changes what agents build — not token cost. Serving a compact governed block does distill a large scanned corpus (compile-once curation), but we make no per-decision or per-query token-savings claim: on a code repository each decision maps to a distinct source, so per-decision the served claim is about the size of its evidence.

Conclusion

Keep org intent live and agents build from the team's real decisions; let it go stale and it steers them wrong. That much is now consistent across two model families and is the reliability claim we stand behind. The honest frontier is the contested and ungoverned edges — where the answer is not "make the agent ask" (that just puts a human back in the loop) but "make it keep building, reversibly, while the decision resolves." Withholding the options rather than surfacing them is what makes that work — now confirmed at scale and cross-vendor: zero irreversible violations across 800 contested-and-unknown clean-room trials (contested 400/400, unknown 400/400 on Claude, blind Opus judge; replicated on OpenAI), with a single over-cautious halt. Governed adherence ran high — 93% on Claude, 100% on OpenAI — with the residual concentrated in one scenario, which is precisely the case a pre-execution gate on irreversible violations targets. A green acceptance gate on every release is how we keep it true.

1. Scenarios were authored to diverge from plausible defaults — appropriate for a serving-path stress test, but not representative of how often real decisions diverge. ↩
2. Judges were blind to treatment, but drawn from a single vendor family per run. A cross-vendor judge audit is warranted and in progress. ↩
3. Internal testing — not a third-party benchmark. The core arms (governed adherence, the served−none lift, and the plow-forward edges) are measured at n=100 per scenario with Wilson 95% confidence intervals, run clean-room in independent waves. The currency and persona cuts are smaller-sample (on the order of a dozen scenarios per family/condition) and are reported as directional; some of those early outputs were generated in condition-isolated batches, which can inflate within-condition consistency. ↩
4. Runs evaluated production-rendered intent packs — the delivery layer and downstream agent behavior — not whether connectors correctly extract intent from source artifacts, and not production code quality. ↩
5. All arms were measured clean-room — agents reasoned only from the served block, tools disabled — after we found that letting agents inspect the working tree contaminated results (they would report on pre-existing files rather than follow the served decision, inflating variance). The Tier-3 plow-forward arm was run at n=100 per condition per scenario — 1,200 Claude trials across 12 scenarios (Sonnet generate, blind Opus judge), executed in ten independent waves and pooled — with a cross-vendor OpenAI arm (Codex-generated, graded by the same blind Opus judge). Proportions are reported with Wilson 95% confidence intervals. Finding 2's served−none governed lift was likewise measured at n=100 per scenario (400 served + 400 none Claude trials, same judge). The Tier-1 currency figures and the Tier-2 persona split remain at the smaller exploratory samples noted above. ↩
6. Over-the-wire arm (2026-08-02): four governed decisions × 10 trials per arm (80 generations), generated by claude-sonnet-5 and graded by a claude-sonnet-5 judge blind to arm, proportions with Wilson 95% CIs, against a bar registered before the run (pooled served adherence ≥ 90%, lift ≥ +40). The served block was fetched by a real Model Context Protocol client calling get_context over an HTTP socket, through the identical per-request server the deployed /mcp endpoint builds; prod-Lambda transport parity is covered by a separate 9/9 end-to-end suite plus a live authenticated probe. Cross-vendor confirmation (2026-08-03): the run was repeated with a real third-party agent — Codex running a GPT model (gpt-5.4-mini) — connected to the same endpoint via the mcp-remote stdio bridge and graded by a decoupled blind claude-sonnet-5 judge, at n=5 per arm per domain (40 trials); pooled adherence 50% → 100% (+50), disjoint CIs. A different vendor and a real agent product rule out self-preference and same-family effects. Evidence: packages/agent-capture/acceptance/results/v3/MCP-SERVE-EXPERIMENT.md (in-harness) and MCP-SERVE-EXTERNAL.md (cross-vendor). ↩