AI coding agent guardrails & proof loops

For years the boundary around source code was physical: it lived in your repo and left only when you pushed it somewhere you chose. Agentic tools quietly redrew that line and didn't tell anyone where the new one is. To answer a prompt, an agent gathers context — the open file, files it imports, files it greps, sometimes a broad sweep of the repo — and sends that context to a model endpoint. The scope of what gets sent is opaque, automatic, and usually far larger than the one function you were thinking about. The "I think the agent sent our core product to a cloud model" moment is really the moment someone realizes they never knew where the boundary was.

You don't need to ban AI tooling. You need to answer three concrete questions:

1. What context leaves the machine per request? Just the open file, or a wide retrieval sweep? The blast radius of a single prompt is the thing most people never check.
2. What's the retention and training policy on the receiving end? Is input stored, for how long, and is there a real, verifiable opt-out from training — not just a checkbox? Business and enterprise tiers usually have stronger guarantees, but only if you're actually on them.
3. Does this work even need a frontier model? A rename, a regex, a boilerplate test, a stack-trace explanation — a competent local model handles the routine 80%. Reach for the cloud API on the hard 20%.

The fix is a posture, not a product: tier your work by sensitivity, route the sensitive tier to a local or self-hosted model (where the retention and training questions become moot because nothing leaves), reserve cloud frontier models for hard problems on a tier whose terms you've read, and make the boundary visible instead of letting the default decide for you.

A confident answer is not evidence. A working screen is not evidence by itself. Evidence is the reviewable trail that connects the task to the patch. The canonical failure is the hallucinated endpoint: the agent says it added a route, the UI calls it, and the route doesn't exist or has the wrong contract — because no evidence gate ever forced the claim to meet the code. The related tax is agents that mark work done without proof, and the broken deploy or hallucinated completion that surfaces later.

Claims die at the end of a session. Evidence compounds — it gets reused in PR review, incident debugging, onboarding, and future agent runs. So the minimum evidence packet should be boring enough to generate every time:

• Scope — what was the agent asked to change, and what was explicitly out of scope?
• Diff — which files changed, and can each be accepted or rejected independently?
• Commands — what did it run: tests, builds, linters, migrations, scripts?
• Logs — what failed before it passed, and what output should the reviewer inspect?
• Handoff — what risks remain, what needs human judgment, what should the next session know?

The load-bearing property: this trail has to be visible while the run is still active, not reconstructed after the fact. If the workflow waits until review to expose a missing boundary, the user has already paid the context, time, and trust cost. Remote and background agents raise the bar further — a DoneCheck-style proof gate should require tests, screenshots, diffs, logs, and unresolved-risk notes before a ticket or release is called done. And it's not only a coding concern: disaster-recovery runbooks and bot-traffic investigations need the same replayable proof layer, because when the system is messy the team needs evidence it can replay, not a stale checklist.

Ask for a one-line swap of to and bcc in an email function and the agent comes back with 31 files: the fix in line 14, plus a renamed variable, a new extracted type, a logger refactor, two unrelated test files updated to use a helper it decided to write, and a README example. It compiles. Tests pass. Now you have to reason about thirty unsolicited changes to land the one you asked for. This is not a model failure — the agent read "fix this bug" as "improve this codebase," because nothing told it otherwise.

The more dangerous version is the invisible second edit: the agent fixes the requested bug and quietly changes a shared utility, a migration, or a config file on the way. A visible failure can be rejected; a silent out-of-scope edit ships because it looks like part of the solution. That's the real enterprise blocker — organizations need proof that generated code stayed inside the approved boundary.

The fix is upstream of every prompt trick: a four-line task boundary at the top of the prompt.

1. Scope — the exact files the agent may edit; everything else is read-only or off-limits. Be ruthless: writing src/ says nothing.
2. Prohibition — concrete actions it's prone to: "do not rename variables, extract types, write new helpers, or update unrelated tests." Name actions; avoid evaluative words like "unnecessarily," which is an opinion the agent will have.
3. Reporting protocol — "if a change outside the allowed file is required, stop and tell me; do not edit it." Phrased this way, stopping becomes the success state, so the agent blocks on you instead of routing around the boundary.
4. Test verification scope — which suites to run, and only those, so it doesn't "fix" a flaky unrelated test on the way.

Restate the boundary on every follow-up turn, not just the first. Tool grants (Read(src/**), Edit(src/billing/**)) are the OS-level floor that stops an inattentive agent; the boundary prompt is the ceiling that stops an attentive one from over-improving. You want both.

There's a real incident behind this: a Cursor agent ran a broken recursive delete that, because of how the path and the recursive flag interacted, wiped far more than the one folder it was meant to. There was no confirmation step between the model emitting the command and the filesystem executing it, and no snapshot to fall back to. Cost caps and task boundaries don't catch this — cost caps watch the meter, task boundaries watch the plan, but neither watches the command string as it leaves the agent and heads for bash -c. That gap between intent and execution is where a preflight policy lives.

A workable policy classifies on three signals the runtime already has: shape (pattern-match rm -rf outside the workspace, git reset --hard, git push --force to a protected branch, DROP/TRUNCATE, DELETE without a WHERE, curl ... | sh, chmod 777), target (rm -rf $TMPDIR/build is fine; rm -rf $HOME is not), and side-effect class (data loss, availability loss, security posture change, or external exposure — highest class wins). Then gate by class:

• Silent allow for anything that doesn't match a risk pattern (ls, cat, npm test). If a policy makes you approve ls, the policy stops being read.
• Single-line preview for state changes that stay inside the workspace (git commit, npm install) — one keystroke, one execution.
• Typed confirmation that names the target for destructive or production actions. delete branch feature/auth-rewrite beats a reflex [OK] you click by muscle memory on the third appearance.

Rollback is the other half. Git stops being an undo button during long agent sessions that never produce discrete commits — you end up with one enormous, unreconstructable change. Auto-committing at short intervals gives you a checkpoint from 11:38 when the agent goes off the rails at 11:40, instead of last night's main. Treat recoverability as something you build before the session, not a heroic reconstruction after.

Generation got fast; verification did not. An agent can scaffold a feature, wire it through three files, and hand back a clean diff in under a minute — but nothing in that minute started a server, clicked a real button, or confirmed the flow you shipped last week still works. "It builds," types check, and the summary reads like a passing test report, so a change that looks finished is really only half-finished: written, but never exercised. The regressions from that gap are the worst kind because they're invisible at merge — a form still submits but a side effect on an adjacent page is now broken, or the new route works but the post-login redirect silently changed — and they surface days later when context is cold.

There's a costlier version: the same bug three times, on the meter. You hand a simple site to Claude Code with explicit instructions to test before declaring done; it reports success, the bug remains, you point it out, and it confidently introduces a new bug while fixing the old one. Each round burns a slice of a monthly plan. The failure isn't raw model ability — it's the absence of a real verification step between "I changed something" and "I am done." When the only signal the agent gets back is its own narration, every iteration is a guess.

The fix is structural, not motivational — you don't solve it with a sterner system prompt. Browser-driving tools like Playwright and MCP-style test harnesses let an agent actually load the app, fill a field, click submit, and read back the result instead of imagining it. The harder part in practice is keeping a durable record of what was run, so "I tested it" becomes an artifact instead of a claim. Builders shipping real apps need the ordinary work too: device matrices, smoke tests, regression checks, review notes, and release gates — an app can be generated quickly and still fail the boring work of shipping.

One developer described roughly $4,000 of Cursor usage across two months — not toy problems, real day-to-day work. The instinct is to reach for cheaper models or Auto routing, and the instinct keeps failing on reliability: the cheaper tier introduces enough subtle errors that the time spent catching and re-prompting eats the savings, and sometimes more. So it becomes a tier question (is Cursor Ultra worth it?) made on shifting ground — quotas change and feature boundaries move, so you're asked to commit to a price while the thing you're pricing gets redefined underneath you.

Model switching doesn't fix bad execution. A cheaper model repeating the wrong tool calls is still expensive; a stronger model working from stale context still wastes time. Before you can decide whether the model, the prompt, or the workflow is the bottleneck, you need to see where tokens, time, and retries are actually going. That's why execution observability comes before cheaper models: without a clear record of which models and which tasks produced something worth keeping, the tier decision is a guess dressed up as a budget.

The practical move is to make budget part of the run contract. Before a task starts, the operator should know the model route, the context budget, whether subagents are allowed, and what requires approval. During the task, spend should be visible enough to stop a bad loop before it becomes the whole afternoon. Routing is a habit, not a one-time choice: send expensive reasoning to the task that deserves it, a smaller model to the routine work, fall back to Kimi or a local model when Claude time runs out, and stay quota-aware when a provider's usage credits are about to kick in. The one rule underneath all of it — evidence and context have to survive the switch, or you haven't switched tools, you've started over.

This is the most common misdiagnosis. After enough audit, refactor, and hardening loops, regressions creep back, old bugs you'd already fixed reappear, and the architecture drifts a little further from the plan each session. The agent is fast and capable on any single task, and somehow the project keeps getting harder to trust. The instinct — "I need a smarter agent" — is the wrong diagnosis. An agent is a per-task optimizer: it makes the change and moves on, with no native memory of the project's history. Three failure modes follow, all the same shape:

• Regressions reappear because the agent re-derives a "cleaner" solution that happens to reintroduce the original bug; nothing remembered that this was already decided and fixed.
• Architecture drifts because each task is locally reasonable but no force holds the whole back toward its intended shape.
• Your own judgment erodes — lean on the agent for everything and you end up approving changes you couldn't have written and couldn't defend in review.

None of these are reasoning failures. A more capable model in the same empty workflow makes the same mistakes faster. What's missing is a workflow layer: durable memory of prior fixes and decisions that survives the session, scoped tasks with a clear definition of done, and review checkpoints where a human sees the change before it's load-bearing. Static instruction files (CLAUDE.md, AGENTS.md) help until they silently rot — the danger isn't the model ignoring context, it's the model obeying context nobody validated recently.

It gets sharper with parallelism. Running four to six agents across worktrees, context fragmentation becomes real: one agent solves what another already handled, another edits a file it didn't know was claimed, a third follows the plan superficially and skips the tests that would prove it. That needs an operating board — shared memory, file ownership, explicit task boundaries, and a record of verification — showing what each agent owns, touched, is blocked on, and spent. Without it, parallel agent work is just terminal sprawl with better autocomplete. (This is also why a prospect who rejects a tool because it "looked AI-generated" is really flagging a delivery layer that never communicated care or evidence.)

AI-built apps often fail at the exact moment they look finished. The interface loads, the local demo works, the screenshot looks plausible — but the meaningful content is locked behind client-side state that a crawler, reviewer, or second agent can't read. If the route ships an empty shell plus a bundle, a browser user sees a polished page after hydration while a raw HTML fetch shows little more than a root div. That's not an SEO footnote; it's an engineering review problem. When the output isn't inspectable without running the same JavaScript path, every downstream check gets more expensive: search visibility, social previews, accessibility review, documentation, agent handoff, and human trust.

The old assumption — if Chrome renders it, the page exists — is now too narrow. AI-built work is inspected by preview panes, link unfurlers, search and LLM crawlers, accessibility tooling, code-review bots, and agents asked to summarize or critique it. Those readers don't all execute the same scripts, wait for the same requests, or hold the same auth state. The result is a strange invisibility: the app works for the builder and disappears for the systems that decide whether it can be trusted. Hidden state also raises the cost of the next agent, which burns context asking basic questions — what does this page say, did the pricing card render or is the browser masking a data failure — that each turn into another screenshot or prompt.

The remedy is a boring pre-handoff test. Fetch the page without relying on the browser UI. Confirm the title, primary heading, core copy, canonical URL, and critical links are present in the document. Open the screenshot and compare it against the raw output. Save the command, result, and any caveats next to the task. Then ask the review question: could someone understand this page's purpose without your dev server, your memory of the prompt, or your screenshot narration? If not, it may be ready to demo, but it's not ready to be trusted.

If your code touches HIPAA-protected health data, PCI cardholder data, GDPR personal data, or SOC 2-scoped systems, AI assistants become a compliance question, not just a productivity one. You can't tell an auditor "we use AI to write code, we hope it's fine." In a real audit the questions cluster into five areas:

1. Data flow — does any regulated data leave the company perimeter when developers use AI? (Most teams fail here, not on policy.)
2. Code review — how do you ensure AI-generated code meets the same security bar as human-written code?
3. Audit trail — can you reconstruct what tool was used, by whom, on what code, when?
4. Training data — is the vendor training on your inputs, and can you prove the opt-out?
5. Incident handling — if an AI-generated change causes an incident, can you isolate the cause?

The practical, defensible answers are concrete. Production data is never shared with AI tools during normal development — synthetic, anonymized, or local-only fixtures only. The approved-tools list is pinned in a CLAUDE.md at the repo root, and every approved tool has a signed BAA (HIPAA), DPA (GDPR), or equivalent living in your compliance vault. Outbound AI calls route through a proxy that logs requests and rejects regulated-data egress patterns. Every AI-generated commit is human-reviewed before merge to a regulated branch, and SAST runs on every PR regardless of authorship. Usage is recorded per-developer per-day, and conventional commit prefixes (feat(ai-assisted): ...) let you answer "which PRs in the last 90 days were AI-assisted?" from one query. Free-tier or personal-account use of AI on company code is forbidden by policy.

The honest bar isn't "no AI involvement" — it's known and bounded: you know which tools are used, where, by whom, with what data and review process, and you can produce the evidence. Meet those five questions and AI tooling is just another category of supplier risk you've already managed before.