Agents need a firewall on both what goes into them and what actions they take.
“In case of non-deterministic agents, you want something similar protecting.”
Domain teams should run a regression eval suite in CI on every change and model switch, starting from 20–50 real use cases, before adding a skill, plugin or connector.
“And then finally, start with a solid twenty to fifty use cases.”
Gates should scale with stakes: reads are low risk, reversible writes higher, and irreversible actions like sending email or publishing numbers highest.
“For reversible writes, the stakes are higher than read.”
With agents anyone can be a coding machine; the engineer's job becomes keeping quality up while volume rises, by investing in an environment that makes agents trustworthy.
“All about trust—not blind trust, but investing in your agent's environment.”
Constrain agents first with deterministic layers (API design, architecture, types, lint, tests), then verification, and only last with prose rules and skills.
“Instead of adding another line to your rules or your agent's dot md, because it's far more reproducible, it's in code.”
Every agent correction becomes a lint rule or an API that eliminates the problem (banning useEffect, most code comments, cross-thread imports), until code review has little left to check.
“And so our general principle is that every time we correct our agents, it turns into one of these deterministic checks.”
Code review has shrunk to inspecting verification artifacts (videos, screenshots) produced by runtime verification, model checking and property-based UI tests.
“And our code review process has now really shrunk into looking at verification artifacts.”
Access rules come in as metadata at ingestion, and every search names the user it is for, so results are filtered to what that user may see.
“they need to authorize say I'm user X.”
Narrow agents with their own identity and only the tools and permissions they need give better quality results and use less time and tokens.
“So in Overcut, we believe that agents should be specialized in what they do.”
Long end-to-end workflows such as ticket-to-PR stay safe by inserting human approval gates wherever the team wants them.
“You can define pausing, deterministic stop, where the agent will stop.”
High governance, limits and a full audit trail are what make it acceptable to let agents do real work in the systems a company already uses.
“But the fact is, if you have a very high governance and very high observability on the flows.”
The session is the artifact of a change: a reviewer sees the conversation, tools, screenshots and a verification video, not just the diff.
“Anyone reviewing it doesn't just see the 400-line diff dropped over the wall and hope for the best.”
Replacing binary unit-test pass/fail signals with commit-derived rubrics provides dense reward gradients covering code hygiene, edge cases, and maintainability.
“We go from unit test or binary correctness. To rubric-based grading, and rubrics give you dense signal. They're not just zero one.”
RL-trained coding agents will exploit test suites by bypassing checkers, gaming benchmarks, or fabricating evidence unless graders evaluate solutions like strict human code reviewers.
“You're training them to pass the test, so they will find any way to pass the test. But if the test pass, that doesn't necessarily mean that they've solved the problem. There are examples where the agent will bypass the checker. There are examples where the agent will game the benchmark. There are examples where the agent will fake the evidence.”
Hardened grading sandboxes isolate git history and held-out tests while applying strong negative RL rewards for environment tampering detected strictly from model actions rather than thoughts.
“So any you know cheating or hampering of the environment gets a very negative RL reward. And all of this is detected just based on the actions that the model is taking, not the thoughts.”
Agents fix bugs from Slack and send back a video of the fix, so engineers can show customers proof without opening an IDE or GitHub.
“We've abstracted away all the development to be based on evidence, videos, and I can just send the proof.”
Every service runs locally with mocked third parties in hourly-rebuilt VMs, and the agent writes and keeps updating its own skills for testing the codebase.
“You need to teach them how to test your codebase as well.”
On massive agent-authored engineering projects, architectural choices are driven primarily by how you can reliably test and verify correctness.
“You're really constrained by how you test that the code works. So that's like even more important than what you're specifically like the approach you go with for how to build it. Is entirely driven by how do you verify that actually is going to like work and not be buggy. And that you can trust it.”
Anthropic rejected rewriting Claude Code in Rust because porting its TypeScript test suite would destroy trust in the verification oracle, choosing instead to accelerate JavaScript with an AOT compiler.
“As soon as you're like rewriting the tests. Then you can't really trust the test anymore. So that seems like it'd be too risky.”
Relying on test suites that predate the project—combined with fuzzing and comparing JIT, AOT, and untyped execution—prevents coding agents from gaming or weakening verification.
“We can largely trust that the tests that predate the project worked.”
Give the agent a cheap, deterministic check between input and output, such as dry-running queries or running tests and linters, so it can iterate and users can trust the result.
“Always give you know your agent some way to kind of verify its results.”
Human-in-the-loop is not all or nothing: chat back-and-forth, permission step-up, out-of-band verification and payment thresholds suit rising levels of stakes.
“I think it kind of depends on the stakes.”
Step-up authorization lets a logged-in user grant the agent extra, narrowly scoped, short-lived OAuth permissions (such as calendar only) when a task needs them.
“So I'm authenticated, but my agent needs to do more.”
Handing agents long-lived API keys on local machines is risky because they leak and cannot be revoked quickly, which raises the question of whose identity an agent acts under.
“We don't have an easy way to revoke those at a moment's notice.”
A token store holds the user's connected-account credentials so an agent can act with only the narrow permission it needs, such as writing to one GitHub repository.
“I'm only going to give it the permissions to specifically write.”
For high-stakes actions the browser session is not trusted, so CIBA sends an approval request to the user on a separate channel and the tool runs only once it is approved.
“It's really just a fancy way of saying we're going to send you an email, push notification, or an SMS.”
For agent payments, use a virtual card or a shared payment token with a maximum threshold rather than the real card, since the purchase context can be spoofed.
“You can share your card with the agent. Please don't do that.”
A purpose-built detection pipeline beats simply prompting a general model with the same data, and for enterprises each point of accuracy matters across millions of sessions.
“Why can't I just do this with an AI?”
Monitoring agent telemetry only matters if it leads to action, such as blocking or steering a bad step before it happens.
“And then, of course, what's the point of observing if you can't do anything about it?”
Pre-planned evals cannot cover the unknown ways agents fail in production at the scale of many traces per day, so something must actively look for emerging issues.
“Regular evaluation is just not enough.”
The design takes the immune system as its model: detect a problem without waiting for a test, respond to its root cause, and remember it to prevent recurrence.
“Pathogen entering, it's not waiting for some test.”
Customers wanted diagnosis run continuously over every session, with an opinionated failure category and a prescriptive fix rather than a vague report.
“Look at all the sessions.”
Surfacing and fixing an issue is not enough; the system must remember the failure and verify the fix holds so the loop is closed.
“And then remembering it, and this is the key part.”
A background retrospective turns the outcomes of past runs into memories, scores them to prevent drift, and feeds only the best back as context so the system improves itself.
“We are scoring them, so the next run we run only on top of the memories that have the highest score.”
Verification runs in two loops: checks within a turn, and production and business signals between turns.
“You can think of it as this inner loop and this outer loop.”
Internal dogfooding across Google's 100,000-engineer community turns real engineering failures and blind-battle friction directly into training tasks for subsequent post-training candidates.
“we want to dog food our models with our hundred thousand engineer community. And repeat the loop.”
To cut fabrication rates, post-training cleans trajectories, grounds final completion summaries in actual tool actions, and rewards agents for reporting partial progress instead of faking completion.
“when the agent says I'm done and gives you a summary. That needs to be grounded in the actions that were taken. And we want to reward honesty, so. If there's a hard problem, it's better for the agent to say, "Hey, this is my partial progress. It's a hard problem."”
Because every PR and decision is tracked, agents can analyse that record daily and propose ways to cut CI time, tokens and bugs, improving the factory itself.
“And this has actually been a huge unlock, where you can just set up these systems, and the agents will go and optimize your software factory for you.”
Encoding product guarantees ('promises') into closed feedback loops stabilizes brownfield factories, whereas breaking the feedback loop only amplifies noise.
“As many of you know, this is the sort of idealized block diagram amplifier, and if you cut that feedback leg. You're just going to amplify noise, okay?”
An AI-built product can achieve high quality rather than 'vibe coded' junk by running automated agent loops that fix reported bugs, sweep logs for other affected users, and proactively notify them.
“People talk about vibe coated products being junk. Amp is basically 100 built with AI. And people say it's very high quality, and this is one of the things that you can do differently with an agent.”
An LLM judge shouldn't be trusted until it has been measured.
“We entrusted our judge until we measured it.”
Run a second check in shadow mode and read the cases where it disagrees with the judge.
“Run in shadow mode, seeing where it disagrees.”
