Judgment when code is free

Across 13 talks and 28 claims, the closing message of AI Engineer Code Summit inverted the old software bottleneck: typing and wiring syntax are effectively free, while human review and architectural understanding remain stubbornly constant. Unfiltered generation does not produce leverage—it produces an overwhelm of plausible diffs.

“AI cannot replace human judgment. It can only amplify our judgment, or maybe perhaps our lack of judgment in certain cases. And the slop is asymmetric. The cost of producing a thousand line PR or a ten page doc is approaching zero. But the cost of reviewing it is still barely constant.”Dexter Horthy · 12-Factor-Factories: Design Patterns for Agentic Coding · 00:14:43

Because human attention cannot scale alongside token throughput, teams have to triage where judgment goes. Dexter Horthy multiplies the likelihood of future change by the difficulty of changing it later—ignoring button colors while guarding database schemas. Ankit Jain found that human review comments naturally pivot from syntax to intent:

“Most of the comments that humans are providing are actually not about bugs or correctness. It's about decisions.”Ankit Jain · One team's path to trusting AI generated code · 00:00:29

Even when an agent runs deterministic tests cleanly, humans still read outputs “for us to build mental models” (Ankit Jain, 00:02:56). Without that shared mental model, as Horthy warned, skipping review altogether guarantees the exact same generic output every competitor generates.

If implementation drops toward zero cost, bad product decisions scale just as cleanly as good ones. Lauren Tan put it bluntly: if your loop lacks discipline, automating it just manufactures more waste faster.

“Then automating that loop just produces more slop.”Lauren Tan · All Code Is Slop (Except Mine) · 00:16:10

Justin McCarthy drew the same boundary for factory-style engineering: factories work on clear customer promises, never on vague greenfield wishlists.

“And if we're trying to industrialize something where we don't know what we want. We're certainly we're not going to get the right thing, or we're going to take too long, or we're going to spend too much.”Justin McCarthy · Brownfield Factories · 00:02:12

Ishaan Dey distilled the entire afternoon into a single design imperative: when production is ubiquitous, taste and direction become the scarce edge.

“So, anything can be made by anyone. Then it's more important than ever to know what the right thing to even make is.”Ishaan Dey · Judgement is the Last Job · 00:00:39

Ishaan Dey added that finishing a first working prototype in seconds creates a dangerous illusion of completion: teams must deliberately pause, explore multiple directions, and apply human judgment before locking a direction in.

Moving from writing code to supervising five concurrent agents introduces a real cognitive tax. Scott Tolinski surveyed working developers on why high-speed AI coding leaves them drained despite higher raw output.

“Obviously, it's decision fatigue.”Scott Tolinski · The True Cost of AI Coding · 00:09:42

Context-switching across asynchronous agents erodes flow state just as easily as endless review queues. Thariq Shihipar described the trap of fanning out work only to come back alienated from the result:

“And I find that like the ball just makes a bunch of choices for me, and then I try it out. I'm like, wow, I have no idea what that actually does. I get distracted pretty easily. I don't feel in the zone very much. You know, I sort of send off a prompt and they start doing something else and I come back. And I honestly make less progress because of these things, right?”Thariq Shihipar · Thariq Shihipar, Independent · 00:11:06

Both toolmakers and practitioners converged on protecting the human attention budget. Tolinski advised capping active threads—“Less paralization makes for a calmer mind.” (00:19:50)—while Ishaan Dey proposed hiding noisy raw tool streams behind semantic summaries so humans only spend cognitive load on meaningful forks.

Several speakers noted that once routine CRUD, boilerplate, and easy tickets evaporate, daily work does not feel lighter—it simply compresses toward the hardest unresolved problems. Simon Willison stated the conference thesis directly during his keynote:

“and I think it's the theme of this conference: is to be more ambitious now that we have all of these new tools.”Simon Willison · 2026 in LLMs · 00:01:58

Later in the same talk, Willison captured why senior engineers still feel stretched even as models get stronger: “I think the truth is that the easy stuff is all handled now.” (00:17:44). Walden Yan backed that ambition thesis with hard numbers from Cognition—merged PRs up 40.4x alongside 2.7x headcount:

“So I think really most teams are limited by the amount of ambition they have.”Walden Yan · A New Age of Software Engineering · 00:13:07

Because models default to safe, incremental compromises, human taste shows up in how builders steer multi-hour runs. Recounting five rewrites of an ahead-of-time JavaScript compiler in 46 days, Jarred Sumner explained that the human role is relentless questioning—forcing the agent to defend slow paths or timid shortcuts:

“what you actually do is like. A bunch of leading questions, like why are we doing this? Can we do this? Also, I end up saying like, can you do this faster? Or like, what's taking so long?”Jarred Sumner · Jarred Sumner, Founder of Bun, Anthropic / Bun · 00:18:48

To keep that steering grounded in real understanding rather than guessing from diffs, Thariq Shihipar recommended maintaining an English-density spec alongside generated code so the builder always holds an editable mental anchor:

“It's supposed to feel like English. It's kind of like a spec, and I'll show you what it looks like. But it's at the density where, if I just read this code, I know what's happening, and I can edit it. I can tweak it. But more importantly, I just like you know can have a bit of a ground truth of like what the what my game looks like.”Thariq Shihipar · Thariq Shihipar, Independent · 00:13:27

Diogo Almeida reminded the room that generic output is often just reliability failure in disguise (“The root of slop is that AI is unreliable.”, 00:19:45). When reliability is solved, Thariq Shihipar argued software enters a Renaissance rather than a factory line: every extra ounce of craft becomes economically valuable.

“Something that is happening with software, but at the same time, there's also kind of a Renaissance happening, right? Where people want and higher and higher quality software, and every marginal. Bit of like creativity, utility, like spirit. Like soul that you put into it is like is useful and like economically valuable, right?”Thariq Shihipar · Thariq Shihipar, Independent · 00:05:09