40× the work · Walden Yan

The result

40× the PRs on 2.7× the people

“This 40x. Is a 40x in the number of merged PRs we are producing on our engineering team.”00:00:00

“that two point seven x. That is our engineering headcount growth in the same period.”00:00:00

So each engineer is more than ten times as productive, and he claims the work hasn't got worse:

“And I would dare to say that these PRs are just as high quality, if not higher quality, than they were a year ago.”00:00:00

The machinery is a software factory: a pull request goes from bug report to merge, monitoring and maintenance without a person, escalating when human judgment is needed.

“By default, everything should be self-driving.”00:01:18

Evidence: transcript 00:00:00, 00:01:18

The org

Organized by vertical, not role

In March they told the team that handing tickets to an agent wasn't the job any more:

“You're not going to really be set up for success in the age of AI because anything that is described and can be implemented will be able to send to Devin.”00:01:48

Large teams became small pods, each with swarms of agents:

“So instead, you might have only two or three engineers on a team at a time.”00:01:48

“But they're responsible for everything beyond engineering as well.”00:01:48

Customer bug reports, QA testing, product management: each engineer runs their own board and sends the work to agents.

From the notes: “they've restructured the team, organized by vertical, no longer by role”. The reading: if implementation is cheap, what's scarce is knowing what to build and whether it works, so the engineer becomes the owner of a product area. It's the same move as Justin McCarthy's “we have a new job” and his demand factory in “Brownfield Factories” this afternoon.

Evidence: transcript 00:01:48

The workflow

Proof, not review

“Another thing I'd like to highlight is we work from Slack everywhere.”00:03:09

A customer sends a video of a bug; it's forwarded to an internal channel with Devin tagged; Devin opens a pull request and sends back a video of the fix, which he can send straight to the customer.

“We've abstracted away all the development to be based on evidence, videos, and I can just send the proof.”00:03:09

“A point where you can do your entire development lifecycle without opening up an IDE or without opening up a single coding engine.”00:03:09

From the notes: “they are sharing the videos etc to show that things work”. The reading: the same idea as Ramp's “the session is the artifact” and Lauren Tan's review by verification artifacts: evidence travels with the change, so people check proof instead of reading code.

Evidence: transcript 00:03:09

The infrastructure

Teach the agents to test

“It's not just you purchase a coding agent.”00:04:06

“You got to go change the way you build.”00:04:06

They used to test against staging with people clicking through flows. Now every service runs locally with Docker Compose, every third party has a mocked version, and it all runs in one virtual machine each agent gets, rebuilt from the latest code every hour; when a rebuild breaks, Devin fixes it first.

“And every code agent in our company now does not need to touch any live services at all to test their changes.”00:04:06

“Devon will write its own skills for how to test our codebase.”00:04:06

As the codebase changes, Devin updates those skills. Engineers no longer need their laptops to test changes; several have moved to iPads, which is also more secure, with no secrets on developer machines. Agents even test multi-user Zoom calls inside one VM.

From the notes: “teach the agents how to test the code base”. The reading: the “Guardrails people own” theme as infrastructure: a reproducible environment the agent can run anything in, and testing knowledge the agent keeps current itself. It's StrongDM's digital-twin idea (mock every third party) and Lovable's golden checkpoints, built for one company's own stack.

Evidence: transcript 00:04:06

The loop

The factory improves itself

Every line, pull request and decision is tracked, so the record becomes input.

“So there's no reason why you should also be able to run agents on top of that.”00:06:41

Agents analyse where time and tokens go and what tests would help, and every day post ideas to a channel: change the Node setup and save a minute of CI; change a lint script and catch 20% more bugs before AI code review.

“And this has actually been a huge unlock, where you can just set up these systems, and the agents will go and optimize your software factory for you.”00:06:41

From the notes: “they are using sessions for self improvement”. The reading: the outer loop of the “Close the loop” theme, pointed at the factory instead of the product: Overcut's scored retrospectives and Lance Martin's dreaming over transcripts, applied to the engineering process itself.

Evidence: transcript 00:06:41

The shift

From tasks to jobs

“moving from the world where you're giving tasks to your agents to giving jobs to your agents.”00:07:41

“You really have to start treating these things like coworkers.”00:07:41

Instead of instructions, say “improve P99 latency”: the agent adds telemetry, sets targets, breaks down the work, creates Datadog alerts and weekly Slack reports, and updates you daily. His slide shows it: from 840 ms to a 500 ms target, 5,000 ACUs approved, 14 issues filed, and a report that P99 is down 65% with no regressions.

“We want you to just be giving your agent goals that they can work.”00:07:41

Anything that matters but isn't measured, measure it: click latency, layout shifts, onboarding retention.

“And I encourage you that if you can do that, you should just set up an agent to go optimize that number twenty four seven.”00:07:41

Evidence: transcript 00:07:41

Example

A week on the new GPUs

Cognition got early access to NVIDIA's Vera Rubin GPUs and had to build an inference stack from scratch. They gave an agent a dedicated set of GPUs and one goal, throughput:

“Go figure out how to push that to its limits.”00:09:27

Over about a week it found eight or nine improvements, and each went to a person before shipping:

“It was sent to a human engineer for final approval.”00:09:27

His slide: 27,336 tokens per minute per GPU, 2.8× the starting point, against the GB300's best of 17,185.

The reading: it's Simon Willison's “Fable class” in practice: define the goal, provide the tools, and the agent works it for days. The human role is the last step: approving each change.

Evidence: transcript 00:09:27

The close

Android parity in 44 days

Cognition is small and can't staff everything. An engineer, Samuel, had an idea:

“Can't we just use that as a blueprint for our Android application?”00:10:45

Not one session but a standing job: hooks on the iOS codebase, the Slack channel of user feedback, and live QA against the iOS app, pixel for pixel.

“and over the course of 44 days, we have a working application, which we'll be putting out soon.”00:10:45

“most days there's not a single human message going to the stage I'm working on it.”00:10:45

Some days it merged forty or fifty pull requests, escalating only the most important decisions.

“it's not a slo—it's not a slop application.”00:10:45

“So I think really most teams are limited by the amount of ambition they have.”00:13:07

“So it's time to be ambitious.”00:13:07

From the notes: “he asked the dev to make an android app and keep it parity to the ios app” and “most teams are limited by the amount of ambition they have”. The reading: your later note sharpens it: being ambitious isn't doing more. The Android app isn't more volume; it's a thing they wouldn't have attempted with the people they had. For a client, the question isn't how many tickets agents can close but which standing jobs, with a measurable goal and a way to check, they'd hand over. It's also the opposite of Steve Yegge's experience: forty PRs a day with almost no human messages, because the iOS app is the spec and pixel-level QA is the check.

Evidence: transcript 00:10:45, 00:13:07