---
title: "The Model Ate the Coding Harness. Now What?"
speakers:
  - name: Jediah Katz
    affiliation: SpaceX Super Intelligence
room: "Shannon Room (NY Ballroom E)"
starts_at: "2026-10-14T11:55:00-04:00"
ends_at: "2026-10-14T12:13:00-04:00"
format: "talk"
recording: "2026-10-14_1155_SpaceX"
duration: "18m 02s"
source: "qwen3-asr-1.7b"
---

# The Model Ate the Coding Harness. Now What?

### 00:00:00 — From Prompt Engineering to the Agent Harness

Cursor, early in two thousand and twenty-five, as the longest tenured engineer working on our agent harness, and recently, I came along to SpaceX as part of the acquisition of Super. So today I want to talk to you all about how the Asian harness has evolved to where we are today, and where I think things are going. And there's a motivating question that I have for us to answer today, which is, is the agent harness dead? So, let's get started.

Here's a more natural and flowing version of that script, which reads like a cohesive conference talk rather than a series of short points. Oops, just making sure everyone's still got their soft detectors on before lunch. No, for real. So the story starts in early 2023, when prompt engineer was the hottest title out there. By mid 2025, that had evolved to context engineer, someone who programmatically puts the best context for the model together on the fly. And then, only at the beginning of this year, did the industry land on the term agent harness and harness engineering, to describe work that my team and I have been doing for a while.

### 00:01:22 — Runtime, Context Orchestration, and Peak Harness

So, what's in an agent harness? The harness is like the eyes and the hands for the model. It wraps around the model, steers it, and connects it to words. Users connect to our harness through apps, and then the harness connects the model to computers, and also to third-party services like MCPs.

I like to divide the harness into two layers: the runtime and the context administration layers. The runtime is kind of like the foundation of the harness. It controls where the core agent loop runs and how state is stored so that work can resume. Building this foundation is a lot like traditional software engineering.

And then the context orchestration layers are kind of like the Wild West. This is what a lot of people refer to when they talk about the harness. These parts of the harness control which models are called when and what context they see. That includes things like prompts and tools, of course, but also memory, compaction, and a bunch of tricks you might not know that we do to make the agent behave in the way that we want. So this is the area that I primarily work in.

In the new year, we kind of hit peak harness. Cursor announced that we had created a browser entirely from scratch, using a new long-running harness called Grind Swarm. This was a multi-model harness that orchestrated a lot of a lot of agents and a lot of machines, and it was very expensive to run. It actually took down cursor maybe once or twice to get this demo.

Now, like anything in the world of agents, it only took a few months of P harness for people online to start saying that the harness was dead. To read from Anthropic talks about how they had killed much of the system prompts from Claude Code. Gary from YC thought that the harness would fade away, and only skills would matter. Even myself and my colleagues Stefan, we wrote about how we had removed all these guardrails that we had in Cursor to patch over gaps in the model's intelligence.

We used to have things like a small model just to apply code edits to a file, because the frontier models aren't good at this. We also had all sorts of coding-specific instructions like how to explore the codebase and how to check your work. So, if all these people are saying the harness is dead, they were saying that smarter models are what killed it, and there's a lot of evidence for that.

### 00:04:14 — Coding Evals and the Model Data Flywheel

I want you all to think about a very simple harness, where the model only has access to the shell and to all of my secrets. Please don't screenshot this time, by the way. So who here thinks that this really simple harness would do just as well on coding evals as a more sophisticated one? Maybe some hands, two things of wood, two things of wood, two things of wood, if you raise anything to wood. A few hands. Most of you know. So, a few recent studies try to investigate exactly that question. The first one out of UC Berkeley tried several different frontier models on a series of harnesses including Py Codex and Cloud Code, and what they found, particularly on the newer, smarter models, is that the choice of harness did not actually make a difference significantly in the score on SuperIntLight. And then another study from Apple UPL found similarly that a dedicated MLE harness did not make a big difference on an MLE bench.

So, what is going on here? Is it is it actually true that the harness is dead? Are they right? Here's what I think that the evidence shows. A fancy harness in this day and age no longer adds coding intelligence. I think that is true. I know that's surprising, but these things that we used to do in the harness to patch over gaps in the raw coding muscle memory of the model, like teaching me how to score the codebase, how to work persistently for a very long time with things like slash goal, how to run tests to check your work — the model has natively absorbs these skills.

So why is this happening? It turns out the harness is actually making itself obsolete through what I call the data flywheel. So in this process, the harness brings harder and harder coding tasks within reach. At that point, more users can productively attempt those tasks with agents. That means more coding data is shared with the labs, and eventually, the models learn natively how to do these skills well. Eventually, it's so good that the model can actually delete the things that needed to do to make those tasks work in the first place. It's a very interesting process. So that's how you get people saying that the harness is dead. And if the models are good enough to code well in any harness, then why should you care? Does it really matter which harness you use?

### 00:06:55 — Unlocking New Frontiers with Grokbot and Software Factories

Is the harness really dead? I say no, and there's two reasons why. First of all, and I'm going to go into both these points: the harness actually opens the new frontier of tasks for the model to solve; and then second, there are some tasks or some parts of the harness, especially outside the context layer, which the model cannot observe. So, to first talk about how the harness unlocks the next frontier, I want to talk about Rockbot. Who here has tried Rockbot? Can I see some hands? Awesome. Let's go. That's great.

So, for those of you who haven't, Rafa is our autonomous colleague who gets work done for you, not just coding, but all kinds of knowledge work, and I really recommend you try it. I'm not going to sell too much, but it's a groundbreaking new product. It's already created a new category. Let's just talk about the harness, though. So Rockbot has a new kind of harness that operates not based off of one-off chats, but long-lived coordinator bots that orchestrate a fleet of worker agents. You can kind of think of it like hiring a high-powered lawyer who deals with you, you know, directly and owns strategy, but they farm out their tasks to a team of partlees. It's like that.

Now, something interesting about the Rockbot harness is, I mentioned how in Cursor we found ourselves removing things from the harness, removing prompting, removing guardrails as the models get smarter, but curiously, in Grokbot, we've actually found ourselves putting back a lot of these things, adding new prompting, adding new guardrails. And that's because there's a lot of new tasks in this new way of working in the new domains that Rockwise is meant to solve that the models have not nailed yet. See a few examples of those kind of things.

Just a few things that we're focused on right now when it comes to robot. There's key website flows like maybe slide decks and booking travel. Communication, enough time is probably needs to keep up to date on what it's been doing without overwhelming you. Acting versus asking is another big one. Rock can do a lot of things, which are kind of risky in some sense, like emailing your colleagues. You want to be able to do that, but you only want to do it in the way that you intended. And then, of course, owning over time the way that you work, the memories is another big focus.

So, those of you who might have pushed the boundaries with coding agents might have noticed that none of these tasks are new, but for Grokbot they're much more important because everybody is needing them in some way, and right now in Grokbot, it is — the hardest thing is making these tasks feasible. Of course, the model is a big part, but the harness is helping bring it within reach.

So, going back to our data flywheel, robot, that example fill in the last arrow in our cycle, which is when a capability like coding gets commoditized by the model, it's actually the harness which unlocks a new frontier for the model to then solve, and kicks off that flywheel again. So that might look like, you know, taking on new tasks outside of the coding domain, as we just saw with Chatbot, or it could just be taking on tasks in the software engineering job beyond pure coding. I'm sure some of you here are you know pretty senior software careers, and you might know the more senior you get, the less coding you're actually doing. So there's a lot of other things in the job that matters. This is the core idea that I would like you to understand: this flywheel and how the model and the harness together drive it.

One more example of how the harness can unlock new frontiers is some of you may have heard of this idea of the software factory, where you have multiple agents coordinating together to solve the full software development lifecycle. Now, this is far from a solved vision, but there's a lot of industry, or there's a lot of innovation happening across the industry at the harness. To give an example of how this might manifest, you know, internally, we have a lot of channels where someone might report a bug to that channel, and then an automation immediately auto triages that bug. Then it might kick off a cloud agent to try to fix the bug, and that agent uses our tooling to show before and after videos to prove that it's fixed. And then once that fix is deployed or is merged and starts deploying, we have our Rollouts product, which kicks off another agent to watch in production for any issues.

So you could think of all this as almost like a meta harness, which is orchestrating many AIs together. That's how I'd like you to think about this. So that's the first reason that you should care about the harness, because a frontier harness is needed to be at the frontier of agentic tasks. And then the next reason is there are some parts of the harness that the model can't absorb. So let's talk about those.

### 00:11:59 — Cloud Runtime, Scoped Security, and Token Efficiency

First of all, there is the infrastructure and runtime needed to make an autonomous colleague reliable. Over the past year plus, we've gone from agents that work on a typical task, for minutes to hours to days and even longer. And there's just no way that this can work on one person's laptop, which might go to sleep. It has to be running in the cloud if you really want, as my colleague.

And then next we have trust and security. Yes, like we saw before, a simple harness with just Bash and all your all your API keys in a file can solve coding tasks, but you probably don't want to give all your attack keys in the file to the model, or if you're okay with that, I'm sure that your head of security would have something to say. So, this is obviously really important companies where we want to have permissions and scope down to specific people, but it's also important to have guardrails for anyone who doesn't want to wake up one morning to early catastrophic action, or to a ten thousand dollar API fee, right?

And then finally, there's efficiency. If you think back to those two slides I showed, where the choice of harness did not make a significant difference in eval scores, both of those studies actually found that there was a significant difference in costs for tasks. I personally have focused over the past month on optimizing the efficiency of the blockchain. Do you want to invite Ben? Now, through tweaking variables like — actually, really improving that token efficiency, model mix — we were able to bring down the costs and the speed of Grokbot significantly for users without any hit to quality. Users are really, really appreciating this. It's been awesome to see. Now it could be an entire different talk about how we did this, but the key takeaway here is that the more and more that you use agents to do your work, the more it matters, the more valuable it will be for you to have the most efficient harness.

### 00:14:01 — Hyrum's Law, Event Subscriptions, and Self-Cleaning Shims

So by now you might be convinced that the harness still matters. Then you might be asking, what's next? Here's a few trends that I'm anticipating. First of all, that flywheel is speeding up. And right now, we've entered a new age where the product is once again moving faster than the model. That means that the harness is going to get more messy before it gets cleaned up — is doing more things. So because of this, we're expecting people to build their harnesses, design them so it's easier to ship and unship new features.

One way we're already seeing that manifest is with both Rockbot and other similar products, less and less of the inner workings are being exposed to the user. Some of you may have heard Hayr's law, which says that anything that you expose to users, they will come to depend on. Now, obviously, there are trade-offs to giving these users less, you know, less visibility into what's happening, but one advantage is that there's less change management to deal with, and you can iterate much faster.

Now, another way that we've been seeing this trend manifest is a bigger focus on designing primitives for agents. Primitives are like building block interfaces that can be composed and extended into powerful. When you have primitives, you can add features without making things more complicated for the agents, and you can remove features without having to do a complicated refactor in your codebase. It's just good engineering practice.

One example of a primitive we've added is called subscriptions. Agents need a way to wake up in response to events. Those might be someone posting in Slack, status going red, how you are that they care about, or just a certain time of day. Now, rather than having one separate integration, or you having separate integrations for each of these things, we have one framework and one tool that the model can use, that the agent can use to subscribe to an event.

And then another cool thing that we do: I mentioned earlier that we have these deterministic guardrails to patch over bad behaviors in the model. That might be, for example, if Rockbot doesn't respond to the user fast enough, and we might give it a little nudge and say, "Hey, users waiting on you." Now, we'd rather not do these things. They're kind of like hacks, but we would rather fix them now than, you know, wait for the next model checkpoint to get released. So in order to make our codebase, you know, be less messy and have less of a worse time iterating in the future, we track all these things, which we call chains, essentially. And every time the model uses one, it gets logged. So now we have a bot who watches this dashboard, and whenever a shim stops firing, that model or that agent will put up a PR and say this shim can be cleaned up.

Another thing that I think is next for the harness is moving up to a higher level of abstraction. Yesterday's generation of harnesses were built from composing a bunch of LLM calls. Now the new harnesses we're seeing, like Grokbot and the Software Factory, are using agents as the core building block. And then finally, I predict that within six months, ninety percent of agent work is going to run off the laptop. This is already where things are going. And I think there's just no way that, you know, this is not necessary in order to have truly autonomous agents who can you know really work with us in the way that we want.

So, is the harness dead? The parts that made it smarter at coding, yes, that is the flywheel at work. That is the final work. But the hardest, as we saw, is how the agents can unlock the next frontier of work. And then also the trust, the efficiency, the infrastructure that the hardest breaks; those things are not going anywhere. So yesterday's harness may be dead, but long live the agent harness of tomorrow. Thank you. Okay.