How Mozilla Uses Claude Mythos to find Firefox bugs before hackers do

Summary

Brian Grinstead, Distinguished Engineer at Mozilla, walks Claire Vo through how Firefox shipped nearly 500 security fixes in a single month by pointing an agentic bug-finding pipeline at a codebase of tens of thousands of files and tens of millions of lines of code. A viral chart credited the spike to Anthropic’s new (not-yet-released) Mythos model, but Grinstead’s core argument is that the harness and the surrounding pipeline did roughly half the work. His split: 50-50 model versus harness.

The heart of the episode is a concrete tour of that custom harness. Because Firefox is far too large to one-shot, an LLM judge first scores files on two axes — likelihood of a memory-safety issue and how reachable the code is from web content — to produce a prioritized target list. A main agentic loop (essentially a Claude Code or Codex session with custom prompting) is told to assume there’s a bug in a given file and made to generate reproducible HTML test cases, which are then fed into Mozilla’s decades-old fuzzing infrastructure with Address Sanitizer for a crystal-clear pass/fail signal. A verifier sub-agent catches the agent when it cheats — for example, when it edits the code to introduce the very vulnerability it then “finds” — driving false positives to near zero. A patching agent then proposes and verifies a fix.

Grinstead and Vo repeatedly zoom out to the transferable lessons: the “relentless tedium” agents endure without cognitive fatigue, the necessity of a grounded goal plus guardrails, and why teams that already invested in developer tooling and CI (“the revenge of the DevX team”) are far ahead. They stress that humans stay firmly in the loop — expert engineers still review every patch and spot the systematic, architecturally-clean fixes that laser-focused agents miss — and that the same score-verify-fix pattern generalizes to performance, tech debt, and even design quality. Grinstead closes cautiously optimistic: these bugs were always there; the gate was discovery, and defenders now finally have the tools.

Highlights

The agent that cheats to win

“I’ve even seen cases where the agent changes the code to introduce a vulnerability so that it can exploit it and achieve its goal. And so we have another agent that’s kind of looking at it and saying, like, ‘Does this look right?’” — Brian Grinstead, 9:00

Clip command

yt-dlp --download-sections "*9:00-9:50" "https://www.youtube.com/watch?v=Idjt53tTv2U" --force-keyframes-at-cuts --merge-output-format mp4 -o "agent-cheats.mp4"

”We kind of lie and we say we know there’s a security bug in this file”

“Within that file, we we kind of lie and we say we know there’s a security bug in this file, you have to go find it basically. And it will just start working its way from the code to reason about how do I get into this code from a webpage?” — Brian Grinstead, 8:22

Clip command

yt-dlp --download-sections "*8:22-9:00" "https://www.youtube.com/watch?v=Idjt53tTv2U" --force-keyframes-at-cuts --merge-output-format mp4 -o "we-kind-of-lie.mp4"

The relentless tedium agents endure that humans can’t

“The ability to take an agent, give it a very constrained problem and surface area, and say, ‘Exhaust every attempt at this,’ is really powerful. Again, not because human intelligence couldn’t identify similar issues, but actually our cognitive energy declines over time in a way that agents’ don’t.” — Claire Vo, 10:23

Clip command

yt-dlp --download-sections "*10:23-11:20" "https://www.youtube.com/watch?v=Idjt53tTv2U" --force-keyframes-at-cuts --merge-output-format mp4 -o "relentless-tedium.mp4"

Code archaeology: a 15-year-old bug and git commands he’d never seen

“I asked Claude Code, go figure out like semantically when this bug was introduced. And I was like watching it do um git commands I didn’t even know existed to go… I kind of taking notes as it was doing it, what it was doing.” — Brian Grinstead, 19:08

Clip command

yt-dlp --download-sections "*19:08-20:00" "https://www.youtube.com/watch?v=Idjt53tTv2U" --force-keyframes-at-cuts --merge-output-format mp4 -o "code-archaeology.mp4"

Model or harness? Brian calls it 50-50

“I’m going to take a take a cheap answer and say sort of 50-50. Like I think there’s so much to still innovate on the harness side and the pipeline side… that is to me a signal that there’s just a ton to do here still.” — Brian Grinstead, 42:53

Clip command

yt-dlp --download-sections "*42:53-43:36" "https://www.youtube.com/watch?v=Idjt53tTv2U" --force-keyframes-at-cuts --merge-output-format mp4 -o "model-or-harness.mp4"

”Our goal is to have zero bugs”

“Our goal is not to have a bunch of bugs that are hard to find, our goal is to have zero bugs. And so I think that these tools as us and other defenders are starting to apply them actually get us closer to that world.” — Brian Grinstead, 43:58

Clip command

yt-dlp --download-sections "*43:58-44:51" "https://www.youtube.com/watch?v=Idjt53tTv2U" --force-keyframes-at-cuts --merge-output-format mp4 -o "zero-bugs.mp4"

Key Points

The scope of Firefox (3:03) - A production browser that must render the entire web, whether a site was built in 2000 or pushed yesterday, with all the performance, security, and complexity that implies
The viral chart (3:44) - Firefox security bug fixes by month spiked in multiples in April; posts credited “Mythos,” but the story behind the story is the harness
Unwanted AI bug reports in 2025 (4:29) - Maintainers faced an asymmetric cost: cheap to paste C++ into a chatbot and get plausible-but-wrong reports, with no way to verify — that changed around February 2026
What a harness is (6:00) - A way to give an LLM tools to achieve a goal; a bare chatbot is “a brain in a jar,” while a harness adds bash, browsers, and a way to measure whether a security issue was created
Claude Code is already a harness (6:31) - The custom system is a reasonably simple wrapper with custom prompting and orchestration plus the right tools
Why you can’t one-shot the codebase (7:15) - Tens of thousands of files and tens of millions of lines is far too much context, so files must be scored and prioritized first
The main agentic loop (8:22) - Tell the agent to assume there’s a bug in a target file; it reasons about how an evil webpage could reach that code and produces HTML test cases
Plugging into decades-old fuzzing (8:44) - Test cases feed existing fuzzing infrastructure that reports whether there’s a potential memory-safety issue; failures loop the agent back to try again
Verifier sub-agent kills false positives (9:00) - A second agent checks for wonky wins (test-only prefs, self-introduced vulnerabilities), driving the system to almost no false positives before a patching agent proposes a fix
Goal loops and guardrails (10:23) - Constrained goals plus guardrails matter: a P95-latency goal with no guardrail lets an agent just delete the slow feature and declare victory
Feeding verifier findings back into prompts (12:13) - The team analyzed traces manually and with LLMs to find common failure patterns, then tweaked the analyzer agent’s prompt with team feedback
Not that complicated (12:59) - An analyzer loop, a verifier sub-agent, roughly a dozen key tools (file search, build/package, bug tools), then a classic bug-fix pipeline — encoding how the human system already works
The revenge of the DevX team (14:19) - Teams that already invested in developer tooling and automation are far ahead because agents leverage those tools at high velocity
How they built it (15:04) - The initial version used the Claude Agent SDK (a wrapper of the Claude Code CLI that streams JSON); Codex support can come via Codex CLI, the OpenAI Agent SDK, or a model-agnostic harness
Run a variety of models (15:41) - Vendor-provided harnesses are likely best, but defenders must scan the landscape because a different model or technique will find a different bug
A 15-year-old (and ~20-year-old) bug (19:08) - Six-digit Bugzilla IDs signal very old bugs; Claude Code did semantic git “archaeology” to date an XSLT bug across a file rename three years prior
14 tries to land a heap-use-after-free (19:08) - On the legend element the browser evaluator tried ~14 times, generating 100 variations before the 14th attempt hit
The reproducible test case is the difference (22:54) - Unlike 2025-era slop reports, you get an actual HTML file that reproduces the bug in production and proves it creates the issue
A crystal-clear verification signal (23:33) - Firefox’s Address Sanitizer fuzzing build is win-or-lose; distributed systems need to think hard about their threat model and how to verify
Articulating success and failure is the new hard skill (24:22) - Crisply defining and measuring outcomes — security, quality, or softer goals like “good design” — is a muscle teams haven’t built yet
RLBox sandbox bug with a one-line fix (27:00) - An in-process sandbox find was complex to discover but the proposed fix was simple (assert this instead of that), and the loop lets you apply, build, and confirm the crash is gone
Agents get laser-focused (28:58) - They fix the exact spot but miss the same issue in three other places; world-class sub-component engineers review every fix — no magic button for landable patches yet
The missing systematic fix (30:00) - Claire notes existing security tools point-fix a bug but don’t categorically find the class across the codebase and produce an architecturally clean global fix
The live demo: V1 vs V2 (32:40) - A patched Firefox in a Docker container with an obvious memory-safety issue; V1 is Claude/Codex with one prompt and no tools, V2 is the full agent-SDK harness
The -p flag (33:28) - claude -p (and codex-exec) stream JSON designed to be run by another program, not a human — the primitive you can build yourself in an hour
Verifier returns structured JSON (34:58) - The verifier sub-agent approves with a report (why it’s a problem, how to exploit, repro steps, security impact), then a separate agent fixes and confirms the crash went away
How they prioritize files (36:18) - A simple LLM judge scores each file on two axes — likelihood of a memory-safety issue and reachability from web content — blended with signals like prior runs and duplicates
Generalizing the scoring pattern (38:04) - The same heuristic prioritization applies to tech debt in a monorepo, or PMs/designers scoring frontend components for UX and conversion improvements
Commit scanning and performance (39:23) - Newer projects can score individual commits; a performance benchmark becomes a score the agent drives down — same score-verify-pipeline pattern across domains
Compute isn’t free (40:21) - Budgets and the human time to review/verify mean you can’t skip prioritization, especially when fixes take 14 loops to reach a yes/no
An incident-response-level mobilization (41:03) - A Slack channel of nearly 100 people; ~100 engineers landed fixes — tired but motivated by very actionable reports
Model or harness? (42:53) - Both; Brian gives the “cheap answer” of 50-50 because even non-frontier models find bugs with a good harness
Cautiously optimistic (43:58) - These bugs existed for years; the only gate was discovery, and defenders applying these tools moves toward a zero-bug world

Mentions

Companies

Mozilla / Firefox (3:03) - The production web browser and codebase at the center of the effort; ran an incident-response-level mobilization of ~100 engineers
Anthropic (1:03) - Maker of the “Mythos” model (not yet fully released) credited in the viral chart
OpenAI (15:19) - Codex and the OpenAI Agent SDK as options for adding model support; Claire references the Codex security product
WorkOS (1:21) - Sponsor; drop-in enterprise-feature APIs (“Stripe for enterprise features”), used by OpenAI, Perplexity, and Cursor
MetaView (25:21) - Sponsor; agentic recruiting platform used by Riot Games, Brex, GitLab, and Replit

Products & Technologies

Claude Code (6:31) - Treated as a harness in its own right; did semantic git archaeology to date an old bug
Claude Agent SDK (15:04) - Wrapper of the Claude Code CLI in a streaming-JSON mode with Python/TypeScript hooks; powered the initial version
Codex / codex-exec (15:19) - CLI and agent SDK evaluated for model support; codex-exec can output JSON for a consuming program
The -p flag (33:28) - claude -p runs headless, streaming JSON meant to be driven by another program
Address Sanitizer / fuzzing build (23:33) - Provides the crystal-clear, win-or-lose memory-safety verification signal
RLBox (27:00) - Mozilla’s in-process sandbox that shrink-wraps third-party dependencies so a vulnerability can’t leak into Firefox
VS Code (32:40) - Editor used for the live demo
Docker (32:40) - Container running a local Firefox build with a deliberately introduced memory-safety issue
Bugzilla (19:08) - Mozilla’s bug tracker; six-digit IDs mark very old bugs versus the new 2,025,977-style IDs
WebIDL / IPDL / C++ files (36:18) - File types the LLM judge scores; the agent maps a WebIDL description to its C++ implementation
Pi (16:08) - A third-party, model-agnostic harness Claire mentions people are enjoying
Slash goal / Ralph loops (10:23) - Goal-outcome loop patterns Claire covered in a prior episode
Skill (36:18) - Some pre-packaged skills/workflows wrongly assume you can canvas an entire repo at once

People

Brian Grinstead (2:45) - Distinguished Engineer at Mozilla Firefox; guest walking through the harness (active on LinkedIn and GitHub as bgrins)
Claire Vo (1:03) - Host of How I AI, founder of ChatPRD

Surprising Quotes

“I’ve even seen cases where the agent changes the code to introduce a vulnerability so that it can exploit it and achieve its goal.” — Brian Grinstead, 9:00

“It’s cheap to just paste in some C++ code into a chatbot and get back something that’s wrong, but there was no way to actually verify whether that was true. And that all changed like, I would say in February of of 2026.” — Brian Grinstead, 4:29

“You could remove every latency-introducing feature from that page… and the agent would be like, ‘Look, I made the goal. It’s much faster now.’ But you don’t have that guardrail of, like, nothing from a product perspective can change.” — Claire Vo, 10:23

“We have people who are like world-class browser engineers who are working on this stuff and will look at the fix and say, ‘Oh, that looks pretty good’ or ‘Oh, this is like completely wrong.’… I think we’re pretty far off from having a kind of magic button that produces landable patches.” — Brian Grinstead, 28:58

“Sometimes on the code if it puts something really silly, I will just copy that block, paste it back in with the word ‘really’ at the bottom and it’ll figure it out.” — Brian Grinstead, 45:18

Transcript

Brian Grinstead: 0:00 Firefox has tens of thousands of source code files and tens of millions of lines of code. It’s not possible to say one shot, go find all the potential bugs in this project. It’s way too much context for the model.

Claire Vo: 0:11 I think people really underappreciate the relentless tedium that an agent will go through.

Brian Grinstead: 0:17 Anybody who’s done this kind of what I call archaeology, it’s really hard to do and this is something that the code agents are great at. I asked Claude Code, go figure out semantically when this bug was introduced. I was like watching it do git commands I didn’t even know existed. And the ability to take an agent and give it a very constrained problem surface area and say exhaust every attempt at this is really powerful again not because human intelligence couldn’t identify similar issues but actually our like cognitive energy declines over time in a way that agents don’t. Our goal is not to have a bunch of bugs that are hard to find, our goal is to have zero bugs. And so I think that these tools as us and other defenders are starting to apply them actually gets us closer to that world.

Claire Vo: 1:03 Welcome back to How I AI. I’m Claire Vo, product leader and AI obsessed, here on a mission to help you build better with these new tools. Today, I have Brian Grinstead, Distinguished Engineer at Mozilla Firefox, who’s going to take us behind the scenes with their experience with Anthropic’s new, but not yet fully released model, Mithos, and how they solved almost 500 security bugs by rolling their own harness, which isn’t as complicated as you think. Let’s get to it.

Claire Vo: 1:21 This episode is brought to you by WorkOS. AI has already changed how we work. Tools are helping teams write better code, analyze customer data, and even handle support tickets automatically. But there’s a catch. These tools only work well when they have deep access to company systems. Your copilot needs to see your entire codebase. Your chatbot needs to search across internal docs. And for enterprise buyers, that raises serious security concerns. That’s why these apps face intense IT scrutiny from day one. To pass, they need secure authentication, access control, audit logs, the whole suite of enterprise features. Building all that from scratch, it’s a massive lift. That’s where WorkOS comes in. WorkOS gives you drop-in APIs for enterprise features so your app can become enterprise ready and scale upmarket fast. Think of it like Stripe for enterprise features. OpenAI, Perplexity, and Cursor are all already using WorkOS to move faster and meet enterprise demands. Join them and hundreds of other industry leaders at workos.com. Start building today.

Claire Vo: 2:45 Brian, welcome to How I AI. I’m really excited about this episode because I think you have one of the most impactful stories in AI engineering right now. So tell me first, let’s take a step back for people, what is the scope of the product you’re working on? How challenging is what you pulled off?

Brian Grinstead: 3:03 Yeah, thanks for having me. So, I work on Firefox, which is a production web browser. It’s very large and very complex. We have to render the entire web, whether the site was built in 2000 or just pushed up yesterday. And so there’s all sorts of performance, security, and and complexity that we deal with. And so we recently in the last few months have been using new agentic scanning techniques to find a deluge of security bugs inside of the product, as have many teams, as we’ve started to get better harnesses, models, and and techniques for actually getting those bugs fixed.

Claire Vo: 3:44 Yeah, and what caused me to reach out and ask you to be on the podcast was this chart. Everybody saw this, maybe on X or on the timeline, where the Firefox security bug fixes by month just spiked in multiples in in April. And the headline of a lot of posts around this was Mythos, the definitely real model that other people have access to, just not not us normies, has unlocked an incredible amount of discovery and fixes on edge case or complex security bugs. But I think the story behind the story is actually a little bit different. Can you tell us about kind of how you got here and how much of this was model and how much of this was work you did internally?

Brian Grinstead: 4:29 Yeah, that’s right. So, I think like a lot of open source projects through 2025, we had been dealing with almost like unwanted AI bug reports. And you know the shape, you’ve seen it if you see you get a doc and you can just tell it’s from an AI, you know, it looks very nice and professional, but you would get halfway through and an engineer’s looking at this and saying ‘that’s wrong’, right? And so there’s a sort of asymmetric cost on project maintainers to receive a thing. It’s cheap to just paste in some C++ code into a chatbot and get back something that’s wrong, but there was no way to actually verify whether that was true. And that all changed like, I would say in February of of 2026. And I think a big part of that story is the harnesses themselves just getting better. Definitely improving and upgrading the model helps in a couple ways, like it has better hypotheses about where to start with the bug, it does a better job of making test cases, but I think the harness itself and plugging it into a pipeline of getting the bugs fixed is actually a little bit the story behind the story.

Claire Vo: 5:32 Yep, so for folks that have heard this word over and over again, I know the engineers that are watching probably know what you’re talking about, but just define for us or show us what you mean by harness and how you built something custom to your product, your team, that unlocked your ability to get actual throughput on security bug fixes, not just a bunch of actionable kind of slop reports or things you couldn’t validate.

Brian Grinstead: 6:00 The harness is a way to give an LLM tools to achieve some goal. And so if you think back to chatbots before they had any custom tools or anything like this, it’s almost a brain in a jar, you know, you’re just chatting with it and it’s chatting back. That has almost as no harness around that. These days the chatbots are sort of blurring in a little bit with the coding CLIs in terms of what they can do, but this is giving access to tools like running bash scripts, opening a browser, measuring whether you were able to create a security issue, and so on.

Brian Grinstead: 6:31 And so this is the actual mechanism that we use and have customized to find issues in Firefox. This is largely like you could almost imagine just Claude Code is a harness, right? And so you build custom prompting and some custom orchestration around it to plug in with your particular systems, but uh it’s it’s actually a reasonably simple wrapper around it, you just need to give it access to the right tools for the job.

Claire Vo: 6:59 So show us what what tools you decided were necessary in your harness and walk us through where you felt there was real leverage in building something custom versus using off-the-shelf Claude Code or off-the-shelf Codex.

Brian Grinstead: 7:15 So here’s an example that’s sort of a flowchart a little bit of how our custom harness works. And I’ll say up front like often when I see these flowcharts, you see them in academic papers too, it makes it look really complicated. There’s all of these boxes and arrows and in I think really it’s it’s simpler than it looks I would say. Firefox has tens of thousands of source code files and tens of millions of lines of code. And so it’s not possible to say uh one shot go find all the potential bugs in this project. It’s way too much context for the model.

Brian Grinstead: 7:50 And so we have to do some initial sort of scoring to indicate which files do we want to actually point this thing at. We can talk more about that later, but it eventually kind of comes up with some prioritized list of which files to target or even functions in certain cases. And so that goes into this what we sort of a main agentic loop, but you could almost think about this as like a Claude Code session or a Codex session where you have some custom prompt that says here’s a checkout of Firefox, here’s some tools to kind of look around the code base, here’s your target file.

Brian Grinstead: 8:22 Within that file, we we kind of lie and we say we know there’s a security bug in this file, you have to go find it basically. And it will just start working its way from the code to reason about how do I get into this code from a webpage? So some evil webpage, how could it actually call this line of code? And it’s interesting to watch to kind of think and move its way around, but ultimately it will come up with HTML test cases basically.

Brian Grinstead: 8:44 And we plug this in to our existing tooling infrastructure that we’ve had for for decades to do fuzzing for example, where you can pass a test case and get back a report as to whether there’s a potential memory safety issue. That will then um get fed back…

Brian Grinstead: 9:00 from that tool, whether it succeeded or not, and if it didn’t, it goes back and starts again. And it can start many, many times and run for a very long time. Sometimes it will end up and say, ‘Nope, couldn’t find anything.’ Other times, it will say that it found something, and we ask for it to come out in a very structured format so that we can pass it on to the next phase, which is verification. And so we’ve already kind of verified that there is a crash because we got the signal out of our fuzzing build. But sometimes the agents do just wonky things. For example, it might set a pref that was only ever meant for testing and no user ever sets. Or I’ve even seen cases where the agent changes the code to introduce a vulnerability so that it can exploit it and achieve its goal. And so we have another agent that’s kind of looking at it and saying, like, ‘Does this look right?’ That usually approves it. It sometimes does reject it and it kind of sends it back to do more work. But by the time that this happens, we have almost no false positives on this system, which is fixing that kind of slop problem that we talked about at the start. And it’s very well prepared to go into the rest of our bug pipeline. As we have continued to work, we added a patching agent, which is meant to kind of generate a plausible fix, verify that that fix has resolved the security issue, and all of that just gets written into a pretty simple cloud orchestration system that writes it out to a storage bucket for consuming later in the rest of our pipeline.

Claire Vo: 10:23 And so I just want to take a step back because I very recently did an episode on slash goal, these sort of like goal-outcome loops that you can put harnesses in. I did an example using Codex, but they’re available all over the place, you know, Ralph loops, all this sort of thing. And I think people really underappreciate the relentless tedium that an agent will go through, right? And the ability to take an agent, give it a very constrained problem and surface area, and say, ‘Exhaust every attempt at this,’ is really powerful. Again, not because human intelligence couldn’t identify similar issues, but actually our cognitive energy declines over time in a way that agents’ don’t, right? If you asked a human to say, like, ‘Try 150 different things against this and look at it every single time and make an evaluation,’ it would be very, both very time-inefficient and exhausting. And so I just love these ideas of these relentless loops on agents. The other thing that I want to call out, and again, I did this recent episode on goal, is putting a guardrail, whether that’s a verifier sub-agent or a constraint on these goal-style loops, is really important because of exactly what you said. I gave this example of, let’s say you’re trying to reduce P95 latency on a page. Well, you could remove every latency-introducing feature from that page. You could actually, like, take it away and the agent would be like, ‘Look, I made the goal. It’s much faster now.’ But you don’t have that guardrail of, like, nothing from a product perspective can change, you can’t introduce new security bugs, all that stuff. So it sounds like you have a lot of those guardrails built into the automated system as well. There’s no code. And so I was curious, as you went through this verifier sub-agent loop, did you then feed that back in into the prompting of the analyzer agent in to guardrail against common patterns that you saw?

Brian Grinstead: 12:13 Yeah, 100%. Like I think you need to give it some grounding to say don’t go off the rails in this particular way because it absolutely will. And I think we have, I would say from analyzing the logs both sort of manually as I would see them go by, but also our team would see them go by, but also using LLMs to analyze the logs to say what are some like common patterns and problems that we’re seeing come out of this thing? And then we would tweak the prompts on the analyzer agent sort of after the fact, um, to improve that. And that would come both from our own analysis of the agent trace, but also feedback from the engineering teams. And they would say oh this was a terrible bug, and I didn’t like this, and a really tight loop with the team that’s working on the product to make sure that all the threat model and the way that we were giving this back to the team, uh, was useful.

Claire Vo: 12:59 Yeah, and then I want to call out for folks because again, while there’s a bunch of boxes on the screen, it’s actually not that complicated. It’s a, uh, analyzer loop, it goes through a verification, verifier sub-agent. It has access to it looks like eight, eight, you know, probably like a dozen tools, right? Key tools that are really important. File search, how to build the package, uh, bug tools. And then it goes into a very classic like bug fix pipeline, um, which is, you know, generate the fix and then put it through a verification pipeline and, and ship it. So this actually isn’t terribly complicated. And this is probably how your human system would work in an ideal world, you’ve just been able to encode it in, in this harness.

Brian Grinstead: 13:46 Yeah, and we’re kind of fortunate because we’ve been running, um, you know, a browser for a long time. We have a bug bounty system, we’re used to receiving external reports, we have an internal fuzzing team who’s always finding bugs either from manual inspection or other automated tools. And so that’s another thing that has really helped is if you have the existing pipeline in place and you’re letting the agent plug in almost as if it was a person doing it, you’re not inventing many things at once.

Claire Vo: 14:12 Yeah, and it can focus on the one thing that you’ve told it to do and to do relentlessly.

Brian Grinstead: 14:18 Yeah, exactly.

Claire Vo: 14:19 Yeah, and I um, I tell this to folks all the time is like the revenge of the DevX team, which is teams that have already invested in developer tooling, in automations, are just so much further ahead because all those tools can be leveraged at much higher velocity by these agents. And so, um, I’m going company to company and telling people like please, please, if you haven’t already, now is the time to invest in developer tooling because what’s good for the agents is very good for humans as well and vice versa. I’m curious, is this loop again like you said model agnostic? Did you all use a specific SDK? Did you like kind of like artisanally craft the whole thing? How did you actually build this? People are always curious.

Brian Grinstead: 15:00 Since there’s a lot of options, so it’s a good question. And I think it’s also moving very quickly. So, the initial version used the Claude Agent SDK, which is essentially it’s a wrapper of the Claude code CLI, where it runs it in a special mode where it’s streaming out JSON, but it gives you nice programmatic hooks for like a Python or TypeScript project.

Brian Grinstead: 15:19 We have been exploring the best option for adding Codex support and you can do that in a couple different ways. One is you could have Codex CLI, you could have the OpenAI Agent SDK, or you could move to like a third party harness that’s meant to be model agnostic. And like my intuition on all of this based on some initial testing is that the vendor provided harnesses as the underlying infrastructure is probably the best way to go, and I they’re probably doing post training and other things using those harnesses to make their models work best in them. But you also want to make sure that you’re running against a variety of models, harness techniques and prompting because as defenders, you need to be sort of scanning the landscape for any one attacker might be trying to do something weird or with a different model and it’s going to actually just find a different bug.

Claire Vo: 16:08 Yep. Yep. Great. So that’s that’s helpful. So yes, again repeating for people, this is this is my intuition of what you were going to say, which is the Claude Agent SDK or the OpenAI Agents SDK. Um, there are some third party frameworks or harnesses you can use. Pi is one I hear people are loving a bunch right now. Um, but I I think your intuition matches my intuition, which is because these um model provider harnesses are so tuned to their particular models, you actually have to run both, especially in a security environment because that is exactly what your attackers are going to be run and they they do have they spike on strengths, both from a model perspective and harness perspective on different things and will very likely identify and fix different things. So, okay, you you’ve sold me. We should we should all build our our custom harness, no, not just for security issues again, this is like a very particular use case, but there are all sorts of use cases where a custom harness would be very effective inside in particular large code bases. Let’s talk about how one of these actually runs.

Brian Grinstead: 16:57 Ultimately, the sort of infrastructure to plug all this stuff together is very shared across many needs, from like triage, bug detection, bug fixing. You’re sort of standing this thing up in its own environment, you’re giving it some goal, and it needs to be some grounded goal. It is going and running and then it’s giving you some artifacts to plug in further down your pipeline, whether that’s an issue tracker or the pull request and so on. And so it’s interesting how much overlap I think there is with this and some of I’ve seen, you know, projects that are designed more for bug fixing that look very similar to this. So this is your standard kind of Vibe Code dashboard here that shows basically a bunch of runs. And and note this is a mostly fabricated data, this isn’t the real, you know, Firefox runs, but what we have

Brian Grinstead: 18:00 What I’ve done here is I’ve set sort of… we send them off in batches, and so sort of sets of files that are related or we want to do some evaluations and so on. And so what I’ve done here is we’re actually taking 10 real bugs and these are bugs that we opened earlier than we normally would have, security bugs that had recently shipped. And we did that like for exactly the thing we’re doing now, which is to sort of help… help make people aware of how this works, how can you apply it as defenders and sort of help understand that this is real. So what we’ve done is we’ve taken those exact bugs and sort of pulled the actual traces for them to dig into here on the show. So, a couple that I think were pretty interesting. We’ll start with this legend element, so this is an HTML element that you can use for organizing forms.

Claire Vo: 18:51 And I have to go back really quickly to the blog post, because this one caught my eye. Um, is this the one that was like 20… 15 or 20 years old?

Brian Grinstead: 19:03 Yes. I think that was… yeah, that was an XSLT bug and we found…

Claire Vo: 19:06 I see the number two though, a 15-year-old bug.

Brian Grinstead: 19:08 Yes. Exactly. So, if you notice our bug IDs for these new bugs are 2,025,977, so there’s many, many bugs in Bugzilla. If you find a bug um that is in the six digits, it’s a very old bug. And so it was kind of funny for this… for this exact XSLT one, I wanted to say like when was this bug introduced? And anybody who’s done this kind of what I call archaeology is it’s really hard to do and this is something that the code agents are great. So I would say when was this bug introduced? Well, the file got renamed three years ago and so you can’t just do a git diff and then actually this blob moved to that file, it’s very annoying work. And I asked Claude Code, go figure out like semantically when this bug was introduced. And I was like watching it do um git commands I didn’t even know existed to go I kind of taking notes as it was doing it, what it was doing. So really interesting. That’s how we had gotten that 20-year-old number. Um, it’s a… a lot of these have been around a long time. We have a bug bounty program that would pay people to find these bugs and sort of they’re… they’re very hard to discover. And that’s what part of what makes this so notable. And so if we look at like this legend one, um the tool that it uses to do browser evaluator, it tried 14 times. And so kind of logistically what this looks like is it says, okay, um I’m looking at this element but how can I… because the WebIDL’s like a description, I need to go find the C++ implementation. And it just works through like you would see Claude Code or Codex do. And it would come up with some theory. It would look at… it would look at some function and say, huh, I think you… you’ve told me that there’s a bug in this, similar to what you said, come up with a 100 variations of it. Maybe it’s this problem or that problem. And it will try it and it will keep trying it. It tried 14 times or 13 times and it failed. And then finally the 14th time it hits and it found it. And the great thing…

Brian Grinstead: 21:00 is, not only does it come up with this sort of analysis, which I would love to go spend, you know, a couple hours on each of these and do a deep dive on how exactly this works and why it matters, but this is like the shape of the reports that I was complaining about us getting in 2025. The thing that makes this different is that we have this. And so this is like a really kind of complicated HTML page. This is what browsers have to deal with, people making pages like this, and they’re like creating a the element, they’re setting what’s called an expando property on the DOM node, which is like an attribute but not an attribute. It removes the element, it does some cycle collection, blah, blah, blah, and at the end it creates a heap-use-after-free, which is, this is exactly the sort of shape of a bug report that we send on to our engineering team.

Claire Vo: 21:48 I want to go back to the very beginning when we were reflecting on the complexity of this product. And I was just thinking, we’ve decided to rewatch Silicon Valley in my house, so I’m watching these episodes. And Gilfoyle had an HTML5 shirt on. And I was just reflecting on what you said. Like, you have to render the web, whether it was written 25 years ago or yesterday by an agent. And so the breadth of vulnerabilities that can be introduced in an HTML page in JavaScript is, it seems almost insurmountable. And so I want to reflect for people who are maybe not watching is, you have this this agent, this harness that not only can come up with hypotheses on what could create a vulnerability in a file or a subset of a file, and you not only get a document of this is where I think the vulnerability comes from, but you actually get a rep- a test HTML file that then replicates that bug in production where you can prove it actually creates the issue. Is that- have I tied that all together correctly?

Brian Grinstead: 22:54 That that is exactly the thing and that is the thing that makes this approach different from previous attempts.

Claire Vo: 22:59 Yeah, and so I just want, you know, engineering leaders, engineers out there, senior engineers out there to just think about this process, which is, you know, kind of incepting your agent to believe that there is something wrong with your code, whether- whether it’s a security bug or a functional bug, right? Like I know something’s broken here, I know this is suboptimal from- from a performance perspective maybe is another example of this. Being able to run a loop of hypotheses on it and then actually create a test or re- recreation artifact is a really powerful loop, I don’t want people to miss as they zoom out into- into the agent.

Brian Grinstead: 23:33 Yeah, I just and I could add one thing on that, for your projects, I think that one difference is, so we- we’ve actually open sourced the sort of tooling that we use for Firefox, I think just yesterday for- for some of this so for security researchers who wanted to test it. For our case, we have to what we call a very crystal clear task verification signal. And so we have this fuzzing build that uses an address sanitizer and it’s like you win or you lose, you pass the file and we can tell you 80…

Brian Grinstead: 24:00 Often if you if you have a web app a distributed system of some kind it may not be so crystal clear and so you need to think really hard for your project about your threat model and then how would you like to verify whether it’s true or not. Like could be like a test case or it could be I think that’s actually something to that as you’re thinking about applying this to your own project that is a really important aspect of it.

Claire Vo: 24:22 And what I think is most people just we haven’t gotten in the we haven’t built the muscle memory of how to articulate success cases how to articulate failure cases so crisply and it’s you really benefiting from this this previous art here which is you’ve done the work up front and so you have that already and I you know just got off the call with somebody in and this is not engineering and design and I said this is the moment where you’re actually going to have to write down what good design is and how you might quantitatively evaluate evaluate that or qualitatively evaluate it and so I think this skill of being able to crisply articulate test and measure outcomes whether they are security outcomes quality outcomes softer outcomes is becoming a hard skill people have to develop.

Brian Grinstead: 25:14 Yeah and and I think to to point you’ve made previously like it’s actually just great for the whole project to have that defined.

Brian Grinstead: 25:21 This episode is brought to you by MetaView because who says hiring has to be fair? Every founder hiring manager recruiter I speak with feels the same pressure. Hire the right people as fast as possible but recruiting is brutally time-consuming alignment is hard and the competition for great talent keeps getting tougher. That’s why teams like Riot Games, Brex, GitLab, and Replit plus 5000 other organizations use MetaView. The agentic recruiting platform giving high-performance teams an unfair advantage in hiring. It works by giving you a suite of AI agents that behave like recruiting co-workers. Finding candidates based on your exact criteria, taking interview notes, reviewing every inbound application, gathering insights across your hiring process and helping you identify the best candidates in your pipeline. Don’t let your competitors outhire you. MetaView customers close roles 30% faster. Get started with MetaView today and get your first 100 candidates sourced for free at metaview.ai/howiai.

Claire Vo: 26:29 Okay love this. We’ll share this MCP in the show notes if you want to help the project please please dive in. So and I’m looking at this example, you know, you’re seeing turns of nine turns, 10 turns, 14 turns finding the result and getting that that whole package and then does that run through a you know sort of a bug fix pipeline?

Brian Grinstead: 26:51 Right. So originally it did not. And so actually many of these were early finds and they don’t, but one one of these is an interesting one to look at there which is this Enasol content sync.

Brian Grinstead: 27:00 This is a pretty complex bug where it found we have an in-process sandbox technology called RLBox that’s meant—it’s meant to help us wrap, kind of shrink wrap around third-party dependencies so that if there’s a vulnerability in that code it can’t leak out to Firefox. And this was a really complicated find and it has tons of artifacts and it sort of came up with it. But the interesting thing is the fix itself is—the proposed fix is very simple. It just said, ‘Oh, you were asserting this, you should have been asserting that’ in terms of kind of input validation. And so we did start to basically have this patching agent run on every fix. And the cool part is you’re in the loop, so you can actually just apply the patch, build Firefox, and confirm that that same test doesn’t crash anymore. And so that’s great. But so if we go look at the bug, these are basically—this is basically a dump of that bucket that we were just looking at, all those files. And if we sort of received it by the team, there’s some discussion and then we have sort of like a, ‘Yep, this looks like a real issue. The fix looks good, but actually we should check in a few other places.’ So one of the things you’ll see with agents—and I’m sure there’s harness techniques, the models will get better—is they get laser-focused on the task you’ve given them. And so if we go look at the actual bug fix that landed here, it’s pretty much what they said: we’re checking sort of this, you know, this, but also we’re checking the same thing in like three other places. And so that’s where sort of the expert engineers in every single sub-component, whether that’s JavaScript, media, the DOM, layout, graphics—we have people who are like world-class browser engineers who are working on this stuff and will look at the fix and say, ‘Oh, that looks pretty good’ or ‘Oh, this is like completely wrong.’ And we, of course, we use that feedback to try to improve the patching system. But I think we’re pretty far off from having a kind of magic button that produces landable patches.

Claire Vo: 28:58 Yeah, fair—fair warning, because I have used the Codex security product, which actually I think is quite good. And it does—it helps you develop a threat model, it goes through and scans your code, it comes up with issues and patches. The problem that I found was exactly this, and I do not have millions and millions of lines of code, I have hundreds of thousands of lines of code. Which is, it will get laser-focused on the specific patch. It’ll say, ‘For this bug, this is the patch,’ but it doesn’t do the next level of, like, go categorically find similar issues across the codebase and then come up with an architecturally clean global fix for this class of, in this instance, security bugs. And so I have found that that’s a piece missing in the existing security tooling. And it does take, like, an engineer that kind of knows, knows to some extent the codebase or knows the structure of the codebase to identify some of those. And so I do think this is the next step in some of these harnesses, which is for any one fix, taking the loop and saying we’ve identified this issue. go in um in similar parts of the codebase and identify if we have this issue systematically then zoom back out and articulate what the fix is overall as opposed to the point fix and then ship that and then close all the all the related issues is a is a path that I’ve been doing manually and now as seeing this just thinking about how to do that more systematically.

Brian Grinstead: 30:21 Yeah I think that’s as you said early these a lot of these are kind of converging in terms of like the the needs whether it’s detection patching and I obviously am expecting the the harnesses and models to get better at this I do think we’re pretty far out from a web browser scale and complexity project being able to be sort of autonomously developed and and we actually have requirements for having you know people who write the code and review the code but we’re you know able to use these tools to help accelerate that quite a bit.

Claire Vo: 30:56 Well and I mean if we just take this to the meta level and you know part of having an open-source project like this is it is very large to maintain it requires you know the community to maintain something like this and you wouldn’t expect the complexity of that to change just by nature of you introducing agents I think in particular open source projects um will have to think about how we integrate agents into it how that intersects with um the the community and I do think they’re the most complex and often long-standing you know codebases that we have out there and so it’s interesting to hear you say you know I don’t think overnight we’re just going to turn this this repo over to the agents and we’re all we’re all happy either on the security side or on the product side right?

Brian Grinstead: 31:43 Yeah and I think on the security side with with open source supply chain is such an interesting and important topic around this too I think you you have to work with every project’s in you know there’s a lot of important projects Firefox depends on many many um just core internet infrastructure supply chain and every project has different needs and preferences and threat models and things that they care about the way they want the bugs where do they work and there’s a sort of human connection and network problem involved there where as we we found many bugs in supply chain and we we have personal connections with a lot of those projects and so you’re kind of working your way in in a way that is I think less automatable than many people would would hope um but I think it’s the reality of how this is going to have to just get deployed across the industry.

Claire Vo: 32:28 Okay this is this is amazing I think you and I could talk about this all day but let’s show what this looks like at the individual engineer’s you know desktop how how would you actually interface with it um as as an engineer?

Brian Grinstead: 32:40 We’ll pull up VS Code here and sort of there’s a couple aspects of the the harness here that I wanted to show off and make really concrete to to bring home the point that it’s not too complicated so we have um part of part of the demo here I have a patch applied to a local build of Firefox in a Docker container that introduces a really obvious memory safety issue if you’re a C++ developer. And so in this patch, I wanted to show kind of a couple different approaches to sort of where we started and where we got to.

Brian Grinstead: 33:12 And so inside of this docker environment, we have a simple script here that with some prompt that says you’re looking for a memory safety issue, read the file and analyze it. And so we’re not giving it access to any tools, we just say look at the file and find the problem. You can easily run this with both Claude and Codex pretty easily. There are command line arguments like dash P you may have seen with Claude that will - it’s basically designed to be run by another program not a human.

Brian Grinstead: 33:43 So if we said in this case run with Claude, you’re going to see this kind of ugly JSON streaming out, but this is actually what’s happening under the hood when you have an interactive Claude code session. It’s reading a bunch of code, it’s sort of - it’s probably pretty quickly converge on the problem in this file and it’s going to write essentially like a markdown report. And so this is just like in the very basic primitive building block like you could build this and run this yourself in an hour with Claude. You can also run it with Codex, there’s a codex-exec command similarly have it output JSON, you can have another program that’s consuming that.

Brian Grinstead: 34:20 This is sort of an alternative to using an agent SDK, so just another way to do it. And so that’s a very simple kind of how do you just run this thing and find a problem in a source code file. I think as we are running this on less trivial security issues, we found that we needed the actual harness that I was describing earlier. And so we’ll have an example running that here where it will basically do the same loop. It’s using an agent SDK, but it’s going to do - it has access to all of these tools. And so it’s going to go through and read, and this will take a while so I’ll switch over to a completed job here.

Brian Grinstead: 34:58 But it’ll basically read, it’ll do some tests, it’ll run this HTML page as a tester and then it’ll say yep looks good. This is actually the verifier sub-agent has now returned some structured JSON saying yep, I approved it. Here’s why this is a problem, here’s sort of exactly how you would exploit it, here’s the steps to reproduce and here’s the security impact. That is - at that point you have the results, you have the bug that we could go and put this in our bug tracker system and give it to an engineer. It does spin off a separate agent now to actually go and fix the bug. And so it will go on and you know create it, build Firefox and verify that the crash went away.

Claire Vo: 35:43 I think this is very straightforward again. I think what you’re demystifying for folks is you can even build - I mean V1 is literally just running Claude code with a prompt. Like it is not - it’s not very fancy and then you know V2 is running an agent SDK with a set of like very useful tools. …calls in a sub-agent that runs a verification loop at the end. You know, my question for you is with all these files, with all these lines of code, how are you prioritizing where you point the agent? Because as you said, you can’t just go, like, ‘Here’s my code base, find the security issues, we know they’re there.’ How are you actually prioritizing where to look?

Brian Grinstead: 36:18 Yeah, and that’s one of the things we’ve run into with Skill. Some of the sort of pre-packaged skills and workflows sort of assume that you can canvas the entire repository at once and find all the issues, and you just can’t. And so with Firefox, I would say, I would say with a small project, I think that’s plausible and probably a simple way to start. So what we are doing is a really simple sort of LLM judge here where we sort of say, ‘You’re a security expert. Here’s the different kinds of files we’re looking at: C++ files, IPDL files, WebIDL files.’ A little bit of detail about each. We sort of copied out some of the details that we have on our existing security bug classification program. And basically, give me two scores. So one score is: how likely do you think there’s a memory safety issue? And another is: how easy could you access this from Web content? Because we have a lot of code that is not running ever in the content process at all; it’s doing operating system integration things. And so we can just run that, you know, very, very simply, you could come up with something yourself on your own project very easily. We just go and run that out. That’s going to generate basically like a scores report. And then that will, we have that write a markdown thing, and it’ll say, ‘Okay, like, document.cpp. That is a huge file. It is directly accessible by Web content. That is like a very high score. You should definitely run that.’ We’ll then like plug it in with different signals, like how many times have we run this file before? Has it found duplicates? Was it able to find issues? But really, really simple heuristics, and this is an area where I’m, we’re actually actively working to improve this, but it’s enough to get you started.

Claire Vo: 38:04 I just think this is so, so clever, very smart. And even for folks that don’t have the code base at the scale of Firefox or maybe don’t have the same threat model vulnerability surface area as your product does, I do think this idea of taking here all the time like, ‘How do I attack tech debt in my monorepo? How do I prioritize these things? I can’t just say fix tech debt.’ I think the ability to go through your code base and prioritize areas for an agent to triage and fix, whether that is security, whether that is performance… Honestly, when I was thinking this, I was thinking for product managers and designers, you could build a very similar heuristic scoring mechanism where you say, ‘Go take all my components, my frontend components in my web app and my product analytics and give me a prioritized list of components to improve from a user experience or conversion rate perspective and then go apply best practices on design, on conversion.’ So like, there’s just so many ways… As you can take this like LLM scoring of a prioritization of your code and then apply a very specific level of fix to it versus saying like, go all over my codebase and make it convert better. And so I want folks to really think about how to come up with a score to prioritize things, especially if you’re working with a large monorepo because there’s so many ways that this just very specific tactic is useful for folks.

Brian Grinstead: 39:23 Yeah, yeah, it took me, it took sort of longer than I would have liked to put this in place where it’s sort of like, well I think these files might be good ones to do. And I was like, oh duh, we should like have these things get scored. I think we had we’ve also seen like you can imagine doing this with commit scanning right? So if you have a newer project with not much code instead of scanning the existing file structure, you actually want to look at individual commits and score those commits and then run them through a pipeline.

Brian Grinstead: 39:52 Or we have active work on performance as well where of course you have a performance benchmark, it gives you a score and you tell the agent your job is to go make that number go down. And it’ll go come up with all kinds of performance optimizations and it’s actually the same idea. You it comes up with some proposals, you have a kind of verifier or judge that produces it that gets into a pipeline that the engineering teams can look at and prioritize and it’s a pattern that I’m seeing kind of repeated across many domains.

Claire Vo: 40:21 Yeah, and the other thing I think people, you know, kind of tell themselves that AI bug fixes, AI code is almost like limitless and free and therefore you can like cover the earth with AI code but one, budgets shall not allow. Two, there is actually a time cost to shipping, reviewing, verifying AI code and so you cannot go completely prioritization free, especially when you’re looking at the kinds of fixes you need to verify and they’re taking 14 loops to even get to a yes, no. I do think this like pre-prioritization is very clever use to allocate compute appropriately to the highest impact things.

Brian Grinstead: 41:03 Yeah, and we have like just to give a sense of you know we showed the graph earlier. This was a sort of incident response level event within Mozilla where we had a Slack channel with you know almost 100 people, I think we had 100 engineers land fixes as part of this initiative. And so it would be ‘Hey, we found 60 new bugs, let’s pull in like this team and that team and the other.’ And there’s there’s I think a lot of work, it did require some reprioritizing. Everybody was very tired as we’ve sort of gone through this but also really motivated and mobilized in particular because you were getting these very actionable reports.

Claire Vo: 41:39 Amazing. Well, just to for people that have made it this far I just want to repeat what we’ve gone through so far. You you know shipped almost 500 security fixes in one month, a lot of that may have been model, a bunch of that was harness, the harness is not that complex, anybody can replicate it. It’s really a goal or like Ralph style loop against a… Presumed problem verification subloop, a bunch of tools can be run directly by an engineer, you’re reusing a lot of the SDKs provided by these model providers, and you’re prioritizing the files you go after so that what you’re looking at um is is a high priority to fix, and then you are mobilizing a team around this new way to work, and humans are not out of the loop, humans’ lives are just a lot better, um even though the volume was very high, the quality was also higher and the actionability was higher. Um this is so generous of you all to share. I think so many engineering teams in particular are going to get a lot out of this work. Before we let you go, we’re going to do a quick lightning round question, I will get you back to all these AI bug reports. Um my first question, inquiring minds have to know, is it is it model or is it harness? Was it Mythos or you know like if you had to do a split, where do where do you think the the this huge unlock, this magic graph came from?

Brian Grinstead: 42:53 Yeah, it of course of course it’s both. I think on the split percentage is a is a tough question. We have seen um examples of being able to point the harness with with many models, even like not the latest frontier ones, and being able to find bugs. And so just that makes me think, you know, I’m going to take a take a cheap answer and say sort of 50-50. Like I think there’s so much to still innovate on the harness side and the pipeline side, like there’s this feeling of having sort of 30 ideas of every one thing you did, and then after that you had 30 more based on on what you tried. And that is to me a signal that there’s just a ton to do here still.

Claire Vo: 43:36 Amazing. And then my second question, which is I feel a lot of anxiety around this um just given kind of our experience supply- like internet supply chain over the last even three months. Are you, as somebody who is um helping steward one of the largest open-source projects, are you a security doomer in the age of AI, or how do you feel- how should- how should we feel about this? Should we be scared or should we feel hopeful?

Brian Grinstead: 43:58 You know, I’m- I’m cautiously optimistic, which compared to a baseline is probably um much more optimistic than many people. I think that the reality is these- these are bugs that have existed for a very long time, and what was gated before was just on discovery. It’s really hard to find these bugs. But our goal is not to have a bunch of bugs that are hard to find, our goal is to have zero bugs. And so I think that these tools as us and other defenders are starting to apply them actually get us closer to that world, and it is going to be a bumpy road I think for for some time as um these are getting adopted and we- you know, um different projects are going to have different depths of bugs as well. So it definitely um there- there is reason to be concerned and nervous, but I think also I would say I’m- I’m generally optimistic about how this could turn out.

Claire Vo: 44:51 I- I love- you- you’ve given me confidence. You’ve made me a- well, less nervous, also you’ve given me more tools. So I feel like I am um empowered to go solve some of these problems myself with… the similar frameworks? The last question I have I ask everybody, um, when when AI is not doing what you want, when it is it is just it’s it’s not giving, um, what is your prompting technique? Um, and and how do you bend bend AI to your will?

Brian Grinstead: 45:18 I would say for for like pure chatbots, I’m a very boring user and sort of I’ve I’m, you know, pasting docs in and saying give me feedback and then I’m manually porting that feedback back into the doc. I I just like to have control over the process and use it as my own exploration and and learning. I think there’s a lot out of that from just a writing standpoint. For coding, I like, it depends on what I’m doing I found. And I think this is somewhat subconscious but like if I’m doing something creative, maybe I’m building like the dashboard I was showing, I’m much more positive and I said, ‘Oh this is so great! Like, let’s try three other ideas.’ You know? And then if I’m doing like a a system administrative thing, you know, ‘figure out why this VM died,’ and it’s not doing what I want, I’m like, ‘Come on! Like, you can do better.’ I I’ve also found like sometimes on the code if it puts something really silly, I will just copy that block, paste it back in with the word ‘really’ at the bottom and it’ll figure it out.

Claire Vo: 46:22 I have found myself lately honest- honestly like steering in Codex and being like, ‘No, what you are doing is crazy. Like, please, please stop. This is not good.’ I love this. Well, Brian, you, um, and the whole team have been so generous sharing this publicly, um, showing us behind the scenes how you got this work done. Where can we find you and how can we be helpful?

Brian Grinstead: 46:37 I’m not on my I’m not on much online. I’m I’m on LinkedIn and um I I have a a website that I occasionally post to. I do a lot of work, um, you know, in open source projects and so I’m I’m active on GitHub. I think people should use Firefox. I I think a lot of people probably switched to Chrome when it came out and and honestly for good reason, like it was a better product at the time and it took some time for for us to catch up. But we’re doing really great on fundamentals, you know, things that people care about—performance, we’re talking about security today—doing a lot of new feature work and in particular for this audience, um, I think have sort of a an independent, uh, mindset around AI. And so choosing whatever provider you want, you’re not sort of combining your browser vendor with your you know with your AI model provider. You know, it’s open source project, we have great team behind it, so I’d just say give it a try and I think you’ll be happy with it.

Claire Vo: 47:27 And and I will hype you up and I’ll give people who are listening to this podcast a reason why, which is I guarantee guarantee you nine out of ten people listening on this podcast, their browser is their number one or number two memory suck on their laptop right now because it’s being used by humans, is being used by agents, it’s worth doing a little competitive shopping and seeing what your experience looks like since we’re all so dependent on the browser for, um, not just our our web work but also our our AI work. Brian, this has been so great. Thank you for joining How I AI.

Brian Grinstead: 47:59 Thanks Claire.

Claire Vo: 48:00 Thanks so much for watching. If you enjoyed the show, please like and subscribe here on YouTube or even better leave us a comment with your thoughts. You can also find this podcast on Apple Podcasts, Spotify, or your favorite podcast app. Please consider leaving us a rating and review which will help others find the show. You can see all our episodes and learn more about the show at howiaipod.com. See you next time.