Notion's Token Town — Sarah Sachs, Notion
Notion’s Token Town — Sarah Sachs, Notion
⚠️ Data note
The uploaded video file behind this URL does not match its own title/description for most of its runtime. Two independent, fresh transcription runs (fully re-downloaded audio, not a cache artifact) both confirm that from 0:00 to roughly 17:00 the audio is actually a duplicate of a different AI Engineer talk — “The Unreasonable Effectiveness of Separating the Task from the Model” by Maxime Rivest and Isaac Miller (DSPy), published as GgLQ02aO-hs — misattributed to Sarah Sachs by the transcriber’s diarization since she was the only seeded speaker name. Only the final ~6:45 (17:10–23:44) is genuinely Sarah Sachs’ “Token Town” content, and it matches the official description’s closing beats (governance, the lethal trifecta, the live software-factory demo). The official YouTube description promises a full ~24-minute talk (token economics, contract negotiation, the “AI Switzerland” routing strategy, etc.) that simply is not present in the actual video file as hosted.
Because of this, the Highlights, Key Points, Mentions, Surprising Quotes, and Transcript below cover only the verified-genuine 17:10–23:44 segment. No content is attributed to Sarah Sachs unless it was actually spoken by her in that window. If AI Engineer re-uploads a corrected video, this file should be rebuilt from scratch with the full talk. The JSON transcript at -I5W5QVAT8E.json is kept as-is (including the misattributed portion) as the literal transcriber output/source of truth for what the current video file contains — do not treat its 0:00–17:00 segments as Sarah Sachs’ words.
Summary
Only the tail end of this video (17:10–23:44) could be verified as Sarah Sachs’ actual “Token Town” talk (see data note above). In it, she argues that teams go “token poor” by routing deterministic work — CSV-to-PDF conversion, CLI tool calls, SQL queries — through an LLM when discrete code would do, and frames model optionality and governance (visibility into who’s using data, maintainability, control) as what lets a company offer more to its customers.
She pivots to security, introducing Simon Wilson’s “lethal trifecta” — private data access, exposure to untrusted content, and the ability to communicate externally — as the core risk shape of increasingly autonomous agent systems, and argues the next six months are a security challenge more than a capability one. She closes with a live demo of Notion’s internal “software factory”: a managed Claude agent scopes a task inside an active Notion document, tags in a teammate (or a Decagon agent) when it lacks context, opens a PR once the team has iterated, and gets a second opinion from Codex — a workflow she says already runs internally at Notion, coordinating most of the company’s own polish and feedback work, and saving real time at scale.
Highlights
”You Don’t Need an LLM to Turn a CSV Into a PDF”
“Bringing discrete pieces of code, like you don’t need an LLM to turn a CSV into a PDF. You don’t need an LLM to talk to notion tool calls if we have a CLI. You definitely don’t need an LLM to do deterministic SQL queries. This is where people become token poor very quick.” — Sarah Sachs, 17:10
Clip command
yt-dlp --download-sections "*17:10-17:51" "https://www.youtube.com/watch?v=-I5W5QVAT8E" --force-keyframes-at-cuts --merge-output-format mp4 -o "csv-to-pdf-token-poor.mp4"
”The Lethal Trifecta”
“And the ability to communicate externally, and that can include like payloads in a web search, the second you have that system, you’re exposing risk. And in fact, the more autonomous your system is, the more unsupervised this risk is.” — Sarah Sachs, 18:30
Clip command
yt-dlp --download-sections "*18:16-18:48" "https://www.youtube.com/watch?v=-I5W5QVAT8E" --force-keyframes-at-cuts --merge-output-format mp4 -o "lethal-trifecta.mp4"
”This Is Real — How It Works Internally at Notion”
“This is real. I asked Rajeev if I could post this. This is how it works today internally at Notion. Almost all of our polish and large feedback like this is actually coordinated through our software factories, both in terms of writing to the right teams and also having coding agents take the first step.” — Sarah Sachs, 22:12
Clip command
yt-dlp --download-sections "*22:12-22:45" "https://www.youtube.com/watch?v=-I5W5QVAT8E" --force-keyframes-at-cuts --merge-output-format mp4 -o "real-internally-at-notion.mp4"
Key Points
(Only the verified-genuine 17:10–23:44 segment is covered — see data note above.)
- Don’t spend tokens on deterministic work (17:10) - CSV-to-PDF conversion, CLI tool calls, and SQL queries don’t need an LLM; using one anyway is how teams go “token poor.”
- Governance and model optionality (17:31) - Visibility into data usage and maintainability, paired with the ability to route across models, is what lets you offer more to customers.
- The lethal trifecta (18:16) - Private data access + untrusted content exposure + external communication capability = exposed risk, credited to Simon Wilson; more autonomy means less supervision of that risk.
- Persistence of enterprise knowledge (19:00) - What agents see, do, and retain across a multi-agent system is an underrated, under-discussed problem.
- Notion’s software factory demo (19:52) - A managed Claude agent scopes a task inside a live Notion document, tags in a teammate (or a Decagon agent) when context is missing, then opens a PR that Codex reviews.
- Internal adoption at Notion (22:12) - Sarah says most of Notion’s own polish/feedback work is already coordinated through this software-factory pattern, saving roughly three minutes per task at scale.
- Closing framing (23:07) - The market is young and opaque; Sarah frames the responsibility as being critical thinkers on behalf of customers rather than chasing raw capability.
Mentions
(Only the verified-genuine 17:10–23:44 segment is covered — see data note above.)
Companies
- Notion (17:10) - Sarah Sachs’s employer; the “Token Town” application section centers on Notion’s AI stack.
- Vercel (22:34) - Cited alongside Notion as using the same staging-to-shipping-to-closing software-factory pattern.
- Decagon (20:35) - Partner whose agents are tagged in for customer-voice tasks in the live demo.
Products & Technologies
- Claude / Claude Code (19:52) - Managed agent used in the demo to scope a task and open a PR inside Notion.
- Codex (21:37) - Used to review the PR Claude opened and surfaces two issues.
- Notion AI (19:34) - Referenced in Sarah’s “T-Pain” joke slide.
- The lethal trifecta (18:16) - Security framework: private data access + untrusted content exposure + external communication capability.
People
- Sarah Sachs (17:10) - Speaker (verified segment only); leads AI engineering at Notion.
- Simon Wilson (18:16) - Credited with coining the “lethal trifecta” security framework.
- Rajeev (22:12) - Notion colleague whose internal example Sarah got permission to share.
- MJ (20:25) - Notion PM tagged into the live demo.
Surprising Quotes
“So, okay, this is me telling T-Pain to tell people to buy Notion AI. And the reason I included this slide is I am going to sell Notion for a second.” — Sarah Sachs, 19:34
“Claude put up a PR, maybe that’s not enough. Maybe I want to go ahead and ask CodeX what it thinks. Great, found two issues.” — Sarah Sachs, 21:37
“It’s really wild out there, you guys. I get it. The market is really young, it’s exceptionally opaque, it’s moving fast.” — Sarah Sachs, 23:07
Transcript
(Only the verified-genuine 17:10–23:44 segment is included — see data note above. The video’s first ~17 minutes contain misattributed audio from a different talk and have been deliberately omitted rather than presented as Sarah Sachs’ words.)
Sarah Sachs: 17:10 Bringing discrete pieces of code, like you don’t need an LLM to turn a CSV into a PDF. You don’t need an LLM to talk to notion tool calls if we have a CLI. You definitely don’t need an LLM to do deterministic SQL queries. This is where people become token poor very quick. And I think the last option here besides open weight CPUs and optionality is actually governance. Um, there’s a lot of AI governance, one is visibility, understanding who’s using the data, understanding its maintainability and control. When you have model optionality you can offer a lot more to your customers.
Sarah Sachs: 17:51 Um, here’s an example of how that governance works in Notion. So final tips again, think about architecture, think about open weight, and build value that transcends tokens. So, we’re going to depart Token Town. I know I said welcome to Token Town. We’re going to spend the next 10 minutes really thinking about what to do next.
Sarah Sachs: 18:09 So, I think the challenge of the next six months doesn’t have to do with capabilities. I think it has to do with security. Let’s start there. There’s this concept called the lethal trifecta. Simon Wilson, I think, crafted this.
Sarah Sachs: 18:22 If you have access to private data, exposure to untrusted content, whether it be through ingestion, MCP, email, right? And the ability to communicate externally, and that can include like payloads in a web search, the second you have that system, you’re exposing risk. And in fact, the more autonomous your system is, the more unsupervised this risk is. I think that this is what builds valuable products, not just capability.
Sarah Sachs: 18:48 Same with sandboxes and computers. We talked about this, but it really is something that builds better determinism in your product and also better token economics for your customers. In multi-agent orchestration, understanding what agents see and do and what persists, I think persistence of enterprise knowledge is something that’s actually really not discussed enough, starting to be with some recent launches. You know, oh there is audio. So don’t have your workflows look like this. And I think this is where most software factories are today, right? It’s like actually your entire engineering time just spends time babysitting the factory. Right? I mean I get it, ours started off like this. Agent orchestration is one of the most difficult tasks of making factories work.
Sarah Sachs: 19:34 So, okay, this is me telling T-Pain to tell people to buy Notion AI. And the reason I included this slide is I am going to sell Notion for a second. It’s my job. Always be closing, always be selling, always be hiring, come find me. But I’m going to talk for a second about how Notion does this. Today we already have the ability to inspect tasks, and you can imagine any task that you look at in a Notion document, you can have Claude actually go ahead and scope out what you need.
Sarah Sachs: 20:02 We’ve launched this managed agent capability today. So if I go ahead to the top of this task, I can actually ask Claude agent to scope out the task. Right? Ideally it’s working. And you’ll see it’ll actually populate an entire spec of what needs to be done. In this example, it’s not ready. It’s going to ask me a question. Keep in mind this isn’t a markdown file. This is an active document.
Sarah Sachs: 20:25 Let’s say I don’t actually know the question and I go ahead and I ask my team what to do. Imagine that you can kind of tag in your team into these system. MJ’s our PM. So in this example she doesn’t know, usually she does, but multi-agent orchestration is important. Maybe Claude Code isn’t the best at customer voice, but Decagon is.
Sarah Sachs: 21:00 Right. You can ask Decagon agents, we’re proud partners with them as well, to collect the right data that you need. Okay. In this example, we think we know enough. We’re going to go ahead and actually iterate through some of this flow. We’re going to skip ahead a little bit. We asked our TL what we needed. He replied, again, it’s a collaborative file, not just a markdown, and we can have Claude actually go ahead and spin up the PR.
Sarah Sachs: 21:29 Hopefully this is looking a little familiar now. This is kind of the vision of software factories. That’s what we’re trying to host. Okay. Claude put up a PR, maybe that’s not enough. Maybe I want to go ahead and ask CodeX what it thinks. Great, found two issues. You can think about this scaling in an actual factory. So today in Notion, you’re actually able to orchestrate these agents together, and you’re not committing to a lab, you’re committing to the concept that AI is augmenting and automating what you do.
Sarah Sachs: 22:12 This is real. I asked Rajeev if I could post this. This is how it works today internally at Notion. Almost all of our polish and large feedback like this is actually coordinated through our software factories, both in terms of writing to the right teams and also having coding agents take the first step. Vercel does this as well, from staging to shipping to closing, and we see massive ROI gains from our customers. That’s over three minutes saved on a given task, imagine that at scale. So, I think we’re trying our hardest to think about the factory lens. We cannot do this without optionality, and we cannot do this without conviction that we understand what models are required for which task.
Sarah Sachs: 23:07 It’s really wild out there, you guys. I get it. The market is really young, it’s exceptionally opaque, it’s moving fast. I’m super grateful for communities like AI Engineer to bring us together and like talk openly about these things and how we navigate it. I think we owe it to all of our customers to get it right and to be critical thinkers about how we navigate this together. I’m chronically online, unfortunately. Um, you can always DM me on Twitter, you can email me, you can find me after this. But thank you for yapping with me and thinking about this problem and have a good day.
