End of Scaling? Why Ilya Sutskever Walked Away from the Rat Race and What He Is Building Now

Summary

This video analyzes Ilya Sutskever’s dramatic philosophical shift between his 2023 interview (when he was OpenAI’s chief scientist) and his 2025 interview after founding Safe Superintelligence (SSI). The host contrasts Sutskever’s previous certainty about scaling - “bet on deep learning, more data, more compute” - with his current humility, frequently saying “I don’t know” and expressing wonder that AI feels like science fiction.

The core thesis is that Sutskever has recognized the limits of the scaling paradigm. He now believes the recipe of “transformers plus internet data” has hit an asymptote, and that achieving superintelligence requires engineering models that understand the physics of the world before reading tokens - a “synthetic biology thesis” inspired by how humans have 3 billion years of evolutionary priors hardcoded into them. The video argues Sutskever is essentially building a “digital limbic system” - not another chatbot, but a reasoning kernel with internal value functions.

The business strategy is framed as a high-stakes bet: SSI is taking $3 billion to go “full waterfall” - disappearing for years to build a “straight shot” to superintelligence, rejecting the agile/product-first approach. The video concludes that the AI market may be bifurcating into useful chatbots (OpenAI, Anthropic) trained on user data, and reasoning kernels that work for hours using search and value functions to produce single exact answers.

Highlights

”Pre-Training Will Run Out of Data - It’s Very Clearly Finite”

yt-dlp --download-sections "*2:46-3:45" "https://www.youtube.com/watch?v=oNC3363P_5s" --force-keyframes-at-cuts --merge-output-format mp4 -o "oNC3363P_5s-2m46s.mp4"

“At some point though pre-training will run out of data. The data is very clearly finite. Scaling sucked out all the air in the room.” — Ilya Sutskever, 2:58

”All of This AI Stuff - Isn’t It Straight Out of Science Fiction?”

yt-dlp --download-sections "*1:18-2:10" "https://www.youtube.com/watch?v=oNC3363P_5s" --force-keyframes-at-cuts --merge-output-format mp4 -o "oNC3363P_5s-1m18s.mp4"

“You know what’s crazy? That all of this is real… Like all this AI stuff… isn’t it straight out of science fiction?” — Ilya Sutskever, 1:36

”To Get Superhuman Performance, Disconnect from Human Feedback”

yt-dlp --download-sections "*7:41-8:40" "https://www.youtube.com/watch?v=oNC3363P_5s" --force-keyframes-at-cuts --merge-output-format mp4 -o "oNC3363P_5s-7m41s.mp4"

“If you train on human data, you get human level performance. And if you want a superhuman performance, you need to disconnect from the human feedback loop.” — Ilya Sutskever (paraphrased), 7:46

”$3 Billion That Research Taste Is Better Than Internet Data”

yt-dlp --download-sections "*8:05-9:00" "https://www.youtube.com/watch?v=oNC3363P_5s" --force-keyframes-at-cuts --merge-output-format mp4 -o "oNC3363P_5s-8m05s.mp4"

“Investors are betting over $3 billion that research taste is better than the collective data of the entire internet.” — Turing Post Host, 8:05

”The Age of Research Is About Discovery - It Needs Quiet Time”

yt-dlp --download-sections "*9:48-10:40" "https://www.youtube.com/watch?v=oNC3363P_5s" --force-keyframes-at-cuts --merge-output-format mp4 -o "oNC3363P_5s-9m48s.mp4"

“If the age of scaling was about execution, the age of research is about discovery. And discovery needs some quiet time.” — Turing Post Host, 10:03

Key Points

• Sutskever’s Transformation (0:50) - In 2023, Sutskever was OpenAI’s chief scientist speaking with absolute certainty; in 2025, he’s a pure researcher unafraid to say “I don’t know”

• The Scaling Era’s End (1:06) - Sutskever’s 2023 message was “bet on deep learning, more data, more compute” - now he sees different limits

• Data Situation Shift (2:51) - In 2023: “data situation is still quite good”; in 2025: “pre-training will run out of data, data is very clearly finite”

• Transformers Hit Asymptote (3:05) - The recipe of transformers plus internet data has tapped out; “scaling sucked all the air in the room”

• Synthetic Biology Thesis (3:20) - Sutskever’s new obsession: engineering models that understand world physics before reading tokens

• The Teenager Metaphor (3:25) - A teenager learns to drive in 10 hours because they have 3 billion years of evolutionary priors; robots need millions of miles of data because they start from scratch

• Physics Understanding Required (4:00) - Without understanding world physics, we won’t achieve truly smart AI - aligned with Yann LeCun and Fei-Fei Li’s work on world models

• Emotions as Engineering (4:32) - The most controversial part: Sutskever uses Patient Elliot (who couldn’t choose socks due to lost emotional processing) to argue intelligence is useless without values

• ChatGPT as Patient Elliot (5:37) - Current LLMs have massive cortexes but no internal value function to make decisions - they’re like the brain-damaged patient

• Building Digital Limbic System (6:00) - Sutskever wants to replace refusal (“I can’t do that”) with consciousness where the model feels negative reward signals for bad ideas

• Product-First to Waterfall (6:55) - In 2023 Sutskever defended OpenAI’s product approach; now he’s rejecting the “rat race” for years of silent research

• User Data Teaches Imitation (7:41) - User data creates imitation, not reasoning; human data yields human-level performance, not superhuman

• Self-Play for Superhuman (7:52) - To achieve superhuman performance, must disconnect from human feedback and create self-play loops like AlphaGo

• $3 Billion Research Bet (8:05) - Investors betting research taste is better than collective internet data - “terrifying” but following the physics trend

• AI Market Bifurcation (8:39) - Future split: useful chatbots (OpenAI, Anthropic) on one side, reasoning kernels (SSI) on the other

• Reasoning Kernels Defined (9:05) - Not bots to chat with, but expensive silent engines working hours to give one exact answer using search, value functions, and emotions

• Certainty as Red Flag (9:30) - In this new research era, when the paradigm’s inventor says he doesn’t know what’s next, we should listen

• Age of Discovery (10:03) - The age of scaling was about execution; the age of research is about discovery, which needs quiet time

Mentions

Companies

• OpenAI (0:53) - Where Sutskever was chief scientist in 2023; defended their product-first approach
• Safe Superintelligence (SSI) (1:32) - Sutskever’s new company building reasoning kernels with $3B+ funding
• Anthropic (8:49) - Listed alongside OpenAI as companies building useful chatbots that may need to merge
• Lunar Society (0:35) - Dwarkesh Patel’s previous podcast name from 2023 interview

Products & Technologies

• GPT-3 (7:06) - Referenced as starting point of OpenAI’s API product approach
• ChatGPT (5:40) - Diagnosed as “Patient Elliot” - massive knowledge but no internal value function
• DALL-E (7:12) - Referenced as example of OpenAI’s product success
• AlphaGo (7:57) - Example of self-play loop achieving superhuman performance
• Transformers (3:12) - Architecture that combined with internet data has hit its limits
• LLMs (5:37) - Diagnosed as having fundamental flaws - cortex without limbic system

People

• Ilya Sutskever (0:15) - Former OpenAI chief scientist, now SSI founder; subject of the entire video analysis
• Dwarkesh Patel (0:18) - Interviewer for both 2023 and 2025 conversations with Sutskever
• Alex Krizhevsky (2:03) - Co-authored AlexNet with Sutskever, demonstrating deep learning’s potential
• Geoffrey Hinton (2:05) - Co-authored AlexNet, helped establish deep learning
• Yann LeCun (4:09) - His work on world models aligns with Sutskever’s physics-first thesis
• Fei-Fei Li (4:12) - Her world maps work aligns with understanding physics before training
• Patient Elliot (4:43) - Neuroscience case study of patient who couldn’t make decisions after losing emotional processing
• Antonio Damasio - Neuroscientist who documented the Patient Elliot case (referenced in video description)

Surprising Quotes

“At some point though pre-training will run out of data. The data is very clearly finite. Scaling sucked out all the air in the room.” — 2:58

“You know what’s crazy? That all of this is real… Like all this AI stuff… isn’t it straight out of science fiction?” — 1:36

“If you train on human data, you get human level performance. And if you want a superhuman performance, you need to disconnect from the human feedback loop.” — 7:46

“Investors are betting over $3 billion that research taste is better than the collective data of the entire internet.” — 8:05

“If the age of scaling was about execution, the age of research is about discovery. And discovery needs some quiet time.” — 10:03

Transcript

0:00 Welcome to Attention Span where we pay close attention to details that matter and I will give you clarity understanding complicated interviews with vague concepts. And I’m talking about the recent interview of Ilya Sutskever that he gave to Dwarkesh Patel, but this interview cannot be analyzed separately from an interview he gave to Dwarkesh 2 years ago and it’s incredible how the only two years brings such a huge difference.

0:32 First, it’s a funny detail, but Dwarkesh Patel was called Lunar Society back then. Now, it’s all forgotten and he’s just Dwarkesh Patel. But Ilya Sutskever looks like a different person back then and now. Back then in 2023, he was a chief scientist of OpenAI and he sounded like a man who knows the future. His answers about limits of AI was absolute.

1:02 “In 2015, my thinking was a lot more I just don’t want to bet against deep learning. I want to make the biggest possible bet on deep learning. Don’t know how but we’ll figure it out.” And back then the road map for him was clear. More data, more compute. Reliability would come from scale.

1:18 But what we saw this week is a completely different Ilya Sutskever, a pure researcher who is not afraid to say I don’t know as many times as needed. This is his first major interview since he started Safe Superintelligence. And the god of scaling sounded different. “You know what’s crazy? That all of this is real.” “Yeah. Meaning what?” “Don’t you think so?” “Meaning what?” “Like all this AI stuff and all this area. Yeah. That it’s happening like isn’t it straight out of science fiction?” “Yeah.”

1:51 He’s not talking about betting on deep learning anymore. And this is crazy because he is the person who basically made it possible for us to believe in deep learning. He demonstrated with AlexNet, co-authored with Alex Krizhevsky and Geoffrey Hinton that deep learning is such a big thing that can take us to this magnificent future of actually smart artificial intelligence.

2:15 And now he’s talking about science fiction and strangeness. Let’s go through what he said and understand through all his “I don’t know” and all his vague wording. What is it that he’s building? What are his ideas? What is his aspiration and where if everything works as he supposes to work where will it take us and what type of AI Ilya Sutskever is building in Safe Superintelligence.

2:46 To understand the pivot we need to see what he said about data. Before in 2023 he was very optimistic: “I would say the data situation is still quite good there’s still lots to go.” But in 2025 his tone shifted completely: “At some point though pre-training will run out of data. The data is very clearly finite. Scaling sucked out all the air in the room.”

3:05 And this is the first signal that we can notice. Ilya is telling us that the recipe transformers plus internet data has tapped out. We have hit the asymptote and his new obsession is what I called synthetic biology thesis. He mentions a specific metaphor. You know how a teenager can learn how to drive in only 10 hours and the robot needs millions and millions of miles and videos and data to learn?

3:34 Why? Because the teenager is not learning from scratch. It’s not just 10 hours. It’s 10 hours plus 3 billion years of evolutionary priors hardcoded into this teenager. And Ilya is betting that we cannot scale our way to superintelligence anymore. We have to stop training models to predict the world and start engineering models that understand the physics of the world before they read a single token.

4:05 And that’s very very much aligned with what Yann LeCun says and Fei-Fei Li does in her world maps - physics of the world before and without understanding the physics of the world. We will not get to a really smart and clever and maybe even sentient artificial intelligence. Though I still have doubt that we can get to sentient AI.

4:32 And this leads to the most controversial part of this recent interview. Emotions as engineering. Ilya tells a story about a patient with brain damage. Most likely he refers to patient Elliot who couldn’t decide which socks to choose because he lost his emotional processing. And Ilya argues that that intelligence, the cortex, is useless without values, the limbic system.

4:59 Now, I want to pause here for a second because a bunch of neuroscientists out there who watching this video and who was watching the interview was screaming that he was wrong because he gets the anatomy wrong. He implies that the damage was to the limbic system. But in patient Elliot’s case, the damage was to ventral prefrontal cortex, the VMPFC. And that’s the bridge between logic and emotion.

5:26 But please don’t let the anatomy distract you from the point that Ilya tries to make. He is bringing the engineering insight. He is diagnosing the fundamental flaws in LLM. Basically ChatGPT is this patient Elliot. If you think about it, it has a massive cortex. It knows everything about socks, but it has no internal value function to tell which socks matter and how to make a decision about it.

5:58 And if my analysis is correct, Ilya is not building another chatbot. What he is building is a digital limbic system. He wants to replace refusal where the model is trying to say I can’t do that with consciousness where the model feels a negative reward signal for bad ideas. What he’s trying to do, he’s trying to hardcode in silicon what evolution hardcoded in our DNA.

6:25 And that’s why he talks about people who would want augment themselves with the AI. This is part of this synthetic biology, this is part of this digital limbic system because if he hacks, if he engineers into this area, if he hardcodes this into the model, then the model will be much closer to who we are as humans and then it will be much easier to merge AI and human.

6:52 But that brings us to the business strategy, which frankly is terrifying. If you remember in 2023 Ilya defended OpenAI’s product first approach. “So we’ve had a product for quite a while now for back from the GPT-3 days from 2 years ago through the API and we’ve seen how it grew. We’ve seen how the response to DALL-E has grown as well and so you see how the response to ChatGPT and I think all of this gives us information that allows us to make a relatively sensible extrapolation.”

7:22 He relied on user data, but now he’s rejecting the rat race of releasing products. He’s going full waterfall instead of going agile development. And waterfall means going dark for years to build a straight shot superintelligence. Why the flip? Because he believes user data teaches imitation but not reasoning.

7:46 And if you train on human data, you get human level performance. And if you want a superhuman performance, you need to disconnect from the human feedback loop and create a self-play loop the way they did with AlphaGo. And this is why this business strategy is terrifying because the investors are betting over $3 billion that research taste is better than the collective data of the entire internet.

8:12 And he might be wrong and he might be right. The trend is certainly going towards the understanding of physics of the world. But what he wants is that he wants to hardcode it in the model’s DNA like it is hardcoded in us. He wants to compress somehow this three billion years of evolution, $1 per each year, and hardcode it into a model.

8:39 So how does the future look like if Ilya is right? We are certainly heading toward a split. On one side, there will be very useful chatbots like OpenAI and Anthropic and Ilya thinks that at some point they probably will need to merge and do something together and these chatbots would be trained as they are on user data.

9:00 On the other side, the reasoning kernels will be the thing and that’s what Safe Superintelligence team SSI is building. This wouldn’t be the bots to chat with, but reasoning kernels will be expensive silent engines. They will work for hours, maybe longer, using search and value functions, emotions, and combining this all together to give one exact answer.

9:24 And the funny thing is that in this new age of research, I don’t know, in this “I don’t know” era, certainty is a red flag. And when the man who invented the current paradigm tells you he doesn’t know what happens next, we should listen and think and do not just brush it off. He is in deep thinking, in deep research mode himself right now.

9:48 And if investors bet on that, that means something. Not because investors necessarily know what to do, but because he is an incredible thinker and researcher who demonstrated them something that made them believe. If the age of scaling was about execution, the age of research is about discovery. And discovery takes some time. It needs some vague time. It needs some silence.

10:15 And discovery is not a commercial product. The discovery itself needs to be quiet. It can be vague in the process. But then it delivers results. Thank you for watching Attention Span. It was an incredibly interesting thought journey to see behind the “I don’t know” what actually he’s working on. If you enjoyed it, please share it with other people if it helps them. This is the best thing you can do if you like this podcast.