Andrej Karpathy and Dwarkesh Patel – Popping the AGI Bubble, Building the AI Aristocracy

Summary

This video from Turing Post analyzes Andrej Karpathy’s recent conversation with Dwarkesh Patel, focusing on two major themes: the AGI bubble and the concept of AI aristocracy. The host argues that while there’s much hype around AGI, Karpathy’s interview demonstrates we should temper expectations - the real bubble isn’t in AI itself, but in the inflated promises around artificial general intelligence. Karpathy views current AI systems as “cognitively lacking” and frames progress as “the decade of agents, not the year,” emphasizing that significant engineering work remains.

The second half of the analysis introduces the concept of “AI aristocracy” - the idea that as AI automates both manual and cognitive labor, humans must shift toward philosophical and educational pursuits similar to how historical aristocrats cultivated higher faculties when freed from physical work. The host draws parallels between historical leisure classes who had time for cultivation and the potential future where AI frees everyone to pursue education and self-improvement.

The video concludes that AI is not a bubble about to crash - it’s evolving exactly as it should, quietly integrating into the economy. The real concern isn’t superintelligence emerging, but ensuring humans develop a “super educated civilization” capable of understanding and governing increasingly complex AI systems.

Highlights

”I almost want to reject the question entirely”

yt-dlp --download-sections "*1:59-2:50" "https://www.youtube.com/watch?v=-lRBpyPt79c" --force-keyframes-at-cuts --merge-output-format mp4 -o "-lRBpyPt79c-1m59s.mp4"

“I’m almost tempted to reject the question entirely because again, I see this as an extension of computing. Have we talked about how to chart progress in computing, or how do you chart progress in computing since the 1970s or whatever? What is the x-axis?” — Andrej Karpathy, 2:01

”Every new nine costs the same as all previous progress”

yt-dlp --download-sections "*5:10-6:05" "https://www.youtube.com/watch?v=-lRBpyPt79c" --force-keyframes-at-cuts --merge-output-format mp4 -o "-lRBpyPt79c-5m10s.mp4"

“Every new nine - like 99% to 99.9% to 99.99% - costs the same amount of work as all the previous progress combined.” — Andrej Karpathy, 5:15

”Leisure was not idleness - it was a technology of reflection”

yt-dlp --download-sections "*9:56-10:50" "https://www.youtube.com/watch?v=-lRBpyPt79c" --force-keyframes-at-cuts --merge-output-format mp4 -o "-lRBpyPt79c-9m56s.mp4"

“Historically, the aristocrat was defined by the absence of necessity. Freed from physical labor, aristocrats cultivated what they considered higher human faculties: education, rhetoric, governance, aesthetics, and philosophy. Leisure was not idleness. It was a technology of reflection.” — Host, 9:56

”It’s not a superintelligence emerging - it’s a super educated civilization”

yt-dlp --download-sections "*13:32-14:15" "https://www.youtube.com/watch?v=-lRBpyPt79c" --force-keyframes-at-cuts --merge-output-format mp4 -o "-lRBpyPt79c-13m32s.mp4"

“It’s not a superintelligence emerging. It’s a super educated civilization we need to become.” — Host, 13:48

”AI is exactly what it’s supposed to be”

yt-dlp --download-sections "*8:19-9:05" "https://www.youtube.com/watch?v=-lRBpyPt79c" --force-keyframes-at-cuts --merge-output-format mp4 -o "-lRBpyPt79c-8m19s.mp4"

“AI itself, AI is fine. AI is exactly what it’s supposed to be. It’s evolving. It’s diffusing. It’s blending into the economy, massively blending into economy and our daily workflows. It’s not breaking reality. It’s upgrading it one autocomplete at a time.” — Host, 8:26

Key Points

• Introduction to the Analysis (0:00) - The host introduces the video as a follow-up to previous Dwarkesh Patel interview analyses, focusing on two themes: the AI bubble and AI aristocracy.

• Karpathy’s Credentials (0:34) - Describes Karpathy as an exceptional thinker with deep expertise in computer vision and deep learning, who has the luxury of independence and endless curiosity.

• AGI Bubble Thesis (1:28) - Argues that Dwarkesh Patel pushes the AGI narrative while Karpathy frames the current moment as “early, impressive, and cognitively deficient.”

• Rejecting the AGI Question (2:01) - Karpathy says he almost wants to “reject the question entirely” about AGI timelines, viewing AI as an extension of computing rather than a discrete milestone.

• OpenAI’s AGI Definition (3:03) - Discusses the original OpenAI definition: a system that can perform any economically valuable task at human performance or better.

• Automation of Fragments (3:34) - Current AI does code suggestions, text generation, and image synthesis - these are tools, not general minds.

• AI as “Ghosts” (3:51) - Karpathy calls current systems “digital spirits imitating humans” that talk and sound like us but don’t remember, reflect, or actually know anything.

• The March of Nines (5:10) - Every new nine of reliability (99% to 99.9% to 99.99%) costs as much work as all previous progress combined.

• Reinforcement Learning Controversy (5:38) - Karpathy called RL “terrible” but clarified it’s just one layer of a multi-layered solution - we still need layers four, five, six and beyond.

• The Decade of Agents (6:35) - We’re in a decade of hard engineering, data refinement, memory systems, multimodal cognition, and safety alignment.

• AI’s Gradual Integration (7:52) - Silicon Valley doesn’t like this narrative because it’s not cinematic - AI ends with hard work and smarter tools, not sentient assistants.

• Karpathy’s Real Concern (8:42) - Not that AI fails, but that it succeeds too invisibly - gradual loss of understanding as systems run faster than our comprehension.

• Eureka Labs and Education (9:10) - Karpathy’s educational project aims to create a “Starfleet Academy” for the AI age.

• Historical Aristocracy Parallel (9:56) - Historically, aristocrats cultivated higher faculties (education, rhetoric, philosophy) because they were freed from physical labor.

• Industrial Education Shift (10:34) - Industrialization democratized but reduced education to function - reading, arithmetic, compliance for the machine age.

• AI Returns Abundance (11:04) - AI returns us to conditions of abundance, but the servants are machines, not people - the human role must become philosophical.

• Wall-E Warning (11:58) - Reference to the movie where humans become passive and dependent - this laziness must be fought with education.

• Cultivating to Govern AI (12:45) - Just as nobles cultivated themselves to govern others, humans must cultivate themselves to govern AI.

• Super Educated Civilization (13:32) - The goal isn’t superintelligence emerging but a super educated civilization - we need to catch up education-wise.

Mentions

Companies

• OpenAI (2:59) - Referenced for their original AGI definition
• Eureka Labs (9:12) - Karpathy’s educational project

Products & Technologies

• Reinforcement Learning (5:38) - Discussed as one layer in a multi-layered AI solution
• Self-driving cars (5:25) - Used as example of the “march of nines” taking decades
• Language models (7:32) - Continuation of automation curve from steam engines to computers

People

• Andrej Karpathy (0:42) - Former Tesla/OpenAI, computer vision specialist, main subject of analysis
• Dwarkesh Patel (1:28) - Podcast host known for pushing the AGI narrative
• Sam Altman (2:41) - OpenAI CEO who recently said AGI definition “doesn’t really matter”
• Yann LeCun (7:20) - Referenced as having similar views to Karpathy on gradual AI progress

Surprising Quotes

“I’m almost tempted to reject the question entirely because I see this as an extension of computing - have we talked about how to chart progress in computing since the 1970s? What is the x-axis?” — 2:01

“Every new nine - like 99% to 99.9% to 99.99% - costs the same amount of work as all the previous progress combined.” — 5:15

“It’s not a superintelligence emerging. It’s a super educated civilization we need to become.” — 13:48

“Leisure was not idleness. It was a technology of reflection.” — 10:18

“AI is fine. AI is exactly what it’s supposed to be. It’s evolving. It’s diffusing. It’s blending into the economy.” — 8:26

Transcript

0:00 Hello everyone and welcome to attention span. Since I’m kind of a commentator on Dwarkesh recently and many of my new subscribers came from my analysis of the Sutton and Dwarkesh interview, I feel obliged to analyze Karpathy’s conversation as well. And I think this interview, already circulating widely through quotes, is very important from two points of view: the AI bubble and the AI aristocracy. Let me unfold.

0:34 First, thank you Andre for the speed of your talking. I actually enjoyed it. I didn’t need to put 2x speed on the YouTube videos that I usually do. So that was perfect. Andrej Karpathy is an amazing thinker, not only because he comes up with the terms that steer up the whole community and then stick forever, but he is an extremely knowledgeable computer vision specialist with a deep understanding of deep learning as well. He possesses a higher percentage of real AI knowledge that not many people have out there.

1:06 Also because he has the luxury of being extremely wealthy, fully independent and endlessly curious. I even have a recurring monthly topic in my newsletter called “What does Karpathy see?” Because he often describes, analyzes, and philosophizes about AI on a level that not many people can do.

1:28 Okay, now to the AGI bubble. Well, Dwarkesh Patel is famous for pushing the AGI narrative. Even during this interview, he asked Andrej Karpathy at least three times in different wordings about AGI and the timelines. Karpathy’s entire framing of the current moment is that it’s early, impressive, and cognitively deficient. He repeatedly says during the interview, “they just don’t work, they’re cognitively lacking” and “it’s still the decade of agents, not a year.”

1:59 At some point he even says “I almost want to reject the question entirely.” Quote: “I’m almost tempted to reject the question entirely because again, I see this as an extension of computing. Have we talked about how to chart progress in computing, or how do you chart progress in computing since the 1970s or whatever? What is the x-axis? So I kind of feel like the whole question is kind of funny from that perspective a little bit.”

2:17 And since currently we are entering another wave of “oh no, this is AI bubble,” I think this interview proves the opposite. What it actually shows is that we should stop with the AGI narrative maybe altogether, because if there is any bubble at all, it’s an AGI bubble. Too much promise around a vague concept we still don’t know how to define.

2:41 Even Sam Altman, who first pushed the AGI and then the superintelligence narrative, recently said “I think the point of all of this doesn’t really matter. It’s just this continuing exponential of model capability that we rely on for more and more things.”

2:56 Karpathy’s own definition goes back to the original OpenAI definition that they came up with back then: “AGI was a system you can go to that can do any task that is economically valuable - any economically valuable task at human performance or better. And the important word here is ‘any’ - like picking up the phone and actually calling someone, and actually standing up and picking up the trash and taking it to the garbage bin. Not there yet.”

3:24 And this definition is a perfect way to see why we’re not there yet, because most economically valuable tasks are not being done by AI today. And what’s happening is automation of fragments. We do code suggestions, text generation, image synthesis, summarization. These are tools but not general minds as in general intelligence.

3:51 And Karpathy calls these current systems ghosts, digital spirits imitating humans. He says they talk like us, sound like us, and even appear to reason like us, but they don’t remember, they don’t reflect, and they don’t actually know anything. And I think the better name for them is mirrors or avatars, not ghosts since it assumes some spirituality.

4:13 And all of it is fine because that’s AI. As Karpathy many times said during the interview, AI is a powerful continuation of computing. It’s a new form of automation, one that rewires human work, but not one that replaces human cognition. And he explains very interestingly how in terms of how the brain works - we might need a little bit of understanding how the brain works, but there is so much more to explore and we don’t even know how our own brain works.

4:44 So we’re not even close to building it and the whole cognition part. And when people look at these tools and say we are close to AGI, they’re making the same mistake we’ve made before - confusing the first 90% of a demo with the last 10% of a product. And it was funny that Karpathy says that he hates demos and I totally agree with him because they’re not representative. They do not actually tell you if the product will work.

5:10 Karpathy calls it the march of nines, and he says that every new nine - like 99%, 99.9%, 99.99% - costs the same amount of work as all the previous progress combined. So it’s still a huge amount of work. That’s why self-driving took decades and is still taking time to build. It’s not there yet also. That’s why agents will too take a lot of time to be fully autonomous, if ever.

5:38 At some point he calls reinforcement learning terrible, and that line went viral. But he later clarified he wasn’t dismissing reinforcement learning entirely, just pointing out that it’s one layer of a multi-layered solution: base model autocomplete, then fine-tuning for style, then reinforcement learning for behavior. And then as he said, we’ll still need layers four, five, six and so on.

6:03 The same as with the analogy of the brain. There is one layer we uncovered and tried to replicate. There are other layers. There’s so much more. There are new ideas that we need to be able to build better AI and to understand our brains better. So that’s his point. We’re not done. We’re layering cognition, not discovering it.

6:23 This is another way of saying that the process we’re betting the entire AGI narrative on - the self-improving loop of intelligence building intelligence - still doesn’t work. So where are we actually? We are in what Karpathy says is “the decade of agents, not the year” - 10 years of hard engineering, iteration, data refinement, memory systems, multimodal cognition, and safety alignment.

6:48 At some point he says that datasets are absolutely terrible still and that will require a lot of work to make datasets much more accurate and precise, because now all the models are built on the whole internet and it’s just as noisy as hell. So AGI is still a myth we keep redrawing on the horizon every few years.

7:09 And if the AI bubble narrative misunderstands progress, the AGI bubble misunderstands intelligence. Karpathy’s version of the future is not one of a singularity - basically all very close to what Yann LeCun thinks - but it’s a slow continuous automation curve. The same exponential that began with steam engines, electricity, extended through computers, and now continues through language models.

7:38 He even says that he tried to find iPhone or computer in GDP but he couldn’t find it, the same way he couldn’t find AI. And I think we might see it now, but it’s still to be proven. Yeah, I would say exactly that - quietly emerging into everything.

7:52 This is the truth that Silicon Valley doesn’t like because it’s not cinematic. Sorry Dwarkesh and the crew. But it doesn’t end with a sentient assistant or a conscious cluster. It’s a technology. It’s an algorithm. Layers of different algorithms. Some of them we still didn’t know, we still didn’t invent, we still didn’t discover. And it just ends up with a lot of hard work, better infrastructure, and smarter tools.

8:19 So yes, it’s an AGI bubble. We’ve inflated too much meaning into an unfinished mirror. But AI itself, AI is fine. AI is exactly what it’s supposed to be. It’s evolving. It’s diffusing. It’s blending into the economy, massively blending into economy and our daily workflows. It’s not breaking reality. It’s upgrading it one autocomplete at a time.

8:42 What Karpathy actually worries about, what does concern him, is not that AI fails, but that it succeeds too invisibly. He says the most likely future is not a sudden takeover, but a gradual loss of understanding. We’ll keep layering automation until the systems run faster than our comprehension. And that’s a little scary because people can just get lazy, and the only way to deal with it is education.

9:10 He currently works on his educational project called Eureka, Eureka Labs. And he wants to create some sort of a Starfleet Academy. But the point is that I actually think this is the part of the interview that starts around 1 hour 58 minutes, which is the most important part of the interview, because there he talks about how education is transforming, how important and crucially vital it is for the future society with evolving AI.

9:39 And here comes the concept of AI aristocracy. I’ve been thinking about this concept for a while. At some point, Andrej Karpathy mentions that if we think about aristocrats or Romans, they had these pockets of time where they could evolve their knowledge, dedicate to things.

9:56 I just go back a little bit historically because I think this is a very important point. Historically, the aristocrat was defined by the absence of necessity. Freed from physical labor, aristocrats cultivated what they considered higher human faculties: education, rhetoric, governance, aesthetics, and philosophy. Leisure was not idleness. It was a technology of reflection. The ruling class was trained to think because others were compelled to work.

10:25 I don’t want to idolize it. That structure produced the birth of civilizations, high culture, but also the moral corruption of privilege. Later with industrialization, that dynamic inverted. Education was democratized but reduced to function: reading, arithmetic, functionality, compliance. Mass schooling replaced the tutors. Curiosity gave way to curriculum.

10:48 The goal is no longer cultivation but coordination - creating workers for the machine age. And there, the aristocratic ideal of learning for life became a luxury. Even today the system remains built to feed production, not introspection.

11:04 AI returns us to the condition of abundance but with a crucial difference: the servants are not sentient. And when machines perform manual and cognitive labor, the human role cannot remain economic. It must become philosophical. Also, to have control over this evolving AI, humans need knowledge. They need curiosity. They need understanding of how things work.

11:33 So in a sense, what we have now completes a historical loop. For thousands of years, education was a privilege. The aristocrats studied because others worked. The industrial worker was educated only to function. Now AI automates necessity again - at least this is the hope. And the vacuum that opens must be filled by meaning, by learning that is not instrumental but existential.

11:58 Karpathy mentions Wall-E, a cartoon where people are on this spaceship and they’re fed, they don’t even know what to wish for. They don’t know who they are. They’re just fed by robots. This laziness should be fought with education and knowledge. We’re just in the very, very beginning of it. But this is a crucial point to start building the system, to start building this understanding that we need new education.

12:23 We soon will have this abundance of workers that free our time, and this time can be used as other privileged groups used before - for education. Because when we think about aristocracy, nobles were expected to cultivate themselves so they could govern others.

12:45 And that’s exactly the same point with AI. We humans will need to cultivate ourselves in a very different educational paradigm, in a very different educational approach that will boost our curiosity, to cultivate ourselves to be able to govern AI. So such change is needed and will be a civilizational alignment.

13:11 It’s a big thing that maybe almost no one even thinks about it, but people like Karpathy - and that’s why he’s so important, he’s such a great thinker because he looks ahead. He looks a few decades further and we just need to listen to people like this and start re-channeling our resources and our attention to that.

13:32 So yeah, going back to AGI, it’s a bubble. We are overpricing the idea of general intelligence, but there is no AI crash coming. It’s still a lot of work to do. And while we’re doing that, we need to understand how we catch up education-wise. It’s not a superintelligence emerging. It’s a super educated civilization we need to become.

13:53 Thank you for listening. I hope to see your comments. I’m ready to continue this discussion and I keep working on educating about AI because I think this is a crucial part of our future success as humans and as a civilization. Every subscription and share is noticed.