Why I don’t think AGI is right around the corner

Summary

Dwarkesh Patel presents his personal assessment of AGI timelines as of July 2025, arguing against the bullish projections of many of his recent podcast guests. His central thesis is that LLMs fundamentally lack continual learning — the ability to build up context, improve organically on the job, and learn from experience the way humans do. While LLMs are impressive at short-horizon, language-in-language-out tasks, they cannot learn and adapt over time, which Dwarkesh considers the key bottleneck preventing them from replacing human workers at scale.

He extends this argument to computer use agents, noting that we lack the pretraining data corpus for computer use that existed for language, and that longer-horizon tasks compound errors in ways that are fundamentally harder to solve. Despite his skepticism about near-term AGI, Dwarkesh emphasizes he is not dismissing AI’s potential — he takes a 50/50 bet that transformative AI arrives this decade, and believes the timelines are lognormally distributed, meaning it is either this decade or potentially not until the 2030s or 2040s.

The video is adapted from a blog post Dwarkesh wrote after disagreeing with bullish AI timeline forecasts from guests on his podcast, including Anthropic researchers and others who predicted reliable computer use agents by 2027.

Highlights

”The saxophone analogy for why LLMs can’t learn”

yt-dlp --download-sections "*2:17-3:10" "https://www.youtube.com/watch?v=nyvmYnz6EAg" --force-keyframes-at-cuts --merge-output-format mp4 -o "nyvmYnz6EAg-2m17s.mp4"

“How would you teach a kid to play the saxophone? They’d see how it sounds, and they’d adjust. Now imagine trying to teach saxophone instead: A student reads your notes and when they fail, you refine your instructions. It wouldn’t work. No matter how well honed your instructions, nobody can learn to play saxophone from reading your instructions.” — Dwarkesh Patel, 2:17

”Even Claude Code reverses hard-earned lessons”

yt-dlp --download-sections "*5:12-6:10" "https://www.youtube.com/watch?v=nyvmYnz6EAg" --force-keyframes-at-cuts --merge-output-format mp4 -o "nyvmYnz6EAg-5m12s.mp4"

“Even Claude Code will often reverse a hard-earned lesson after you hit /compact — because the explanation for why something was done gets compressed away. I just think that titrating all this rich feedback into a compact format will be brittle in domains outside of software.” — Dwarkesh Patel, 5:12

”Have you read the reasoning traces from o3?”

yt-dlp --download-sections "*11:30-12:50" "https://www.youtube.com/watch?v=nyvmYnz6EAg" --force-keyframes-at-cuts --merge-output-format mp4 -o "nyvmYnz6EAg-11m30s.mp4"

“Have you read the reasoning traces from o3? It’s breaking down a problem, thinking through what the user wants, and correcting itself when it notices something is off. How are we just like, ‘Oh yeah of course a machine can do that’? At this point, part of you has to step back and say: We’re making machines that are intelligent.” — Dwarkesh Patel, 11:30

”AI timelines are lognormal”

yt-dlp --download-sections "*15:04-16:10" "https://www.youtube.com/watch?v=nyvmYnz6EAg" --force-keyframes-at-cuts --merge-output-format mp4 -o "nyvmYnz6EAg-15m04s.mp4"

“I am forecasting a pretty wild world within the next decade. The timelines are very lognormal. It’s either this decade or it could be the 2030s or even the 2040s. AI progress has been driven by scaling training compute. It’s been over 4x a year. After 2030, AI progress has to mostly come from algorithmic improvements.” — Dwarkesh Patel, 15:04

”I’d take a 50/50 bet on doing my taxes by 2028”

yt-dlp --download-sections "*12:59-14:00" "https://www.youtube.com/watch?v=nyvmYnz6EAg" --force-keyframes-at-cuts --merge-output-format mp4 -o "nyvmYnz6EAg-12m59s.mp4"

“I’d take a 50/50 bet that AI can do my taxes end-to-end for my small business, including chasing down all the receipts, emailing back and forth with anyone who we need invoices from, and sending it to the IRS. This I’d say 2028.” — Dwarkesh Patel, 12:59

Key Points

• Continual learning is the core bottleneck (0:00) - LLMs don’t learn on the job the way humans do; you’re stuck with whatever abilities the model was trained with
• LLMs are 8/10 at short-horizon tasks (1:18) - They excel at language in, language out tasks but are only 5/10 at tasks requiring deeper understanding
• The saxophone analogy (2:17) - You can’t learn saxophone from reading instructions, just as LLMs can’t truly learn from prompt engineering
• RL fine tuning is not the same as learning (2:59) - While RL exists, it’s not continuous or organic like human learning
• Writing assistance hits a wall (4:17) - LLMs can help with outlines but can’t learn your preferences and style over time
• Disagreement with Trenton Bricken (5:32) - Dwarkesh pushes back on the claim that even stalled AI progress would be transformative
• Less than 20% of labor gets automated without continual learning (6:00) - Without the ability to build context, AIs can’t serve as actual employees
• Intelligence explosion requires continual learning (6:55) - Once solved, AI agents could share learnings across copies and rapidly learn every job
• Computer use agents face two problems (8:58) - Error compounding over longer horizons and lack of pretraining data
• Quote from Mechanize on data scarcity (9:28) - “For the past decade of scaling, we’ve been blessed with freely available data” — computer use has no such corpus
• DeepSeek R1 took 2 years after obvious insight (10:36) - Even seemingly simple innovations require enormous engineering effort
• We’re making machines that are intelligent (12:38) - Despite skepticism, Dwarkesh acknowledges the genuine impressiveness of current models
• 50/50 bet on AI doing taxes by 2028 (12:59) - His concrete prediction for reliable computer use
• 2028 as GPT-4 moment for computer use (13:53) - Expecting something impressive but not yet practically useful at that scale
• Lognormal AI timelines (15:04) - Either this decade or the 2030s/2040s, with training compute scaling maxing out around 2030
• After 2030, algorithmic improvements must drive progress (15:48) - Hardware scaling has limits; low-hanging fruit in algorithms may already be picked

Mentions

Companies

• Anthropic (8:08) - Researchers predicted reliable computer use agents by 2027
• DeepSeek (10:36) - Their R1 RL procedure took 2 years from concept to execution
• Mechanize (9:28) - Quoted on data scarcity for computer use

Products & Technologies

• Claude Code (5:12) - Example of how context compaction loses important lessons
• o3 (11:30) - Reasoning traces show impressive problem-solving abilities
• GPT-4 (9:48) - Referenced as a milestone for language; computer use needs its own GPT-4 moment
• GPT-1 (14:53) - Came out 7 years ago, setting pace for what might happen in next 7

People

• Trenton Bricken (5:36) - Anthropic researcher whose claim about stalled AI progress Dwarkesh disagrees with
• Sholto (5:55) - Referenced in context of data collection filling capability gaps
• Stephen Kotkin (16:46) - Dwarkesh mentions reading thousands of pages of his writing after their interview

Surprising Quotes

“If AI progress totally stops today, I think less than 20% of labor gets automated. Sure, many tasks will get automated. But their inability to build up context will prevent them from serving as actual employees at your firm.” — 6:00

“I am not saying that there won’t be impressive demos in 2026 and 2027. GPT-4 was impressive, but it was not that practically useful. I expect 2028 to be for computer use what GPT-4 was for language.” — 14:00

“The RL procedure which DeepSeek explained in their paper seems obvious in retrospect. And yet it took 2 years from the development of the idea to execution. That’s precisely my point — we’re underestimating the difficulty of solving problems when you’re operating in a totally different modality.” — 10:36

“Part of the reason some people are too pessimistic is they haven’t played around with the smartest models. Giving Claude Code a vague spec and just watching it zero-shot a working application is a wild experience.” — 12:06

Transcript

0:00 I’ve had a lot of discussions on my podcast about when AGI will arrive. Some guests think it’s 2027, others think it’s 2030 or later. Here’s where my thoughts lie as of July 2025. Even if AI progress totally stopped, the systems we have today would still be more transformative than the internet. I disagree. But the reason that the Fortune 500 aren’t using AI to replace workers isn’t because the management there is too stodgy. It’s because you still can’t get normal humanlike labor out of these LLMs. There are real capabilities these models lack.

0:46 I spend dozens of hours trying to build these little LLM automations here at the Dwarkesh Podcast. And the experience of trying to get them to work has shaped my views. I’ll try to get an LLM to rewrite autogenerated transcripts in the way a human would. Or I’ll try to get it to identify clips from a transcript that I feed in. Or I’ll ask it to go through an essay with me, passage by passage. These are short horizon, language in, language out tasks — the sweet spot in the LLMs’ repertoire. And they’re 5/10 at many of these.

1:30 But the fundamental problem is that LLMs don’t learn on the job. This lack of continual learning is the core handicap. The LLM baseline at many tasks might be 7/10 or even 8/10. But there’s no way to give it feedback and have it improve. You’re stuck with the abilities it was trained with. You can keep messing around with the system prompt, but you’ll never get anything even close to the kind of learning and adaptation that humans do.

2:04 The reason humans are so useful as employees is not their raw intelligence. It’s their ability to build up context, to learn your preferences, and pick up small improvements and adjustments over time. How would you teach a kid to play the saxophone? You’d put the instrument in their hands, they’d see how it sounds, and they’d adjust. Now imagine trying to teach saxophone instead: A new student arrives, and the moment they make a mistake, you stop them and write instructions about what went wrong. And the next student reads your notes and tries again. And when they fail, you refine your instructions. This wouldn’t work. No matter how well honed your instructions, nobody can learn to play saxophone from reading your instructions.

2:59 That’s essentially what we’re doing when we try to teach LLMs anything. Yes, there’s RL fine tuning. But it’s not continuous or on-the-job in the way that human learning is. And my best producers wouldn’t have gotten that way if they hadn’t spent months decomposing the different subtasks involved in their work, practicing each one themselves and thought hard about what kind of content I like, and how they can serve the podcast better.

3:30 Now, it’s possible to imagine ways in which each model could have its own dedicated RL loop for itself which just runs in the background. I give some high level feedback, and the model generates its own practice problems to RL on — maybe even a whole curriculum targeting skills that it thinks it’s lacking. And I don’t know how well these approaches will work across different kinds of tasks and feedback. But I doubt they can substitute for learning on the job in this organic way that humans can. And I doubt that this will happen within the next few years, given that there’s no obvious architecture into which to slot in continuous learning.

4:14 Now sometimes LLMs actually do get kinda smart and learn in context. For example, sometimes I’ll be working on a blog post. I’ll give it an outline, and I’ll ask it to suggest changes paragraph by paragraph. And all its suggestions up till now have been bad, so I’ll just rewrite every single paragraph from scratch and say “This is what I wrote instead.” And then it’ll start giving good suggestions for the next paragraph. But internalizing my preferences and style will just never happen through this mechanism.

4:47 Maybe there is an easy solution to this that I’m not seeing. Something like what Claude Code already has, which just compacts the conversation into a CLAUDE.md file. I just think that titrating all this rich feedback into a compact summary will be brittle in domains outside of software engineering, in which you already have this external scaffold of tests and linters. Again, think about what it would be like to teach someone saxophone through written instructions only.

5:12 Even Claude Code will often reverse a hard-earned lesson the moment you hit /compact — because the explanation for why something was done a certain way gets compressed away. This is why I disagree with something that Trenton Bricken said on my podcast. He said “Even if AI progress totally stalls, the capabilities are really spiky, and they don’t have general competence, each individual task is sufficiently well-defined and sufficiently easy to collect data on, such that to Sholto’s point we should expect to fill in all those gaps.”

6:00 If AI progress totally stops today, I think less than 20% of labor gets automated. Sure, many tasks will get automated. LLMs already rewrite autogenerated transcripts for me. But I’m never going to be able to have it improve over time and learn my preferences and style. So even if we get more data, without progress on continual learning, we’ll be in a substantially similar position with respect to AI. Yes, technically AIs will be able to do more and more individual tasks, but their inability to build up context will prevent them from serving as actual employees at your firm.

6:48 If I’m wrong about the difficulty of getting transformative AI in the next few years, it makes sense to prepare now. When we do solve continual learning, we’ll see a dramatic phase shift. Even if there isn’t a software-only intelligence explosion where models build smarter and smarter successor systems, we’ll still see a broadly deployed intelligence explosion. AI agents will be out in the economy, doing different jobs and learning how each one works. However, unlike humans, these agents could share their learnings across all their copies. So you’d rapidly converge on an AI that knows how to do every single job in the economy. An AI with continual learning might rapidly become a superintelligence in that way.

7:42 However, I’m not expecting to watch some YouTube live stream where continual learning has been totally solved. If the labs come up with these innovations quickly, we’ll see a broken early version of continual training, or whatever you want to call it, before the polished final version. I expect to get lots of heads up before the singularity.

8:08 When I interviewed Anthropic researchers on my podcast, they said that they expect reliable computer use agents by 2027. We already have computer use agents right now but their forecast is something quite different. Their forecast is that you’ll be able to tell an AI, “Go do my taxes.” And it’ll go through your Amazon orders, and Slack messages, and figure out which person you need to get invoices from, it’ll figure out which of your expenses are actually business expenses, and it will fill in the right tax forms and then will just submit Form 1040 to the IRS. They know a lot more than I do so it’s hard for me to contradict them on technical details.

8:58 But I have two arguments against this forecast. One: As horizon lengths increase, errors compound multiplicatively. The AI needs to do two hours worth of work before you can even see if it did it right. The tax task requires processing images and video, working across multiple applications — and that’s even if you don’t factor in the longer rollouts.

9:22 Two. We don’t have a large pretraining corpus for computer use. I like this quote from Mechanize’s post: “For the past decade of scaling, we’ve been blessed with huge amounts of high-quality data that was freely available for us to use.” That data was great for language processing, but not for getting models to navigate UIs. Imagine trying to train GPT-4 on all the world’s screenshots — that data would have been nowhere near enough.

9:58 Again, I’m not at the labs. Maybe current models already have a great prior over how different UIs work, and what buttons to click and where. Maybe RL fine tuning is so sample efficient that it can learn computer use quickly. But I haven’t seen any public evidence which suggests that these models have gotten less data hungry, especially in domains outside of language. Alternatively, maybe these models are such good generalizers that they can generate millions of toy UIs for themselves and learn from that synthetic data.

10:36 But consider that innovations which seem quite simple in retrospect often take years to actually implement. The RL procedure which DeepSeek explained in their R1 paper seems obvious after the fact. And yet it took 2 years from the development of the core idea to actually making it work. Now of course I know that it’s insanely and enormously difficult — it requires a ton of engineering, debugging, and pruning of the reward function to get this solution. But that’s precisely my point! If even the apparently simple idea of “We should train a model to solve verifiable math problems with RL” took that long, we’re underestimating the difficulty of solving computer use when you’re operating in a totally different modality.

11:30 Now I’m not going to be like one of those spoiled children who got a golden-egg-laying goose and would still spend all their time complaining. Have you read the reasoning traces from o3 or Claude? You can watch the model breaking down a problem, thinking through what the user wants, exploring different approaches, and correcting itself when it notices that something doesn’t work. How are we just like, “Oh yeah of course a machine can do that”? It can brainstorm with a bunch of ideas, and come back to me with something genuinely novel.

12:06 Part of the reason some people are too pessimistic about AI is they haven’t played around with the smartest models in their most capable modes. Giving Claude Code a vague spec and just watching it zero-shot a working application is a wild experience. And the simplest explanation for why reasoning models produce such concise, and accurate explanations is simply that they understand the material. At this point, part of you has to step back and say: “We’re making machines that are intelligent.”

12:50 And I want to emphasize that I do think AGI is coming. Which means that work to prepare for a world with transformative AI is extremely important. I think this is a totally plausible outcome. I’d take a 50/50 bet. That AI can do my taxes end-to-end for my small business as well as a human accountant, including chasing down all the receipts on Amazon and elsewhere, emailing back and forth with anyone who we need invoices from, and sending it to the IRS. This I’d say 2028. I think that’s aggressive.

13:28 But we have no pretraining corpus, and the RL needs to learn to optimize reward over a much longer time horizon using a completely different modality. That being said, the base model is extremely smart and probably already has a good prior over computer use tasks, and the labs have some of the best researchers in the world, so it might even out. I expect 2028 to be for computer use what GPT-4 was for language.

14:00 Just to clarify, I am not saying that there won’t be impressive demos in 2026 and 2027. GPT-4 was very impressive, but it was not that practically useful. I’m talking about the level of end-to-end handling a week-long and quite complex task reliably.

14:19 Ok, and as for the forecast of when AI will be able to learn organically, seamlessly, and quickly from experience on the job — for example, if I hired an AI video editor, after a few weeks it would have a deep understanding of my preferences, my style, my audience, and edit as well as a human would.

14:44 While I don’t see an obvious way to build continual learning into the kinds of models these LLMs currently are, GPT-1 had just come out this time 7 years ago. I would not bet against the possibility that in the next 7 years, we’ll find some way to get models to learn continuously.

15:04 At this point you might be reacting, “This doesn’t sound like someone who thinks continual learning being such a huge handicap. You’re describing a world not that far away from what, at a minimum, looks like a broadly deployed superintelligence.” And you’d be right. I am forecasting a pretty wild world within the next decade. The timelines are very lognormal. It’s either this decade or possibly the 2030s or even the 2040s (technically like a 15% probability per year — but that’s less catchy).

15:35 AI progress so far has been driven by scaling training compute. It’s been over 4x a year. But that rate of scaling is slowing whether you look at chips, power, or even the willingness of investors. After 2030, AI progress has to mostly come from algorithmic improvements. But even there all the low-hanging fruit may have been picked, at least under the deep learning paradigm. This means that if we end up on the longer side of the timeline distribution, we could have a relatively normal world up till the 2030s or even the 2040s.

16:15 Regardless of when you think AGI arrives, thinking carefully about the current limitations of AI helps us prepare for and navigate the transition. This was originally a blog post that I wrote on my Substack. And it was obviously inspired by the discussions I had with recent podcast guests where I ended up disagreeing with them about their AGI timelines. I often write blog posts afterwards, sorting out exactly where I agree and disagree. And I do this for other episodes as well. For example, after the Stephen Kotkin episode I read thousands of pages that Stephen Kotkin has written — things I couldn’t exhaustively cover in that one 2-hour interview. If you want to see the artifacts and writing that I produce as a result of these conversations, you should subscribe to my blog and newsletter. Otherwise I will also see you next week on the podcast.