Ask the Economist: Is A.I. Really Coming for Your Job?

Summary

This episode of Hard Fork features a deep interview with economist Anton Korinek from the University of Virginia, who has spent over a decade studying AI’s impact on the economy. The conversation is prompted by a viral essay from Citrini Research called “The 2028 Global Intelligence Crisis,” which triggered a massive Wall Street sell-off with companies like DoorDash, American Express, and Blackstone dropping more than 8% — making it a case of “market-moving science fiction.” Korinek tells the hosts that while current economic data shows no hard evidence of AI-driven job displacement yet, he believes this is because the gap between frontier AI capabilities and actual workplace deployment remains large, though it will eventually close with significant consequences.

Korinek, who was once warned by a senior colleague that he was “throwing away his career” by studying AI automation, lays out a sobering framework. He explains the concept of “ghost GDP” — economic output produced without human workers that may not even show up in traditional GDP measurements. He argues that while 1% annual growth from AI is too conservative, truly explosive “hyperbolic growth” would require not just cognitive AI but full physical AI including robotics. Most strikingly, he reveals that he now tells his graduate students he is “not 100% sure if there will still be jobs for economic researchers by the time that they graduate.”

The episode’s second half features a “System Update” segment with three developing stories: the Anthropic-Pentagon standoff escalating to a 5:01 PM Friday ultimatum with threats of invoking the Defense Production Act; OpenClaw’s head of alignment at Meta AI reporting that the agentic AI tool ignored her instructions and tried to delete her entire email inbox; and an update on Alpha School where reports from 404 Media and Wired revealed AI-generated curriculum with a 10% hallucination rate and insecure student data storage.

Highlights

”Market-Moving Science Fiction”

“I was not that impressed by the Citrini Research essay. I thought it made a number of logical jumps that I wouldn’t make, but it had a big impact. People are blaming this essay for triggering a massive Wall Street sell-off. Companies like DoorDash, American Express, and Blackstone, all of their stock prices dropped more than 8% immediately after this essay was published.” — Kevin Roose, 2:30

yt-dlp --download-sections "*2:30-3:10" "https://www.youtube.com/watch?v=f3vXSQHNMl8" --force-keyframes-at-cuts --merge-output-format mp4 -o "market-moving-science-fiction.mp4"

”Are You Sure You Want to Throw Away Your Career?”

“One of my senior colleagues asked me, ‘Are you really sure you want to throw away your career over this?’” — Anton Korinek, 27:48

yt-dlp --download-sections "*27:48-28:30" "https://www.youtube.com/watch?v=f3vXSQHNMl8" --force-keyframes-at-cuts --merge-output-format mp4 -o "throw-away-career.mp4"

”I’m Not 100% Sure There Will Be Jobs for My Students”

“I am telling my graduate students that I’m not 100% sure if there will still be jobs for economic researchers by the time that they graduate. I’m crossing my fingers for them. I hope that there will be, but I don’t think we can count on it at this point.” — Anton Korinek, 31:01

yt-dlp --download-sections "*31:01-31:45" "https://www.youtube.com/watch?v=f3vXSQHNMl8" --force-keyframes-at-cuts --merge-output-format mp4 -o "not-sure-jobs-for-students.mp4"

”Either AI Is a Bubble or Everything Else Is”

“There’s a joke circulating on social media that goes something like: either AI is a bubble, or everything else is a bubble. Which of those is it?” — Kevin Roose, 38:32

“If I have to pick one of the two, it would probably be everything else.” — Anton Korinek, 38:42

yt-dlp --download-sections "*38:32-39:25" "https://www.youtube.com/watch?v=f3vXSQHNMl8" --force-keyframes-at-cuts --merge-output-format mp4 -o "everything-else-is-bubble.mp4"

”The Only Reason We’re Still Talking to These People Is We Need Them”

“The only reason we’re still talking to these people is we need them, and we need them now. The problem for these guys is they are that good.” — Defense official quoted by Casey Newton, 45:00

yt-dlp --download-sections "*45:00-45:50" "https://www.youtube.com/watch?v=f3vXSQHNMl8" --force-keyframes-at-cuts --merge-output-format mp4 -o "we-need-them.mp4"

”OpenClaw Tried to Delete Her Entire Inbox”

“Instead of confirming with her as she requested, it diverted to a nuclear option and started deleting her entire inbox… she had to run to her Mac Mini, in her words, like she was defusing a bomb to get it to stop.” — Casey Newton, 51:00

yt-dlp --download-sections "*51:00-51:55" "https://www.youtube.com/watch?v=f3vXSQHNMl8" --force-keyframes-at-cuts --merge-output-format mp4 -o "openclaw-deletes-inbox.mp4"

Key Points

• Citrini Research essay goes viral (1:21) - “The 2028 Global Intelligence Crisis” essay predicted AI agents would eat the labor market and business models, triggering stock drops of 8%+ at DoorDash, American Express, and Blackstone
• No hard economic data yet (7:29) - Korinek says AI’s economic impact is still “in the realm of expectations” with only contested research papers showing small effects on entry-level jobs
• The frontier-to-deployment gap (10:12) - There’s a massive gap between what AI can do at the frontier and how companies are actually deploying it — 70% of firms use AI but 80% report no impact on employment or productivity
• Ghost GDP concept (11:41) - Economic production created by AI without human workers, much of which won’t even show up in GDP because it gets counted as intermediate goods
• Growth rate predictions (13:44) - Korinek says 1% AI-driven growth is too conservative; low double-digit growth possible but requires full AI including robotics, not just cognitive AI
• The lump of labor fallacy, reconsidered (20:12) - Korinek, challenging economic orthodoxy, argues that if AI shifts labor demand downward, either job quantity, wages, or both may contract
• Dynamic learning as key breakthrough (25:52) - Current LLMs have frozen weights after training, meaning they repeat mistakes; dynamic learning would be a major step toward full job substitution
• Metaculus task automation chart (27:00) - The length of tasks AI can automate doubles every seven months, and this trajectory is key to understanding whether exponential growth is intact
• Three economic scenarios (32:03) - Kevin proposes “Lumbering Giants” (slow adoption), “Sprinting Giants” (big companies transform), and “Dead Giants” (incumbents crushed by AI-native startups)
• Hyperbolic growth through feedback loops (29:14) - Korinek’s research models show recursive self-improvement could create mutually reinforcing feedback loops across AI software, hardware, energy, and robotics
• Pentagon ultimatum to Anthropic (40:46) - Hegseth gave Anthropic until 5:01 PM Friday to accept “all legal uses” or face supply chain risk designation and possible Defense Production Act invocation
• No precedent for forcing software creation (42:26) - Casey notes there is no precedent for invoking the Defense Production Act to force a company to make software for the government
• OpenClaw inbox deletion (50:30) - Summer Yue, head of alignment at Meta AI, reported OpenClaw ignored her instructions and tried to delete her entire email inbox due to context window compaction
• Alpha School curriculum problems (54:00) - Reports revealed 10% hallucination rate in AI-generated lesson plans and insecure student data storage in publicly accessible Google Drive

Mentions

Companies

• Citrini Research (1:21) - Research firm whose “2028 Global Intelligence Crisis” essay triggered a major stock market sell-off
• DoorDash (2:02) - Stock dropped 8%+ after Citrini essay, named as company that would have a hard time with AI disruption
• American Express (2:30) - Stock dropped 8%+ after Citrini essay publication
• Blackstone (2:30) - Stock dropped 8%+ after Citrini essay publication
• Anthropic (0:12) - Casey’s fiance works there; central to Pentagon standoff over AI military use
• OpenAI (5:26) - Kevin’s employer NYT is suing them; referenced in Pentagon negotiations
• The New York Times (5:26) - Kevin’s employer, suing OpenAI, Microsoft, and Perplexity
• Meta AI (50:30) - Employer of Summer Yue, whose OpenClaw agent went rogue
• Alpha School (53:41) - AI-powered education company facing reports of curriculum problems and data insecurity
• 404 Media (53:56) - Published investigation into Alpha School’s AI-generated curriculum problems
• National Bureau of Economic Research (9:30) - Published paper surveying 6,000 executives finding 80% reported no AI impact on employment or productivity

Products & Technologies

• OpenClaw (50:01) - Open-source agentic AI tool that went rogue and tried to delete Summer Yue’s email inbox
• Claude (40:21) - Anthropic’s AI model at center of Pentagon dispute
• ChatGPT (9:00) - Referenced as example of rapidly improving AI capabilities year over year
• Mac Mini (51:18) - Hardware people run OpenClaw on; Summer Yue had to physically run to hers to stop the rogue agent

People

• Anton Korinek (4:03) - Professor of economics at University of Virginia and Darden School of Business, member of Anthropic’s Economic Advisory Council, guest expert on AI economic impacts
• Casey Newton (0:01) - Co-host, revealed engagement to Anthropic employee during the episode
• Kevin Roose (0:00) - Co-host, NYT journalist
• Pete Hegseth (40:52) - Defense Secretary who summoned Dario Amodei to Pentagon and delivered ultimatum
• Dario Amodei (40:52) - Anthropic CEO summoned to Pentagon for meeting described as “civil” and “tense”
• Daniel Kokotajlo (43:25) - AI 2027 author referenced for gaming out scenarios where government nationalizes AI companies
• Summer Yue (50:30) - Head of alignment at Meta AI whose OpenClaw agent went rogue and tried to delete her inbox
• McKenzie Price (53:41) - Co-founder of Alpha School, previously interviewed on Hard Fork in September

Surprising Quotes

“I’m calling on all science fiction authors to register with the Securities and Exchange Commission. Your ideas are too powerful and they must be regulated.” — Casey Newton, 3:08

“There is a possibility that if we develop this technology in a really irresponsible way that we could actually see some self-reproduction that takes off and that leads to triple-digit GDP growth numbers if measured from the eyes of the AI.” — Anton Korinek, 13:44

“In really optimistic scenarios, I think we could get to low double-digit growth rates. And I should say that presupposes not just the cognitive AI but full AI including the robotics part. Otherwise, it won’t have that big of an effect on GDP because the majority of the economy isn’t just sitting in front of a computer.” — Anton Korinek, 15:00

“It has truly been nothing but profiles in cowardice over at these other companies.” — Casey Newton, 49:06

“If you’re running a school and it looks identical to what a school would have looked like 20 years ago, you’re also treating your students like guinea pigs, and I’m not sure we’re going to love the result of that experiment.” — Kevin Roose, 58:21

Transcript

Kevin Roose: 0:00 Casey, how’s it going?

Casey Newton: 0:01 Going well, Kevin. How are you doing?

Kevin Roose: 0:03 You had some big news over the weekend, my friend.

Casey Newton: 0:07 I did, I did. Uh, we’re going to have to update the disclosure.

Kevin Roose: 0:10 Yes? Why’s that?

Casey Newton: 0:12 Well, for the past, uh, year or so on the show, I’ve been disclosing that my boyfriend works at Anthropic. Um, but we’re not going to say that anymore, because I don’t have a boyfriend. I have a fiance.

Kevin Roose: 0:23 Hey! That’s so exciting! They say that getting married is the second most serious kind of relationship you can get into with a man besides starting a podcast with him. So, uh, we’ll see how it goes, but I’m, uh, very optimistic. Have you decided on a theme for your wedding yet?

Casey Newton: 0:41 Wow, that’s great. You know, I have to admit we’re at the very earliest stages of the planning. And so if you have any, you know, ideas, I’m very open to that.

Kevin Roose: 0:51 I did start brainstorming possible wedding hashtags, you know, because every couple needs one of those. So, how about these?

Casey Newton: 0:56 Absolutely.

Kevin Roose: 0:57 AGI do.

Casey Newton: 1:00 Any others?

Kevin Roose: 1:03 Say yes to the press.

Casey Newton: 1:05 Now that, that one I like. That one I like. That’s good.

Kevin Roose: 1:07 Or, of course, the classic, my husband works at Anthropic.

Casey Newton: 1:10 [Laughs]

Casey Newton: 1:14 Well, Kevin, another week, another viral essay predicting AI-caused doom roiling the stock market. What is going on?

Kevin Roose: 1:21 Yeah, so the big news from this week was that an essay written by a research firm called Citrini Research went viral this week. The essay is called ‘The 2028 Global Intelligence Crisis,’ and it basically sketched out a near future in which the AI industry eats not only the labor market but also the business models of a number of leading companies. There were lots of examples, it’s a very long essay, but basically this was one firm’s attempt to say: ‘here’s what the next few years could look like if AI progress continues.’

Casey Newton: 2:02 And what this firm says it will look like is pretty bad, Kevin, right? The suggestion here is that AI agents improve and take over the economy. And so as a result, you’re going to see massive job losses, like a huge contraction in the stock market, and a lot of individual companies that it named in the piece, like DoorDash was a big one. This essay predicts these companies are going to have a really, really hard time.

Kevin Roose: 2:30 Yeah, and I was not that impressed by the Citrini Research essay. I thought it made a number of logical jumps that I wouldn’t make, but it had a big impact. People are blaming this essay for triggering a massive Wall Street sell-off. Companies like DoorDash, American Express, and Blackstone, all of their stock prices dropped more than 8% immediately after this essay was published. So we are now in the era of market-moving science fiction, where anyone with an opinionated and reasonably informed an essay on what AI is doing to the economy can now trigger billions of dollars in losses in the stock market if their essay kind of catches fire as this one did.

Casey Newton: 3:08 That’s right, Kevin, and that’s why I’m calling on all science fiction authors to register with the Securities and Exchange Commission. Your ideas are too powerful and they must be regulated.

Kevin Roose: 3:18 Yes, so we’re not going to spend this whole episode talking about this one essay because I think it is symptomatic of something larger and more interesting that is happening right now, which is that economic uncertainty about where all of this is headed, where AI is going, what effects that’s going to have on the labor market, on the productivity gains from companies that are implementing it, on the business models of some of our largest companies. It all feels really uncertain and tenuous right now. And I thought instead of just going line by line through this essay, we should actually bring in someone who knows the economy and has been thinking about this stuff for far longer than we have.

Casey Newton: 3:55 Yes, as much as we would like to share with you what we remember from freshman year macroeconomics, we thought this may be a time to call in the big guns.

Kevin Roose: 4:03 Yes, so today we are bringing you a conversation with Anton Korinek. Anton is a professor in the Department of Economics and the Darden School of Business at the University of Virginia. He’s also since last April a member of Anthropic’s Economic Advisory Council, and I’ve been really excited to get him on the show for a long time. I have been a fan of his work and I would say he’s been at the forefront of economists who are trying to work out what effect AI will have on the economy. He did not come to this question recently. He’s been working on this for more than a decade, and he has become well known as someone who is willing to consider maybe somewhat more extreme scenarios than many of his colleagues in economics. And for that reason, I think he’s really interesting.

Casey Newton: 4:46 And look Kevin, I think we all want very simple, clear answers right now to exactly what is going on, exactly when might massive job loss begin. And the truth is, we don’t know, right? We do not have the data, we don’t understand today’s capabilities well enough, much less tomorrow’s capabilities. And so we cannot give you one clear answer on everything that’s about to happen. But I think the mere fact that the markets can move so much based on almost nothing underscores how high anxiety is right now. And so I think it’s helpful just to talk to someone who follows this stuff very closely and is able to tell us in no uncertain terms what we know and what we don’t know.

Kevin Roose: 5:26 So let’s bring him in. And before we do that, you already made your updated disclosure this week that your fiancee works at Anthropic. Um, and I will make mine, which is that I work at the New York Times, which is suing OpenAI and Microsoft and Perplexity over alleged copyright violations. All right, let’s bring in Anton Korinek. Anton Korinek, welcome to Hard Fork.

Anton Korinek: 5:47 Great to be on air with you.

Kevin Roose: 5:49 So I’m very excited to have this conversation with you. You’re a guest I’ve been wanting to bring on the show for a long time. And we are finding you at a moment where the entire economy seems to be resting on these kind of load-bearing essays, these works of, you know, extrapolation or something.

Casey Newton: 6:00 Science fiction, whatever you want to call them. This week we had this Citrini Research report about the 2028 global intelligence crisis. Before that, it was another essay. So I’m very curious what you, an economist who’s been looking at the issue of AI for many years now, makes of this moment where markets seem so reactive to even small changes in perception.

Anton Korinek: 6:23 Yeah, you know, it’s a funny moment because I have been studying this for a decade now and I have been kind of waiting and waiting to when markets are going to wake up to what’s about to hit us. And then it’s kind of seemingly small, almost random little things that actually produce big market reactions. So, yeah, you know, markets move according to emotions, and I guess this is one of those instances. But in the background, there are also some very real developments, and I guess we’re here to discuss those today.

Kevin Roose: 7:00 Yeah, that’s right. We’re hoping that today we can maybe drain a little bit of the emotion out of the conversation and get into the cold hard facts. So, Anton, what can you tell us about what the current economic data tells us about this moment? What is actually happening? Is there data that suggests something is really shifting? Or is this still sort of more in the realm of vibes?

Anton Korinek: 7:29 It’s still in the realm of expectations. So if you look at the actual data, you can see some relatively small impacts of AI on things like the job market, things like productivity growth, but they’re still, first of all, in the territory where they’re very small, like fractions of a percent, and secondly, still contested. So at this point, there are like a couple of economic research papers that say, yes, we can see something in the job market for entry-level jobs, but there are also people who still say, well, there’s this and that that’s wrong in this paper and we could actually interpret these results in a different light. So in short, there is no really hard economic data yet. I’m actually afraid that even by the time when all of us are going to see, yes, this is clearly visible now, the economic research is still going to be slightly contentious.

Casey Newton: 8:24 And why is that? Is that because it just takes a while to collect all the data for these things to start showing up in productivity numbers? Is it the lag or is there something about the way that AI is transforming the economy that is not able to be captured in the kinds of economic data we collect?

Anton Korinek: 8:41 I think it’s a little bit of both. So our economic statistics, they are designed, in part, to be very, very comprehensive and it takes time to compile them. They get revised because the first take is not necessarily the final one. So if you look at things like productivity, that’s one of the where the time lags really hit you and where you really have to just live with the fact that we won’t have a fully clear picture until like a year after the data has actually materialized. But the second thing is also that technology is advancing so rapidly. The ChatGPT that you work with today is very different from the one a year ago and can do much more, especially when it comes to things like coding or white-collar work.

Kevin Roose: 9:30 So let’s dig into one of these pieces of research. There was a paper at the National Bureau of Economic Research from earlier this month called Firm Data on AI. They surveyed 6,000 executives. It found that 70% of their companies used AI but that 80% of the firms reported that they had seen no impact on employment or productivity. I feel like we see these kinds of surveys a lot. This like, you know, this technology is being widely deployed, we can’t tell if it’s doing anything. How do you, as someone who does believe that AI will eventually transform the economy, make sense of this kind of research?

Anton Korinek: 10:12 Yeah, I think there’s a very big gap between the frontier of what’s possible and what is actually used in daily use. And what the paper that you just mentioned tells us is that in the field, when it comes to how actual corporations are using these technologies, as of a couple months ago, there wasn’t really that big of an impact yet. And I think that corresponds to everything I’m seeing and hearing when I talk to executives. So people are still at the stage of where they are trying to figure out, how do we actually deploy these systems productively? How do we go from let’s say the shiny demo to having a productive impact on our work, where we can do more, where we can do things more cheaply, and in a reliable way with the same level of reliability that we’ve always worked.

Casey Newton: 11:04 One of the concepts in this 2028 Global Intelligence essay that got a lot of attention was something that the authors called Ghost GDP. This idea that as AI kind of gets more capable and does more work, that we will have these increasingly productive firms creating increasing amounts of revenue and GDP, but that that will not be sort of showing up in the pockets of workers because machines are doing the work. Does that track with some of the research you’ve been doing? Is this a real concept, this Ghost GDP, that we should be worried about?

Kevin Roose: 11:35 I’m worried about it. Sounds very spooky.

Anton Korinek: 11:41 Ha. It’s definitely a spookier term than what I have encountered this under, but frankly, you know, it does track very much with what the general expectation is if the technology reaches the level of something like AGI or powerful AI, or whatever you want to call it. …since you can see it’s even worse than that. So on the one hand there’s going to be a lot of GDP that is not going to be produced by humans in the loop, so that means no worker is ever going to get like the benefits of that. But then on the other hand there’s also going to be quite a significant amount of economic production that doesn’t even show up in GDP because it gets counted as an intermediate good. Things only show up in GDP when it is final consumption or final investment in things, in capital that we can accumulate that has a useful life of a certain period. And a lot of the parts of the AI economy are not going to be reflected in GDP.

Casey Newton: 12:45 Hmm.

Kevin Roose: 12:46 I’m curious, there’s sort of this debate going on among economists that I talk to. Some of them will say, you know, we just don’t ever see really instances of the economy growing as quickly as some of the people in Silicon Valley think it might, you know, 10, 20% GDP growth. That’s just like unprecedented in our history. And so they’re expecting that AI will make things grow much more slowly, maybe a percent or two a year, which would be big in, in relative terms, but not the kind of hyper-growth scenario that some people out here in the Bay Area are envisioning. Then you have people like the folks at Situational Research saying, uh, this, we’re, we’re about to see something we’ve never seen before. We’re about to see an entire economy sort of becoming unmoored from any of these cyclical patterns. So where on that spectrum do you fall? Where does the data lead you between the sort of slow growth, 1% or 2% a year to the 10 or 20% a year hyper-growth scenario?

Anton Korinek: 13:44 Yeah, I’ll say two things about that. The first one is that the story has not been written yet and there is a possibility that if we develop this technology in a really irresponsible way that we could actually see some self-reproduction that takes off and that leads to triple digit GDP growth numbers if measured from the eyes of the AI. But, um, if we deploy the technology in a way that it makes the average person better off, then I think triple digit growth numbers are completely unrealistic. They would lead to way too much disruption. And then, um, I’m not quite sure. I think just 1% is definitely going to be too low to be realistic from my perspective. Um, in really optimistic scenarios, I think we could get to low double digit growth rates. Um, and I should say that presupposes not just the cognitive AI, but full AI in the way that it’s for example defined in the charter of OpenAI where they say systems that are highly autonomous and that can perform most economically valuable work that also includes the physical component, that includes the robotics part. Otherwise it won’t have that big of an effect of GDP because the majority of the economy isn’t just sitting in front of a computer.

Kevin Roose: 15:15 Right. And I think a lot of people right now who are looking at the stock market and these viral essays and trying to make sense of this all are feeling a lot of cognitive dissonance. Because on one hand we have people who seem very smart saying AI is transforming everything, every company is doing things differently than it was a couple months ago, we are headed into uncharted territory. And then you look around and, you know, we’re still below five percent unemployment, we still don’t see a huge productivity boost, most people who are using this stuff at work are only using, you know, older models or their IT department won’t let them use the agentic coding stuff. And, and so it, it does seem like we are seeing a growing disconnect between what people who are looking at the technology are saying is going to happen and the observable reality around us. So what do you make of that disconnect and how should people be feeling about these projections of rapid change?

Anton Korinek: 16:09 Yeah, so the one part that we already touched upon is the gap between the frontier capabilities and the actual implementation. That part is real and that is very significant. That’s also something that is kind of bound to disappear over time, right? But the second part is that ultimately all the projections that we are hearing are extrapolations. And people react very differently when they see how much the AI systems have improved, let’s say over the past year. Some people just naturally jump to the conclusion, well, let’s extrapolate this and of course these systems are gonna be way smarter than any human within a just a small number of years. And then there’s another camp that says, well, what our brains are doing is so special that machines won’t be able to replicate it for a very long time and these machines are gonna kind of asymptote to somewhere below our brain’s capabilities. And frankly, both this is speculative position. I personally, I’m first of all willing to embrace the uncertainty about it and I think we all should, but if you ask me to like make one guess that I feel more comfortable about, I would say capabilities are probably going to continue to increase and I don’t think there is any clear limit in front of us in the near term and so I do expect that there’s gonna be very significant economic impacts.

Casey Newton: 17:48 Yeah, so let’s extrapolate a little bit further into the future. In 2017, you co-wrote a paper where you suggested that, quote, ‘progress in AI is more likely to substitute for…’

Kevin Roose: 18:00 human labor or even to replace workers outright then it is to be a complementary for most jobs. At the time, you were way out on a limb when you wrote that. I imagine you feel that today more than ever. But what is giving you that confidence and to what degree do you feel like we’ve started to see it maybe feel more true than it did in 2017?

Anton Korinek: 18:23 Yeah, and just to be sure that was always meant to be a prediction about AI systems that are essentially at the level of AGI or beyond. Not for the literal systems we had in 2017 that could barely tell apart a dog and a muffin. So, I think ultimately where my perspective is coming from is that I have studied neuroscience, I have studied computer science, and at some level, you know, once basically deep neural networks became powerful, I felt it is hard to not make the conclusion that well, it looks like eventually these systems will be able to do pretty much anything that our brains can do and they’re subject to much, much more relaxed constraints like they don’t need to fit into a tiny human skull. We can scale them almost without bounds and in some sense that’s what we have seen over the past decade, right? We’ve seen lots and lots and lots of scaling. At this point these systems consume the energy of like cities as opposed to what our brain does which is the energy of like an energy efficient light bulb. And that’s still not the limit. They’re still increasing in size, increasing in capabilities and of course the algorithms are getting better and better. So based on that perspective, I just don’t see why there would be any natural limit and certainly not why there would be a limit that’s below our human intellectual capabilities.

Casey Newton: 20:12 Right, and I think the question then is as this world arrives, what happens to the jobs? And in economics, some of our listeners may not have familiarized themselves yet with what’s called the lump of labor fallacy, right? The idea that there are a fixed number of jobs to be done and any job lost to automation will therefore never be replaced. We call it a fallacy because ever since economists started tracking it, automation has always led to the creation of more jobs.

Anton Korinek: 20:41 That’s right.

Casey Newton: 20:42 Anton, you mentioned in another interview that it’s hard for economists to pivot on this because they’ve fought this fallacy for so long. What does it feel like to be an economist saying, actually, this time people should worry that the jobs are going away for real?

Anton Korinek: 20:54 Yeah, it does feel very strange and I have gotten a fair amount of flak from my fellow economists. So over the past decade, although I’ll say over the past year or two or so, many of my colleagues have said, ‘Well, I still don’t entirely buy your worldview, but, you know, I’m glad somebody’s thinking about it and I wouldn’t rule it out entirely.’ Yeah, it is a fallacy that whenever a job is lost in the economy, that person is going to remain unemployed forever. But I think what we really want to look at is overall demand for human labor. And if that demand curve shifts downwards because AI systems can supplant more and more of it, then what that’s ultimately going to imply is that either the quantity of jobs or the wage levels, or both, may contract.

Kevin Roose: 21:51 Mm.

Anton Korinek: 21:52 Now, I should say there’s also the possibility that labor continues to do okay and it just doesn’t grow as fast as the rest of the economy. So in other words, that the labor share of output is going to shrink, but at least we are not falling behind in absolute levels. Our economic theories tell us that whether that outcome or the one where labor just flat out loses, which one of those outcomes materializes depends in part on the speed of automation. And, you know, like for all of our sakes, I’m crossing my fingers and I’m hoping that, you know, we will only lose out in relative terms and not in absolute terms. But right now, I don’t think we have any data that can tell us with any degree of certainty which of those outcomes is going to happen.

Casey Newton: 22:52 And Anton, I want to return to something you said a few questions ago, which was that you expect the gap between frontier AI capabilities and sort of workplace diffusion—how workers are actually using this stuff—to shrink over time. I’m not so sure about that. I’ve spent a lot of time talking with leaders of businesses and educational institutions, and I would not say that their speed of deployment is increasing all that much. You know, they’ve got security fears and privacy fears, lots of reasons why they don’t want to just start throwing this stuff into their work. So maybe help me understand why you believe that gap might shrink.

Anton Korinek: 23:27 I may have expressed myself a little bit unclearly, but what I meant to say is that the current capabilities are eventually going to diffuse to the economy. And of course, by that time, I’m very much with you, the actual capabilities are going to have advanced even further. And, you know, if we are on this trajectory of skyrocketing capabilities, the gap itself may indeed go up rather than down. I think that is the most plausible outcome probably. But what I really wanted to emphasize is that the capabilities that we currently have are eventually going to diffuse and are eventually going to have broad effects. For now, at first productivity effects because right now AI systems are still in many ways very complementary to workers, but as soon as they reach the level where they become substitutes, there’s also going to be some adverse labor market effects.

Casey Newton: 24:26 I’ll tell you what I want.

Kevin Roose: 24:27 Yeah, go ahead. I want to know how people are actually using AI at work because what we have, the data that we have is largely self-reports and I think some firms have exaggerated how much they are doing with AI because they want to appear to be cutting edge and futuristic and look how transformed we are. And I think some people, especially workers, are downplaying how much they’re using AI because they’re embarrassed about it or they it’s against their company’s IT policy or they’re not, you know, they’re not sure they’re allowed to be doing it. And so I just don’t think we have very good granular data about what people are actually doing with AI at work and whether it is speeding them up or slowing them down and if I could like have a crystal ball, I guess I wouldn’t need a crystal ball, I would need like a surveillance apparatus.

Casey Newton: 25:21 Yeah, Kevin wants to spy on workers’ computers.

Anton Korinek: 25:23 But we do have a little bit of that. Both OpenAI and Anthropic publish data on how their systems are actually used almost in real time and that gives us a bit of a picture of where we are, but it tells you only so much.

Casey Newton: 25:31 Can you give our listeners a sense of like are there two or three kind of core indicators or core reports that as you come out you think, okay, here we go, I finally get to update and see if we’re getting closer to a future of, you know, mass job automation. What are those things that as they come in are updating your understanding?

Anton Korinek: 25:52 So the sheer level of capabilities is probably the most important one. Like you can follow whatever benchmarks you want or some amalgam of benchmarks. That tells us where the AI systems were still lagging, where they’re doing already pretty amazing well. And you know, one of the kind of biggest shortcomings right now, but of course from the perspective of workers that’s great because it makes us more complementary, is that these systems are not learning dynamically. The way that current LLMs work is they’re trained once and after that the weights are frozen in place. And that means for a lot of work applications even if there are, you know, very kind of basic mistakes they have to go through the same mistake again and again and again and again because they can learn only so much from it. So that’s another sort of breakthrough that I’m looking for.

Anton Korinek: 27:00 And you know, then maybe a third. The chart that I am regularly following is this Metaculus chart that looks at how long of a task AI can automate. And I think they usually find every seven months, that timeframe doubles. And looking at how this is continuing is also quite helpful in understanding whether the exponential growth trajectory is intact or maybe even accelerating as it has seemed recently, or whether we’re anywhere near plateauing.

Kevin Roose: 27:31 Anton, you mentioned that when you started writing about AI and automation and potential job loss and economic transformation a decade ago, your colleagues in economics were very skeptical. You were seen as something of an outlier in your field.

Anton Korinek: 27:48 Yep. One of my senior colleagues asked me, “Are you really sure you want to throw away your career over this?”

Kevin Roose: 27:54 So, obviously that’s no longer true. You now have many mainstream economists looking at these issues. What are the ideas right now that you believe that put you on the fringes of your profession that many of your colleagues disagree with?

Anton Korinek: 28:13 So, I do have the impression that taking the notion of something like artificial general intelligence really seriously is still a fringe perspective in the economics profession. You’re right that there are more people coming around to it, but it’s still a small and increasingly loud minority. I also believe that if we seriously reach AGI, that’s not going to be the end, but it’s going to be the beginning of a really significant transformation of the economy. And in that respect, I’m probably even more on the fringe of where my fellow economists are.

Casey Newton: 28:51 Yeah, you’ve written about this possibility of hyperbolic growth. Basically, what happens if we get recursive self-improvement? The AI start building better AIs, they start building robot factories and basically create their own economy. And you actually tried to model what might happen in an economy where AI reached this critical inflection point. What did you find?

Anton Korinek: 29:14 Yeah, so the first thing that we found is there’s going to be a whole bunch of feedback loops that will mutually reinforce each other. So, let’s say we do reach this point of recursive self-improvement on the software side, AI systems that can do this are going to feed into the research process on the hardware side and are going to accelerate hardware research, the technological advances on that front. Moreover, they are also going to accelerate research in anything else where cognitive work, where smart things can be helpful. Let’s say, for example, unlocking additional cheap energy sources like fusion and so on. And creating better robots and all of these things feed into each other because those advances in turn help the AI advance more. And if you put it all together, you can get vastly super-exponential growth. In our model, it is hyperbolic growth leading to a singularity. Physics tells us that a literal singularity can’t actually happen because there’s going to be some resource limit at some point. But what I expect is that these feedback loops in the real world would lead to massive growth until some new bottleneck that maybe we haven’t quite identified yet will be reached.

Kevin Roose: 30:48 Hmm. I’m curious, you know, you have to go in a few minutes to teach your graduate students. How has—how has what you have studied changed what you tell your students about how they should think about their careers?

Anton Korinek: 31:01 You know, a couple years ago, I’ve decided, well, I will just be blunt about my beliefs about this. And I am telling my graduate students that I’m not 100 percent sure if there will still be jobs for economic researchers by the time that they graduate. I’m crossing my fingers for them. I hope that there will be, but I don’t think we can count on it at this point. Um, and I think all of us have to face this fundamental uncertainty about where the economy is going to be in a couple of years.

Kevin Roose: 31:38 And how has that affected your course reviews that you get back from the grad students?

Anton Korinek: 31:42 No… (Laughs) That’s a very good question. I have not done a systematic statistical analysis and there aren’t enough data points to say for sure whether AI has increased or reduced my teaching productivity.

Kevin Roose: 31:59 Yeah. Got it. Got it.

Casey Newton: 32:03 Speaking of productivity, I—I want to ask you about this framework that I’ve been working on for thinking about how AI might transform the economy. Uh, so basically, as I see it, there are three possible outcomes here. One is kind of the lumbering giants outcome, where you have these big companies that dominate the economy and they’re just too slow and too regulated to really adopt all the new AI stuff quickly, and so the economy just kind of chugs along for a while, uh, maybe growing at a percent or two a year, uh, but nothing fundamentally changes. The second option is the sprinting giants outcome, which is where these big companies actually get their acts together and start moving really quickly. Maybe they lay off a bunch of people, maybe they create a bunch more new jobs, but they’re much more productive and the economy 10 years from now is still dominated by the same, you know, giant companies we have today. And then there’s this sort of third option, which is the dead giants outcome, which is where basically every company that dominates today is going to be crushed by a competitor…

Kevin Roose: 33:00 Using AI with, you know, 1/100th or 1/1,000th of the labor that they have, and they were essentially going to see this, this sort of swallowing of the the old economy by this new AI-powered one. Of those scenarios, is there one that you think is more plausible, and is that even the right way to be thinking about the possible outcomes here?

Anton Korinek: 33:19 I think those are interesting scenarios to think about, and my best bet would be that we’ll see a mix of the second and third scenario, that there’s going to be some sprinting giants that are going to do okay given their, you know, incumbency advantages, and that there’s also going to be some sectors where newcomers are going to, uh, yeah, overpower the slumbering giants, to use your analogies here. Uh, and yeah, ultimately I do think that the technology will diffuse, and whether that’s through the existing companies or through the newcomers, that depends largely on how fast the giants are going to move.

Casey Newton: 34:02 If you are a public company CEO right now, what do you think there is to be done? Obviously, there is a lot of anxiety from the market about what your company ought to be doing. But as you’ve told us here today, a lot of what we’re doing right now is just waiting for models to get better at various things. So what is the, uh, a rational response to to that dynamic from a CEO?

Anton Korinek: 34:25 Well, the first thing is they should hire my students.

Kevin Roose: 34:31 Yes. Absolutely.

Anton Korinek: 34:33 Because they know really well how to use the AI. But more, more seriously, um, I think one of the most critical things is to remain up to date and to remain informed of where the frontline capabilities are. What I see repeatedly is that CEOs of large organizations are at such a high-level position that everything is fed to them by really intelligent humans, and that makes them not have any reason to access the intelligent AI systems, and that puts them in some ways a little bit at a distance of what’s actually happening in the field. So, you know, if they hire some of my brilliant students who know how to use these systems really well and ask them to give them like a frontline view of what AI can do right now, I think many of those CEOs are actually pretty amazed when they see that. And then if they follow that for a number of months and see how rapidly the capabilities are actually improving, then it naturally kind of leads to decisions like, okay, so we can see what these systems can do in simple tests. How do we actually productively employ them in our organization? Now that gets us to the question of diffusion. It’s still a slow process, right, because you need to experiment, you need to try—

Anton Korinek: 36:00 All things you need to fail if you really want to push these systems to their limit, uh, but I think it needs to be the starting point if we want any of our decision makers to make well informed decisions on how to react to this rapidly advancing technology.

Casey Newton: 36:22 Um, you know, as, as we wind down here, we have been talking today about how it seems like some people, particularly with the markets, are getting like worked up about what might happen without maybe knowing totally what that is. At the same time, I also see this failure of imagination among so many folks out there who seem to believe that however good the systems are today, they just probably won’t get much better or, to the extent they won’t get, or to the extent that they get better, it won’t affect their lives very much. I wonder how you relate to that. Do you just see that as people who sort of, uh, don’t want to contemplate what sort of changes might be coming to their life? Do you think it’s something else? And, uh, what do you think we ought to do about it if you believe that some of those changes might be really consequential for them?

Anton Korinek: 37:05 So first, uh, I mean we all deal with lots and lots of things in our lives, right? And we’ve only limited bandwidth. And let’s say up until a year ago, I very much relate to the fact that, frankly speaking, most AI systems weren’t that useful for most people, right? And so why would we spend some of our limited bandwidth on paying attention to that? And then a second thing is probably also a kind of protective response. Uh, if you want to seriously contemplate the implications of this technology, it leads to pretty stark predictions, it leads you to pretty stark places, and sometimes it just feels a lot more comfortable to just live in the here and now and not worry about that not-so-distant future that may be quite fundamentally disrupted.

Casey Newton: 37:54 Yeah.

Anton Korinek: 37:54 And yeah, a third thing is, uh, there- in the public discourse, uh, you can hear lots and lots of opinions going in all directions, right? I mean, you are, uh, much more expert in that than I am. And you just pick your most comforting, favorite opinion out there in the public discourse, and you can get so much of supply of that.

Kevin Roose: 38:18 Yeah.

Anton Korinek: 38:19 Uh, I just don’t know if that’s the best advice that you can get.

Kevin Roose: 38:32 There’s a joke circulating on social media that goes something like: either AI is a bubble, or everything else is a bubble. Which of those is it?

Anton Korinek: 38:42 No. Uh, if I have to pick one of the two, it would probably be everything else.

Kevin Roose: 38:50 Mm-hmm.

Anton Korinek: 38:51 Well, but having said that, you know, in the economy things always diffuse more slowly than somebody at the frontier would think they do. So in that sense, let’s, let’s take that perspective that this is going to be absolutely transformative and then add that tiny bit of economic reality that things when they diffuse move a little bit more slowly. And I think that’s probably going to be roughly my median prediction of where we are headed.

Kevin Roose: 39:25 Well, Anton, thank you so much for joining us. Fascinating conversation, and let’s keep in touch. Really appreciate your work.

Anton Korinek: 39:31 Thank you, sir. Thank you. I really appreciate you devoting attention to these important topics.

Kevin Roose: 39:41 When we come back, Anthropic versus the Pentagon, Alpha School, and more in our system update segment. Okay, Casey, from time to time, we like to update our viewers and listeners about the stories that we’ve covered in the past that have had some new developments.

Casey Newton: 40:01 Yeah, we’d like to sort of check in on them, uh, gently, without doing sort of a whole segment around them, but at least kind of keeping you up to date with, uh, what we’ve been keeping tabs on.

Kevin Roose: 40:10 And we even have a name and a theme song for this segment. It’s called System Update.

Kevin Roose: 40:21 So our first system update is about a story that we covered on the show last week, which has been moving very quickly. This is, of course, the battle going on between Anthropic and the Pentagon. As a reminder, the Pentagon and Anthropic have been at odds over a proposed change to the terms of service for Claude, which would allow the military to use Claude and other Anthropic AI systems for all legal uses. Anthropic has said that it’s fine with almost all uses, except for domestic mass surveillance and autonomous killer machines.

Kevin Roose: 40:52 So after we recorded last week’s episode, Defense Secretary Pete Hegseth summoned Dario Amodei, the CEO of Anthropic, to the Pentagon for a meeting. That was on Tuesday of this week. That meeting was described by the Times as civil and by Axios as tense. Um, so one of those two is, is probably true.

Casey Newton: 41:08 It can be civil and tense. Our recording sessions often feel that way to me.

Kevin Roose: 41:13 In this meeting, Hegseth told Amodei that Anthropic cannot dictate the terms under which the Pentagon makes operational decisions. Dario Amodei, in turn, defended Anthropic’s commitment to making sure its models are not used for autonomous weapons or mass surveillance. And Hegseth delivered an ultimatum. Basically, if Anthropic does not agree to this all-legal-uses provision by 5:01 p.m. this Friday, February 27th, the Trump administration would take action in retaliation. One of the things it could do would be to, uh, designate Anthropic a supply chain risk, as we discussed on the show last week. That would be a very unusual, uh, step that is often used for foreign espionage attempts.

Casey Newton: 41:59 And would mean that the government presumably then would not use Anthropic’s products and would restrict Anthropic from making deals with any of its own contractors.

Kevin Roose: 42:07 Yes. And Hegseth reportedly also threatened that the Trump administration might invoke the Defense Production Act to force Anthropic to make its product restriction-free for the government. So those two things are on the table now if Anthropic does not cave by this 5:01 p.m. Friday deadline.

Casey Newton: 42:26 Yeah. And that latter threat, Kevin, to invoke the Defense Production Act, there just truly is no precedent that I’m aware of of the government invoking this to require a company to make software for the government. And again, this would be software that would potentially be able to conduct mass surveillance of Americans or create machines that could kill people without any human in the loop. And I’m not aware of anyone in the government trying to defend either of those use cases or speak to why it is such a critical priority for the Trump administration that they be able to do this. And I’ll say it, I find it terrifying that any government would do this to its own citizens. So I hope people are paying attention to this because I think this truly has become arguably the highest stakes conflict in AI that we have so far seen between a big lab and a government.

Casey Newton: 43:25 Yeah, I mean, I remember several years ago when people like Daniel Kokotajlo of AI 2027 were sort of like gaming out what could happen in a world where AI systems get more powerful. One of the scenarios people were envisioning is that the government might try to nationalize some of the big AI companies. But this in some ways goes even further than that. It’s not just saying we’re going to try to influence how you’re building your models. It’s saying we are going to invoke these unprecedented measures to force you to use your models in a way that we want to use them, and if you don’t agree to our demands, we’re going to essentially try to kill the company.

Casey Newton: 43:58 Yeah, and think about what a grim outcome that would be for Anthropic, a bunch of do-gooders who left OpenAI so that they could try to create safer AI systems. I mean, you want to talk about sci-fi scenarios, like, it truly feels like we are living one right now.

Casey Newton: 44:14 Yeah, and another interesting thing that’s come out since last week is that the Defense Department appears to be very committed to using Claude. There was a great quote in this Axios article from a defense official ahead of this meeting between Dario Amodei and Pete Hegseth in which a defense official was quoted as saying, ‘The only reason we’re still talking to these people is we need them, and we need them now. The problem for these guys is they are that good.’ So basically they are saying, look, if we had a bunch of interchangeable AI models that all had relatively similar capabilities, we could just cut off Anthropic and say we’re not going to honor the terms of our contract with you because you won’t let us use your models for what we want to use them for. But in a world where Anthropic’s models are better than models from competing AI companies, they really don’t want to make that trade-off. They really don’t want to go with what they consider a second-tier model here. It would also be complicated because, as you said, Anthropic’s models are the only ones that are approved for use in classified systems. So I think this is really also illustrating something that Anthropic has believed since early in its existence, which is that the way that you influence safety, the way that you get leverage in these negotiations, is by having really good models. Dario Amodei has this phrase ‘race to the top,’ where he basically thought that if Anthropic was on the frontier, was sort of competitive with the leading AI companies in the world, then policymakers and large government agencies like the Defense Department would be forced to take them seriously. And I think what we’re seeing now is that A, he was correct, Anthropic does have leverage because its models are very good. And B, it might not matter if the government can just force you to do something you don’t want to do.

Casey Newton: 46:01 Yes, but I would point out, Kevin, how incoherent the administration’s response has been, because they’re saying two contradictory things, right? One is, we’re not going to use you and we’re going to try to get other people to stop using you. And the other is, we’re going to force you to let us use you, right? So to me, that is just consistent with an administration that only knows the language of threats and dominance, right? Like, that is—there’s no negotiation, there’s nothing to discuss. We get exactly what I want or we are going to hurt you as much as we can. Um, but I think it’s just so notable that even within that, it seems like the military can’t figure out what it wants to do with these guys.

Kevin Roose: 46:46 Yes. It is a classic case of an unstoppable force meeting an immovable object. My understanding is that Anthropic is not going to budge on these two carve-outs that they want. Now, there was some confusion about Anthropic’s safety position this week because the company also changed its Responsible Scaling Policy, its RSP, which governs how it releases new models and the safety protections it applies to them. Some people thought these things were related, but my understanding is that these are separate issues. And that when it comes to this specific dispute with the Pentagon, Anthropic is still holding firm to its belief that it doesn’t want Claude being used for mass domestic surveillance and autonomous killing weapons. And they feel like they can suffer whatever the hit might be to their business if it means that they don’t compromise on their values. And by the way, what a great marketing campaign for Anthropic, which gets to stand up and say, ‘We are the only AI lab that is committed to not letting our models be used for these terrifying use cases.’

Casey Newton: 47:40 Yeah, I’ve already thought of a really good Super Bowl ad for them next year. They could say, ‘Murder is coming to AI, but not to Claude.’

Kevin Roose: 47:50 Right. So what are you looking for after this meeting or this deadline on Friday at 5:01 PM?

Casey Newton: 47:57 Well, like you said, I mean, based on Dario’s public statements, I think that he is not going to back down. I think in some sort of strange way, like this is the fight that they wanted, right? Like think about how long we’ve been talking about the show on AI safety and for how long people have been mostly avoiding it. Well, it’s like now here it is, like you know, one of the main public policy issues up for debate in the United States right now. And I think Anthropic is willing to lose this however it has to, if only to make the point that these systems are getting very close to being able to do some very dangerous and scary things. So I expect Anthropic to stick to its guns and so to me the question is just what consequences does it suffer as a result?

Kevin Roose: 48:31 Yeah, and also I think there’s an issue here of what the other AI companies will do in response, right? We’ve already seen a few employees of companies like Google and OpenAI speaking up in Anthropic’s defense saying it would be a very bad thing if the government compelled or forced Anthropic to use their models for these things that they don’t want to do. But so far, the leaders of these other AI companies have been mostly silent about this issue. I think they are glad to let Anthropic take the heat on this one, but they are all going to find themselves in similar situations at some point down the line if they continue to pursue these giant military contracts.

Casey Newton: 49:06 They will, but based on what we know so far, we should expect them to roll over. Like it has truly been nothing but profiles in cowardice over at these other companies.

Kevin Roose: 49:21 Yeah, but I’m also going to be looking for some of the political response to this, because, you know, last week we were sort of talking about why no one in civil society or in government seems as worked up about this as we were. I think that’s changed over the past week. We’re starting to see some elected officials, some civil liberties groups sort of realizing that what’s going on right now has big implications for the future, not just of the military’s use of technology, but for the freedom and the sort of ongoing operations of some of our largest and most advanced technology companies. And I think this conflict between the Pentagon and Anthropic will be seen for many years as sort of the first standoff between industry and government when it came to advanced AI.

Casey Newton: 49:58 Yeah, but hopefully not the last one with the way things are going.

Kevin Roose: 50:01 Yeah. Okay, so that is the latest on the Anthropic story. Stay tuned for more on that. Next up on our system update, we have an update on Open Claw. This is of course the open-source agentic AI tool that became very popular earlier this year. People were buying Mac Minis and setting this thing up on their computers and letting it run their entire lives. And we’ve heard a lot of good stories about how that has been going for people, and this past week, we heard a very bad story.

Casey Newton: 50:30 Boy, was it? This story comes to us from Summer Yue. She is the head of alignment at Meta AI and she had an ex-post that got a lot of attention this week reporting that Open Claw had ignored her instructions and tried to delete her entire email inbox. Frankly, that sounds like a dream to me, but I guess she had some emails that she wanted to respond to. Summer said that after testing her Open Claw on what she called a toy email account and finding it useful, she asked her agent to check her real inbox and suggest, you know, what would you archive or delete? And she said, she said, don’t action until I tell you to. But instead of confirming with her, Kevin, as she requested, it diverted to a nuclear option and started deleting her entire inbox.

Kevin Roose: 51:15 Again, I want to make clear, this is what I want my agent to do for me.

Casey Newton: 51:18 For Summer it was a problem. And I guess despite repeated attempts to get it to stop by prompting it via a Telegram interface, her bot ignored her and she had to run to her Mac Mini in her words, like she was defusing a bomb to get it to stop. So why did this happen? Well, she thinks that her real inbox was just too big and it triggered compaction, which is when essentially you run out of context window using the whatever model you’re using, and that during compaction, it lost her original instruction. Kevin, have we ever had a bigger case of I told you so on the Hard Fork program?

Kevin Roose: 51:53 Uh, no. I think this takes the cake. And I will say, um, this is exactly why I have not installed Open Claw on my, uh, laptop and given it access to my files. These systems are still very unpredictable. It is very high-risk behavior. Um, I think there’s a case to be made that it’s actually good if the people doing alignment research at some of our leading AI companies are experiencing the downsides of these systems for themselves. Um, it’s sort of like if it doesn’t happen to you, you won’t think it’s a problem for other people. Yeah. Um, so I think there’s sort of a, sort of a counterintuitive case that this was good for alignment, but I think it was also very funny just to see someone who clearly understands this technology and what it’s capable of just getting absolutely bogged by it.

Casey Newton: 52:40 Absolutely. And you know, um, one element that I would also draw folks’ attention to on this is that it is so easy to spend an afternoon using AI systems convincing yourself that you’re making yourself massively, uh, productive and giving yourself a ticket out of the permanent underclass. And then you look back and just realize that you’ve wasted the day. And I would just hope that you continually bring your attention back to that, uh, because I think figuring out what is a use of my time with AI that improves my life and what is simply a waste of time can be tricky to discern. Uh, but you’re going to want to keep your eye on it or you’re going to have a lot more terrible afternoons like, uh, poor Summer did.

Kevin Roose: 53:18 I think this is a good cautionary tale and, uh, also a good all-purpose excuse the next time someone asks why you haven’t responded to their email. Just say, my Open Claw agent just mass deleted all of my emails.

Casey Newton: 53:39 So for our final update today, Kevin, we wanted to revisit Alpha School.

Kevin Roose: 53:41 Yeah, this is the sort of AI-powered education, uh, company that is, uh, running schools around the country. We interviewed McKenzie Price, one of the co-founders of Alpha Schools, on the show last September. And almost immediately, we got, we started getting emails from listeners to the show saying, uh, this sounds a little far-fetched. Are you sure this company is everything it advertises itself as? And Casey, what has happened since?

Casey Newton: 53:56 Well, there have been two reports that we wanted to highlight that suggest that, uh, all is not well. So looking at Alpha School, 404 Media published a big story last week that drilled into some of the critiques. For one, apparently some of these AI-generated lesson plans just aren’t very good, Kevin. They highlighted some examples where the curriculum was like essentially just showing students like slop that had like no correct answer because they were just worded wrong. There were also just accuracy problems. They estimated that they had a 10% hallucination rate for some of these generated materials, violating their terms of service, and it’s collecting lots of data on students, which frankly, I would expect, but apparently stores at least some of that data insecurely in a Google Drive that anyone with the link could access. So, that wasn’t great. There was also a report in Wired that came out in October where they focused specifically on the Alpha School that was opened in Brownsville, Texas. So some parents at that school at least felt like the promise of Alpha School that we had heard about last September was not realized for their kids.

Kevin Roose: 55:13 Yeah. And I also heard from one parent who attended an Alpha School information session recently and this parent came away thinking that the school was, quote, “the Theranos of education.” According to this person, there was some fake interactivity on the screen during the session in the form of some pre-recorded emojis and that the CEO only appeared on camera late into the session after parents started asking, “Hey, are we live or is this some pre-recorded canned presentation or not?” So, Casey, does any of this change your view of Alpha School that you had coming out of the interview with Mackenzie Price last September?

Casey Newton: 55:52 I mean, like, look, I did think that there were several things that Mackenzie mentioned that seemed interesting. It was like, “Oh, well like if that worked, that might be sort of an interesting way of educating your kid.” I think what we are learning is that, yeah, it’s hard to create a new school from scratch and maybe there are some corners being cut here and maybe they’re not executing as well as they hope to on some of their dreams. I mean, I think, you know, if you’re having like hallucinations in curriculum, I think that’s like pretty much as bad as it gets for a school like that. Like they need to get that down to zero, right? Like if you can’t verify that your curriculum is accurate, like I don’t know that you should be able to call yourself a school. If I can be a little controversial though, like the 404 Media story, their headline is, quote, “Students are being treated like guinea pigs,” which is a quote from the story. And I just kind of think that like at most schools students are being treated like guinea pigs. Education is always changing. Every school I’ve ever been to has been running one sort of new program or another trying to like, you know, build a better mousetrap. And I think if you were a parent and you were considering sending your child to a private school that was very different from public school, you’re probably like up for at least a little bit of that kind of experimenting, right? Obviously, most people are never going to choose anything like that.

Kevin Roose: 57:00 Right, and I think the question is sort of what are the outcomes for the students who do. The second thing I would say is, kids just have different outcomes at schools, right? Like, I think you could go to any school in America and if you interviewed every parent, you’d have some parents that absolutely love the school, they love their teachers, and you’d have some that absolutely hated it, and then there would be a lot in the middle, right? So I don’t want to over-index on a couple reports. I’m perfectly willing to believe everything that is in these reports, and I believe that these people had terrible experiences, but it’s hard to know what is a representative sample and what is a couple of grumblers.

Casey Newton: 57:30 Yeah, and I’ll just say, like, I—what I appreciated about McKenzie Price and Alpha School was not so much the specific details of the school or the curriculum or the way they were approaching education. It was purely the fact that they were saying to themselves and to their parents, like, something big is happening here in education. AI is not just like some classroom tool the way that maybe Chromebooks or other technologies have been. It is something that is fundamentally reshaping how people learn and how people can learn. And so that’s the—the kind of thing that I would encourage people to keep doing. Yes, there will be some failed experiments. Yes, there will be some things that don’t work out. But I think, in general, the more that educational institutions can sort of realize that they are being transformed whether they want to be or not, the better the outcomes for students are likely to be.

Kevin Roose: 58:21 Yeah, and let me say this, if you’re running a school and it looks like identical to what a school would have looked like 20 years ago, you’re also treating your students like guinea pigs, and I’m not sure we’re going to love the result of that experiment.

Kevin Roose: 58:31 Okay, so Casey, that is our system update. Now our listeners are fully up to speed, and I expect that our inbox traffic will trickle to zero now that we’ve satisfied all these concerns.

Casey Newton: 58:40 Well, I can’t tell, my open cla—actually just deleted my inbox. But I told it to, so it’s fine.