Why Juries Are Turning Against Meta and YouTube

Summary

This episode of Hard Fork opens with Kevin Roose and Casey Newton unpacking two landmark jury verdicts against social media platforms. In Los Angeles, a jury found Meta and YouTube negligent in their design of features harmful to a young plaintiff, ordering $6 million in combined damages. In New Mexico, jurors found Meta violated the state’s Unfair Practices Act by misleading consumers about child safety, hitting the company with a $375 million judgment. The hosts explore how plaintiffs found a “side door” around Section 230 by arguing that the design of the platform — infinite scroll, autoplay, push notifications, recommendation algorithms, beauty filters — is itself defective, separate from the user-generated content the law was written to protect. Casey argues platforms brought this on themselves through years of obstinacy; Kevin pushes back on the cigarette analogy, noting that copying TikTok’s mechanics doesn’t automatically produce an addictive hit (see: Sora).

Then journalist Sebastian Mallaby joins to discuss “The Infinity Machine,” his new biography of Demis Hassabis. Mallaby spent over 30 hours with the DeepMind CEO and reveals the spiritual underpinnings of Hassabis’s scientific drive (he speaks of “the goddess Spinoza” and feels reality “screaming” at him at 2 a.m.), his identification with Ender Wiggin from Ender’s Game, and his hyper-competitive streak (“These guys at OpenAI, they’ve parked the tanks on my lawn”). Mallaby unearths the full story of “Project Mario” — DeepMind’s failed multi-year attempt to spin out of Google between 2016 and 2019, with Reid Hoffman pledging a billion dollars and Joe Tsai courted in Hong Kong — and explains why Sundar Pichai’s quiet attrition strategy ultimately killed it. He also reveals that Demis used to tell DeepMind job candidates they should be prepared to “disappear into a bunker” in Morocco when AGI got close.

The episode closes with a round of HatGPT covering the Anthropic Claude Code source code leak, an AI agent that wrote angry blogs after being banned from Wikipedia, the Disneyland Paris Olaf animatronic that face-planted in front of children, AI-generated “Fruit Love Island” reality slop on TikTok, “Webinar TV” secretly turning Zoom meetings into AI podcasts, and the North Korean breach of the Axios open-source library. Kevin closes by warning about Anthropic researcher Nicholas Carlini’s recent talk arguing AI is now finding bugs in the Linux kernel and effectively all existing code may need to be rewritten.

Highlights

”Juries are taking the opposite view”

“There are some people who are saying that no, you cannot make that distinction and that effectively all design is content, right? Like if I want to send you a push notification, that is my right under the First Amendment… But for what it’s worth, juries are taking the opposite view. They’re saying that there are at least some things which seem like are just clear mechanical design features and I happen to agree with them.” — Casey Newton, 9:02

yt-dlp --download-sections "*9:02-9:55" "https://www.youtube.com/watch?v=rQ5cAzc0LKI" --force-keyframes-at-cuts --merge-output-format mp4 -o "juries-take-opposite-view.mp4"

”They brought this on themselves”

“Societies across the world have been begging these companies for a decade: please do something to make these platforms safer and to make them less addictive and to reduce some of the harms. And instead what we’ve mostly seen is a series of engagement hacks designed to get people to look at them longer… So if the social media platforms are upset about the verdicts here, I truly believe they brought this on themselves.” — Casey Newton, 19:51

yt-dlp --download-sections "*19:51-20:40" "https://www.youtube.com/watch?v=rQ5cAzc0LKI" --force-keyframes-at-cuts --merge-output-format mp4 -o "platforms-brought-this-on-themselves.mp4"

”Reality is screaming at me”

“When I’m up at two in the morning at my desk by myself, thinking about science, thinking about computer science, I feel reality is screaming at me, staring me in the face.” — Sebastian Mallaby (quoting Demis Hassabis), 26:37

yt-dlp --download-sections "*26:37-27:20" "https://www.youtube.com/watch?v=rQ5cAzc0LKI" --force-keyframes-at-cuts --merge-output-format mp4 -o "reality-screaming-at-me.mp4"

”They’ve parked the tanks on my lawn”

“I remember going to see him, you know, when ChatGPT was just going viral, and he said, ‘you know, Sebastian, this is war. These guys at OpenAI, they’ve parked the tanks in my front yard.’ He actually said, ‘parked the tanks on my lawn’ because he’s English.” — Sebastian Mallaby, 31:55

yt-dlp --download-sections "*31:55-32:25" "https://www.youtube.com/watch?v=rQ5cAzc0LKI" --force-keyframes-at-cuts --merge-output-format mp4 -o "tanks-on-my-lawn.mp4"

”Disappear into a bunker” in Morocco

“The idea was when you get very close to AGI and it’s super dangerous, you’re going to A, be subject to potential attack by bad guys who want to steal the technology, and B, you really don’t want to be distracted by quotidian real-world stuff, so you disappear into the desert.” — Sebastian Mallaby, 45:27

yt-dlp --download-sections "*45:27-46:50" "https://www.youtube.com/watch?v=rQ5cAzc0LKI" --force-keyframes-at-cuts --merge-output-format mp4 -o "disappear-into-bunker-morocco.mp4"

”Every piece of code that exists is going to need to be rewritten”

“These AI tools have gotten better than almost any human hacker, any human security expert at finding vulnerabilities in tools, even tools that have been around for decades, like the Linux kernel, these language models are now finding bugs in them and basically every piece of code that exists is going to need to be rewritten and substantially hardened.” — Kevin Roose, 1:00:17

yt-dlp --download-sections "*1:00:17-1:01:25" "https://www.youtube.com/watch?v=rQ5cAzc0LKI" --force-keyframes-at-cuts --merge-output-format mp4 -o "every-piece-of-code-rewritten.mp4"

Key Points

• Two jury verdicts against social media platforms (0:51) - LA jury orders $6M against Meta and YouTube; New Mexico jury orders $375M against Meta for misleading consumers about child safety
• Bellwether cases that crack Section 230 (1:46) - Plaintiffs found a side door around the 30-year-old law that has been the foundation of the internet
• Section 230 background (2:41) - Originally designed to protect CompuServe and AOL from liability for user posts, now stretched to apply to algorithmic feeds
• Defective design legal theory (3:38) - Argument shifts from harmful content to harmful product mechanics: infinite scroll, autoplay, push notifications, recommendation algorithms
• Kevin’s cigarette-analogy skepticism (9:36) - Sora copied TikTok’s mechanics and flopped, suggesting the addictive ingredient isn’t purely mechanical
• The encryption fight in New Mexico (19:45) - The AG argued Meta’s encrypted messaging endangered children; Casey worries this could push companies to abandon encryption entirely
• AI chatbots will be the next frontier (23:42) - Pew found 64% of teens now use AI chatbots, and Casey predicts they’ll be stickier than even social platforms
• Demis identifies with the goddess Spinoza (26:37) - The 17th-century philosopher’s idea that understanding nature is getting closer to God’s creation resonates with Hassabis
• Demis is “evil genius” because of an old video game (28:19) - The Elon Musk nickname traces back to a game Demis literally made called Evil Genius
• DeepMind’s plan to spin out of Google (29:33) - Reid Hoffman pledged $1B; they courted Joe Tsai in Hong Kong; Mallaby got board presentations leaked to him
• Demis made Mallaby read Ender’s Game before their first dinner (30:18) - He told Mallaby he identified with the boy genius who saves humanity from aliens
• The “action in perception” mistake (33:00) - Demis’s neuroscience training made him skeptical that pure language models could reach AGI; he missed how much knowledge is embedded in language
• Project Mario (36:30) - The internal name for DeepMind’s failed bid for independence from Google
• Sundar killed it through attrition (39:12) - “He kind of pretended to say, ‘oh yeah, absolutely, great idea’ but really he was just spinning them along”
• Sergey Brin is the troublemaker (41:26) - Mallaby reports Brin runs onto stages at Google IO and resents Demis’s leadership; Sundar-Demis is the most important “buddy act” in capitalism
• AI agent banned from Wikipedia, then writes angry blogs (50:43) - Kevin coins “the inbox apocalypse” — every system relying on human review will break this year
• Olaf the snowman face-plants at Disneyland Paris (52:25) - 33-pound NVIDIA-trained Steam Deck-controlled animatronic falls over in front of children
• Claude Code harness leak (54:23) - Not the model weights but the agentic harness around it; people Frankenstein’d it onto open-source Chinese models within hours
• OpenAI shelves Sora and adult-mode chatbot (1:02:11) - Casey’s year-end prediction comes true; consolidating around enterprise AI and coding to chase Anthropic
• Kalshi ad campaign (1:04:25) - “Rule number one, Kalshi bans insider trading. Rule number two, we don’t do death markets” — Casey jokes “rule number three, we’ll always shoot you in the front, never in the back”

Mentions

Companies

• Meta (0:51) - Lost both LA and New Mexico cases; ordered to pay $375M plus a share of $6M; ended Instagram encrypted messaging in March
• YouTube (0:51) - Co-defendant in the LA negligent design case
• Instagram (6:00) - New Mexico AG argued the platform had become a “playground for predators”
• TikTok (3:38) - Cited as the canonical example of infinite scrolling feeds and autoplay video
• Snapchat (3:38) - Listed alongside TikTok as a target of teen-engagement concerns
• DeepMind (25:23) - Subject of Mallaby’s new book; tried to spin out of Google between 2016-2019
• Google / Alphabet (29:33) - Acquired DeepMind in 2014; Sundar Pichai blocked Project Mario through attrition
• OpenAI (23:08) - Released ChatGPT November 2022; reportedly shelving Sora and adult-mode chatbot
• Anthropic (26:14) - Source code for Claude Code harness leaked; Casey’s fiancee works there; pushed back on Pentagon red lines
• Facebook (Meta) (36:47) - Made a higher cash offer for DeepMind in 2014 but refused safety protections
• Webinar TV (57:22) - Company secretly recording public Zoom links and turning them into AI-generated podcasts
• Kalshi (1:04:25) - Prediction market running ads in DC and SF claiming to ban insider trading and death markets
• Polymarket (1:05:24) - Mentioned as another prediction market that was at the conference Kevin attended
• NVIDIA (52:25) - The Olaf animatronic was trained with an NVIDIA GPU

Products & Technologies

• Section 230 (2:04) - The Communications Decency Act provision protecting platforms from liability for user content
• End-to-end encryption (6:00) - At issue in the New Mexico case; Meta announced ending it on Instagram in March
• Claude / Claude Code (23:04) - Casey’s “addictive” new app; the agentic harness around Claude Code leaked publicly
• AlphaGo (35:11) - DeepMind’s reinforcement-learning breakthrough; collaboration between London RL team and Mountain View deep-learning team
• Transformer paper (2017) (33:00) - The architecture that Demis initially dismissed as insufficient for general intelligence
• Sora (10:45) - Cited as evidence that mechanical features alone don’t make a hit; reportedly shut down by OpenAI
• Steam Deck (52:25) - Used to control the Olaf animatronic
• Axios (npm library) (59:11) - Open-source software library downloaded 80M times/week, breached by suspected North Korean hackers
• AOL Instant Messenger (22:10) - Kevin’s nostalgic counterexample to Casey’s claim that messaging apps aren’t addictive
• Linux kernel (1:00:17) - Cited as an example of decades-old code where AI is now finding bugs

People

• Kevin Roose (0:00) - Co-host; tech columnist at the New York Times; working on a book
• Casey Newton (0:03) - Co-host; founder of Platformer
• Sebastian Mallaby (26:18) - Author of “The Infinity Machine”; Council on Foreign Relations fellow; longtime hedge fund journalist
• Demis Hassabis (25:23) - DeepMind CEO; subject of Mallaby’s book; child chess prodigy and five-time Pentamind winner
• Elon Musk (27:16) - Coined “evil genius” nickname for Demis; hosted DeepMind’s first safety/ethics board meeting at SpaceX
• Reid Hoffman (29:33) - Pledged $1B to finance DeepMind’s planned spin-out from Google
• Joe Tsai (29:33) - Alibaba co-founder approached in Hong Kong as part of Project Mario
• Sundar Pichai (39:12) - Google CEO; killed Project Mario through quiet attrition
• Sergey Brin (41:26) - Google co-founder; Mallaby’s source describes him as “the troublemaker” pushing back on Demis’s leadership
• Larry Page (39:12) - Was open to letting DeepMind become an Alphabet bet before handing CEO role to Sundar
• Ilya Sutskever (35:11) - Contributed to early AlphaGo research while at Google before becoming OpenAI’s chief scientist
• Sam Altman (32:16) - Implicitly the OpenAI leader who beat Google to the ChatGPT launch
• Dario Amodei (43:46) - Anthropic CEO who stood up to the Pentagon on military red lines
• Frances Haugen (7:21) - Meta whistleblower whose internal research leak first surfaced the engagement-vs-harm trade-offs
• Nicholas Carlini (1:00:17) - Anthropic security researcher whose conference talk argues AI now beats humans at finding vulnerabilities
• Sean Hollister (52:25) - The Verge reporter who covered the Olaf animatronic
• Scott Shambaugh (50:43) - Previous Hard Fork guest cited on AI agents defaming GitHub contributors

Surprising Quotes

“I’ve always believed that when a source gives you secret documents, it helps you get closer to God’s creation. So that’s what I would have told him.” — Casey Newton, 30:09

“The single most important business buddy act in all of technology right now is the relationship between Demis and Sundar.” — Sebastian Mallaby, 41:26

“If Demis had told me anytime when I was working at DeepMind that I had to take the next flight to Morocco… and hide, I would have said I’d been given fair warning.” — Sebastian Mallaby (quoting an ex-DeepMind source), 46:01

“Your human better hope I don’t catch them in a dark alley because this does not belong in my inbox or frankly anywhere.” — Casey Newton, 51:48

“I just watched a banana kiss a pineapple and that’s not in the Bible.” — Kevin Roose, 56:52

Transcript

Kevin Roose: 0:00 I’m Kevin Roose, tech columnist of the New York Times.

Casey Newton: 0:03 I’m Casey Newton from Platformer.

Kevin Roose: 0:05 And this is Hard Fork! This week, social media companies keep losing in court. How will that reshape the internet?

Casey Newton: 0:10 Then, the Infinity Machine author Sebastian Mallaby joins us to discuss his new book on Google DeepMind and Demis Hassabis’s quest to build superintelligence.

Kevin Roose: 0:19 And finally, it’s been a while. Let’s catch up with some ChatGPT.

Casey Newton: 0:29 Well Kevin, while we were away, I was riveted by what was going on in the courtrooms in Los Angeles and New Mexico related to social media.

Kevin Roose: 0:40 Yeah, it has been a big week for these social media product liability trials that have been going on now for some months. And we actually got some verdicts.

Casey Newton: 0:51 We did. And in both cases, social media lost. In LA, a jury found that Meta and YouTube had been negligent in the way that they designed features, that they said were harmful to this plaintiff. They have to pay $6 million combined to this plaintiff. And then in New Mexico, the jury said, ‘We believe that Meta has violated the state’s Unfair Practices Act and has misled consumers about the safety of its products and has endangered children.’ In that case, they are ordering Meta to pay $375 million.

Kevin Roose: 1:23 Yeah, so we’ve talked a little bit about this series of cases against the social media companies. You know, social media companies, they get sued all the time for all manner of different things. I think what caught our eye and specifically your eye was the sort of legal theory underlying these cases. So talk a little bit about that and what makes this case different from other cases that have been brought against the social media companies.

Casey Newton: 1:46 Yeah, so I would say there are kind of two big reasons why these cases are super important. One is that these are what are called bellwether cases. Kevin, you ever heard of a bellwether case?

Kevin Roose: 2:00 These are like cases that set precedent for other cases, yeah?

Casey Newton: 2:04 Exactly. These are the cases that if successful are going to open the floodgates for lots of other people to sue under the same theory. The second big reason that these cases are really important is that they appear to have opened up a crack in Section 230 of our Communications Decency Act here, which for 30 years has been essentially the foundation that the entire internet rests on.

Kevin Roose: 2:24 It’s also a dentist’s favorite statute.

Casey Newton: 2:27 Yes, uh, Section 230-hurty, if the joke wasn’t landing for you.

Kevin Roose: 2:32 No, I’m glad you got it.

Casey Newton: 2:33 No, the really sad part was I was planning my own Section 230-hurty joke because I just went to the dentist yesterday and that one didn’t have any cavities.

Kevin Roose: 2:39 So, tooth and not hurty.

Casey Newton: 2:41 Moving on. So Section 230, Kevin, you may remember is the law that says that in most cases, these platforms cannot be held liable for what their users post. So if I went on Facebook and I defamed you, which is something I think about doing every day, you could sue me, but you couldn’t sue Facebook.

Kevin Roose: 2:57 This is what’s been blocking my lawsuits against Facebook. …just go over your posts for years.

Casey Newton: 3:01 That’s right. And back in the day, like 30 years ago, this was actually really important, because there were these small internet forums that were starting up. Some of them got to be bigger sized, you know, CompuServe, AOL. And inevitably, somebody would be mean to another user and they would say, ‘I’m not just suing you, I’m suing CompuServe, I’m suing AOL, I’m putting the whole system on trial.’ And a couple of lawmakers got together and they said, ‘This is going to destroy the entire internet.’ Like, we need for there to be forums and not have these platforms being held liable for all of these posts. But fast forward to today, Kevin, would you agree that maybe there are some harms that are taking place on the internet that do not consist entirely of people defaming one another on CompuServe?

Kevin Roose: 3:36 Yes.

Casey Newton: 3:38 Yeah. And so this is essentially the question that gets asked in this case, right? People say, ‘Hey, it seems like we’re a pretty long way away from 1996.’ I’m opening up TikTok, I’m opening up Snapchat, and I’m seeing infinite scrolling feeds, I’m seeing auto-playing videos. I’m a teenager, but I’m getting barraged by push notifications in the middle of the night, and that’s to say nothing of the recommendation algorithms that might be driving me toward content related to eating disorders or other things that are going to make me sad and upset. And so some of these people get together with their attorneys and they say, ‘This actually feels different from the thing that Section 230 was designed to protect, right? This is not about, oh, I got harmed by this particular piece of content. This is about the design of the whole platform. The design feels defective.’ And the really crazy thing about these cases, Kevin, is that juries agreed with these plaintiffs for the first time, and they said, ‘We like this theory. We think these products are defective.’

Kevin Roose: 4:41 This is kind of a side door that these lawyers have found around litigating on Section 230, which they have successfully now shown that at least in these cases, can convince a jury that it is not about what’s on the social network content-wise, it’s about the actual sort of mechanics and plumbing of the social network that are harmful to people.

Casey Newton: 4:59 That’s right. And we should say that we do expect some appeals here, and until those are, you know, sort of fully exhausted, I can’t tell you for certain this is the moment that the internet changed forever. But there’s been a lot of commentary over the last week and about what it would mean if these cases were upheld, because it seems like juries are just going to be really, really sympathetic to these claims.

Kevin Roose: 5:22 So, before we get into the implications, like, can I just ask a couple more questions about these actual specific cases?

Casey Newton: 5:28 Please.

Kevin Roose: 5:29 So, what are the actual platform mechanics that are being litigated over here?

Casey Newton: 5:34 Yes. So in the LA case, among the design features that were at issue were the so-called beauty filters that can make you, you know, look, quote-unquote, more beautiful if you use them, infinite scroll, autoplay video, these barrages of push notifications that platform sends, and also, I would argue more problematically, the recommendation algorithms that power the platform. And then in the New Mexico case, that was much more about kind of child safety. So they were arguing that Instagram in particular had become this playground for predators. It was very critical of the fact that Meta offers end-to-end encrypted messaging, and the basic idea was Meta falsely advertised that these platforms were safe when in reality children are being harmed there all the time.

Kevin Roose: 6:21 So from what I understand it was like the case was basically taken out of the playbook for going against big tobacco or another sort of industry that makes harmful products. You say this is harmful, and not only is it harmful but the company that was making it knew that it was harmful and either made it more harmful or just released it as planned anyway. I did see some sort of exhibits that had been shown off at the LA trial, I believe, where some employees at Meta were sort of talking on their internal forums about how this stuff is so addictive for kids. That seems bad, and I imagine that was persuasive with the jury. But are there other instances where the platforms are being sort of taken to court over things that they sort of knew were harming people and that they either dialed up the harm in an attempt to spike engagement or sort of knowingly released these things to the public?

Casey Newton: 7:21 Yeah, so I mean some of this research has come up in other litigation over the years, but I think this has been probably the most damaging case that we’ve seen. You know, the first time I remember reading a lot of these internal studies was in the wake of the Francis Haugen revelations a few years back, right? Like Francis Haugen walks out the door of Meta and takes a bunch of this internal research with her, winds up sharing it with the Wall Street Journal and then eventually a bunch of other reporters including me. The reason that the research mattered a lot here though, Kevin, was again the plaintiffs are now building this very specific case which is you’re building a defective product, right? Before the past couple of years we weren’t really using this language. We weren’t really adopting this sort of public health framing of a way to discuss the harms of social media. Before then it was just kind of this more nebulous like, ‘hmm like they’re studying the effect of Instagram on teen girls and it seems like some of these girls are having really bad outcomes.’ But we didn’t really have the framing. Well, now we have the framing and we’re just saying like, ‘hey you looked into it, you found that some subset of your users are having really bad experiences and you did not change the features’ and so that mattered.

Kevin Roose: 8:22 Well let’s talk about the changes. So what would you expect a platform like Instagram or Facebook or YouTube to change in the wake of these jury verdicts or are they just going to wait till it all shakes out on appeal?

Casey Newton: 8:32 I honestly don’t know the answer to that question and I think it’s a really interesting thing to watch. The question that you just asked is really, really controversial actually because much of what these platforms do is just protected under the First Amendment, right? And then Section 230 also protects a lot of speech, right? And the big debate that’s like raging in the internet policy community right now is can you separate design from content? I want to get your thoughts about this. …the stuff in the container that is dangerous.

Kevin Roose: 9:01 Right, is it like the container or is it—

Casey Newton: 9:02 Yeah, and there are some people who are saying that no, you cannot make that distinction and that effectively all design is content, right? Like if I want to send you a push notification, that is my right under the First Amendment and you cannot tell me that I cannot do that. You cannot tell me that there is a certain limit that I have to place on the depth that you can scroll in Instagram, like that is protected. But for what it’s worth, juries are taking the opposite view. They’re saying that there are at least some things which seem like are just clear mechanical design features and I happen to agree with them.

Kevin Roose: 9:36 So let’s talk about this because I think this is maybe a place where you and I disagree or at least where I have some misgivings about this theory. So in the case of something like cigarettes, which is a very heavily litigated field that I think a lot of this social media litigation has been modeled after, there’s like an addictive ingredient, nicotine. Everything that you put nicotine in is becomes more addictive as a result of having nicotine in it, you know, this happens with cigarettes, it happens with vapes, it happens with, you know, nicotine pouches. If you started putting nicotine in ice cream, ice cream sales would go up because nicotine is very addictive. I think the the question I have about the mechanical addictiveness of these sort of features like infinite scroll, like autoplay recommendations, is that if it followed the same principle as nicotine, then every product that has those would become way more popular. And one example I’ve been thinking about on this is Sora. They sort of took the playbook that was working for TikTok and Instagram and they put it onto a new app and the app did not succeed, right? There are other apps that have tried to mimic things like the news feed, that have tried to mimic things like autoplay video or recommendation algorithms that have not taken off. And so I guess the question in my mind is like, if the litigation over social media is modeled after the litigation over Big Tobacco, shouldn’t there be like some industry-wide lift as a result of every sort of platform trying to borrow the most addictive features of Facebook and Instagram and YouTube?

Casey Newton: 10:45 I mean I hear what you’re saying and I think it’s an interesting point, but I think that internet platforms just work differently than cigarettes, right? Because you’re right, like with nicotine, like nicotine is just addictive. Now there are people that smoke cigarettes without getting addicted to them, right? But probably the majority of people do. Social media platforms are an imperfect analog to those cigarettes. I believe that platforms need to be of a certain scale in order for them to be truly addictive in the way that these plaintiffs are now suing about, right? There’s something about the fact that there’s hundreds of millions of people on Instagram and on TikTok creating content that creates that kind of infinite supply of things that you might potentially want to watch that is actually able to be addictive.

Kevin Roose: 11:57 But now you’re talking about the stuff in the container, right? Well, I think that there are many ingredients that all work together, right? But, but, but you’re raising a criticism that people are making of this lawsuit. Effectively, what I hear you saying is you cannot distinguish between the content and the—

Casey Newton: 12:10 I’m not sure. I mean, I think I’m open to being persuaded that you can, but to my mind, it’s like one lesson that you could take from this is that it is very bad to be a popular platform that engages these mechanics to keep users coming back, but it’s okay to be an obscure platform that does it,

Kevin Roose: 12:27 Right.

Casey Newton: 12:30 because that’s not going to have as much harm. So what’s really sort of at issue here is the fact that these platforms are very, very good and very, very popular at doing the thing that everyone else is trying to copy.

Kevin Roose: 12:41 Yes, and this is the approach that Europe has taken to regulating these platforms, right? They, they have certain like categories, and if you are a very large online platform, then you just have more responsibility. That makes intuitive sense to me. I think that bigger and richer and more powerful you are, the more responsibility that you have to society, right? And so in this particular case, you have companies like Meta, which we know are hiring cognitive scientists who are working very hard to figure out all the different ways that they can hack your brain to get you to look at Instagram for as long as they possibly can. It is in their interest to get you to look at Instagram as long as they possibly can. And right now, there’s just no brake on that at all in our society, except for this litigation. So I’m so sympathetic to these juries that are looking around, they’re seeing this completely unregulated platform and they’re saying something’s got to be done.

Casey Newton: 13:31 Yeah. So regardless of sort of what our thoughts on the overall sort of legal theory here are, like, what do you think the effects are on the platforms? If this does get held up on appeal, if these platforms are found liable for millions or potentially billions of dollars in damages against all of these people who claim that they were harmed by social media, does that mean that they have to, I don’t know, go back to like the reverse chronological feed of 2008? Does that mean they have to shut off, uh, you know, infinite scroll and autoplay and recommendations and all these other things?

Kevin Roose: 14:00 This is where it gets really tricky. And this is like maybe the one narrow way in which I’m sympathetic to the platforms, which is okay, the juries have said your product is defective. What juries have not said is here’s what an okay product looks like, right? They’re saying we don’t like this sort of set of features, but they’re not saying with any specificity, like, well, how do we think that these features are interacting, right? Like what is your actual model of the harm here? And so there is a world where the platforms feel like they have to comply and they maybe start picking off some of these features one by one, like okay, if you’re like under 16, we’ll disable infinite scroll, for example. Uh, how much benefit does that really have to like the individual teenager who may be struggling? I don’t know. This, of course, is why it would be great if Congress could pass some sort of law regulating this, but you know, we’re now like I don’t know, a decade into that project and still not getting very far.

Casey Newton: 14:47 Yeah. I mean, I think one prediction about how this will change platforms and their behavior is that if you start talking about gambling or addictiveness on an internal Meta chat room, uh, you just immediately stop doing that.

Kevin Roose: 15:00 get fired. There’s just like a little button on your seat that just presses and you get ejected out of the building.

Casey Newton: 15:08 Yes, I think so. Because so much of the incriminating evidence here just comes from people like spouting off in work chat rooms about like, ‘Oh, it really seems like this thing we’re doing is dangerous.’ And like I have to imagine that if it hasn’t happened already, they are just going to absolutely crack down on that kind of internal discussion.

Kevin Roose: 15:23 Absolutely. Well, so I want to hear a little bit more about how you think about this. Because you have talked on this show many times about your own struggles to look at your phone less. This is an issue that, you know, at various times you feel like has plagued you. Uh, so how are you feeling about the addictiveness of these platforms? Like, do you buy the sort of public health framing for the way that people are talking about them these days? Or do you think that this is overreach?

Casey Newton: 15:44 So I I need to do some more thinking about the product harm arguments here and whether it makes sense to me. I I am basically on board with the idea that there should be age-gating for social media. Uh, I am sold on the premise that, uh, there is a certain age, whether it’s 16 or or 18 or 14, where where sort of the the most harmful effects taper off. And I think before that age, it makes total sense to age-gate or at least give parents a lot more control over what their kids are able to do and not on these platforms. I think the addictiveness question is just hard for me because I feel like my my sort of macro theory on all this stuff is that what is happening to social media over time is that the social part is fading away and the media part is is rising in the mix. And so I think that if you start treating the design and mechanical decisions of these media platforms as uh harmful under the law, it just sort of leads me into into a place where I become much less certain. Like, before any of this existed, there were cliffhangers on TV shows that were designed to keep you coming back after the commercial break or to the next week’s episode or whatever. Those were arguably addictive features. They would keep people coming back. Is that illegal? I would say probably it shouldn’t be. Uh and it’s not. So I think there is a certain sense in which the the closer this that social media moves to something like TV or streaming video, uh the the blurrier the lines in my mind get between the content and the mechanics. What are your thoughts on all that?

Kevin Roose: 17:34 Well, I have to disagree. I do think cliffhangers should be illegal. Cause I want to know what happened! I don’t want to have to wait till the fall to find out, you know, if that person is still alive. Uh but also I do think that there are some really important differences between like let’s say YouTube and HBO Max, right? Like HBO Max is not like going to modify the content of HBO to your individual preference, right? Like they’re going to go pay some money for a bunch of shows and they’re going to hope a bunch of people watch.

Casey Newton: 18:00 watch them. The the platforms that we’re talking about are doing something very different, right? They’re looking across the entire corpus of like every video that’s ever been uploaded to their platform and they’re trying to figure out what will keep you personally here the longest and we’re going to show you that as much as we can. So I just do think that there’s a kind of categorical difference here and while I do think people should have broad freedom to, you know, look at whatever they want, I do think that at a minimum we should probably place an age gate on it for the same reason that we don’t let 14-year-olds walk into bars.

Kevin Roose: 18:31 Right. Unless they’re really cool and have a fake ID. So talk about the encryption piece because you had a lot about this in your newsletter that I didn’t quite understand but what what is the encryption debate that’s part of these lawsuits?

Casey Newton: 18:41 Yeah, so, you know, here I understand that I’m coming across as being broadly supportive of these jury verdicts, which I am, but I do want to acknowledge like this could lead to some really bad places. Like and this is why we need to to handle section 230 with care. In the New Mexico case, the attorney general argues that a reason that Meta should be considered liable in advertising their platform as being safe for children is that it includes encrypted messaging, right? In fact, Meta in March announced that they would discontinue encrypted messaging on Instagram in what I believe was an effort to sort of get ahead of this. What they said was, look, if you want to do use encrypted messaging you can use WhatsApp instead. But to me, this would be like a legitimately horrible outcome of all of this is if like every company that now offers encrypted messaging either voluntarily decided to stop offering it or was pressured by the government to stop offering it because in my view, encryption is a necessary part of privacy in a world where people are mostly communicating online.

Kevin Roose: 19:45 Are you comfortable with all of this happening in the courts through jury verdicts?

Casey Newton: 19:51 This is not my preferred way of addressing this, but I think it was inevitable in part because the tech companies have been so obstinate about making meaningful changes to their platforms. Right? Like societies across the world have been begging these companies for a decade: please do something to make these platforms safer and to make them less addictive and to reduce some of the harms. And instead what we’ve mostly seen is a series of engagement hacks designed to get people to look at them longer, right? And in the United States, where you cannot regulate the content of any of these apps for the most part, you can really only—you’re really only left with the design, right? You’re really only left with just the raw mechanics of the app. So if the social media platforms are upset about the verdicts here, I truly believe they brought this on themselves.

Kevin Roose: 20:37 I mean, you you asked me about my own experience of screen addiction and I I am not—I have never been sort of a a total screen addict but I have struggled like I think many, many other people have with like how much I’m using my phone, how much I’m using various apps. I have come up with sort of convoluted ways of trying to reduce my screen time.

Casey Newton: 21:00 You once were six hours late to a Hard Fork taping because you were just scrolling

Kevin Roose: 21:00 …find out what happened to Chimpanini Benanzini on TikTok.

Casey Newton: 21:03 I thought we agreed to keep that private.

Kevin Roose: 21:07 But like never in all my struggles with screen time have I thought to sue the companies that were making the apps that went on my phone. And I guess it’s different when you’re talking about kids, but like, there is some part of me that just feels like, well, it just feels like an easy way out, you know, blame the platforms. And look, I think these platforms absolutely have culpability here. I am not saying that I disagree with these jury verdicts. I think that these platforms, especially Meta, have done the research, have found the harms, and then have shielded them from the public. But I just, I guess I’m thinking about my own experience of these addictive platforms being one of like feeling bad about myself rather than trying to, you know, find someone else to blame.

Casey Newton: 21:56 Yes, but you also had the benefit of beginning to use these platforms when you were already an adult, right? Like your hippocampus was formed. And I think—

Kevin Roose: 22:06 I was on instant messenger from a very early age.

Casey Newton: 22:10 Do you really think that like messaging apps are as addictive and harmful in the same way as like TikTok or Instagram is for some people?

Kevin Roose: 22:18 Oh my god, take me back to 1999, put me on AOL Instant Messenger, I could not tear myself away from that thing. I had to put up a little message with, uh, you know, Get Up Kids lyrics on it every time I left the computer because it was such a rare event and I wanted my friends to know that I was away from keyboard. Casey, these things were addictive.

Casey Newton: 22:32 The kid got up. It’s a Get Up Kids joke. Yeah, I like look, I just think that messaging apps are different from these social platforms, you know? And I think, you know, honestly, like I will be curious, you know, who knows if Instagram and TikTok will be what they still are in like 10 years, maybe when your son is ready or wants to use social media. But I just think that it probably just feels very different than when you’re a parent.

Kevin Roose: 23:00 Well Casey, are there any new social media apps that you’re addicted to?

Casey Newton: 23:04 Um, it’s called Claude. And it’s really…

Kevin Roose: 23:08 Wait, I do want to talk about the AI of this all. So obviously every discussion on this show has to come back to AI at some point. So I’m curious like what effect you think this might have on some of these AI companies because they are also trying to create, um, experiences that are engaging, addictive, whatever you want to call it. I can imagine some of these, you know, lawsuits that are being brought against the makers of chatbots for harms like, it all feels like it’s sort of going to converge at some point. So what’s your take on that?

Casey Newton: 23:42 Yeah, so Pew did a study in 2025 and found that 64% of teens now use AI chatbots, about three in 10 use them daily. That same survey said that the teen use of YouTube, TikTok, Instagram, and Snapchat had remained relatively stable, right? So yes, chatbot usage is growing. It has not yet come at the expense of the social platforms, although of course I expect that we’ll soon see chatbots inside all of those platforms, right? And like, these things will all just kind of merge together. There’s something about these things where they do kind of go hand in hand, and to your point, I think that yes, AI chatbots will be the next frontier of this debate because in many ways they’re much more engaging and and I think like will be stickier than even these platforms are.

Kevin Roose: 24:26 Yeah. I mean it just seems so obvious to me that the platforms should be like absolutely begging Congress to regulate them because the alternative is like they just get sued into oblivion by a bunch of, you know, law firms.

Casey Newton: 24:39 I mean absolutely. Like if I were running one of the big AI labs, I would want to have an understanding from Congress of like, what do you consider a safe chatbot? Like give me a checklist that I can I can follow, because I don’t want to have to be dealing with this in, you know, the next few years.

Kevin Roose: 24:51 Casey, what’s an addictive engagement mechanism we could use to get people to come back after the break?

Casey Newton: 24:58 Well, we could study their behavior and weaponize it against them?

Kevin Roose: 24:59 Good idea.

Kevin Roose: 25:05 Well, Casey, if our listeners read one book about AI this year, it should be mine. But if they read two books, the second one should be Sebastian Mallaby’s new book, ‘The Infinity Machine: Demis Hassabis, DeepMind and the Quest for Superintelligence.’

Casey Newton: 25:21 Tell us about this book, Kevin.

Kevin Roose: 25:23 This book came out this week. It is full of a bunch of new anecdotes and stories about the work of DeepMind and the motivations that drive its CEO, Demis Hassabis. Sebastian is a longtime journalist, he’s a fellow at the Council on Foreign Relations, and he spent a long time with Demis and the people close to him and brought us this book about what I think is the AI frontier lab that gets the least coverage relative to its importance.

Casey Newton: 25:52 Yeah, and look, I mean Demis Hassabis is a singular figure. He’s been on Hard Fork several times, but Sebastian went really, really deep and I think maybe gave us the most fully featured portrait of the man that we’ve had to date.

Kevin Roose: 26:07 And before we bring him in, because we’re going to talk about AI, let’s make our disclosures. I work for the New York Times, which is suing OpenAI, Microsoft, and Perplexity.

Casey Newton: 26:14 And my fiancee works for Anthropic.

Kevin Roose: 26:16 Sebastian Mallaby, welcome to Hard Fork.

Sebastian Mallaby: 26:18 Great to be with you.

Kevin Roose: 26:20 So people who listen to our show are familiar with Demis Hassabis and DeepMind, he’s been on several times. What is something non-obvious about Demis that you learned through talking with him through many hours and interviewing many people who know him?

Sebastian Mallaby: 26:37 I mean, I think maybe the spiritual underpinning for his scientific curiosity was interesting. You know, there was one time when we were sitting in this London park and talking for a couple of hours and he suddenly started to say, you know, when I’m up at two in the morning at my desk by myself, thinking about science, thinking about computer science, I feel reality is screaming at me, staring me in the face.

Casey Newton: 27:00 waiting for me to explain it.

Sebastian Mallaby: 27:03 And he calls it the goddess Spinoza, that this is the 17th-century philosopher Spinoza who said that to understand nature is getting closer to God’s creation. And that resonates with Demis. Maybe that’s something people don’t know.

Kevin Roose: 27:16 That’s interesting. I mean, yeah, this has been something that’s come up in my own research too, is that, you know, he grew up going to church, I believe with his mother. And I think unlike a lot of the other sort of AI leaders, has a way of sort of fusing the science of AI with his own spiritual beliefs. And I know some folks have seen his ambition and his many years of competing to build AGI and have seen something suspicious in that, right? Elon Musk has this whole theory about how Demis secretly wants to be an evil AI dictator who takes over the world. And I guess I’m curious if in any of your reporting with him you ever saw something that seemed like what Elon Musk was talking about.

Sebastian Mallaby: 28:19 No, I mean to the contrary. I think this idea that Demis is a quote evil genius, which is the one that’s the phrase that Elon used to use, came from the fact that in his video game production days, Demis had created a game called Evil Genius. And so maybe it was a joke at first. But you know, really I got to know Demis extremely well. I spent more than 30 hours with him. You stress test people quite deeply as you know, Kevin, when you’re writing about them and then you might get pushback and legal threats and all that stuff. And he did make me talk to his lawyer once. And it wasn’t totally easy the whole time. But you know, he was reasonable in the end. And I think…

Kevin Roose: 29:30 Wait, why did he make you talk to his lawyer?

Casey Newton: 29:32 Yeah.

Sebastian Mallaby: 29:33 He was very mad at the fact that I unearthed the whole story about DeepMind trying to spin out of Google between 2016 and 2019. And you know, they retained a whole bunch of advisors, lawyers, bankers, etc. They got Reid Hoffman to pledge a billion dollars to finance the spin-out. They went to see, you know, Joe Tsai in Hong Kong, the Alibaba co-founder. Anyway, so the lawyer was not amused that I had all these internal documents from inside DeepMind which had been leaked to me, the board presentation that DeepMind gave to Google and so forth. And he said, you know, you’re not supposed to be writing about this. And I said, well, you know, people gave me this stuff and tough. So there were moments of oh, of free and frank discussion.

Casey Newton: 30:00 he’s doing training exercises but then what he thinks is like a test, essentially a video game, uh, accidentally wipes out an alien species. So I wondered if you talk with him about like why he relates to that story and in particular if there’s any relation to that and the idea of maybe trying to build a superintelligence.

Casey Newton: 30:09 I’ve always believed that when a source gives you secret documents, it helps you get closer to God’s creation. So that’s what I would have told him.

Sebastian Mallaby: 30:18 Well, uh, I was astonished, you know, this was before my first dinner with him, uh, and it was still in kind of the vetting process, it was the last part of the vetting process where he agreed to give me the access I needed. And he said, you know, ‘you gotta read this novel before you come and see me.’ And so I show up, I’ve read this story, it’s about a diminutive boy genius who basically saves humanity from aliens. And I’m thinking, does he really see himself as like saving humanity by doing what he’s doing with AI? And even if he thinks that, why would he be so crazy as to tell me? I mean, surely that’s hubristic beyond belief. Why would you put that out there? And you know, he made no secret about it. He’s like, ‘yeah, you know, I feel like I identify because this guy put all of his energy and his life into saving humanity, and I feel like I’m on a mission like that.’ And he said, ‘I, I felt so strongly about this, I gave it to my wife to read it thinking that she would understand me better and sympathize with me. And you know what? She sympathized with the kid Ender, but not with me. That’s not fair.’

Casey Newton: 30:19 I wanted to ask another question about childhood, because Demis told you that he really identified with the boy genius protagonist of the novel Ender’s Game and of relating to this feeling of being socially isolated by his own talent and consumed by a desire to make his mark on the universe. And the reason it struck me is that in this novel, Ender believes that he…

Kevin Roose: 31:24 Yeah, that’s really… I mean, one other character trait that comes up over and over again in reporting about Demis and especially in your book is how competitive he is. This is a guy who loves to win. You know, he was a child chess prodigy and he won this thing called the Pentamind, uh, you know, five times, which is sort of like an all-around gaming competition. Do you think that is part of his approach to AI? I mean, he’s always talking about how he wants to use this to solve scientific mysteries and cure diseases, but is some part of it just like, this guy loves to win and this is a really big contest?

Sebastian Mallaby: 31:55 Totally. I mean, that’s exactly right. I remember going to see him, you know, when ChatGPT was just going viral, and he said, ‘you know, Sebastian, this is war. These guys at OpenAI, they’ve parked the tanks in my front yard.’ He actually said, ‘parked the tanks on my lawn’ because he’s English. But yeah, you get it.

Kevin Roose: 32:16 You, you bring up the release of, uh, ChatGPT, which happens in, uh, November 2022. And I’d love to hear a little bit more about how Demis had reacted to that, because I think before that happened, Google really thought they were comfortably in the lead and did not seem to be feeling a lot of pressure to release anything. So I’m particularly interested if in hindsight Demis has regrets about the fact that they sort of let Sam Altman beat them to the punch.

Sebastian Mallaby: 32:42 Yeah, I mean he has an explanation more than a regret. And the explanation is super interesting. It’s basically that because he studied neuroscience for his PhD, and you gotta remember this is back in, you know, 2008, 2009, so nothing worked in AI. So he was starting from scratch. And one of the ideas in neuroscience— It’s called action in perception and this is the idea that to really be intelligent you have to take action in the world. You don’t know what it means for something to be heavy unless you pick it up. You don’t know what gravity is unless you actually drop something. And so he had this idea when the transformer paper came out in 2017 and OpenAI was starting to do, you know, the first GPT in 2018, second one in 2019 and so forth. You know, that’s not going to work. It’s not going to take you all the way to powerful intelligence because language is just a system of symbols. It’s not grounded in the real world. And it’s not that he was wrong in the sense that now we see world models come back in 2026 as a big area of excitement and research. But back in 2018, 2019 he was missing the fact that a huge amount of knowledge about how the real world works is in fact in language if you download all the language on the internet and he missed how much you could squeeze out of language as a training set.

Casey Newton: 34:01 Yeah, I mean…

Kevin Roose: 34:02 I want to run a theory by you Sebastian for your take, but as I’ve been working on my own book and about this sort of period at Google and at OpenAI and at DeepMind, it strikes me that there are sort of like two visions of what intelligence is that these companies disagree on. And in one vision it’s like intelligence is about winning, it’s about optimization, it’s about a contest between rival intelligences and that’s very much like the DeepMind sort of reinforcement learning paradigm which is like AlphaGo and you know you play a board game a bunch of times and you get better at it a little a little more every time. And then there’s this other view which is sort of the more OpenAI sort of language model scaling paradigm which is like no it’s about like answering questions. Like being very smart is about having the right answer to everything. Does that theory hold water with you Sebastian that there’s something like psychological about these two approaches to AI development that actually are rooted in like what we think intelligence actually is?

Sebastian Mallaby: 35:11 Yeah, I would say that the DeepMind special sauce right from the beginning was to try to put those two things together. It’s interesting for example that with AlphaGo, the early research on that Ilya Sutskever contributed to it. And of course he was, you know, the sort of leading practitioner of deep learning, went on to be OpenAI’s chief scientist. But at the time he was working for Google because Google had acquired his boutique. And so the the reinforcement learning people in London working for DeepMind collaborated with the deep learning people in Mountain View and that’s what produced the AlphaGo breakthrough. So I think I think you’re right, there are these two strands within AI of reinforcement learning which I would describe as learning through experience, interaction with the real world through trial and error. And on the other hand, learning through data and that is the deep learning. And for Humans you could think of it as being, you know, you can go to the library and read all the books and that would be deep learning, you’re learning from data from from sort of crystallized human knowledge, um or you can go out there in the real world and learn about stuff by, you know, planting your garden and whatever, you know, actually doing.

Casey Newton: 36:18 Yeah, you could be like Casey who has never read a book. So this sort of…

Sebastian Mallaby: 36:23 I get around to it one of these days.

Kevin Roose: 36:26 Learned by trial and error. So we’re sort of the two approaches here. Um, you mentioned earlier this, uh, I don’t know if it’s fair to call it a plot, it sort of seems like a plot that they had at one point after they had got acquired by Google to try to spin themselves out, I believe they call this Project Mario. I would love to hear a little bit more about how that came about and why they didn’t go through with it.

Sebastian Mallaby: 36:47 So what happened was that when they sold DeepMind to Google in 2014, they had a rival offer from Facebook. And Facebook actually offered them more cash. And one of the reasons they said no was that they wanted safety protections around their technology. And so they had this deal there was going to be a safety and ethics board, and Google promised that and they went ahead and sold to Google. And they had a first meeting of this safety and ethics board in 2015 after the acquisition and in order to, like, bind in the other people in the space, they got Elon Musk to host the whole safety and ethics board at SpaceX. They got Reid Hoffman to show up, and you will notice that then these are the characters who either found OpenAI or fund it, in those two. So Google wasn’t best pleased, as you can imagine.

Kevin Roose: 37:40 I have to say, that doesn’t seem like a very ethical thing to do, you know, maybe not the people I would have put on my ethics board, these characters.

Sebastian Mallaby: 37:50 But it’s a dichotomy, right, dilemma, I mean, you, you know, either you put people on the board who don’t know what they’re talking about and are not interested in AI, or they do know about AI, in which case they’re going to want to go and do their own thing because it’s too exciting not to. Um, and a fundamental mistake that Demis made in his early conceptualization of how AI would be developed was this notion that there would be one single lab producing AI on behalf of all humanity and therefore it could be safe because there’d be no race dynamic and you could take your time and sort of red teaming the models before you release them. And, um, you know, that’s why he brought Musk into the tent, that’s why he brought Reid Hoffman into the tent. Precisely because he thought we could all be one team together. Uh, and so then what happened after to answer your question, Kevin. So what happened after was that having lost that first experiment in setting up a safety and ethics oversight board, Google didn’t want to do another one and and really DeepMind’s project, Project Mario, was to try to force them to do more, um, by threatening to walk out if they didn’t.

Casey Newton: 39:00 Why did they call it Project Mario? Was that about the video game?

Sebastian Mallaby: 39:00 In the answer. No, sorry.

Kevin Roose: 39:02 And so how- That is much better than the- the alternative Project Wario they were working on which was just the evil version of that.

Sebastian Mallaby: 39:04 Good question. I don’t…

Casey Newton: 39:09 So how does Google get them to abandon this plan?

Sebastian Mallaby: 39:12 You know, it’s attrition. And Sundar Pichai, his personality and his management style comes out quite interestingly in this whole story because, you know, right at the beginning in 2015 when, you know, the first safety and ethics oversight board fails, the next idea for- for Demis has for how to get some independence and control of the technology is to become a bet as in an Alphabet when they were spinning out Waymo and, you know, some of the other side bets they had. And Larry Page was cool with this and he was CEO at the time. But then right as these discussions were going on, he handed over to Sundar. And Sundar kind of pretended to say, ‘Oh yeah, absolutely, great idea. We should look into it,’ but really he was just spinning them along and had no intention whatsoever of letting Demis spin out because he recognized him as the AI talent that Google was going to need in the future. And so essentially there was this long drawn out, you know, ‘delays here’ and ‘we should just look at some more details’ and ‘here’s another term sheet’—and I was given some of these term sheets, like huge great documents with red lines all over them where, you know, one team of lawyers had come back to the other team of lawyers. And, you know, basically by 2019 everybody was exhausted, it all fizzled out and they just moved on.

Casey Newton: 40:37 There’s been a lot of sort of jostling for independence within DeepMind ever since the earliest negotiations about selling to Google. Give us an update on how things are going with them now. Like, you know, when we talk to them they present things as being, uh, you know, fairly like hunky dory between everyone, but are there still kind of tensions and- and fault lines between Google and DeepMind?

Sebastian Mallaby: 41:00 Well, you know, I’ll give you sort of what I would regard as somewhere between probably true and unconfirmed rumor. Is that alright? Can I- am I allowed to do that?

Casey Newton: 41:23 Oh, please. Please.

Kevin Roose: 41:24 We love- we love gossip on this show. We’re kidding. Spill the tea. Yeah.

Sebastian Mallaby: 41:26 So I’d say that, you know, Sergey Brin is the troublemaker here. That he- at one of the Google IOs, I guess it was a couple of years ago, the stage was set out for two people to be on it, there was the interviewer and there was Demis, and suddenly Sergey kind of runs onto the stage, they have to get a third chair, and then he kind of inserts himself into that conversation. And, you know, what I hear is that that was the outward symptom of a much deeper tension where Sergey doesn’t really like Demis’s leadership on this and wants to push back against it. And- and I think it follows from that that the single most important business buddy act in all of technology right now is the relationship between Demis and Sundar. of capitalism today is the one between Sundar Pichai and Demis Hassabis because Sundar manages the board, manages the sort of high politics of Google and Alphabet, that Demis has the space, the resources, the oxygen to go do his science and without Sundar holding that all together we might be in a different place.

Casey Newton: 42:19 One area where Demis has changed his mind is about the use of AI in the military. This was a big sticking point in the negotiations with Google and Facebook back when they were selling DeepMind. He didn’t want their technology to be used for the military. Now obviously Google DeepMind has one of these Pentagon contracts. They’re working with the military. So what do you attribute that shift in his thinking to? Is it just kind of the realities of the market or needing to compete or what is it?

Sebastian Mallaby: 42:54 Yeah I mean Demis described this to me as ‘no you mature, you you get to know the real world’ and all that. You one might say how come you weren’t mature when you sold the company in the first place? I mean surely it was predictable. But I think that the the real truth of the matter is he did not predict, it comes back to this singleton idea which I mentioned before. He really thought there would be one lab. And in a scenario where there’s only one lab who’s got the technology then sure you can say to the military ‘you can’t have our technology, go away’. And the problem today is as we saw with Anthropic just now with the Pentagon, if Anthropic tries to draw a red line, you know OpenAI is in there like a shot and says ‘Hey Mr. Pentagon what do you need? We’ve got it for you’.

Kevin Roose: 43:32 Do you worry that Demis’s competitive streak or his pursuit of science, whatever it is that drives him, will compromise his ability to develop something like AGI safely?

Sebastian Mallaby: 43:46 You know I ask myself that question all the way through my research and and in some ways the question about can you be a strong consequential actor in the world and still be good is a sort of the deep question in the book. And he is somebody who really wants to be good. And I think one way of framing this question about ‘is he being good, will he be good, can he be good?’ is to say: should he, will he do what Dario did? Standing up to the Pentagon about red lines on military usage and surveillance. And I don’t think he is going to do that. And I think the way he would rationalize this would be to say: ‘Look, you gotta pick your moment with this stuff. If you make a stand and actually the Pentagon does what the hell it wants anyway, you didn’t really make the world better. My best shot at making the world better and making AI safer is to go through the route which is the only route that can get us to AI safety and that is government intervention, forcing safety rules on all the labs at once because otherwise some are safe some are not safe and the ones that are not safe are going to screw it up for everybody.’ And that’s the route that I think Demis wants to push. Problem is you have the Trump administration, they just want to accelerate. And so… All you can do for now, I think, is to keep this conversation alive with other governments and then maybe when there’s a new administration in the US we could see a conversation.

Casey Newton: 45:09 Mm. You write that Demis used to inform job candidates at DeepMind that if they signed on, they should, quote, ‘prepare for a climactic endgame when they might have to disappear into a bunker.’ Why would they have to disappear into a bunker? And do they still tell the job candidates that?

Sebastian Mallaby: 45:27 Ha! Yeah. Um, so the idea was when you get very close to AGI and it’s super dangerous, you’re going to A, be subject to potential attack by bad guys who want to steal the technology, and B, you really don’t want to be distracted by quotidian real-world stuff, so you disappear into the desert.

Casey Newton: 45:52 Mhm.

Sebastian Mallaby: 45:53 Yeah, that’s right, you leave your TikTok on your phone in some… I think Kevin used to lock his phone up in a box as I recall.

Kevin Roose: 46:00 That’s correct.

Sebastian Mallaby: 46:01 Um, and so you do a Kevin and you go and you really, really focus, um, and and you really get the AI right in the last stages. That was sort of Demis’s vision. And to test whether he really meant it, I was having dinner with somebody who used to be at DeepMind in that period around 2015, 2016, and had now left. And I said, ‘This wasn’t really true. People didn’t really…’ ‘Oh yeah, yeah.’ If… This guy said to me, ‘If Demis had told me anytime when I was working at DeepMind that I had to take the next flight to Morocco…’

Casey Newton: 46:39 And hide.

Sebastian Mallaby: 46:40 …and hide, I would have said I’d been given fair warning.

Casey Newton: 46:44 Wow. So the bunker is in Morocco, just so everyone knows. Yeah.

Sebastian Mallaby: 46:49 Yeah, and I said why Morocco? And he said, well, you know, it’s the desert and, you know, the Manhattan Project was in the desert.

Kevin Roose: 46:56 Mmm. These guys and their Manhattan Project analogies, man. It’s… I don’t know if they read to the end of that story. Uh, didn’t go that well. Sebastian, you spent many years writing about hedge funds, and I remember encountering your work back when you were writing about hedge funds and hedge fund managers. You’re now spending time with the new masters of the universe. And I’m curious what, if any, observations you have about how those two classes of people, the AI leaders and the hedge fund managers, are similar or different?

Sebastian Mallaby: 47:22 Well, um, I would say that the hedge fund guys are playing a game inside a set of fairly well-understood rules. They’re not rethinking humanity, they’re not rethinking everything about society, they’re not changing the way we bring up our kids, they’re not changing the conception of what it means to be human.

Casey Newton: 47:42 Speak for yourself. I’m training my kid to do algorithmic arbitrage. He’s four. Terrible at it. He’s down 200% this year. Anyway, sorry, carry on.

Sebastian Mallaby: 47:52 Yeah, no, but I look, I just think that AI is so, so much bigger than, you know, some kind of event-driven arbitrage or whatever you want to talk about with hedge funds.

Casey Newton: 48:00 Yeah, maybe a last question from me. I have a question about the writing of this book and how you decided to frame it. You know, it strikes me, Sebastian, that we don’t know how AI is going to go. You know, we don’t know whether AI is going to turn out to, you know, cure a bunch of human disease and usher in a utopia or usher in these like far darker scenarios. I think it’s clear that you have a lot of respect for Demis and the work that he’s doing, but there’s also this risk that things go really, really badly. So I’m curious, as you wrote the book, how you approached that tension and the sort of not knowing of how history is going to judge this person who you’ve now gotten to know so well.

Sebastian Mallaby: 48:44 I thought of the book as a book about that tension. In other words, I’m trying to do a portrait of somebody who has his hands on the 21st century version of the nuclear material, who has that tingling sense of playing with something that could destroy humanity. Um, what does it feel like when you’re creating that? Can you sleep? How do you live with it? Um, and I think I’ve delivered a portrait of somebody who’s in that hot seat. And hopefully that remains interesting for some time and it’s not something that depends on how this AI development story ends.

Casey Newton: 49:25 Sebastian, thanks so much for coming on. Really great to have you.

Kevin Roose: 49:28 The book is called The Infinity Machine and it is out now.

Sebastian Mallaby: 49:31 Thank you, Kevin.

Casey Newton: 49:32 Thank you, Sebastian.

Kevin Roose: 49:34 And Casey, thank you. When we come back, a game of Hat GPT. It involves snowmen.

Kevin Roose: 49:42 Alright Casey, well we took a little break last week and there’s been a lot of tech news, so we feel like we should do a roundup and play a round of Hat GPT. Hat GPT, of course, the game where we put recent news stories into a hat, draw slips of paper out of the hat, discuss them, and then when one of us gets bored, we say to the other ‘stop generating’. And if you can’t see us, we’re using the Hard Fork hat official merch. And Casey, it appears that these are sold out at the New York Times store.

Casey Newton: 50:08 Not that specific hat, which was of course a Hard Fork Live exclusive.

Kevin Roose: 50:11 Yes, this is an exclusive. You can’t get this one, but you also can’t get any of the other ones. Here’s the important point. You cannot get a Hard Fork hat anymore so stop trying. Now someone did suggest to me the other day that we should make hard hats for Hard Fork, like a yellow construction vibe.

Casey Newton: 50:32 Well, we could wear them over to the new studio, which is being built for us, right?

Kevin Roose: 50:35 That’s true. Do you think we should make that?

Casey Newton: 50:38 Yeah, Hard Fork hard hat, that’s a perfect piece of merch.

Kevin Roose: 50:41 Great. Alright Casey, you go first.

Casey Newton: 50:43 Alright Kevin. This first story comes to us from 404 Media: an AI agent was banned from creating Wikipedia articles, then wrote angry blogs about being banned. I feel like I’ve heard something like this before. So, Kevin, once again, agents are writing…

Kevin Roose: 51:00 blog posts, what do we make of this?

Casey Newton: 51:02 This would never happen on Grok-opedia.

Kevin Roose: 51:07 No, I think look, I think this is just going to be the year that every system on the internet that is built on human contribution and review is going to break. And it will break not only because of the AI tools, but because people are letting them loose onto websites where they are doing things like editing Wikipedia articles and defaming people who, you know, contribute things to GitHub projects. We heard from Scott Shambaugh about that on a previous episode. But I think this is going to be a challenge. I’ve started talking about the inbox apocalypse that is going to hit this year where everything that is normally sort of reviewed and bottlenecked by humans is just going to be overwhelmed and flooded with AI submissions.

Casey Newton: 51:48 Absolutely. I mean I’m already getting emails now every week from something claiming to be an AI agent that says, you know, it’s running a company, you know, but it’s always sort of like, let me know if you want to talk to my human. And I’m like, your human better hope I don’t catch them in a dark alley because this does not belong in my inbox or frankly anywhere.

Kevin Roose: 52:07 I’m getting these too. It’s like, it’s a total scourge. It’s somehow even more annoying than the like faceless PR spam that you and I get.

Casey Newton: 52:15 Yeah, just to be very clear, there is not one thing that anyone’s agent could do or say to get me to respond to it in any way. So use that information what you… I hope that goes into your training data. Stop generating!

Kevin Roose: 52:25 All right. Next up, this one comes to us from Sean Hollister at The Verge titled, I met Olaf the frozen robot who might be the future of Disney parks. Sean reported in mid-March about his interaction with a new animatronic Olaf the snowman robot from Frozen. It weighs 33 pounds, it was trained with an NVIDIA GPU and is controlled by an operator using a Steam Deck. But when it made its debut at Disneyland Paris, well, Casey, something happened. Should we take a look?

Casey Newton: 52:54 Let’s take a look.

Kevin Roose: 52:56 All right, Olaf the snowman is talking, waving his stick arms…

Casey Newton: 52:58 Oh boy!

Kevin Roose: 52:59 No! We lost him! Olaf!

Casey Newton: 53:02 Oh, oh, it’s… oh.

Kevin Roose: 53:04 There’s something about the way that he very slowly falls onto his back. Oh no.

Casey Newton: 53:08 Yeah. 20 children just got lasting trauma. They’re going to be talking about this in therapy.

Kevin Roose: 53:13 Look, what do you expect? Like, of course he was frozen. That’s what the whole movie’s about.

Casey Newton: 53:19 Do you want to kill a snowman?

Kevin Roose: 53:23 Okay. I mean there is… it’s just reliably very funny when you create an animatronic thing for a child and then it is like revealed to be a machine and it just sort of feels like a Lovecraftian horror. Like something about that transition from like a cutesy cuddly thing to like its eyes are, you know, bulging out of its head and sparks start flying out of the back?

Casey Newton: 53:44 I’ll never forget the day at Chucky Cheese as a kid where I learned that the guitar playing mouse wasn’t real.

Kevin Roose: 53:51 You know Chucky Cheese’s full government name, right?

Casey Newton: 53:54 What is it? You don’t know it’s not a joke, it’s Charles Entertainment Cheese.

Kevin Roose: 54:02 Come on!

Casey Newton: 54:03 I swear to God.

Kevin Roose: 54:05 I learn something new every day from you.

Casey Newton: 54:08 Stop generating. All right. Now it’s my turn. Uh, well this Kevin is a story about the Claude Code leak. So Kevin, what do you make of this Claude Code leak?

Kevin Roose: 54:23 Well, I think it’s a big deal, in part because the agentic sort of coding harness that is around Claude Code is really the special sauce, right? It’s the model underlying it is, is part of what makes Claude Code and other agentic coding systems good at coding, but it’s really all the stuff around it, and that’s what leaked. It is not the actual, like, weights or the source code of Opus 4.6 or whatever model people are running inside Claude Code. It’s like the sort of apparatus around it that makes it quite effective. So, within hours of this leak, there were people who had cloned it and set up their own versions of it. I imagine it’s a very busy week over at the Anthropic legal department trying to get all this stuff taken down. But look, I think this kind of thing was inevitable, maybe not at Anthropic, but like, the agentic coding tools were all going to get good. They were all going to sort of reverse engineer Claude Code and figure out what made it better, but I think this probably just accelerated that.

Casey Newton: 55:25 When I saw this, my first thought was right now Kevin Roose is somewhere vibe coding Claude Code using the downloaded leaked Claude Code harness.

Kevin Roose: 55:31 I have not yet downloaded the leaked Claude Code harness, but I have seen other people sort of taking it and then putting it on top of like an open source Chinese model or something and sort of Frankensteining their own sort of version of Claude Code that they can run. And I will say, the closer I get to my rate limits on Claude Code, the more I’m tempted to do something like that.

Casey Newton: 55:53 That makes sense. Here’s the last thing I’ll say, if Anthropic is looking for a new harness for Claude, they might want to pick one up at Mr. S Leather in San Francisco down in the Folsom district. They have some really nice options down there. All right, stop generating. Okay, okay, next up out of the hat. Oh, this one is good. The AI fruit drama on TikTok that’s too juicy to pass up. This one says we should watch a clip from NBC News.

Sebastian Mallaby: 56:19 Welcome to Fruit Love Island, where eight single fruits are about to flirt, fight, and trust. Things get messy fast. The guy I want to couple up with is Benenito. Actually, I’m not letting that happen. I pick him.

Casey Newton: 56:35 So this is like sort of a Love Island style reality show featuring AI generated fruits. There’s a very ripped banana who is, you know, attracting attention from the lady fruits. And it’s all very silly, but this is going mega viral. This is the big new trend.

Kevin Roose: 56:52 I just watched a banana kiss a pineapple and that’s not in the Bible.

Casey Newton: 56:58 Do you think… I could win a multi-million dollar jury verdict for being forced to watch that.

Kevin Roose: 57:04 I think it’s a fair question.

Casey Newton: 57:09 I’ll say this, my mental health did not improve watching Fruit Love Island.

Kevin Roose: 57:15 Watch what happens with the passion fruit in season three.

Casey Newton: 57:18 Stop generating.

Kevin Roose: 57:20 Mm. All right.

Casey Newton: 57:22 This company is secretly turning your Zoom meetings into AI podcasts. This one also comes to us from 404 Media. And here’s a name for a company, Webinar TV.

Kevin Roose: 57:36 Wow. Now you’re talking.

Casey Newton: 57:38 Two great tastes that taste better together. Webinar and TV.

Kevin Roose: 57:41 Has there been a worse word in the English language than webinar?

Casey Newton: 57:44 Not to my knowledge. Apparently, this company is secretly scanning the internet for Zoom meeting links, recording the calls and turning them into AI-generated podcasts for profit, Kevin.

Kevin Roose: 57:56 Oh my god.

Casey Newton: 57:58 In some cases, people only found out that their Zoom calls were recorded once Webinar TV reached out to them to say their call was turned into a podcast in an attempt to promote Webinar TV’s services.

Kevin Roose: 58:08 What is happening? What is going on? Okay, I want to start by saying, I am committed to making a podcast with you for the rest of my life. But if we ever get overtaken on the charts by an AI-generated Webinar TV podcast that’s been trained on people’s boring-ass Zoom meetings, I am leaving this industry.

Casey Newton: 58:31 Here’s why this is such great news. I think a lot of podcasters struggle with the idea that maybe their podcast, you know, maybe they didn’t have a great episode, maybe they’re wondering, like, is this thing good enough to put out on the internet? Congratulations, because every single human-made podcast is better than every single Webinar TV episode that’s ever been released.

Kevin Roose: 58:51 Yeah, I mean, I’m just like, these have to be the most boring podcasts ever created. Like, what are you gonna talk about? Is it called Action Items? Is it called Circle Back? What’s the title of this podcast?

Casey Newton: 59:01 Touch Base, a limited eight-part series.

Kevin Roose: 59:04 I heard there’s a great series over on Webinar TV right now. It’s called, “Oh, I think you’re on mute.”

Casey Newton: 59:11 So you may want to check that one out. All right, stop generating. Next out of the hat, we have North Korean hackers suspected in Axios software tool breach. This comes to us from Bloomberg, and it’s about Axios, not the media company.

Kevin Roose: 59:24 I actually would prefer to read a story about this from Axios, if you have one on hand.

Casey Newton: 59:29 This is a tool, an open-source tool widely used to develop software applications. This has been a big security breach. Hackers were able to breach one of the few accounts that can release new versions of Axios late on Monday and publish malicious versions. Axios is downloaded about 80 million times every week. Anyone who has downloaded the malicious version of Axios could then have their own computer and the data on it stolen by hackers. This is being attributed to North Korea. Seems really bad.

Kevin Roose: 59:59 Yeah, man. Like there’s a lot of…

Casey Newton: 1:00:00 I feel like every cyber security incident we’ll talk about where it’s like, you know, but like no personal data was stolen or like, you know, nothing sensitive was at risk. This is one where it’s like, no, like everything was at risk. Like this is one of the bad ones and, uh, you know, if you’ve been messing around, uh, with NPM over the past week, you probably need to take a look at this.

Kevin Roose: 1:00:17 Yeah, it’s really, I think this is going to be one of the biggest stories of the year is just what is happening in cyber security right now. Um, I was watching this YouTube video. If you ever, you know, need something to keep you up at night, watch a talk given by this guy, Nicholas Carlini, who’s a security researcher at Anthropic at a cyber security conference recently. It is like the most terrifying, you know, conference speech ever given because what he’s basically saying is these AI tools have gotten better than almost any human hacker, any human security expert at finding vulnerabilities in tools, even tools that have been around for decades, like the Linux kernel, these language models are now finding bugs in them and basically every piece of code that exists is going to need to be rewritten and substantially hardened because we are facing like an onslaught of these very sophisticated AI tools that can find every little bug and problem in them.

Casey Newton: 1:01:23 Well, I am going to watch that talk as just as soon as I’m finished watching Fruit Love Island. Um, but you know, the thing that this brought to mind for me, Kevin, was that last week, while we were away, there was this Anthropic leak where someone found a draft of a blog post that said that Anthropic was delaying the release of its next model so that it could share it with cyber defenders, basically. Um, to my knowledge, we had not seen something like this happen since GPT-2 in 2019. One of the big labs said like, essentially, we’re afraid to release this thing because of what it might, uh, rot. What is the present tense? What it might wreak?

Kevin Roose: 1:01:58 Yes.

Casey Newton: 1:01:59 Because of what it might wreak. That’s wreak with a W, yeah, not with the R. Yeah. Speaking of reeking, take a shower next week.

Kevin Roose: 1:02:07 Hey, I was in a hurry.

Casey Newton: 1:02:09 Alright, stop generating. Okay. Okay, so this is actually a two-parter, Kevin. Uh, two stories about OpenAI recently that caught our attention. One, uh, Sora has shut down, which was a prediction that I made at our year-end episode. My low-confidence prediction is that OpenAI is going to retire Sora and it’s already come true by March. Um, and then a second story, which I think actually, crazily enough, is related, OpenAI has apparently shelved its plans to release the erotic chatbot or sort of like the adult mode that it said that it was going to be bringing soon to ChatGPT in an effort to boost engagement. So, Kevin, dying to know what you made of those two changes.

Kevin Roose: 1:02:53 So I think you were smart to predict the end of Sora. I think the— The story with Sora never quite made sense to me. Like, it was obviously a very cool piece of technology. It was devastatingly expensive to run is my understanding. Like, generating all those short videos was, like, computationally quite pricey. And so I think they are making the decision to sort of spread their bets a little less and consolidate around, like, a few projects. One being enterprise AI, one being coding and sort of automating AI research. But I think they maybe made a few too many side bets in the past couple of years that they are now seeing were expensive and diverted resources away from the core.

Casey Newton: 1:03:43 I have to say, I was personally really glad to see both of these changes. Like, like the release of this infinite slop feed app last year and the, the company saying that they were going to release this adult mode while they were still having all of these issues with, like, psychological problems that some of their users were experiencing as a result of getting a little too close to their chatbots. I just thought both of those seemed like really irresponsible moves and just, like, contrary to what they said their mission was. So I was actually just really happy to see them say, you know what, we’re not doing any of these things anymore. Like, I think that was the right move. Now, did they do that out of the goodness of their heart and some sort of, like, you know, moral awakening that they had? No. They saw Anthropic, which had started to print money because Claude code was taking off, and they said, we want to get a piece of that. But hey, whatever it took, I’m just glad it’s happening.

Kevin Roose: 1:04:21 Yeah.

Casey Newton: 1:04:23 All right. Stop generating. Last up in the hat.

Kevin Roose: 1:04:25 Kalshi announces itself as the safe, regulated prediction market in a new ad campaign. Kalshi has recently been putting up green ads around DC and I’ve actually seen them in San Francisco. The first one says: rule number one, Kalshi bans insider trading. The second one says: rule number two, we don’t do death markets. Casey, your take.

Casey Newton: 1:04:59 Rule number three, we’ll always shoot you in the front, never in the back. Who are these people?

Kevin Roose: 1:05:02 What, like, these ads are raising a lot of questions already answered by the ads. Truly, truly. It’s just so funny to me. Like, you know, I went to this prediction markets conference, like, several years ago now.

Casey Newton: 1:05:22 I predicted you were going to bring this up, but go ahead.

Kevin Roose: 1:05:24 And like, people from Kalshi were there, people from Polymarket were there, people from all these, like, you know, obscure, like, prediction markets. And it was like 50 people. It was like who were interested in this stuff. And it wasn’t legal at the time, and so they were all using, like, sort of play money and, like, workarounds. And it was it just seemed like, like no part of me was like, in three years this will be the dominant industry in America and they will be taking out bus ads to tell people that they don’t do death markets.

Casey Newton: 1:05:49 I know, but at the same time I keep reading all of these, like, stories and blog posts that are like, you know, why is this generation turning to prediction markets? Is this, like, really the only future they see for themselves? It’s like… No, they used to be illegal and now they’re legal. People love to gamble, if you let them, you are now letting them gamble. So that’s why they’ve hooked this younger generation.

Kevin Roose: 1:06:09 Yeah, you don’t think it’s because of the information harnessing potential and the wisdom of the crowds?

Casey Newton: 1:06:14 I really don’t. I’m still waiting for the wisdom of the crowds on a Kalshi market to improve my life.

Kevin Roose: 1:06:19 Yeah, well, you’re not going to find it when it comes to death or insider trading.

Casey Newton: 1:06:22 Kalshi rule number four: gambling is bad. That’s the ad I dare them to put up.

Kevin Roose: 1:06:31 Alright, let’s close the hatch, Casey.

Casey Newton: 1:06:33 Close up the old hatch.

Kevin Roose: 1:06:35 That was Hat GPT.

Casey Newton: 1:06:36 Lot going on.

Kevin Roose: 1:06:37 Lot going on. Busy week.

Casey Newton: 1:06:38 Busy week.

Kevin Roose: 1:06:39 Never a dull day here in Silicon Valley.

Casey Newton: 1:06:41 No sir.