How We Got to the Biggest I.P.O. Race Ever | SpaceX, Anthropic & OpenAI

Summary

The biggest IPO race in capitalist history is happening this summer. Kevin Roose and Casey Newton break down what’s at stake as SpaceX, Anthropic, and OpenAI all rush toward the public markets. SpaceX is closest, planning to sell shares at $135 to raise $75 billion at a $1.75–2 trillion valuation — what would be the largest IPO of all time. The combined “Frankenstein” conglomerate now bundles the great businesses (rockets, Starlink) with the terrible ones (xAI and X.com), which Casey argues is Elon Musk hiding losses inside winning ventures. Anthropic just filed a confidential S-1 expected to value the company at over a trillion dollars — staggering for a company that just three years ago, when Casey first visited their Jackson Square walk-up, seemed to actively resist the idea of making money. OpenAI is expected to file imminently.

The hosts then explore three knock-on effects. First, San Francisco inequality: even friends earning mid-six-figures are panicking about people who got in early at Anthropic or OpenAI, and the city’s swinging from a 2010-era “abundance” mindset back into a scarcity one where you either won the lottery or you didn’t. Houses are already being sold for Anthropic or OpenAI stock instead of cash. Second, philanthropy: thanks to Anthropic’s effective altruism roots, all eight co-founders pledged to give at least 80% of their wealth to charity, and Anthropic’s stock-matching program is up to 3:1 for some early employees. Nan Ranshanoff has called this “the third wave of philanthropy” — Gates-Foundation-scale capital flowing every year for the next several. The joke is it’ll be a great year for shrimp welfare. Third, AI safety: Kevin worries that public-market pressure on these companies will make it harder to refuse to ship dangerous models. Both Anthropic and OpenAI’s public benefit corporation structure helps somewhat, but as Kevin puts it, “they’re still corporations, and they still exist at the pleasure of shareholders.”

The episode then welcomes Kevin Hartnett — author of the new book The Proof in the Code and editorial lead at Cursor — to discuss the seismic shifts in AI and math. Last summer, three labs (Google DeepMind, OpenAI, Harmonic) hit gold-medal scores at the International Math Olympiad, but that was still just high school math — “0% of the way to the frontier” of research. Then a month ago, OpenAI announced its model had solved the Erdős unit distance conjecture, an important problem that human mathematicians had genuinely tried and failed to solve. Hartnett describes a polarized field: at the Institute for Advanced Study in Princeton, he met one top mathematician who said Gemini is useless and another who thinks AI will put mathematicians out of business in two years. Terence Tao — the greatest living mathematician — sits in the middle, treating AI as a “jetpack for your thoughts,” but even Tao signed the Leiden Declaration, a worried open letter from 800+ mathematicians about AI’s irresponsible use in their field. The hosts close with HatGPT: Trump’s new (still voluntary, now 30-day) AI executive order, a startup that secretly trashed Airbnbs while training robots, hackers who simply asked Meta AI for celebrity Instagram credentials and got them, a United flight forced to turn around because a 16-year-old’s Bluetooth speaker was named “bomb,” and George Santos getting investigated for insider-betting on his own State of the Union no-show.

Highlights

”The biggest IPOs of all time”

“It really does seem like it is going to be a hot IPO summer, Kevin. I am told that we are on track to see what might be the three biggest IPOs of all time.” — Casey Newton, 0:33

yt-dlp --download-sections "*0:33-1:30" "https://www.youtube.com/watch?v=gIkyiAEpu7E" --force-keyframes-at-cuts --merge-output-format mp4 -o "biggest-ipos-ever.mp4"

”Two great businesses and two terrible ones”

“There are two great businesses in here, right? One is a reusable rocket business… and Starlink is just on fire. So there are two amazing businesses in there, and then they also have two terrible businesses called xAI and X.com. And it’ll be really interesting to see what the interplay of the good businesses and the bad businesses are.” — Casey Newton, 3:16

yt-dlp --download-sections "*3:16-4:30" "https://www.youtube.com/watch?v=gIkyiAEpu7E" --force-keyframes-at-cuts --merge-output-format mp4 -o "spacex-good-and-bad-businesses.mp4"

”Anthropic actively resisted making money”

“When I tell you that this company was not only ambivalent about making money, but seemed to actively resist the idea of making money… But boy howdy did they make a product and decide that they actually liked making money and wanted to make a lot of it. And now they’re going to be one of the largest IPOs of all time just three years after I was hanging out with them in their little like Jackson Square walk-up office.” — Casey Newton, 5:26

yt-dlp --download-sections "*5:26-6:41" "https://www.youtube.com/watch?v=gIkyiAEpu7E" --force-keyframes-at-cuts --merge-output-format mp4 -o "anthropic-actively-resisted-money.mp4"

”A great year to be a shrimp”

“The joke going around the sort of AI circles is that it’s just gonna be a great year for shrimp welfare because shrimp welfare for arcane reasons that are probably not worth going into here, has become like a sort of half-joking pet cause of the effective altruists. But it’s a great year to be a shrimp.” — Kevin Roose, 14:51

yt-dlp --download-sections "*14:51-15:31" "https://www.youtube.com/watch?v=gIkyiAEpu7E" --force-keyframes-at-cuts --merge-output-format mp4 -o "great-year-for-shrimp.mp4"

”We cannot have wealth this concentrated in so few hands”

“I’m gonna go further and saying it’s not just good, I’m gonna say it is necessary. Like, we cannot have a very small handful of companies that are growing this quickly, that are concentrating wealth and power this much into so few hands. It has to be shared more broadly than that, and while an IPO is a very small step in that direction, I do think it is a necessary one.” — Casey Newton, 22:45

yt-dlp --download-sections "*22:17-23:04" "https://www.youtube.com/watch?v=gIkyiAEpu7E" --force-keyframes-at-cuts --merge-output-format mp4 -o "wealth-too-concentrated.mp4"

”AI can do top-tier math research”

“About a month ago, OpenAI came out with this big new result. They’ve solved what many people think of as one of the most important Erdős problems, this thing called the unit distance conjecture… Pretty unanimously people agreed this could be published in the Annals of Math, the top journal in math… this result really said AI can do absolutely top-tier research.” — Kevin Hartnett, 33:16

yt-dlp --download-sections "*33:16-34:56" "https://www.youtube.com/watch?v=gIkyiAEpu7E" --force-keyframes-at-cuts --merge-output-format mp4 -o "ai-top-tier-math-research.mp4"

”Two mathematicians, two opposite views, one afternoon”

“A couple weeks ago, I was at the Institute for Advanced Study in Princeton… I ran into two 40-year-old mathematicians, top of the field. One of them told me he’d just tried to do some math with Gemini and he’s like, ‘that stupid thing told me like XYZ thing… I closed it and I went back to doing math the other way.’ An hour later, a guy who on paper looks a lot like the first guy says, ‘I think in two years, AI is going to put mathematicians out of business.’” — Kevin Hartnett, 36:04

yt-dlp --download-sections "*36:04-37:49" "https://www.youtube.com/watch?v=gIkyiAEpu7E" --force-keyframes-at-cuts --merge-output-format mp4 -o "two-mathematicians-opposite-views.mp4"

”Bluetooth speaker named ‘bomb’”

“Everyone complied except for one speaker that belonged to a 16-year-old boy and was named ‘bomb’.” — Kevin Roose, 57:18

yt-dlp --download-sections "*57:00-58:00" "https://www.youtube.com/watch?v=gIkyiAEpu7E" --force-keyframes-at-cuts --merge-output-format mp4 -o "bluetooth-speaker-bomb.mp4"

Key Points

• The hot IPO summer of 2026 (0:27) - SpaceX, Anthropic, and OpenAI all racing to public markets, potentially the three biggest IPOs ever
• SpaceX numbers (1:48) - $135/share, $75 billion raise, $1.75–2 trillion valuation, the largest IPO of all time
• SpaceX is now a “Voltron-like” conglomerate (2:37) - Rockets, Starlink, xAI, X.com — all bundled together
• xAI pivoting to neo-cloud (4:01) - Now renting compute it originally built for itself to Anthropic
• Anthropic filed confidential S-1 (5:00) - Expected over a trillion-dollar IPO; ARR went from ~$1B in January 2025 to ~$50B more recently
• Casey’s first Anthropic visit in 2023 (5:26) - “Glum and strange” office, safety-obsessed people who hadn’t decided whether they wanted to make a product
• OpenAI S-1 imminent (6:58) - Expected to file with the SEC as soon as this week
• San Francisco status anxiety (7:55) - Even mid-six-figure friends are looking at OpenAI/Anthropic early hires and panicking
• Houses for sale asking Anthropic/OpenAI stock instead of cash (10:21) - Real story in San Francisco this week
• Anthropic co-founders pledge 80%+ of wealth (13:44) - All eight founders pledged at least 80% to charity
• 3:1 employee stock matching (13:44) - Anthropic matches employee philanthropy pledges share for share, 3:1 for some early employees
• “Bigger than the Gates Foundation every year” (14:44) - Scale of philanthropic capital coming, per Nan Ranshanoff’s “third wave” analysis
• Public benefit corporations don’t fully insulate from market pressure (16:14) - They’re still corporations, still exist at pleasure of shareholders
• Securities fraud cuts both ways on safety (18:35) - If you ship a bioweapon-capable model, shareholders can sue you for not releasing only safe models
• Index fund rule changes (20:20) - Nasdaq 100 and S&P loosening their “seasoning periods” so IPO’d AI companies join indexes faster
• Wealth concentration must be broken (22:45) - Casey’s strong statement that IPOs are necessary to distribute AI gains
• Starlink in airplanes (23:04) - Kevin: “this is going to be the biggest company in the world” after first Starlink-equipped flight
• AI hit IMO gold medal last summer (26:51) - Google DeepMind, OpenAI, Harmonic — but it was still just high school math
• 0% of the way to research frontier (27:23) - Hartnett: IMO is “like barely even wading into the field”
• Why labs care about math (28:40) - If you can teach a model to reason about math, it’ll do other commercially valuable things
• Erdős problems (30:43) - 1,200 problems left behind by Paul Erdős, mostly with cash rewards still funded after his death
• Unit distance conjecture solved by OpenAI (33:16) - One that humans had actually tried; result is publishable in Annals of Math
• Three mathematician attitudes (36:04) - “Useless,” “jetpack for thought” (Tao), “putting us out of business in 2 years”
• The Leiden Declaration (37:49) - 800+ mathematicians (including Tao) signed worried open letter
• Math archive banning AI metadata (39:36) - Year-long ban for posting unedited AI prompts in a PDF
• Is math invented or discovered? (45:50) - Hartnett quotes Terence Tao: feels like creation, ultimately an act of discovery
• Robots secretly trashing Airbnbs (48:20) - Bot Company being sued for $12,383.50 in damages after training in SF home
• 30-day AI EO (50:30) - Trump signed scaled-back voluntary model review (down from 90 days) after David Sacks objected
• George Santos under investigation (52:50) - Kalshi referred him for betting against his own State of the Union attendance
• Meta AI hacked celebrity Instagrams (55:03) - Hackers asked Meta’s support bot to change email and got access to Obama White House, Sephora, Space Force accounts
• Survivor spoiled by prediction markets (59:04) - Jeff Probst: Kalshi and Polymarket are “incentivizing people to lie, cheat, and steal”
• Google engineer charged with Polymarket insider trading (1:00:58) - Made a million by betting on what users were searching for

Mentions

Companies

• SpaceX (0:43) - Closest to IPO; rockets, Starlink, xAI, X.com all rolled in
• Anthropic (0:43) - Filed confidential S-1 this week; Casey’s fiancé works there
• OpenAI (0:43) - Expected to file IPO paperwork soon; being sued by NYT
• xAI (2:37) - Inside SpaceX now; renting compute to Anthropic
• X.com (2:37) - Social network inside SpaceX; “two terrible businesses” along with xAI
• Starlink (2:37) - Satellite internet, “on fire”; Kevin praises it after first plane experience
• Blue Origin (3:16) - Lost a rocket on the launchpad this past week
• Microsoft (1:29) - Being sued by NYT alongside OpenAI and Perplexity
• Perplexity (1:29) - Being sued by NYT
• The New York Times (1:29) - Kevin’s employer, suing OpenAI/Microsoft/Perplexity
• Platformer (0:02) - Casey’s publication
• Hobby Lobby (2:46) - Joke target for SpaceX conglomeration
• Stripe (10:21) - Another potential IPO this year
• United Airlines (23:36) - Carrier rolling out free Starlink; also the airline with the Bluetooth “bomb” incident
• Google DeepMind (26:51) - One of three labs that hit IMO gold last summer
• Harmonic (27:00) - Math AI startup that hit IMO gold
• Cursor (25:37) - Where Kevin Hartnett works as editorial lead
• Quanta Magazine (25:37) - Where Hartnett previously was senior reporter
• Institute for Advanced Study (36:04) - Princeton math institution where Hartnett observed polarized views
• Bot Company (50:08) - The robotics startup being sued for trashing the SF Airbnb
• Airbnb (48:20) - Where the secret robot training happened
• Meta (55:03) - Their AI chatbot leaked Instagram credentials
• Kalshi (52:50) - Prediction market that flagged George Santos
• Polymarket (59:04) - Prediction market spoiling Survivor

Products & Technologies

• ChatGPT (30:00) - Mathematicians initially mocked its “finitely many primes” errors
• Gemini (36:04) - The model the dismissive mathematician at IAS tried and gave up on
• Meta AI (55:03) - Support chatbot that was tricked into giving up Instagram credentials
• International Math Olympiad (IMO) (26:51) - Last summer’s gold medal benchmark for AI
• Putnam exam (31:38) - The premier college math competition; next AI target after IMO
• The Proof in the Code (25:37) - Hartnett’s new book about Lean and formal math
• Lean (25:37) - The formal math language at the heart of Hartnett’s book
• Annals of Math (33:16) - Top math journal where the unit distance proof could be published
• The Leiden Declaration (37:49) - 800+ mathematician open letter against irresponsible AI use
• arXiv (math archive) (39:36) - Banning year-long for unedited AI prompts in submissions
• Bluetooth (57:00) - Subject of the “bomb”-named speaker that caused emergency landing

People

• Elon Musk (2:37) - Owner of the SpaceX conglomerate going public; accused of hiding losses in winners
• Kevin Hartnett (25:37) - Guest; author of The Proof in the Code; editorial lead at Cursor
• Nan Ranshanoff (12:00) - Author of “third wave of philanthropy” post analyzing AI IPO charity capital
• Sam Altman (15:31) - OpenAI CEO; cited as having been fired and rehired
• Terence Tao (35:35) - The greatest living mathematician; sees AI as “jetpack for your thoughts”; signed Leiden Declaration
• Paul Erdős (30:58) - “The Bob Dylan of math”; died at math conference; left 1,200 problems with cash rewards
• President Trump (50:54) - Signed the 30-day AI EO
• David Sacks (50:44) - Former White House AI Czar; objected to original 90-day review, blessed 30-day
• George Santos (52:50) - Under federal investigation for betting against his own State of the Union attendance
• Mark Zuckerberg (55:46) - His apparent contact info posted on X via the Meta AI exploit
• Jeff Probst (59:04) - Survivor host complaining about Kalshi and Polymarket
• Aubrey Bracco (59:04) - Survivor winner whose victory was forecast at 80% before the season aired
• Cirie Fields (1:00:00) - Casey’s pick for “greatest player to never win” Survivor

Surprising Quotes

“We are talking about sort of the combined Frankenstein…” — Kevin Roose, 2:18

“Like up until very recently, SpaceX was just SpaceX and Starlink. Like it was just a pure good business, but then it seemed like Elon Musk decided he needed to kind of hide his losses somewhere, and so they inherited the two worst companies he owns.” — Casey Newton, 4:01

“It makes me glad that we shredded the social safety net and did all that, sort of, reductions in pandemic preparedness, so now the San Francisco billionaires can step in and rebuild it hand by hand.” — Casey Newton, 15:21

“AI was still just doing essentially high school math. The hardest high school math in the world, but still just high school math… it’s like 0% of the way to the frontier.” — Kevin Hartnett, 27:23

“These are like the Sudoku of math. It’s like the Wordle of math.” — Kevin Roose on Erdős problems, 33:41

“Most bombs that would blow up planes, you cannot actually connect to them via Bluetooth and are not named ‘bomb’ in the Bluetooth list.” — Casey Newton, 58:10

“I salute you, George Santos. This diva truly will go down in history.” — Casey Newton, 54:00

Transcript

Kevin Roose: 0:00 I’m Kevin Roose, a tech columnist at the New York Times.

Casey Newton: 0:02 I’m Casey Newton from Platformer.

Kevin Roose: 0:04 And this is Hard Fork! This week SpaceX, Anthropic, and OpenAI are all heading to the public markets, but what do their IPOs mean?

Casey Newton: 0:12 Then, author Kevin Hartnett is here to talk about why some mathematicians are sounding the alarm about the use of AI in their field.

Kevin Roose: 0:18 And finally, ChatGPT.

Kevin Roose: 0:27 Well Casey, the big news this week is that the AI IPO race is heating up. It’s hot IPO summer.

Casey Newton: 0:33 It really does seem like it is going to be a hot IPO summer, Kevin. I am told that we are on track to see what might be the three biggest IPOs of all time.

Kevin Roose: 0:43 Yes. So SpaceX is getting ready to go public as soon as maybe next week. And then just this week, Anthropic filed a confidential S-1 with the SEC, noting that it intends to go public, that is sort of the first step in the process, and there’s reports that OpenAI is going to file their S-1 soon as well. This is obviously long-awaited, people have been wondering when these giant private companies were going to go public, and now it seems like they are all racing to do it as quickly as they can and potentially beat each other to market.

Casey Newton: 1:18 Yeah, and the consequences are really important and we’re going to get into them, but before we do that, truly there has never been a better or more important time to do our disclosures.

Kevin Roose: 1:29 Yes. I work for the New York Times which is suing OpenAI, Microsoft, and Perplexity.

Casey Newton: 1:34 And my fiancee works at Anthropic.

Kevin Roose: 1:36 Okay, so those are our extra special AI disclosures this week. Now Casey, let’s talk about these IPOs. Maybe we should start with SpaceX. What is going on with the SpaceX IPO, what are we expecting, and what does it mean?

Casey Newton: 1:48 Yeah, so, you know, as you noted Kevin, SpaceX is just the furthest along right now. They seem like they’re getting very close to the finish line and they just have some staggeringly ambitious plans. They plan to sell their shares at $135 a piece, which would raise $75 billion dollars. That would make it the largest IPO of all time. It would also value the company at between $1.75 and $2 trillion dollars, which would sort of instantly make it among the very biggest companies in the world.

Kevin Roose: 2:18 Yeah, those are crazy numbers. Like, that, those are just numbers that we have not seen before in the history of capitalism. And we should also just remind people that when we say SpaceX, we are talking about sort of the combined Frankenstein…

Casey Newton: 2:35 Yeah, remind us what is actually in SpaceX.

Kevin Roose: 2:37 So, they make rockets. They make Starlink. They also, as of fairly recently, own xAI and X, the social network, and so that is all sort of part of this Elon Musk conglomerate that is going public.

Casey Newton: 2:46 What about Hobby Lobby? Do they, is Hobby Lobby part of it as well?

Kevin Roose: 2:49 Not yet, but don’t give their biz dev team any ideas.

Casey Newton: 2:52 Okay, fair enough.

Kevin Roose: 2:54 We’re going to do Hobby Lobby in space. So this giant conglomerate is being sort of positioned in the market as a way for people to invest in AI. Obviously, they do have an AI company inside of SpaceX, but I would say investors are also, I would say more excited about the space part of it, which is the most developed and they make stuff that people actually use.

Casey Newton: 3:16 Yeah, I mean look, there are two great businesses in here, right? One is a reusable rocket business that delivers satellite into space. It’s very hard to build that kind of company. So like SpaceX just has an incredible moat. There aren’t that many competitors to it. We saw Blue Origin, one of its biggest competitors, lose a rocket on the launchpad just in over the past week or so. So that is what makes that an incredible business. And then Starlink is just on fire — they’re using their ability to deliver satellites into space to also create a really powerful global internet access system that is just growing like wildfire. So there are two amazing businesses in there, and then they also have two terrible businesses called xAI and X.com. And it’ll be really interesting to see what the interplay of the good businesses and the bad businesses are in the months to come.

Kevin Roose: 4:00 Yeah, sometimes you just got to take the good with the bad.

Casey Newton: 4:01 And especially when they’re all packaged in the same stock ticker. Actually, what’s interesting is that you didn’t have to take the good with the bad. Like up until very recently, SpaceX was just SpaceX and Starlink. Like it was just a pure good business, but then it seemed like Elon Musk decided he needed to kind of hide his losses somewhere, and so they inherited the two worst companies he owns.

Kevin Roose: 4:11 Well, and what’s interesting about that is that one of those companies, xAI, appears to be pivoting. So they are now renting out compute that they originally built for themselves to Anthropic, another one of these companies that’s going to IPO this year. So they are positioning themselves as something like a space company with a kind of AI neo-cloud business attached to it and a social network that is going to become the everything app. All of it is a little mysterious, but basically this is the long-awaited time when all of Elon Musk’s sort of Voltron-like companies are going to sort of take a stab at going public together. Okay, so that is the SpaceX IPO. Now let’s talk about Anthropic. They have filed their confidential S-1. They are also expected to go public at something over a trillion dollars. That is not — what?

Casey Newton: 5:15 Oh, I’m just shaking my head at the insanity of that considering what their ARR was one year ago today.

Casey Newton: 5:26 Yes. I mean, this one is just wild to me. Like, I was thinking the other day about the first time I ever visited this company. It was in 2023, so three years ago. And when I tell you that this company was not only ambivalent about making money, but seemed to actively resist the idea of making money. They were like a small group of researchers — very earnest AI safety obsessed people who were like tinkering with and building models for unspecified purposes.

Kevin Roose: 6:09 I remember your story about it and it was just kind of about how glum and strange the office was, which you might expect for a bunch of safety-focused people who hadn’t even decided whether they really wanted to start making a product yet.

Casey Newton: 6:23 But boy howdy did they make a product and decide that they actually liked making money and wanted to make a lot of it. And now they’re going to be one of the largest IPOs of all time just three years after I was hanging out with them in their little like Jackson Square walk-up office.

Kevin Roose: 6:41 No, and like, you know, in January of 2025 this company has, you know, an annualized revenue run rate of about a billion dollars. And recently they’ve said it’s 50, who knows what it’s going to be by the time they IPO. But yeah, I mean, that just is unprecedented growth in Silicon Valley.

Casey Newton: 6:58 Yep. OpenAI we don’t know anything about their upcoming IPO except that they have said that they plan to do it. They may file as soon as this week, their paperwork with the SEC to start that process. People also expect this to be a big gigantic IPO. And so all of this taken together, I think there are a few threads to pull on here. One of which is, what is this going to do to San Francisco? Here we have two, call it two and a half companies because SpaceX has some headquarters and offices in Texas and Southern California and other places. So two and a half companies going public in the same year based in San Francisco minting hundreds if not thousands of new millionaires, decamillionaires, and centimillionaires, and what that will do to the city’s tech scene, to the local real estate market, etc.

Kevin Roose: 7:50 What are your thoughts on that?

Casey Newton: 7:55 I mean, my fear Kevin, is that we are about to see a massive increase in inequality in a town that already had really significant inequality. And I just worry that it’s going to sort of feel even worse. And where I’m already starting to notice it is when I talk to my friends who have like really good jobs paying like maybe even mid six figures. They’re looking at what they’re reading about the folks who got in early at an OpenAI or an Anthropic. And man, the comparison is not feeling good. And they’re starting to wonder, what does this mean for me? Am I going to be able to lead the life that I wanted? And it’s had me thinking a lot about how when I got to San Francisco in 2010, there was a sense of abundance here. There was a sense of anyone can do a startup and anyone can sort of have the the life that they want. And it was true for many, many tens of thousands of people. And I feel like we’re almost swinging back to this sort of scarcity mentality here, which is like well, if you didn’t make it in at one of these two companies, your future is in doubt. So I don’t know how true that is, but I could tell you that that is the anxiety that I’m hearing.

Kevin Roose: 9:02 Totally. And it’s obviously there’s some, you know, it’s hard to feel much sympathy for these people who are very well paid engineers and tech workers who are looking at their slightly richer peers and thinking, I gotta get some of that. But I think this is a real like status anxiety moment in San Francisco and Silicon Valley where even the people who sort of thought they had sort of made it in in the world of tech are now looking at these people who have joined these like insanely fast growing companies that are sort of wanting to get in on that somehow but also like thinking it might be too late. Like people who I don’t think were feeling precarious a year or two ago are now. And I think that’s just a really interesting sort of social marker.

Casey Newton: 9:49 Yeah. And just to say again, like I think what you want in a society is for opportunity to be spread broadly and for everyone to feel like they have a chance to lead the life that they want. And so when you move into a world where there are what essentially amounts to a handful of lottery winners and those are the only people that truly get to live the lives that they want, like that just causes massive social instability and all sorts of other problems. So I have like a knot in the pit of my stomach about this.

Kevin Roose: 10:21 Yep. I do too. And one thing that is making it slightly better is that I did see this coming. The real estate market is going nuts. Houses are going for many multiples of their asking price. There was a story this week, did you see this one about the San Francisco homes that are on sale asking for Anthropic or OpenAI stock instead of cash?

Casey Newton: 10:54 I mean, look, I think those people are smart. Like there’s a very real chance that that stock will appreciate in value even faster than your house. So I say shooter’s shoot.

Kevin Roose: 11:00 Yes. So there are two other things about these IPOs that I want to discuss with you. One of them is the effect it’s going to have on philanthropy. Because one of the strange characteristics of these particular companies is that many of their employees, and I would say this especially applies to Anthropic, but there’s also a piece of this at OpenAI are committed to effective altruism and other sort of similar philanthropic movements that basically teach you that you should if you want to make a maximum impact on the world, you should make a bunch of money and then give it away. And so nonprofits, philanthropies, donor advisory networks in and around San Francisco are now starting to ask like what if we just have this insurgence of new philanthropic capital coming from these IPOs. Our mutual friend Nan Ranshanoff wrote a great post about this the other day calling it the third wave of philanthropy basically where you have tens of billions or possibly hundreds of billions of dollars sort of flooding into these charitable movements and causes. How does that affect like what gets funded? Are there going to be new institutions that need to be built to sort of absorb all of this philanthropic capital? I think this is something that people outside San Francisco don’t quite understand is like how much money is going to be flowing into these philanthropies over the next couple of years.

Casey Newton: 13:03 Yeah, I also really loved Nan’s post and had a chance to talk with her about it in person recently and it was just so fascinating to hear about the sheer volume of philanthropic capital that we’re expecting to emerge as a result of these IPOs and how little infrastructure there is to absorb it. Something that I think some people might not know about Anthropic is that from their start, they have told people as they come on board, if you pledge like a percentage of your equity to philanthropy, we will actually match it. And so this is just a program that is dramatically amplifying the amount of philanthropic capital that is about to become available.

Kevin Roose: 13:44 Well, it was not just matching it. So, there are two things that Anthropic did that sort of speak to their historical ties to effective altruism and that sort of world of philanthropy. One is that all of the co-founders pledged to give at least 80% of their wealth to charity. So right there off the top we are talking about hundreds of billions of dollars potentially that are being earmarked for charity just by the eight co-founders of Anthropic. Then you have this stock matching program which as you alluded to, not only offered to match employees who pledged a certain percentage of their stock to charity share for share, but match them three to one in the case of some early employees. And that just is creating like when you actually lay out the numbers as Nan did in her post, like it is just staggering the amount of money — we are going to see something bigger than the Gates Foundation every year potentially for the next few years.

Kevin Roose: 14:44 And it’s gonna go to some stuff that will seem to the outside world fairly weird, right?

Casey Newton: 14:50 Like what?

Kevin Roose: 14:51 The joke going around the sort of AI circles is that it’s just gonna be a great year for shrimp welfare because shrimp welfare for arcane and reasons that are probably not worth going into here, has become like a sort of half-joking pet cause of the effective altruists. But it’s a great year to be a shrimp. It’s probably also a great year to be working on global health and pandemic prevention, AI safety, all these other sort of cause areas that are very closely affiliated with effective altruism.

Casey Newton: 15:21 Yeah, I mean it makes me glad that we shredded the social safety net and did all that, like, sort of reductions in pandemic preparedness, so now the San Francisco billionaires can step in and rebuild it hand by hand.

Kevin Roose: 15:31 Exactly. I want to bring up one other thing about these IPOs and ask for your opinion about it, which is that the thing that makes me nervous about these IPOs is not that, you know, they could go sideways or people could lose money or these companies are very speculative, all of that is true. What worries me is the safety angle here, because I am of the belief that these AI systems are getting more powerful, that those increased capabilities also bring increased risks. And I just know that many of these AI companies, OpenAI and Anthropic specifically, were started by people who were worried about safety and specifically worried about the ability of a for-profit corporation to develop AI safely. At OpenAI, you’ve had this sort of governance struggle that resulted in, among other things, Sam Altman being fired and rehired. At Anthropic, they have sort of made themselves a public benefit corporation to try to kind of lessen the influence of, sort of, shareholder capital and fiduciary duty on their ability to make decisions related to safety. But I think all that just gets much harder in a world where these are publicly traded companies that, you know, big investors, index fund holders, that retirees are invested in. It was already going to be hard to sort of slow down or maybe refuse to release something that was dangerous because of the enormous sums of money that these companies have raised. But it’s going to be a lot harder when the public markets are also pressuring these companies to race and go as fast as they can.

Casey Newton: 16:06 Does it matter that both OpenAI and Anthropic are structured as public benefit corporations and have, sort of, are allowed to make social commitments that a sort of traditional, like, SpaceX for example, will not be beholden to?

Kevin Roose: 16:14 I’m of a couple of minds on this. I think it’s probably better that they are public benefit corporations than not, because public benefit corporation is sort of this legal designation that allows you to take into account things like, are we being socially responsible? It doesn’t, you know, investors can’t sue you as easily for sort of breaching your fiduciary duty if you do something that is counter to their interests as shareholders. But they’re still corporations, and they still exist at the pleasure of shareholders, and there are certain concessions they can make to social…

Casey Newton: 18:00 Issues and impact, but when it comes down to it, like when the rubber meets the road and one of these companies develops a model that is truly dangerous, they are going to now need to weigh not just what do we think the right thing to do is, or what do our private investors think we should do, or what do our employees think we should do, but they’re also going to have the public markets breathing down their neck. They’re going to have activist investors and things like that. So I’m just nervous about the structure that is now going to grow up around these companies and just push them in the direction of acceleration.

Kevin Roose: 18:35 I think that’s fair, and I may be coping, but when I think about the possibility of a lab developing a really dangerous model and saying, well, due to shareholder pressure, we’re just going to put it out there. You have to remember that the shareholder pressure can work in the other way too, because if you put out a model that can create a new bioweapon, you’re probably going to get sued for securities fraud by a shareholder that said, I trusted you to only release safe models. So I do think that there are going to be some positive pressures here that hopefully keep them from doing anything too silly.

Kevin Roose: 19:08 Yeah. How do you think this will impact the average person who’s not an investor or a shareholder in these companies who may just want this stuff to be developed well and safely?

Casey Newton: 19:22 I mean, I think on balance you’re right that, because these are corporations, we do have to worry that capitalist pressures will lead them to cutting corners and doing things that are unsafe. So that is I think the right concern to have and to keep your eye on it. On the other hand, I could also see an argument that when your company goes public, you are introducing more democratic oversight and governance into it. These folks will now have to like report their earnings. They will have to give us information about their financials. They will have to make certain disclosures as their products come out and as their company changes. And shareholders will be able to maybe vote on certain things. So I think these are all good things because right now we have almost no levers whatsoever that people can pull other than trying to prevent a data center from being built in their backyard. So maybe these give folks some new ones.

Casey Newton: 20:20 Yeah, and I think there’s been a lot of hand-wringing over these indexes and these exchanges that have changed the rules, like the Nasdaq 100 and the S&P have already or are considering loosening their rules around these so-called seasoning periods. Basically, it used to be if you were a brand-new public company that had just IPO’d, you could not be included on these major stock indexes because they sort of wanted to see whether you were stable enough to like become part of the basket of blue chips that people invest in when they buy an index fund. Now, partially because of these looming IPOs, those rules have been relaxed so that these companies are not going to have to wait three months or six months or a year anymore to be included on these exchanges.

Kevin Roose: 21:00 Some people have said, well that sounds bad and we’re exposing retail investors to these like volatile and risky stocks. I’m not that worried about it. I think investors want exposure to these companies. I know several people in San Francisco who have been sort of like devising these crazy hair-brained schemes to get pre-IPO stock in one of these companies. And I think that letting the public benefit from the upside of the AI boom I think is going to do more help than harm, but I could be wrong if all of this goes up in a conflagration and people lose their shirts.

Casey Newton: 21:38 I mean, yes, absolutely. Kevin and I are not financial advisors.

Kevin Roose: 21:46 I am actually a certified financial advisor.

Casey Newton: 21:47 I am not a financial advisor. But I do think if you’re a retail investor and you believe in this stuff, you should have the ability to make that bet. Because we live in a country where you can bet on Bitcoin in an exchange-traded fund, you can go on a prediction market as a member of the military and bet on an operation that you’re a part of. So in a world where those are our restrictions on financial gambling, if you want to buy a share of OpenAI, I say godspeed.

Kevin Roose: 22:17 Yeah, I think part of the icky feeling that people are having about the AI industry now is that so much wealth is being concentrated in so few private companies and so few hands. And so in an optimistic scenario where these IPOs go off without a hitch and these companies keep growing at hyper scale, I think maybe having the benefits shared a little more broadly through things like index funds could be good for people’s feeling like, ‘oh, there’s something for me in this’.

Casey Newton: 22:45 I’m gonna go further and saying it’s not just good, I’m gonna say it is necessary. Like, we cannot have a very small handful of companies that are growing this quickly, that are concentrating wealth and power this much into so few hands. It has to be shared more broadly than that, and while an IPO is a very small step in that direction, I do think it is a necessary one.

Kevin Roose: 23:04 Okay, well that is enough about the IPOs for this week. We will continue to cover these IPOs and everything that comes out of them, including possible space data centers. I don’t know, I’m excited to learn more about those. I will say, when it comes to Starlink, I was not a believer. And then I went on my first Starlink-equipped airplane last week. And Casey, this is going to be the biggest company in the world.

Casey Newton: 23:22 It’s very good. When people get a taste of 200 plus megabits in the air on a plane, you’re never going back.

Kevin Roose: 23:30 Yeah, you can actually watch YouTube if you have Starlink on your plane. You can watch so many YouTube videos.

Casey Newton: 23:36 And the deal they struck, apparently I’m told, is they struck the deal where at least with United, they’re like, ‘we’ll put Starlink on your plane and we will charge United for that, but you can’t charge your passengers.’ So everyone’s experience of Starlink is it is a free miracle that’s being delivered to me in my airplane seat, which is not bad marketing strategy.

Casey Newton: 24:00 Not a bad marketing strategy.

Kevin Roose: 24:09 Well, Casey, get out your TI-83 graphing calculator, because today we’re going to talk about math.

Casey Newton: 24:14 Can I play Snake on it, or do we actually have to talk?

Kevin Roose: 24:16 No, we have to talk because today, we are going to talk about what is going on with AI and math. Now, this is a subject that we have talked about before on this show, but there’s actually been a lot happening just over the past couple of weeks. So two weeks ago on May 20th, OpenAI announced that one of their models had reached this big mathematical milestone. Basically, it had disproved this long-standing geometry conjecture by identifying a new way of thinking about this famous math problem, one of these Erdős problems that basically no human mathematician had considered before. That was considered a very big deal in the world of mathematics. And at the same time, there is also this backlash brewing in mathematics to the use of AI. Just this week, a group of mathematicians have been passing around and signing something called the Leiden Declaration, which is basically an open letter about the use of AI in mathematics from people who are concerned that maybe they’re sort of eroding the human foundations of this sort of academic mathematical discipline.

Casey Newton: 25:25 Yeah, and when I asked a mathematician why, they said Y equals MX plus B.

Kevin Roose: 25:30 Okay, very good. That, of course, is the classic slope-intercept formula for a straight line.

Casey Newton: 25:35 Which model did you use to look that up?

Kevin Roose: 25:37 I’ll tell you later. So we just thought it was a really good idea to check in on the state of AI and math. And to help us make sense of what is going on right now, we have turned to one of the best guests I can imagine for this subject. Kevin Hartnett is a journalist. He has covered math and computer science for many publications, including most recently as a senior reporter for Quanta Magazine. He’s also the author of a book that comes out next week called The Proof in the Code, which is sort of about this formal math language called Lean and how it’s transforming math and AI. Today he works as the editorial lead at Cursor, the AI coding platform. And I just thought he would have a really good view of this situation. Also, we just like to bring on people named Kevin because they tend to be really smart.

Casey Newton: 26:22 Okay, let’s bring in Kevin Hartnett.

Kevin Roose: 26:23 Kevin Hartnett, welcome to Hard Fork.

Kevin Hartnett: 26:25 Thanks Kevin, great to be here.

Kevin Roose: 26:27 Well Kevin, we’ve brought you here today to talk to you about AI and math. You just wrote a book called The Proof in the Code, which is, I’ll just say it, the most interesting book I’ve ever read about math.

Kevin Hartnett: 26:41 It’s a short list.

Kevin Roose: 26:43 I won’t ask the follow-up question.

Casey Newton: 26:45 It narrowly edged out One Fish, Two Fish, Red Fish, Blue Fish.

Kevin Roose: 26:51 So when we last checked on the field of mathematics and AI, it was last summer, and three of the big AI labs — Google DeepMind, OpenAI and Harmonic had all reported that their math models had achieved a gold medal score at the International Math Olympiad. That was something that, you know, people had been saying for years would be impossible for or would take many, many decades for computers to be able to do, but their AI models did it last summer. What has been happening in the field of AI and math since then?

Kevin Hartnett: 27:23 Yeah, I mean, virtually everything. So like the IMO had been a benchmark for a long time. In my book, there’s a whole chapter about the previous year’s IMO, the 2024 IMO, where Google DeepMind got a silver medal score. And that was kind of considered a small watershed. And then as you said, last year, three labs got this gold medal level score, which had really been the kind of the benchmark that had been set out. At that point, AI was still just doing essentially high school math. The hardest high school math in the world, but still just high school math. And I think for people who never went beyond high school math, it’s hard to really appreciate how far that is from the frontier of research math. Like forever far. That’s like barely even wading into the field. It’s like 0% of the way to the frontier. So it was a proof of concept maybe, but it certainly didn’t mean much in terms of can these models actually do research.

Kevin Roose: 28:12 That’s very hard for me to hear as someone who was not that good at high school math. I have to say. I’m feeling a little defensive. But I get what I believe you. It’s just making me defensive.

Kevin Roose: 28:23 Why were the labs so focused on the IMO and on math in general? Was it because that was just like a very hard challenge that they liked? Or was it because they thought that being able to do math at a high level would enable their models to do other important things?

Kevin Hartnett: 28:40 Well, it’s definitely both. I think the challenge, this IMO Grand Challenge, which was the name that a researcher at Microsoft Research gave to it, was really about like can we just create models that can do amazing math research. It was just really kind of research for research’s sake. At a certain point, the labs and these startups you mentioned adopted that challenge themselves. And their motivations were a little different. There’s very much this belief that if you can teach a model to reason about math problems and solve math problems, it will be much better at other things. And I always think about this statement like my high school math teacher would give when people would ask like why are we learning this, what’s the point of this? It’s like they would either say so you can balance your checkbook, or to teach you how to think. The “to teach you how to think” is really the point here. If you can reason through a math problem, think logically, then you can apply that kind of skill to all of the parts of your life. And I think the labs believe that if you can teach a model to reason through math problems, it’s going to be able to do all these other things that are much more probably like commercially valuable as well.

Casey Newton: 29:43 So, I’m having a flashback to when we first started to talk about AI and math, and the knock on these models was that they were actually quite terrible at it. And that if you would try to get them to do basic addition or multiplication, they would utterly fail. So Kevin, sketch out for us a little bit what the labs did to sort of navigate through that problem and get to a place where they could kind of credibly try to advance the frontier of the science.

Kevin Hartnett: 30:07 Right. And when ChatGPT came out in November 2022, mathematicians were like passing around all these like haha look at this stupid model telling me that there are only finitely many primes when we all know there are infinitely many primes. And like 2+2 is 5, basically that kind of thing. I mean I think essentially the models got better. There’s definitely an element of reinforcement learning on math problems that make these models better like RL on math. But I think it is just like the general improvement in these models that we all experience kind of in a lot of ways we use them has led to these kind of incredible reasoning tasks that they’ve become capable of.

Casey Newton: 30:43 So let’s talk about one of the areas where it seems like we’ve seen some creativity in math with AI lately, which are these Erdős problems. Kevin, can you tell us who this Erdős was and why he left us with so many problems?

Kevin Hartnett: 30:58 Yes. I mean, so Paul Erdős was a really colorful mathematician. He was essentially the Bob Dylan of math in that he spent his life on the road. He died at age 83 actually at a math conference. He slept on mathematicians’ couches his whole life. And as he went around, he compiled lists of problems that he thought were interesting. Either he’d find them in the wild or he invented many of them, and he created this Erdős list. He endowed them with these tiny little rewards, like $20 for solving this problem, $500 for solving this problem. And that fund still exists after his death and these things are paid out. Actually I don’t know if the LLMs have received the money or who gets the money when they do it, but anyway.

Casey Newton: 31:34 Don’t give them the money. They don’t need the money.

Kevin Hartnett: 31:38 When a human solves them, they get the money. So Paul Erdős collected over a thousand problems, 1,200 problems that he thought were interesting and just kind of left them out there. The AI labs and these startups have looked for kind of benchmarks, kind of mountains they can climb, things they can do to prove that their models work well. The IMO was one of the most prominent ones. Once they got the gold medal there, they needed to move on. They moved on to the Putnam exam, which is the premier college math competition, and started to do quite well there. And then they just started looking for new targets. And these Erdős problems are sitting out there, 1,200 or so problems that for the most part mathematicians had never looked at or had received very little attention. And so they essentially set their models to work on, you know, all of them. Like see what you can do on them. And it would cook up answers to them. And through the beginning part of this year, we would see like on Twitter one announcement after another. Solved Erdős problem 737, solved Erdős problem 63. And I think mathematicians viewed those very kind of like hypey announcements differently than the energy behind the announcements themselves.

Casey Newton: 32:47 How did mathematicians view them?

Kevin Roose: 32:50 They said something’s not adding up here.

Casey Newton: 32:53 Thank you, Kevin.

Kevin Hartnett: 32:55 I’m going to have to definitely think of some puns. I’m going to get one.

Casey Newton: 33:00 burn out before we finish this podcast. That’s my goal.

Kevin Hartnett: 33:03 Yeah, so mathematicians… there are just a lot of unsolved math problems in the world. And just because a math problem is unsolved and has been around for like decades and was dreamed up by a famous mathematician does not make it an important problem. Like an important problem is one the field kind of in its like collective wisdom determines either like the answer to the problem really will change how we view math, or more significantly, the methods we will need to develop to solve that problem are just going to remake the field. It’s going to create important new math. These Erdős problems were just not viewed that way. They were kind of like sophisticated riddles in a way. Sophisticated arithmetic, numerical riddles.

Kevin Roose: 33:41 These are like the Sudoku of math. It’s like the Wordle of math.

Kevin Hartnett: 33:46 It is like the Wordle of math. I think that’s a fair statement. And so, anyway, mathematicians didn’t spend a lot of time looking at them. I think the conventional wisdom was these Erdős problems are like toy problems, not serious problems. But like it’s not true about all of them. About a month ago, OpenAI came out with this like big new result. They’ve solved what many people think of as one of the most important Erdős problems, this thing called the unit distance conjecture. And what was important about the unit distance problem is it was a problem that a lot of people had looked at, so you couldn’t just say no humans had really tried to solve this. People have looked at it, they hadn’t solved it. The methods underneath it were very sophisticated and surprising. It was not just kind of a clever cobbling together of like obvious techniques. And the result itself was just like so good. Pretty unanimously people agreed this could be published in the Annals of Math, the top journal in math. In a way, over the last year there’s been this kind of shifting of goalposts. AI did this, but it can’t do that. Oh, it did that? No, it still can’t do this. There’s still some shifting going on. There’s still room to shift. They haven’t solved the Millennium Prize problems, but this result, this proof of the unit distance conjecture really said AI can do absolutely top-tier research.

Kevin Roose: 34:56 Yeah. I’ve been following the story of AI and math in part through people like Terence Tao, who is widely considered the greatest living mathematician. He has been sort of experimenting and writing and making videos about his experiments with these AI programs for use in sort of frontier math research for a number of years now. And when he started sort of working with them, he was like, oh, these aren’t that helpful, or maybe they’re like a, you know, a sort of mediocre grad student who you’d have assisting you. And more recently, he seems to be actually saying, like, this is revolutionary for the kind of frontier math research that I and other professional mathematicians do. He recently made a video with OpenAI talking about how he can now just try a bunch of sort of crazy ideas and experiments because the sort of cognitive friction of using these models means that you can just sort of have an idea and give it to the model and say, go test this a bunch of different ways and figure out if there’s anything there. Is that a widely held view among mathematicians or is he just sort of on the extreme AI-pilled end of the field?

Kevin Hartnett: 36:04 Yeah, so Terry is an extremely interesting figure in this because he is so important as you said. He there’s I have a whole chapter in my book about some of the early work that he did with AI, this thing he called equational theories and he’s always been interested in different ways of doing math. He’s very intensely collaborative, he’s just interested in kind of new ways of working. Terry is I think kind of representative of one of three attitudes towards AI and math right now. A couple weeks ago, I was at the Institute for Advanced Study in Princeton, New Jersey, which is like the citadel of modern math, like the biggest, the most dense collection of kind of great math minds in the world live and work there. And in a single afternoon, I was walking the campus there and I had two strikingly different experiences. I ran into two 40-year-old mathematicians, top of the field. One of them told me he’d just tried to do some math with Gemini and he’s like, that stupid thing told me like XYZ thing, which we know is wrong, is true. And then I closed it and I went back to doing math the other way. An hour later, a guy who, on paper looks a lot like the first guy says, I think in two years, AI is going to put mathematicians out of business because it is just going to be strictly better than us at all of it and like we won’t need mathematicians anymore. So those two poles, Terry is squarely in the middle. He, that OpenAI video you described Kevin, that’s like the kind of jetpack for your thoughts view of AI. It’s like the Iron Man suit that will allow me to do more and better and bigger things than I could before. He is clearly the kind of, I don’t know, the leading figure of that point of view. I don’t really know if you were to take a poll right now of mathematicians, like how it would break down. I would guess the it’s going to put us out of business would finish last. And I would guess Terry’s would finish first and I think that it’s good for nothing, that might have actually won a year ago, but I think that’s definitely falling in the polls.

Casey Newton: 37:49 Let’s talk about some potential paths to a world where maybe more people are agreeing with the person who thinks that AI threatens the job of a mathematician. Kevin, do you want to talk a little bit about this letter that these mathematicians put together?

Kevin Roose: 38:04 I do. This was fascinating and this was actually the reason we wanted to have you on today was to talk about this response, this declaration, the Leiden Declaration on Artificial Intelligence and Mathematics. And this is a basically declaration signed by something like 800 mathematicians so far that I would characterize as like a very worried document. They are upset about the use of AI in their field, what they consider the irresponsible or reckless use. They make a bunch of statements about how these technologies are producing plausible but unreliable or even incorrect arguments which are hard to distinguish from sort of correct proofs.

Casey Newton: 38:56 So help us understand this declaration and kind of the perspectives of the mathematicians who are putting their names on it.

Kevin Hartnett: 38:57 Yeah, I think you are right that it reflects a kind of deep concern and worry for the future of the field. Mathematicians I would think are deeply worried in the way of a population or a community that largely was able to run itself and self-regulate for centuries. And now there’s just this massive exogenous force that is shaking it and they want to be able to kind of defend the field, they want to set up some kind of guardrails, and also say like this is our field. You don’t get to tell us kind of what’s important, how it runs, and that this document is I think an effort to try and start making that kind of statement.

Casey Newton: 39:29 Well I think they’ve realized that there’s strength in numbers.

Kevin Roose: 39:32 Oh my god.

Casey Newton: 39:34 The hits keep coming.

Kevin Roose: 39:36 So here’s what I want to know. Like what exactly is so threatening to them about, you know, other people using ChatGPT to do math proofs?

Kevin Hartnett: 39:44 Well let me actually step back one second. I think there are kind of two things the Leiden document’s trying to do. One is the same thing that like all fields are trying to do, which is like what are the new rules for the road in the age of AI? Like basically saying if you use AI when writing a proof you need to tell us about it. Saying the math archive — which is where proofs are posted online before they’re published — recently issued a statement that if we see unedited use of AI in your PDF that you upload, like we see the metadata from the AI prompt, like you copy and pasted it in there and you didn’t even know it was there and you didn’t review it, we’re banning you from the platform for a year. So there’s some effort to set some new rules for the road. That’s one. And then two, I think they worry that the types of math and incentives to do math, the types of math problems that LLMs are good at, are different than the kinds they care about, but their own priorities are just going to be steamrolled by the rapid pace of progress in the particular kinds of problems that AI is good at at the moment. All the attention and money that flows to them, I think they’re worried the field will kind of get squeezed off and they’ll have no say in that direction. They’re trying to have a say.

Kevin Roose: 40:58 Like I’m sure this is a naive perspective, but isn’t this just what people in every discipline feel when there’s a new technology that does the thing that they used to do better than they do? Like wouldn’t the abacus guild have been writing declarations about the dangerous new calculator?

Casey Newton: 41:14 They wouldn’t have bothered, they were very violent. They turned straight to violence.

Kevin Hartnett: 41:20 So I mean, is this just like a purely defensive thing? I think there are definitely large elements of math that are worth preserving and that AI could in a certain way kind of undermine those elements without fully replacing them. And I think that is a valid worry. Math is a very rich discipline, it’s the kind of the greatest flowering of the human mind. I think there are things to worry about that AI could kind of strip a lot of the incentive and value out of it without fully replacing it.

Casey Newton: 41:44 I kind of just want to return because I’m still not sure I totally connected on what the folks who signed the Leiden declaration are worried about. There is like a version of this anxiety that I feel like I’ve seen at other professions, which is essentially AI just enables the instant creation of so much stuff, what is often called slop, that it kind of overwhelms and crowds out the people in the industry who are actually like talented and doing a good job. Like, is this primarily just a slop issue where they feel like they’re not going to be overcome by a tide of AI-generated proofs, or is there something else there that I’m missing?

Kevin Hartnett: 42:22 Oh, I think it’s actually entirely that if AI can generate proofs that are really good and we can read them and we can all be super impressed by them, then there’ll be no more reason for us to like have jobs. Like we will be like hobbyist great chess players.

Casey Newton: 42:34 And is the anxiety there like an economic one of, hey, like this is how I feed my family, or is it something in addition to that of like without humans steering the direction of math, something bad will happen?

Kevin Hartnett: 42:51 I think the anxiety is having been in possession of something for a long time that was like very special, that was like rare in the human population, great ability at math, and now it’s kind of like generally available. I think that’s just kind of a weird thing to reckon with. I think it’s also this belief that math and the way the community has developed, the norms around it, has produced a lot of basic discoveries that are both practically important — math fuels our understanding of the universe, it’s important in engineering, it’s important in all sorts of technology. So there’s a concern that if you kind of squeeze it off in certain ways that we will lose those downstream benefits. And the more serious point is this belief that math is just a quintessentially human endeavor, it’s like the pinnacle of human thought, it’s like writing a sonata, it’s like writing a novel, and that we don’t feel as threatened in those areas by AI because we kind of understand that the human behind the creation is such an important thing.

Kevin Roose: 43:54 Oh, I think writers and musicians feel very threatened by AI, just to offer that. I think there is a very similar response happening in the creative community.

Kevin Hartnett: 44:05 Do you — I guess I wasn’t — I was not so sure of that. I just think that people would not be as interested in a novel written by AI.

Kevin Roose: 44:11 They’re not going to be interested if they know that it was written by AI, right? Yeah, that is true.

Kevin Hartnett: 44:16 Anyway, so like if a proof is not — if there’s no human behind it who kind of sat and wrestled with it, then maybe we lose some kind of very essentially human endeavor, that is the worry.

Kevin Roose: 44:29 Right. I think it’s — programmers are an interesting exception to this sort of defensiveness because for the most part, I think like many programmers are very excited about the tools that allow them to just do their jobs better and faster, and obviously they’re worried about the future of their own jobs, but I don’t think you’re seeing this kind of Leiden Declaration backlash to the use of AI for programming. But I think it’s interesting to see just the list of all-star mathematicians who have signed this declaration, including we should say Terry Tao. Who we just mentioned is like fairly excited about the use of AI also signed this declaration saying, hey, wait a minute, we got to be careful and put some rules on the road here.

Casey Newton: 45:09 Let me ask what I imagine will be a naive question though, which is my extremely limited understanding of math is that mathematics are natural laws of the universe, right? Like this field is not invented, it is discovered. And my sense is that maybe potentially it is a field that could even someday be solved completely because we will just simply understand the structure of math and all of those laws of the universe. So I could imagine a group of mathematicians saying, hey, this is really exciting because this is going to accelerate us to sort of getting to the teleological end of our entire discipline. But I’m not hearing that today. So what am I missing?

Kevin Hartnett: 45:50 Yeah, this idea of is math invented or discovered, right? It’s kind of an ever-ending debate. 10 years ago, a little more than 10 years ago, Terence Tao won a big prize and he was asked that question and his answer, which I very much remember, was when you’re doing math it feels like you’re creating something but ultimately he kind of viewed it, I think he said, as an act of discovery. I just don’t think mathematicians have any concern that we’re about to run out of math to be discovered and AI would have to be a lot better to discover it all. Like maybe we already know effectively 0% of all the math there is to know, there’s a lot more out there. So I don’t think that’s a concern, although that is a fun thing to imagine.

Casey Newton: 46:29 Yeah. I’m just kind of wondering is the future of AI in math more likely to be sort of a bittersweet acceptance or is there going to be this kind of principled resistance pushing back and saying like no this is still going to be the domain of humans.

Kevin Hartnett: 46:43 It’s hard for me to believe and I can only speculate, no one knows the answer to this. I ask mathematicians like frequently where do you think this is going, what is the future for you all? No one knows. It is just hard for me to believe that something that has been so important and central to human activity for so long is just going to completely disappear and be replaced by pushing a button. I think we will be surprised by the way it turns out. Put me kind of voting in that middle camp, the Terry Tao camp, that we’re going to human beings directing these machines in some important ways, choosing which problems to set them on is going to continue to be important. So math is going to look a lot different, there’s just no doubt about it. It’s going to have to adapt in a lot of ways just like everybody in almost any industry is. But I think there will be something quite impressive and different that comes out at the end.

Casey Newton: 47:29 Kevin, thank you for helping us balance the equation.

Kevin Roose: 47:32 Yeah, you’ve been integral to this show.

Kevin Hartnett: 47:35 Thank you guys. It was a pleasure to come on and check your work.

Kevin Roose: 47:40 We really appreciate it. Thanks, Kevin.

Kevin Roose: 47:49 All right Casey, before we go, we did have some other tech news that we wanted to discuss this week in our segment ChatGPT.

Casey Newton: 48:00 HatGPT of course our segment where we take news stories, put them in the hat, draw them out one by one, riff on them a bit and then when one of us gets bored we say: stop generating.

Kevin Roose: 48:12 And then we solve a formal math theorem.

Casey Newton: 48:15 Mhm. That’s a new twist on the game.

Casey Newton: 48:18 All right, what’s our first item today, Kev?

Kevin Roose: 48:20 First out of the hat: A San Francisco startup is secretly testing robots in Airbnbs and trashing them, lawsuit claims. This one comes to us from the San Francisco Standard. Casey, did you hear about this robot testing that’s going on in the Airbnb?

Casey Newton: 48:34 I did, and this is a thing, there are all of these well-funded robotics companies and they need to just practice doing household chores over and over and over again, trying to generate lots and lots of training data, and apparently a lot of Airbnbs are now caught up in the crossfire.

Kevin Roose: 48:53 Yes, so on April 12th, a Ring camera in San Francisco captured footage of people moving large black cases into a home in San Francisco. Two days later, the house’s owner stopped by the house to take out the trash, looked through the window and saw black cables taped to the walls, a man was typing at a laptop sitting next to what appeared to be a robot, and when the guest checked out 11 days later, the house was a mess. Everything had been removed from the kitchen cabinets, and the dishwasher was scratched, and everything was a mess because this startup had been training their robots in this person’s house without his knowledge.

Casey Newton: 49:43 I have to say, I’ve rarely felt less sympathy for anyone on our show than the people who opened up their Airbnb to a robot company.

Kevin Roose: 49:50 Well, they didn’t know they were opening it up to a robot company!

Casey Newton: 49:54 Doesn’t matter, I subscribe to the A-LAB theory, that’s All Landlords Are Bastards. So look, if you have enough money to buy a place and then you just want to rent it out and charge people these usurious cleaning fees, you know, and make ‘em take out the trash on their way out the door, I have no sympathy for you. Let a robot in there. Airbnb was a 2010s phenomenon, everyone’s staying in hotels again. Like we have to do something with these spaces, we might as well let a robot mess ‘em up. Am I wrong? Tell me I’m wrong.

Kevin Roose: 50:08 You’re wrong! You should not be able to just bring in a bunch of robots in body bags to an Airbnb and have them screw up the house. Or if you do, there should be a cleaning fee, and this — the owner of this house is now suing this company called the Bot Company for renting his Airbnb under false pretenses to do robot training, and he’s seeking, wait for it, 12,383 dollars and 50 cents in compensation.

Casey Newton: 50:18 That is now the median cleaning fee on Airbnb anyway! I’m telling you, this is a normal Airbnb situation. It’s the cost of doing business.

Casey Newton: 50:30 Stop generating.

Kevin Roose: 50:32 Okay.

Casey Newton: 50:33 Next up,

Casey Newton: 50:44 Trump signs executive order seeking oversight of AI models, this from Shira Frenkel and Trip Mickle here at the Times. President Trump signed an executive order on Tuesday that asked technology companies to voluntarily give the government oversight of new AI models before releasing them to the public, there is a big change here from an earlier version that got scrapped a couple of weeks ago, which is that the number of days that the government will get to review models is down from 90 days to 30, and apparently that was enough for former White House AI Czar David Sacks to give it his blessing. Kevin, what do you make of the Trump EO?

Kevin Roose: 51:20 Casey, what is going on here? Because when we talked to Sundar Pichai a couple weeks ago, he was going to the White House to be there for this EO signing, but then that got delayed because apparently David Sacks had a fit and didn’t like something that was in the order. So now we have this 30-day provision. I am just very confused about the state of the AI order and whether David Sacks is still kind of running the show from the shadows over there.

Casey Newton: 51:47 Well, he’s not the AI czar anymore, but he definitely still has sway. And there are some within the administration who believe that if the government is able to hold up the release of a new model for 90 days, that could hurt American competitiveness. If you bring it down to 30 days, they’re just not as concerned. So that’s the change that was made. I mean, the part that just makes me roll my eyes at all of this is just the fact that all of this is still voluntary. Like, I think we are now past the point where frontier models actually should have to submit to mandatory required testing before they inflict these models on the public, but we’re still not there yet.

Kevin Roose: 52:21 Which was voluntary? It’s voluntary to do the 30 days?

Casey Newton: 52:25 It’s voluntary to submit it, and then the government has 30 days to, I guess, give you notes.

Kevin Roose: 52:32 I see. I’m still confused about the state of AI regulation in this country, and my ongoing assumption is that until I see something that is passed into law and signed, we are just sort of operating in the vibes universe.

Casey Newton: 52:43 Yeah, if the state of AI regulation is, ‘Hey, what do you guys want to do? What sounds good to you? What would work for you?’

Kevin Roose: 52:49 Stop generating!

Kevin Roose: 52:50 Alright, what’s next? Okay, we’ve got US is said to be investigating George Santos — haven’t heard that name in years — over Kalshi betting. This one comes to us from our colleagues at the New York Times. Federal authorities are investigating whether former US Representative George Santos engaged in insider trading by betting on a prediction market about whether he would show up at President Trump’s State of the Union address in late February. And Kalshi has referred this matter to the Justice Department and the CFTC for further investigation.

Kevin Roose: 53:33 So say a bit more because am I right that there were maybe some indications that he was going to go to this State of the Union, but then it appeared that he may have become aware of that and then gone and placed a bet that he would not go and then did not go?

Casey Newton: 53:44 Yes, so the pattern of facts as we understand them is that just before the State of the Union, George Santos goes on social media and says that he’s going to attend. But he missed the speech, and around the time of the event, Kalshi detected that he had bet against his own attendance. So we salute a legend. This diva truly will go down in history. George Santos, I salute you.

Casey Newton: 54:07 Now look, there’s a very narrow category of crimes, Kevin, for which I think you should not get in trouble, and this is absolutely one of those. If you are out there on social media engaging in trickery to get people to lose money in a prediction market, that I actually think should not be a crime. Or at the very least, I think you should get one for free.

Kevin Roose: 54:23 Yeah. Yes, very funny. It’s always the ones you most expect. And this is going to be just a hilarious genre of like silly crime stories over the next few years, is just increasingly famous people just getting caught with their pants down betting on prediction markets about events that they themselves control.

Casey Newton: 54:44 Yeah, I will say as an aging millennial, it is very appealing to me the idea that I could profit from saying I was going to go to a party and then not go. That we’ve all had that dream.

Kevin Roose: 54:54 We’ve been doing that for free for years.

Casey Newton: 54:56 Truly. What’s wrong with us? Okay, stop generating.

Casey Newton: 55:03 All right, this next one comes to us from 404 Media. Hackers simply asked Meta AI to give them access to high-profile Instagram accounts, and it worked. Hackers say they used a Meta AI support chatbot to break into a host of high-profile Instagram profiles by asking the support bot to change the email address associated with the target account. The claims coincide with a series of high-profile Instagram account takeovers, including the Barack Obama White House account, the Chief Master Sergeant of Space Force’s account, and Sephora’s account. Kevin, what do you make of the fact that you can just apparently access someone else’s Instagram account by asking Meta AI?

Kevin Roose: 55:31 Well, I just want to say my old pal at Meta AI, Nasty Nancy, would never have done this. She would have upheld the integrity of these Instagram accounts. But now they’ve turned it over to this crazy chatbot who’s just giving away people’s passwords.

Kevin Roose: 55:46 Look, I know it seems like this story is bad for Meta, but I actually think it’s good because we have finally found something that Meta AI is good for, and I’m not sure that I could name a second thing. So congratulations to the super intelligence. I learned about this when someone posted the apparent account credentials and cell phone number of Mark Zuckerberg on X. So I have not tried the number yet to verify that it works, but it appears that you can just go on and ask Meta AI for anything.

Casey Newton: 56:14 I’m going to guess that that’s not actually his contact information and is in fact some sort of fishing scheme that would harm you. Although one way to find out. Everyone try calling Mark Zuckerberg at…

Casey Newton: 56:22 Kevin, you know we have a strict no-doxing policy here on the show.

Kevin Roose: 56:25 Okay, well I’ll save that for our bonus content.

Casey Newton: 56:28 All right, stop generating.

Kevin Roose: 56:30 Next up, United Flight — oh this is my favorite story of the week. United Flight forced to turn around because of a Bluetooth speaker name. That’s the Verge headline. A United Airlines flight from Newark to Mallorca, Spain last Saturday night had to turn around about two hours after takeoff and do an emergency landing due to security concerns over a Bluetooth signal. The crew on the flight asked for passengers to turn off their Bluetooth devices multiple times and everyone complied except for one speaker that belonged to a 16-year-old boy and was named ‘bomb’.

Casey Newton: 57:18 It’s pretty funny because it’s like there’s maybe only one word in the English language that you could name your Bluetooth speaker that would force an emergency landing and you picked it, brother. Congrats.

Kevin Roose: 57:33 It’s so true. I was following this story on Reddit where there were like weirdly a number of passengers on this flight who were like active on Reddit. And so during this they were like the pilots are coming on and telling us that we all had to turn off our Bluetooth devices immediately. What’s going on? And then you kind of follow in real time and finally they figure out, yeah, there’s this Bluetooth speaker and when you connect it, it shows up on the little list of Bluetooth devices as ‘bomb’, which is a very bad name for a Bluetooth speaker. Don’t do that.

Casey Newton: 57:58 It’s a very bad name for a Bluetooth speaker unless you plan on playing some bomb ass tunes. You know, it’s all about context.

Casey Newton: 58:10 Also, I have news for people who are running airport security. Most bombs that would blow up planes, you cannot actually connect to them via Bluetooth and are not named ‘bomb’ in the Bluetooth list. These are not discoverable devices that are advertising themselves as what they are.

Kevin Roose: 58:35 It reminds me of like do you remember when people used to give their like home Wi-Fi networks like names like ‘CIA surveillance van’ or something? It’s like the real CIA surveillance van is not named that.

Casey Newton: 58:41 Look, I think we have a very important message to deliver to airline security and that is you guys gotta have a sense of humor, you know what I mean? You guys need to relax, live a little. Take a chill pill. It was a damn Bluetooth speaker. Do you know how irritated I would be if I was two hours into my flight into Mallorca and now we gotta turn around? Trying to get to that beach.

Kevin Roose: 58:57 All right, last one out of the hat.

Casey Newton: 59:00 You didn’t say stop generating.

Kevin Roose: 59:01 Stop generating. Okay, last one out of the hat.

Kevin Roose: 59:04 Okay, Casey this one’s in your wheelhouse. Survivor boss Jeff Probst says Kalshi and Polymarket are ‘incentivizing people to lie, cheat, and steal.’ This one comes to us from Variety. Apparently there was some drama on Survivor recently when one of the episodes was spoiled due to widely circulated reports about the odds on these prediction markets Kalshi and Polymarket. On both platforms, Aubrey Bracco was forecast to have an above 80% chance of winning before the season even premiered.

Casey Newton: 59:35 Yeah, so this is a story that brings together two of my favorite things, which are Survivor and hating on prediction markets. You know, we have talked on the show about the fact that one of the main things that prediction markets do is incentivize you to betray your friends, family, coworkers, and possibly your country. And so what Jeff Probst is noticing is that now like all of his crew members could stand to make a ton of money by betting on one of these prediction markets.

Casey Newton: 1:00:00 I’ll say that it does — we do not currently have information to suggest that this is what happened. Like, we don’t know of somebody in particular on the Survivor crew who may have leaked the information, but now we’re just living in a world where everyone is suspicious and I think it just contributes to the low-trust society that we’re already living in. So this is obviously very bad. But let me take this opportunity to say, Kevin, that while I do think that Aubrey played a great game and had a great season, I have to give a shout-out to the greatest player to never win the game, Cirie Fields, who truly came so close on this season and was amazing to watch every single week and I was absolutely crushed when she lost. So just incredible work, Cirie, and incredible work to the Survivor team for putting out 50 really amazing seasons of television.

Kevin Roose: 1:00:50 Only like one or two of which you could bet on on prediction markets. I mean, they were just doing this for the love of the game.

Casey Newton: 1:00:57 Mhm.

Kevin Roose: 1:00:58 We have to start pretty soon a segment that’s just about people getting caught for doing stupid stuff on prediction markets because, in addition to the George Santos thing and the Survivor thing, there was also a story this week about a Google engineer who was charged with using inside information to make a million dollars on Polymarket by placing bets on, allegedly, what users were searching for. So, truly no corner of society is safe from the corrupting influence of prediction markets.

Casey Newton: 1:01:21 Yeah, when you’ve got that Google money and you’re still trying to make a little extra scratch by, like, corrupting a prediction market…

Casey Newton: 1:01:29 Something has gone wrong in this society.

Kevin Roose: 1:01:31 Yes. So, that is HatGPT. Let’s close the hat.

Casey Newton: 1:01:35 Close up the hat.