The Hidden Cost of OpenAI’s Pentagon Deal? Trust.

Summary

This episode of Hard Fork covers the continued fallout from OpenAI’s deal with the Pentagon. After Sam Altman rushed to announce a deal with the Defense Department that included similar red lines to the ones Anthropic had insisted on — no domestic mass surveillance and no autonomous weapons — OpenAI faced one of the biggest backlashes in the company’s history. Altman admitted the announcement was “slopportunistic” and sloppy, releasing only partial contract language that procurement experts said was insufficient to evaluate. High-profile employee departures followed, including VP of Research Max Schwarz, who left for Anthropic. Kevin and Casey explore the deeper cultural divide at OpenAI between its original safety-minded researchers and newer hires, and why the company’s leadership needs to keep the former group happy to build GPT-6 and 7.

Meanwhile, Anthropic finds itself in a paradoxical “dual quantum state” — its annualized revenue has hit $20 billion (a 20x increase over the past year), driven by Claude Code adoption and enterprise growth, while the Trump administration has formally designated it a supply chain risk and forced federal agencies like the State Department to switch back to OpenAI’s older GPT-4.1 model. The hosts explore the looming possibility of “soft nationalization” of AI companies, drawing parallels to the Manhattan Project and the moment when scientists lost control of the atomic bomb to government decision-makers. They also note the whiplash of an administration that criticized Biden’s gentle AI executive order but is now threatening to destroy companies that resist full compliance.

In the second segment, Kevin and Casey examine the troubling intersection of prediction markets and the US-Israel war with Iran. Platforms like Polymarket and Kalshi allowed users to bet on airstrikes and the fate of the Iranian supreme leader, creating what the hosts call “grim technology number one” for this conflict. More than 150 accounts placed large bets correctly predicting a US airstrike, raising serious insider trading concerns — especially after Israel arrested people for using classified information to bet on military operations. The hosts debate whether prediction markets are fundamentally broken when applied to war, creating direct financial incentives for the worst possible outcomes.

Highlights

”Slopportunistic”

“Or slopportunistic, to coin a phrase.” — Casey Newton, 3:19

yt-dlp --download-sections "*3:03-3:46" "https://www.youtube.com/watch?v=tnv3kIGU8go" --force-keyframes-at-cuts --merge-output-format mp4 -o "slopportunistic.mp4"

”Most Americans Just Don’t Like AI Very Much”

“Most Americans just don’t like AI very much. They didn’t in the first place… When you add into that mix it’s potentially also going to be used by your own government to spy against you or maybe kill you with a murder bot, of course Americans are going to say well this freaking sucks, right?” — Casey Newton, 6:00

yt-dlp --download-sections "*6:00-7:01" "https://www.youtube.com/watch?v=tnv3kIGU8go" --force-keyframes-at-cuts --merge-output-format mp4 -o "americans-dont-like-ai.mp4"

”Anthropic Has 20x’d Over the Past Year”

“Bloomberg reported this week that Anthropic is on track to hit $20 billion in annualized revenue. At the start of 2025, Kevin, they were on pace to earn about $1 billion… So this company has 20x’d over the past year.” — Casey Newton, 9:56

yt-dlp --download-sections "*9:56-10:34" "https://www.youtube.com/watch?v=tnv3kIGU8go" --force-keyframes-at-cuts --merge-output-format mp4 -o "anthropic-20x-revenue.mp4"

”The Average College Freshman Now Has Better AI Than the State Department”

“The State Department is going back to GPT-4.1… that is several generations ago. That’s like an early 2025 model. And basically what that means is that the average college freshman with a ChatGPT subscription now has access to substantially better AI tools than the Department of State.” — Kevin Roose, 10:34

yt-dlp --download-sections "*10:34-11:40" "https://www.youtube.com/watch?v=tnv3kIGU8go" --force-keyframes-at-cuts --merge-output-format mp4 -o "state-dept-gpt41.mp4"

”A Country of Geniuses in Our Data Center”

“If we are headed to a world with very powerful AI systems in it, as Dario Amodei calls it, ‘a country of geniuses in our data center,’ that eventually that will just not be allowed to happen inside a private corporation.” — Kevin Roose, 15:00

yt-dlp --download-sections "*14:44-16:00" "https://www.youtube.com/watch?v=tnv3kIGU8go" --force-keyframes-at-cuts --merge-output-format mp4 -o "country-of-geniuses.mp4"

”150 Accounts Correctly Predicted the Airstrike”

“More than 150 accounts placed hundreds of bets of at least a thousand dollars, correctly predicting that there would be an American airstrike on Iran by Saturday.” — Casey Newton, 27:54

yt-dlp --download-sections "*27:54-28:47" "https://www.youtube.com/watch?v=tnv3kIGU8go" --force-keyframes-at-cuts --merge-output-format mp4 -o "insider-trading-airstrikes.mp4"

Key Points

• OpenAI’s Pentagon backlash (0:00) - OpenAI faced intense public and internal criticism after announcing a Pentagon deal with disputed surveillance and weapons prohibitions
• Sam Altman admits mistake (3:03) - Altman acknowledged the Friday announcement was rushed, “opportunistic and sloppy,” and released partial contract language
• Employee departures (4:29) - VP of Research Max Schwarz left for Anthropic; employee Leo Gao called the contract language “window dressing”
• Jedi mind tricks warning (3:46) - Casey warns that whether you’re surveilled will come down to semantic distinctions between “surveillance” and “intelligence gathering”
• Two camps at OpenAI (7:50) - Casey describes a divide between original safety-minded researchers and newer “Meta people” who are more flexible on military use
• Data center opposition (13:31) - Public distrust of AI companies could fuel political opposition to the data centers OpenAI needs to build
• Anthropic’s paradox (9:00) - Anthropic is simultaneously hitting $20B annualized revenue while being formally designated a supply chain risk
• State Department downgrades (10:34) - The State Department switched from Claude to GPT-4.1, a model several generations behind current offerings
• No statutory authority (11:40) - Lawfare reported there appears to be no legal basis for the president to ban federal agencies from using individual software
• Manhattan Project parallels (18:30) - Kevin rereads “The Making of the Atomic Bomb” and draws parallels to how scientists lost control of their creation
• Soft nationalization (21:45) - The current AI-Pentagon drama may be an “early dress rehearsal” for the government taking control of AI companies
• Prediction markets at war (22:54) - Polymarket and Kalshi allowed bets on airstrikes and the fate of Iran’s supreme leader
• Kalshi voids assassination market (24:51) - Kalshi voided its Khamenei market and reimbursed losers, angering traders who correctly predicted his death
• Israeli military insider trading (27:08) - Israel arrested people accused of using classified military information to place bets on Polymarket
• 17% probability before strikes (31:46) - Prediction markets showed only 17% probability of a strike an hour before it happened, undermining claims of superior information
• Assassination market incentives (34:05) - Prediction markets on war can create literal bounties on world leaders
• Kevin’s physical-only rule (37:07) - Kevin proposes that prediction markets should require in-person betting to add friction, like casinos

Mentions

Companies

• OpenAI (0:00) - Pentagon deal backlash, employee departures, damage control
• Anthropic (0:00) - Supply chain risk designation, $20B annualized revenue, Pentagon standoff
• Polymarket (25:33) - Offshore crypto prediction market that allowed bets on Iran strikes
• Kalshi (24:00) - Regulated US prediction market that allowed proxy bets on war outcomes
• Meta (7:50) - Referenced as example of company that survived user distrust but lost public goodwill
• DraftKings / FanDuel (23:59) - Broader sports gambling industry extending into prediction markets

Products & Technologies

• ChatGPT (1:43) - Users cancelling subscriptions in protest of Pentagon deal
• Claude / Claude Code (9:56) - Driving Anthropic’s explosive revenue growth
• GPT-4.1 (10:34) - Early 2025 model the State Department downgraded to
• GPT-6 / GPT-7 (7:50) - Future models OpenAI needs its core researchers to build

People

• Sam Altman (1:43) - Doing damage control on Pentagon deal, hosting AMA on X
• Dario Amodei (15:00) - Still seeking Pentagon agreement; coined “country of geniuses” metaphor
• Max Schwarz (4:29) - OpenAI VP of Research who left for Anthropic
• Leo Gao (4:29) - OpenAI employee who called contract language “window dressing”
• Senator Chris Murphy (26:12) - Introducing legislation to ban prediction market bets on war
• Ali Khamenei (24:00) - Subject of voided Kalshi prediction market
• Amy Qin (27:54) - NY Times reporter who covered insider trading on prediction markets

Government & Institutions

• Pentagon / Department of Defense (0:00) - Central to both the Anthropic and OpenAI deals
• US State Department (10:34) - Switched from Claude to GPT-4.1 per Trump order
• CFTC (28:54) - Regulates Kalshi but lacks enforcement capacity
• Defense Production Act (9:11) - Potential tool for compelling Anthropic to build military AI

Surprising Quotes

“Has an American technology company ever had such a good week and such a bad week at the same time?” — Casey Newton, 9:00

“We didn’t like that government trying to control the tech industry, but this government trying to control the tech industry, that’s just business as usual. That’s fine.” — Kevin Roose, 21:41

“Their policy was really like ‘imagine the worst thing you could do on our platform. You can do that.’” — Casey Newton, 25:45

“If you were one of the traders who did not get your expected winnings from the death of the Ayatollah, I just want to say I don’t care and it doesn’t matter.” — Casey Newton, 25:21

“Mark my words. One of these bots plugged into a Mac Mini is going to see a prediction market for the assassination of a world leader and it’s going to say, ‘Well, I have some ideas about that.’” — Kevin Roose, 34:40

Transcript

Kevin Roose: 0:00 Casey we are now in week two of this incredible high stakes drama that’s been playing out between the Pentagon and America’s leading AI companies. There’s been a lot going on. We now have more clarity on why the deal between Anthropic and the Pentagon fell apart. We also know how this Anthropic supply chain risk designation is actually going into effect and impacting the way that government agencies are responding. And we have been learning this week about how OpenAI’s deal with the Pentagon is shaping up. So lots to discuss here, but first we should make our disclosures. I work at the New York Times suing OpenAI and Microsoft in perplexity over alleged copyright violations.

Casey Newton: 0:39 And my fiancee works at Anthropic. Okay, let’s start with OpenAI because they are sort of the late arrival into this story but in some ways the most dramatic. Since Sam Altman announced last Friday that OpenAI had arrived at a degree with the Pentagon, we have learned a little bit more about that agreement. As a reminder, according to Sam Altman, this agreement did include some prohibitions on domestic mass surveillance and autonomous weapons systems, basically the same two red lines that Anthropic had set out that were causing them so much trouble with the Pentagon. And I think it’s fair to say like this provoked one of the biggest backlashes in that company’s history. It really did. We’ve seen it across social media. Many sort of top up-voted posts on OpenAI-related subreddits have been condemning this move. OpenAI has been scrambling to try to rebuild trust. But at the end of the day, Kevin, I think both the Pentagon and OpenAI are saying to the public you’re just going to have to trust us and the public is saying well we don’t.

Kevin Roose: 1:43 Right. So there’s been a lot of people cancelling their ChatGPT subscriptions and switching over to Claude as a result of all of this. People who don’t agree with the Trump administration or the stance that the Pentagon has taken here. And presumably because they’re seeing some, you know, some pain in the cancellations department as well as just a general feeling that this narrative is not going well for them, Sam Altman has been doing some damage control. So on Saturday he hopped on X to talk about this and answer questions about the Pentagon deal. He was joined by two other employees and these questions were sort of the kinds of things you’d expect. You know people asking what did you guys agree to that Anthropic didn’t? Where are your red lines? Who’s going to be making the kinds of hard decisions during something like a war about how these models can and can’t be used? What about this domestic mass surveillance thing? So I think he answered these questions but really the thing that they did was also to release the language of this contract that had been in dispute that had been the subject of so much speculation.

Casey Newton: 2:46 Well, they released what they call the relevant portion of the contract, but then we would see later commentary from experts in government procurement that said essentially look, until we see the entire contract, it’s just very difficult for us to know.

Kevin Roose: 3:00 Take at face value the idea that this is the only relevant language here. Right. So they didn’t release the whole contract, but they did release some relevant language from this contract with the Pentagon in a blog post. Then on Monday, Sam admitted that he made a mistake. He said we shouldn’t have rushed to get this out on Friday. He also added that it looked opportunistic and sloppy.

Casey Newton: 3:19 Or slopportunistic, to coin a phrase.

Kevin Roose: 3:22 Yes, it was slopportunistic. And it includes the language, quote, “the department understands this limitation to prohibit deliberate tracking, surveillance or monitoring of US persons or nationals, including through the procurement or use of commercially acquired personal or identifiable information.” I found this all slightly confusing. Casey, do you understand what OpenAI has said and the various evolutions of its position on this?

Casey Newton: 3:46 Well, I think that the key takeaways here is that they are saying that they have put in some amended language that will prohibit certain uses of their systems by the government. So for example, they’re going to prevent the government from using commercial data that they sort of acquire legally and sort of running that through GPT models for domestic surveillance. I just want to say though, that there is always a high risk here for what I would call Jedi mind tricks, Kevin, and for the government, because we have seen Democratic and Republican presidents do this, right, of sort of going to the absolute limit of what the law will allow when it comes to surveillance of Americans and a way that they’ll get around that is by saying, well, we’re not doing surveillance, Kevin, we’re doing some intelligence gathering, right? And so as annoying as it is to fixate on the semantics here, I’m telling you that whether or not you personally are surveyed will come down to semantics, right? And so that’s why we’re digging in the way that we are.

Kevin Roose: 4:21 So still lots of questions about some of the details here, and I think there’s a lot of doubt and concern among some employees of OpenAI that this actually did end up in a place that they’re comfortable with.

Casey Newton: 4:29 Boy is there. Some of that employee discontent spilled over onto X, where you had some employees say essentially that they didn’t trust their leadership either. An employee named Leo Gao, an OpenAI employee, called the contract language window dressing and pointed out that it still seems to give the Pentagon control over when to deploy autonomous weapons and doesn’t do much to address some of the other loopholes. And then maybe more dramatically, Kevin, on Tuesday, Max Schwarz, who was the post training lead for OpenAI, a vice president of research at the company, announced that he was leaving and in his X post, while he was pretty vague, suggested that this was an important time and that he had come to really respect Anthropic’s values. And so he said he’s going over to work there.

Kevin Roose: 4:41 Yeah. So what’s your take on how the damage control is going for OpenAI? Do you think they have warded off the most heated criticism or are people still really mad?

Casey Newton: 4:46 I do not think that they have stemmed the tide.

Casey Newton: 6:00 I put a lot of effort into changing the narrative here when I saw that they were doing that AMA, that they had put up a blog post, that they were quoting at least some of the contract language, I thought these guys are really going for it, that also told me that they were really scared. But here’s the thing to remember Kevin: most Americans just don’t like AI very much. They didn’t in the first place, they didn’t like it for all the normal reasons of well my social media feed is filling up with slop or my manager’s telling me I have to use it every day or I’m going to get fired. When you add into that mix it’s potentially also going to be used by your own government to spy against you or maybe kill you with a murder bot, of course Americans are going to say well this freaking sucks, right? So I think this was kind of the strategic miscalculation that Sam Altman made was that, at least according to him, he thought he was going to get into this dispute and sort of be able to de-escalate it and sort of come in as the white knight and save the AI industry from the overreach of the US government and what he found out instead is they’re still kind of holding the bag of all of the discontent that the Pentagon whipped up with this force policy change.

Kevin Roose: 7:01 Yeah, it’s really interesting to me because I think my assumption had been that we were sort of over the era of like worker empowerment in Silicon Valley, right? With like years ago sort of pre-COVID we had all these like Google walkouts and all these employee protests over these military contracts. And I think a lot of CEOs and leaders of these companies sort of said, we’re not doing that again, like we’re not going to give our employees veto power over the deals that we make or the contracts we sign. And it suggests to me what is going on at OpenAI right now that at least for them in their specific case where you do have, you know, this staff of elite technical talent that are not easily replaceable, there aren’t that many people who know how to like build and train these models, you actually do need to keep them happy and so those people maybe only those people have significant leverage still.

Casey Newton: 7:50 Yeah. Let me make a sort of sweeping generalization, right? Like I think there’s sort of like two major camps at OpenAI. One are the camp that have sort of been there for, you know, let’s say three-plus years, that are the real experts that you just mentioned, that have this kind of critical knowledge for how to build next-generation frontier systems that almost nobody else in the world has. And those people tend to just really care a lot about how the technology is used. These are like people who joined OpenAI in part because it was a nonprofit, right? And like there is like a solid core of those folks who are still working there. And then there’s a group at OpenAI that I’m just going to call the meta people. Like the people that came over from Meta a little bit more recently that are, you know, maybe a little bit more flexible in what they’re willing to see their company do, and I don’t think that they’re going to raise a big stink about this. The problem if you’re OpenAI leadership is you actually need that original core, right? If you’re going to build a GPT-6 and 7 that is going to blow everybody’s minds, those are the people you’re going to need. And so yes, almost everything that we have seen over the past few days as they have tried to do damage control is aimed at those people.

Kevin Roose: 8:52 Okay, so that’s a little bit of the drama going on at OpenAI. What is happening at Anthropic?

Casey Newton: 9:00 Printing money. In two words. Hey, well, I mean you know, I wrote this in my newsletter this week, Kevin, but has an American technology company ever had such a good week and such a bad week at the same time?

Kevin Roose: 9:10 Explain.

Casey Newton: 9:11 Well, so on the bad side, obviously they’re in a very heated fight with the Pentagon that continues by the way. It seems like there is still some risk that perhaps the president will try to invoke the Defense Production Act to try to compel Anthropic to make the version of Claude that it does not want to make that would sort of do its bidding. And it seems also that the supply chain designation risk is now official. We learned on Thursday that the Pentagon sent a formal letter to Anthropic. So if nothing else, this is going to result in a long and costly legal battle as Anthropic tries to ensure that American companies can still use it for non-military purposes, right? So there is actually an existential threat to the company that is buried somewhere inside there and it is by no means over.

Kevin Roose: 9:55 Right. But on the good side?

Casey Newton: 9:56 On the good side, Bloomberg reported this week that Anthropic is on track to hit $20 billion in annualized revenue. At the start of 2025, Kevin, they were on pace to earn about $1 billion in annualized revenue. So this company has 20x’d over the past year. They were on pace to earn about $9 billion by the end of 2025. So it has doubled in barely over two months, which speaks to the rise of Claude Code, right? And the overwhelming adoption of Claude in the enterprise. So in that respect, this really has become maybe the fastest growing American technology company of all time.

Kevin Roose: 10:34 Yeah, and like what’s so strange about this sort of dual quantum state right now of Anthropic is like at the same time that they are printing money and people are signing up for Claude and they’re switching from ChatGPT and like things appear to be going well for them. At the same time, they are also being pulled out of the federal government, forcibly. There was some reporting this week by Reuters that the US State Department has sort of started to comply with this order from President Trump to sort of stop using Anthropic’s models. They have switched the model powering their sort of in-house State Department chatbot from Anthropic’s models to OpenAI, according to this memo seen by Reuters. And furthermore, this Reuters report said that the State Department is going back to GPT-4.1. Now, if you have not been tracking all of the model names and numbers as closely as we have, that is several generations ago. That’s like a early 2025 model. And basically what that means is that the average college freshman with a ChatGPT subscription now has access to substantially better AI tools than the Department of State.

Casey Newton: 11:40 It’s not great for a lot of reasons, Kevin, and one of them, as the blog Lawfare covered this week, is that there appears to be no statutory authority for the president to do what he did. There is not a statute that lets the president just sort of declare that federal agencies cannot use individual software. But because this is just the way the Trump administration works, everyone has just decided to comply.

Kevin Roose: 12:00 Yeah. I want to ask you about this other sort of interesting piece of OpenAI’s response over the last week, which is that Sam Altman has said multiple times that he wants the Pentagon to extend the same deal to Anthropic that it extended to OpenAI. Do you think that is sincere? What is going on here? Why is Sam Altman saying, ‘Hey, if you’re making these terms available to us, you should also make them available to other AI companies’?

Casey Newton: 12:25 I think that that is the part of Sam that appears to be sincere in saying that he wants to de-escalate this conflict. He does not want the United States government to come in and nationalize the AI companies, at least not right now, right? And so maybe if OpenAI could reach some sort of agreement that would provide at least some protections for Americans and other AI companies would sign onto it, that would just release the pressure on the industry overall. Now, of course, at the same time, it would buy him a lot of cover, and all of a sudden people wouldn’t be mounting these ‘quit ChatGPT’ campaigns because Sam could be on X saying, ‘Well, you know, Claude is doing the same thing.’

Kevin Roose: 13:02 Do you think that’s real? Like, how big a deal do you think this consumer opposition is? I mean, I, you know, I am somewhat jaded on this point because I can’t count the number of times that people have said, you know, ‘Oh, we’re all going to cancel our subscriptions to this thing’ or ‘We’re going to delete Uber’ or ‘We’re going to, you know, quit Facebook in protest.’ And like, it never really seems to have much of an impact. But like, do you think in this case that enough people are mad about this at the consumer level that it could actually impact their business?

Casey Newton: 13:31 Not really. I think you’re exactly right. I think that usually these things just tend to blow over in a few days, and I’m sure that OpenAI is counting a lot on that. At the same time though, Kevin, I think back to the lesson that Meta learned, which is that as it had its own series of controversies, by and large, people did not quit Facebook. They did not quit Instagram. But you know what they did do? Just kind of start to hate Meta as a company and develop really low trust in that company, and that winds up hurting Meta in all sorts of ways. And the particular way, by the way, that I think this is going to hurt OpenAI is they’re gearing up to go out and build a lot of data centers around this country. And there’s already enormous backlash, and that we are seeing right — we’re starting to see it creep into our politics. And so if they are not able to sort of reverse the narrative and convince people that AI is going to have, like, hugely positive outcomes in their lives, I think you’re going to see the data center opposition ramp up as a proxy for people’s just kind of distrust of that company in general.

Kevin Roose: 14:30 Right. It’s the visible physical symbol of all of this, and for most people, the only one that is, like, anywhere near them. And so I think you’re right. It could turn into a political problem for them even if people aren’t canceling their ChatGPT subscriptions en masse.

Kevin Roose: 14:44 I want to ask you about something else that I’ve been thinking a lot about this week, which is this idea that you mentioned of nationalization. There’s been a debate happening on social media about this idea that if we are headed to a world with very powerful AI systems in it, as Dario Amodei calls it, ‘a country of geniuses in our data center,’ that eventually that will just not be allowed to happen inside a private corporation. That the US government, whether a year or two years or five years from now, at some point will step in and say, ‘Hey, you guys built this really cool thing that’s really useful and has all these like important geopolitical and national security implications. We’re gonna just take that now. And you work for us now.’ And I’m curious what you make of that as a possibility, because some people who I consider quite serious and credible, have been talking about this threat of nationalization for several years now.

Casey Newton: 15:37 Yeah, if you go to the sort of nerdy AI conferences that Kevin and I do, this comes up a lot at the tabletop role-playing games that people do during lunch, right? Is that at some point, a government of one or more countries kind of steps in and takes over the AI lab. I understand in this moment that that feels like a kind of sci-fi scenario, right? Like most of the time when you’re using ChatGPT, you probably don’t think, ‘this is a dangerous superweapon and we need to ensure that, you know, this is being controlled by the president.’ At the same time, we are now at war with Iran. We know that these systems are embedded in the like command and control operations of the military. And so to some extent, they are already becoming weapons, right? So if you say to me, do I think that once these systems become three, four, five, ten times more powerful, the government will want to take an interest in them and potentially oversee their development and deployment? I absolutely believe that will happen. I see no reason why that wouldn’t happen. And unfortunately, how that goes, I think depends a lot on the quality of the government that is overseeing that AI, right? And like what do they want to do it? Do they want to use it to create opportunity and safety and democracy for all or do they want to, you know, mount an authoritarian takeover of the globe?

Kevin Roose: 16:56 So if you are a leader of one of these companies and you know that at least until 2028, we are likely to have sort of the same administration in power. If you believe that the technology is rapidly accelerating such that a year or two years or three years from now, we might have something like a superhuman country of geniuses in our data center. What does that mean you should do? I mean, one thing that I’ve been thinking about is like, should these companies be doing deals with the government at all? Right? If the lesson of the past couple of weeks is that the federal government is not a trustworthy counterparty in these negotiations and is going to insist on total control and obedience or else they’re going to try to nuke your company. Like, I think a very rational response from these AI companies will be like, ‘Well, we’re just not gonna make any more deals with you. You’re gonna have to use some open source models for your state department and your military and your treasury because it’s just too risky for us as a business risk, and you can’t be trusted with it.’

Casey Newton: 17:52 I could see why that may seem somewhat rational to them, but like, I don’t think that that is the tack that they are going to take.

Casey Newton: 18:00 And this has happened with Anthropic, Dario Amodei is still out there saying we were very close to an agreement with the Pentagon, we liked working with the military, we want to work with the military again. Right, so I think that’s very important to note. Like, Dario did not like throw up his middle fingers like on his way out the door. He is still trying to reach some sort of agreement, and I think in part that likely is to avoid the exact sort of scenario that you are describing, right? It’s you kind of want to like keep the tigers at bay for just a little while longer at least, while you maybe like think through the rest of that scenario, which is admittedly a very difficult one.

Kevin Roose: 18:30 I’ve been rereading The Making of the Atomic Bomb this week, which is Dario Amodei’s favorite book, and he used to give it to all Anthropic employees, and there’s still like a bunch of copies at their headquarters. It’s sort of the company book as far as their mission goes. And they see a lot of parallels between what they’re building and the Manhattan Project. And so I went back and I’ve been rereading it, and the piece that struck me from that experience was just right before the bombs were dropped in 1945, there was this point where the scientists got really worried about how their creation was going to be used, and a number of them from the Manhattan Project sort of created these petitions and reports and tried to get them to the government and say, like, ‘Hey, could you guys like not use this against a city, at least as like a first-line, you know, act of war?’

Kevin Roose: 19:27 And the military and the government sort of like pretended to hear them out and then they just went ahead and bombed Japan anyway. And there was sort of this moment where it was like, we hear you, you’re the scientists, you’re the geniuses who made this all work, but now you’re playing in our turf and so we’re going to control the technology from here, and like thank you for your input. And like, I think the comparison between the Manhattan Project and the AI industry is somewhat overstated, and I think it breaks down in some key ways, one of which is like that was a government project. You know, the Manhattan Project was paid for by the government. These were government employees. What we’re talking about now are private companies that have been developing this thing outside the public sector. So I think there’s some important differences. But I do worry that we are headed toward a moment where this stuff just gets so useful to governments and militaries and confers such a decisive advantage to the countries that control it that the US government, no matter kind of who is in power, is just going to say, like, this thing is too important to be left to the private sector.

Casey Newton: 20:22 Well, I mean, keep in mind that one of the original ideas for OpenAI was that it should be a government funded project. But Sam Altman and his co-founders just came to the conclusion, correctly by the way, that no government would give them the amount of money they needed to build this technology, right? And, you know, they just sort of quickly came to the conclusion that it was just going to have to be a private enterprise. But, you know, going back to the earliest days, there was thinking among the people that created this technology that the government was going to take an interest in it eventually. Another reason though, Kevin, why I find the current situation so vexing is that you and I both covered President Biden’s executive order on AI, which I personally felt like was a pretty gentle way of attempting to regulate industry. It was sort of like, you know, tell us about your safety testing please when you test these new models and sort of told federal agencies to get ready for this technology. And the howls of protest on the right that said, how dare, you know, this administration come in and try to put these fetters on capitalism. We are going to lose to China because of this sort of nanny state behavior. And then to see those same people come to power and now say, we are going to tell you exactly how you are going to build your models, what they are going to do for the military or else we will destroy you is just like the whiplash is insane.

Kevin Roose: 21:41 Yeah, we didn’t like that government trying to control the tech industry, but this government trying to control the tech industry, that’s just business as usual. That’s fine.

Kevin Roose: 21:45 So I guess my worry zooming out from all of all of this stuff that’s been going on for the past two weeks is that we are sort of living through like an early dress rehearsal for what something like nationalization of the AI companies could look and feel like. I don’t think it’s going to be as sort of cut and dry as like it was during World War II where like the government showed up to a bunch of like steel plants and was like, hey, we run these now. I think it’s going to be kind of this soft nationalization like we’ve been seeing over the past week where it’s like a little pressure to build your models differently. Oh, maybe could you remove some of those safeguards? Oh, maybe this is actually so strategically important that we need to be the people putting the clauses in the constitution of Claude or whatever that dictate how it will behave in these high stakes situations. And I think that is a more likely direction, but I would not take full sort of like brute force nationalization off the table entirely. I think there’s a decent chance that something like that happens.

Casey Newton: 22:44 Hmm. Well, maybe we should set up a prediction market for it.

Kevin Roose: 22:47 Speaking of prediction markets, when we come back, we’ll talk about how prediction markets have made it to war. So predictable.

Kevin Roose: 22:54 Okay, Casey, so the other big news from the past week is that the United States is now at war in Iran. And one angle that really has been sticking out to me about this is the role that prediction markets are playing in this conflict because I think that is something that we truly have not seen before.

Casey Newton: 23:13 Yeah, it seems like every new war brings along some grim new technology and I would say that prediction markets are maybe grim technology number one for this conflict in Iran.

Kevin Roose: 23:24 Yes, it’s a grim technology already, even absent the war and now just with the war it has become even grimmer. And we’ve talked about prediction markets on the show. We talked about them way back in 2023 when they were sort of this new thing that was like kind of in this legal gray area that wasn’t really being done at any scale yet. It was sort of an interesting idea. Now, of course, you cannot walk down a street in a major American city without seeing one and probably multiple ads for prediction markets like Kalshi and PolyMarket.

Casey Newton: 23:59 Yeah, this sort of gambling mania that has taken over all media and advertising, from DraftKings, the FanDuel has now extended even further into these prediction markets.

Kevin Roose: 24:00 So, both Polymarket and Kalshi, the two leading prediction markets platforms, took a lot of heat this week on bets they were allowing their users to make on questions related to Iran. So, Kalshi, which is kind of the more regulated U.S. based prediction markets company, does not allow bets on war or assassination, but it did allow the question ‘Ali Khamenei out as Supreme Leader’ basically sort of as a kind of careful proxy for betting on the outcome of a war or a strike on Iran.

Casey Newton: 24:30 Yeah, and ‘out’ I suppose could have, you know, many meanings. You know, perhaps there would be a sort of gentle democratic revolution in Iran, but I’m going to assume that most of the people who were wagering on that one assumed that he was going to be killed in war.

Kevin Roose: 24:51 Yeah, so people got really mad at Kalshi for allowing these bets on the fate of the Iranian leader. They also got mad when Kalshi sort of voided this market and said that it was going to reimburse anyone who may have lost money on this, basically make sure everyone ends up in the black. But people who were supposed to make a bunch of money because they correctly predicted the death of Khamenei were mad that they didn’t get paid out their expected winnings. So, just a big cluster all around.

Casey Newton: 25:21 And I just want to say, if you were one of the traders who did not get your expected winnings from the death of the Ayatollah, I just want to say I don’t care and it doesn’t matter.

Kevin Roose: 25:33 So, Polymarket, the other sort of less regulated offshore crypto-based prediction market, was even more permissive. They allowed people to bet on the dates of strikes on Iran and other details related to the war in Iran.

Casey Newton: 25:45 Their policy was really like ‘imagine the worst thing you could do on our platform. You can do that.’ Actually, they did draw a line when it came down to markets that allowed users to bet on the likelihood of nuclear detonations by specific dates. So, sorry to anyone who was trying to cash in on nuclear war. These woke liberals that won’t let me bet on nuclear explosions need to go, Kevin.

Kevin Roose: 26:12 So, no one was happy about this. Senator Chris Murphy posted that “it’s insane this is legal. People around Trump are profiting off war and death” and also said that he was introducing legislation to ban this. And there are also a bunch of people looking into whether any of this has been done via insider trading. Basically, do you have people in the military or close to the decision makers in this conflict placing bets once they have this sort of non-public information about what is going to be happening?

Casey Newton: 26:38 Yeah, and I think it speaks to why allowing prediction markets to take bets at least around sort of like war and death is so corrosive and bad, Kevin. Because not only is it just kind of like grim and like how do we live in this society where, you know, gambling on war and death is becoming a sort of form of entertainment…

Kevin Roose: 27:00 But also, you’re just creating incentives for like the worst things in the world to happen, which doesn’t seem logical to me.

Kevin Roose: 27:08 Well, and it’s not even a theoretical harm here. Recently, Israel arrested a number of people who were accused of using classified information to bet on military operations on Polymarket. So this is already starting to happen, and I think this is why people like Senator Chris Murphy are so alarmed about this. Not just because it’s sort of like gross and aesthetically offensive to have people betting on wars…

Casey Newton: 27:32 Although it is.

Kevin Roose: 27:33 But yeah, although it is, but also because it could create direct incentives if you’re a member of the military and, you know, your commander gives you an order to go do an airstrike on an Iranian compound, to log onto your phone and head over to one of the prediction market platforms and say, ‘You know what? I could make a couple grand off this.’

Casey Newton: 27:54 Yeah, that’s your little Kalshi bonus. You know, this is not theoretical at all, Kevin. In fact, your colleague Amy Qin at the Times wrote that it is relatively uncommon for someone to bet a significant sum of money that a US strike will happen within the next day. But just last Friday, more than 150 accounts placed hundreds of bets of at least a thousand dollars, correctly predicting that there would be an American airstrike on Iran by Saturday.

Kevin Roose: 28:21 Yeah. So I think one of the interesting things here is like, I am not like a blanket opponent of prediction markets, right? I sort of bought some of the kind of theoretical arguments for why something like a prediction market could, for example, outperform political polls, because it would incentivize people to like come up with really good polling data and like use that to trade on, and you could end up with kind of a better picture of a given election.

Casey Newton: 28:47 Or people will like say what they really think because their money is at stake and they’re not just trying to like impress a pollster.

Kevin Roose: 28:54 Yes. And you’ve actually had some of the people who are in charge of these prediction markets sort of talking about the fact that insider trading can be good because it can get the best information to the markets as quickly as possible and kind of like give people an unfiltered understanding of what the real insiders are thinking. Now, of course, officially you are not supposed to be able to insider trade on these platforms, right? They all have policies against it. Kalshi, the most regulated US platform that allows for prediction markets, says, you know, they’ve investigated people, that it is actually illegal per the CFTC, which is their main regulator, to place bets using inside information. But there are a couple problems with this. One is the CFTC is a tiny agency. It doesn’t have a huge team of enforcers going out to investigate what I assume must be hundreds or thousands of trades using inside information on their platform every day. It’s also not clear what is public information and what is private information. You know, there are certain types of information in the stock market that are considered material non-public information that it is illegal to trade on. But it is also legal to, you know, fly a drone over an oil facility to see how their production is going or to park outside a store and see the foot traffic going in and out and use that to sort of calculate how well their sales must be going.

Casey Newton: 30:12 I find it suspicious how much you know about the insider trading rules, I have to say. I didn’t know you had this much facility with the law here.

Kevin Roose: 30:20 I’m calling my lawyer. But of course, this is part of the appeal of prediction markets in general is that they incentivize people with good information to trade on that information.

Casey Newton: 30:28 Yes, and if you allow people to wager on almost anything, how are you ever possibly going to police the entire platform to understand who is insider trading and who isn’t?

Kevin Roose: 30:39 Yes. So I think in this specific case of war, I think it’s very dangerous for some of the reasons that we’ve talked about. Not only do you have military officers and service people disclosing classified information in some cases to sort of make a little extra for themselves, but you also have just this incredibly strange war profiteering innovation where like you can just go on one of these platforms and try to make a bunch of money from something that involves a lot of devastation and destruction.

Casey Newton: 31:14 You know, the other thing that comes to mind for me, Kevin, is that, you know, as you say, the prediction market backers, their argument is like this just helps us understand the world better, right? This is a new kind of information that helps us see more clearly. And yet as I look across all of the trades that you just described, I don’t understand really what I was supposed to see more clearly, right? Like maybe you get a, you know, a brief heads up about something horrible that is about to happen, maybe that’s, you know, useful in at least some circumstances, but for the most part, I just don’t feel like we actually have a much better understanding of the world because all of these bets are happening.

Kevin Roose: 31:46 Yeah, and I think in this specific case, that’s especially true because if you actually look at the markets that were being traded before this strike on Iran, the conventional wisdom of the crowd was that this was not going to happen. It was a very low probability, I think something like 17% probability on one of these platforms an hour before the strikes. So these markets aren’t actually distributing the best possible information at all times. They’re just kind of like aggregating vibes until like someone with inside information shows up and like makes a fortune.

Casey Newton: 32:20 Well, I think that’s exactly it. It isn’t as if these have been adopted by the mainstream and everybody’s placing these sort of casual bets and now we have this like beautiful, perfect understanding of the world. What we have, as you say, is a bunch of vibes plus some insider trading and it just doesn’t actually seem that useful to me in practice for most things.

Kevin Roose: 32:38 I want to try to like sort of steelman the defense of prediction markets here and see what you make of it. So I think someone who believes that these prediction markets are good in the aggregate might say something like the following. People have been betting on war forever. They bet on the stock prices of defense companies. They bet on things like oil prices. That is all legal, we consider that sort of part of the normal markets. Those things all fluctuate when you have a war break out. How is this any different? Your response.

Casey Newton: 33:08 Well, I think that it is actually really meaningful that these are indirect ways of betting on war, right? It seems very unlikely to me that if I, like, you know, buy oil stocks assuming that they’re going to go up, that I’m creating an incentive for somebody to assassinate the supreme leader of Iran.

Kevin Roose: 33:27 But wasn’t this the whole conspiracy theory about the war in Iraq was that it was just motivated by like Dick Cheney owning a bunch of stock in Halliburton?

Casey Newton: 33:35 Well, I mean, yes, that was like the conspiracy theory. You know, I don’t know that that was what was actually driving it. I think that, you know, as with most wars, at least at that time, there were sort of like a number of interrelated factors that were going on and, you know, maybe oil was one of them. My point here is just that when you have the betting at some sort of meaningful remove from the action, it just like feels better for me. It doesn’t create the same horribly grim incentives that this particular approach does.

Kevin Roose: 34:05 Right. I think the difference for me is the directness that you mentioned and, you know, one thing that came up over and over again when I was talking to people about prediction markets a couple years ago for this story is the assassination markets get really dark because if you have something like, you know, will this world leader, you know, be removed from power in air quotes…

Casey Newton: 34:21 Wink, wink.

Kevin Roose: 34:23 …before a certain date, that can actually create a bounty on that person where someone might go out and say, ‘Hey, if I want to make money on this, I need to like kill this person before this date.’

Casey Newton: 34:34 And you know what is going to be the first thing that actually takes action on that, Kevin? Open-Claw.

Kevin Roose: 34:40 Mark my words. One of these bots plugged into a Mac Mini is going to see a prediction market for the assassination of a world leader and it’s going to say, ‘Well, I have some ideas about that.’ So I think most people agree that like the assassination prediction market is sort of, you know, out of bounds and is a bad idea for lots of reasons. But I think there are still a lot of gray area around these questions about conflict and war and politics and I think it is the risk here is that these prediction markets have gotten so popular so quickly with so little regulatory oversight that it is just kind of legal to do a bunch of stuff on them that it’s not legal to do in the regular stock market.

Casey Newton: 35:19 Yeah. Well, so you mentioned that some lawmakers have talked about introducing legislation. My experience is that that kind of legislation typically doesn’t go anywhere. What if anything do we know about what is going to happen as this war continues to unfold in Iran when it comes to these prediction markets?

Kevin Roose: 35:37 I mean, I think the Trump administration is very unlikely to do anything to sort of stop the growth of prediction markets. We’ve already seen them signal, via these sort of regulatory actions that they’ve dropped against Polymarket, that they are not going to take a firm line against these prediction markets. We’ve also seen them adding members of the Trump family to their advisory boards. So I think all of these prediction markets are sort of becoming entangled with the administration in ways that are going to make it very hard for them to do anything, but I certainly expect like Democratic lawmakers to stand up and say like what the hell are we enabling here? Why are we allowing people to bet on the assassination of world leaders or the outcomes of a war in Iran? This just feels all incredibly fraught to me.

Casey Newton: 36:20 My fear is that we’re in a sort of time race where like if Democrats were able to like somehow advance some legislation, maybe they win some seats in the midterms, maybe they retake the presidency, maybe sometime within the next few years, they could meaningfully rein these prediction markets in. I think though if they continue to grow, my fear is that they will become a massive entrenched interest group like the crypto world and they will then lobby to ensure that Democrats and Republicans both feel like they have a vested interest in these things sticking around. So, you know, my fear is that if we’re to do anything about some of these excesses we’ve been talking about today, it needs to happen soon or otherwise platforms like Kalshi and Polymarket might just have too much money for that to happen.

Kevin Roose: 37:07 I have a proposed rule for these prediction markets. Which is that you should have to go to a physical place like you do for a casino. I think that putting this stuff on people’s phones, making it super easy for them to do it, like if you want to go bet on the war in Iran, you should have to like go to a seedy like OTB betting place to do it. Like you should have to like put in some effort. It should not be as easy as whipping out your phone.

Casey Newton: 37:31 Alright, well, it’s very interesting Kevin, I predict we are not going to try that.

Kevin Roose: 37:38 No, I also predict we’re not going to try that. But it’s a good idea. People should listen to me.