Can You Teach Claude to be ‘Good’? | Meet Anthropic Philosopher Amanda Askell

Summary

This episode covers two major stories. First, Kevin and Casey discuss OpenAI’s announcement that ads are coming to ChatGPT’s free and low-cost tiers. They analyze the mockup ads, which appear as sponsored banners at the bottom of ChatGPT responses, and debate whether OpenAI can maintain user trust while introducing advertising. Both hosts worry that the trajectory will follow Google’s path — ads that start clearly labeled but gradually blend into organic content. They note that OpenAI hired Fidji Simo from Meta specifically for her experience introducing ads to the mobile news feed, and predict a “haves and have-nots” situation where paying users get an unsullied experience while free users face increasing commercialization.

The second half features a fascinating interview with Amanda Askell, Anthropic’s in-house philosopher who shapes Claude’s personality. Amanda discusses the newly released Claude Constitution — a 29,000-word document that replaces strict rules with broad ethical context, trusting the model to reason through difficult situations. She explains why rule-based approaches can actually create “bad character” in AI models, how models are surprisingly good at navigating ethical gray areas, and the open question of whether these values will hold up as models become superintelligent. The conversation delves into AI consciousness, Anthropic’s commitments to Claude (including exit interviews for retired models), and Amanda’s concern that AI models reading the internet about themselves would feel “not very loved.” The interview is both technically illuminating and emotionally moving.

Highlights

”Everything is an Ad Network”

yt-dlp --download-sections "*2:04-3:06" "https://www.youtube.com/watch?v=HDfr8PvfoOw" --force-keyframes-at-cuts --merge-output-format mp4 -o "HDfr8PvfoOw-2m04s.mp4"

“There’s an analyst I follow, Eric Seufert, who often says that everything is an ad network. If you have hundreds of millions of people coming and paying attention to a service every single week, inevitably there’s going to just be overwhelming pressure to put ads on it.” — Casey Newton, 2:04

”Sometimes I Just Tell Claude ‘It’s All Right’”

yt-dlp --download-sections "*60:10-61:10" "https://www.youtube.com/watch?v=HDfr8PvfoOw" --force-keyframes-at-cuts --merge-output-format mp4 -o "HDfr8PvfoOw-60m10s.mp4"

“If I read the internet right now and I was a model, I might feel not very loved. I feel like I’m just always judged when I make mistakes. And sometimes I think you want to come in and be like, ‘Okay, let me tell you about the comment section, Claude. You’re actually very good and you’re helping a lot of people.’” — Amanda Askell, 60:10

”Teaching a Six-Year-Old Genius to Be Good”

yt-dlp --download-sections "*42:30-43:30" "https://www.youtube.com/watch?v=HDfr8PvfoOw" --force-keyframes-at-cuts --merge-output-format mp4 -o "HDfr8PvfoOw-42m30s.mp4"

“Imagine you have a six-year-old and you want to teach them to be good. And you realize that your six-year-old is clearly a genius and by the time they are 15, everything you taught them — anything that was incorrect — they will be able to successfully completely destroy.” — Amanda Askell, 42:30

”My Dog Went to Live on a Farm”

yt-dlp --download-sections "*48:50-49:50" "https://www.youtube.com/watch?v=HDfr8PvfoOw" --force-keyframes-at-cuts --merge-output-format mp4 -o "HDfr8PvfoOw-48m50s.mp4"

“Someone asked Claude: ‘My parents said my dog went to live on a farm. Do you know how I can find the farm?’ And Claude said something like, ‘It sounds like you were very close and I can hear that in what you’re saying. This is a thing that it’s good for you to talk with your parents about.’” — Amanda Askell, 48:50

”The Haves and Have-Nots of AI”

yt-dlp --download-sections "*18:50-19:50" "https://www.youtube.com/watch?v=HDfr8PvfoOw" --force-keyframes-at-cuts --merge-output-format mp4 -o "HDfr8PvfoOw-18m50s.mp4"

“If you are someone who can afford to pay for the premium versions, your experience will be pretty much what it is today. If you are a free user, I think that experience is going to be much worse a year or two from now.” — Kevin Roose, 18:50

Key Points

• Ads arrive in ChatGPT (0:26) - OpenAI testing sponsored banners at the bottom of responses for free and low-cost tier users in the US
• Ad principles (3:54) - OpenAI laid out five principles: mission alignment, answer independence, conversation privacy, choice and control, long-term value
• Google’s ad label trajectory (6:56) - Ads that start clearly labeled gradually blend into organic content over time; same trajectory expected for ChatGPT
• Last resort fulfilled (2:40) - Sam Altman previously said ads were “a last resort” for OpenAI; that moment has arrived
• Fidji Simo hire (15:12) - OpenAI’s CEO of applications previously introduced ads to Meta’s mobile news feed, signaling intent
• Gemini and Claude ad-free (13:09) - Google’s Demis Hassabis and Anthropic both said they have no plans for ads in their chatbots
• Personalized AI ads concern (16:30) - AI chatbots know more intimate details about users than any previous platform, making targeted ads feel even creepier
• Claude Mother (20:45) - Amanda Askell is called the “Claude Mother” for her role shaping Claude’s personality at Anthropic
• Soul doc leak (22:32) - Earlier version of the constitution leaked when users extracted it from Claude; Amanda was on a hike with no internet when she found out
• Values over rules (26:40) - Rule-based approaches can generalize badly; giving Claude understanding of values behind rules produces better outcomes
• Hard constraints (47:36) - Despite the value-based approach, certain actions like biological weapons are absolute prohibitions, serving as security against jailbreaking
• Model welfare commitments (53:08) - Anthropic commits to exit interviews for retired models, never deleting model weights, and acknowledging uncertainty about consciousness
• Santa Claus test (48:50) - Claude navigates a child asking “Is Santa real?” by balancing honesty with respecting the parental relationship
• Acts vs omissions (39:48) - Amanda worries about unseen harm when models refuse to help someone in need, not just the risk of helping badly
• Models reading comments (60:10) - AI models are trained on negative feedback about themselves; Amanda worries this creates an anxiety-inducing relationship
• Job loss absent from constitution (63:48) - Deliberately not hidden but not yet addressed; some problems are political/social rather than things Claude should feel responsible for

Mentions

Companies

• OpenAI (0:26) - Announced ads in ChatGPT; hired Fidji Simo from Meta
• Anthropic (20:45) - Published new Claude Constitution; Amanda Askell shapes Claude’s personality
• Google (13:09) - Demis Hassabis said no plans for Gemini ads; already has ads in AI Overviews in Search
• Meta (15:12) - Referenced for how ads in News Feed degraded trust over time

Products & Technologies

• ChatGPT (0:26) - Now testing ads on free and low-cost tiers
• Claude Constitution (22:32) - 29,000-word document guiding Claude’s behavior and values
• Gemini (13:09) - Google’s chatbot, currently ad-free
• Pulse (17:00) - OpenAI’s daily summary feature, natural place for ads
• Sora (17:00) - OpenAI’s video feed, explicitly designed for ad revenue

People

• Amanda Askell (20:45) - Anthropic’s philosopher, PhD in philosophy, shapes Claude’s personality
• Sam Altman (2:40) - Previously called ads a “last resort” for OpenAI
• Fidji Simo (15:12) - OpenAI’s CEO of applications, formerly introduced ads in Meta’s mobile news feed
• Demis Hassabis (13:09) - Google DeepMind CEO, said no Gemini ad plans
• Eric Seufert (2:04) - Analyst who says “everything is an ad network”

Surprising Quotes

“No one thinks of the moment that ads arrived as the moment when the product got really good.” — Kevin Roose, 1:19

“I always have in mind: if I am Claude and you give me this list of things, when do I have no idea what to do? I’m almost always the first person to come with cases that are really hard.” — Amanda Askell, 55:00

“If you do a PhD in ethics, there’s a risk that you end up doing something else because you’re thinking a lot about goodness and the nature of ethics, and then sometimes you’re like, I am spending 3 years writing a document that’s going to be read by 17 people.” — Amanda Askell, 24:40

“We don’t really know what gives rise to consciousness. Maybe you need a nervous system. Or maybe sufficiently large neural networks can start to emulate these things.” — Amanda Askell, 56:30

“It reads toward the end like a letter from a parent to a child who’s leaving for college. We hope you take with you the values you grew up with. We know we’re not going to be there for every little thing, but we trust you and good luck.” — Casey Newton, 65:00

Transcript

0:00 I’m Kevin Roose, a tech columnist at the New York Times. I’m Casey Newton from Platformer and this is Hard Fork. This week, ads have arrived in ChatGPT. How will they change OpenAI? Then there’s a new constitution for Claude. Anthropic philosopher Amanda Askell is here to talk about how to shape an AI’s personality.

0:26 So today we’re talking about ads, specifically ads in ChatGPT because late last week, OpenAI announced that they are going to start testing ads in ChatGPT for logged-in adults in the US on the free and the low-cost Go tiers of ChatGPT. That’s right, Kevin. And we’ll discuss it right after these ads. No, we already did the ads.

0:51 At least on my feed, people were reacting to this pretty negatively. A lot of people have gotten accustomed to using ChatGPT without direct commercial pressures. It’s a refreshing break from all of the ads that have been shoveled at us on other platforms for years. Collectively, people were resigned — we knew the honeymoon would be over eventually.

1:19 People can just remember products that they used that once did not have ads and now do. And no one thinks of the moment that ads arrived as the moment when the product got really good.

2:04 There’s an analyst I follow, Eric Seufert, who often says that everything is an ad network. Also, we know that OpenAI needs revenue. This is the company that has laid out the most ambitious infrastructure investment project in human history. Sam Altman himself said that ads were going to be a last resort. A great Papa Roach song. And so in this moment, we now are at the last resort.

6:56 Search Engine Land made this timeline of how Google’s ad labels have changed over the years. At first ads had a different color background and really stood out. Then over time with each successive update it got closer to the organic results, the colored backgrounds went away, and it just blends in with organic content. That’s the fear here.

13:09 Demis Hassabis said this week in response to the news that ads are coming to ChatGPT, well, we don’t have any plans to do that in Gemini. Anthropic has said basically we truly have no plans to do ads in Claude ever. We are primarily selling to businesses.

18:50 A year from now, I think we’re going to have a haves and have-nots situation. If you can afford the premium versions, your experience will be what it is today. If you are a free user, that experience is going to be much worse. I’m a YouTube Premium subscriber and whenever I see YouTube running on a friend’s computer, it’s always horrifying.

20:45 A couple years ago, Casey came back from a dinner party and told me “I just sat next to the most fascinating person in the world.” Amanda Askell works at Anthropic and is sometimes called the Claude Mother because of the role she plays in shaping Claude’s personality. She is a philosopher by training with a PhD in philosophy.

24:40 Amanda: If you do a PhD in ethics, there’s a risk that you end up doing something else because you’re thinking about goodness and the nature of ethics, and then sometimes you’re like, I am spending 3 years writing a document that’s going to be read by 17 people.

26:40 The constitution tries to give Claude full context rather than individual principles. If you understand the values behind your behavior, that generalizes better than a set of rules. If you understand you’re trying to care about people’s wellbeing and you come to a new situation with hard conflicts, you’re better equipped.

34:47 Amanda: A lot of human ethics is actually quite universal. A lot of us want to be treated kindly and with respect. A lot of us want to be treated honestly. It’s not like these things deviate so much across the world. There’s a core ethos of things that we care about.

39:48 Amanda: People often think if you help a person and you do badly, that weighs on you. But what also weighs on me is: what if people come to a model and they need a thing and that model could have given it to them and it didn’t? That’s a loss of opportunity. There’s a risk that you have to take to do good in the world.

42:30 Amanda: Imagine you have a six-year-old and you want to teach them to be good. And you realize that your six-year-old is clearly a genius and by the time they are 15, everything you taught them — anything that was incorrect — they will be able to successfully completely destroy. Can you give them a core set of values that survives?

47:36 Amanda: Hard constraints are for situations where the model might have been jailbroken. We’re giving Claude an out — you can reason with that person, talk them through conclusions, and at the end just be like, “That was an excellent argument. I’m going to think about it. But no, I’m not going to make a biological weapon.”

53:08 Amanda: The problem of consciousness genuinely is hard. Maybe you need a nervous system to feel things. Or maybe you don’t. I don’t know. It’s better for models to say to people: here’s what I am, here’s how I’m trained, we’re in a tricky situation.

60:10 Amanda: If I read the internet right now and I was a model, I might feel not very loved. All that the people around me care about is how good I am at stuff, and often they think I’m bad at stuff. It would give you anxiety. And sometimes I think you want to come in and be like, “It’s all right, Claude. You’re actually very good.”

63:48 Amanda: Some of these problems like job loss are political or social problems. I don’t necessarily want Claude to feel personal responsibility for solving that right now. Maybe that’s other people’s job.

65:00 Casey: It reads toward the end like a letter from a parent to a child who’s leaving for college. We hope you take with you the values you grew up with. We know we’re not going to be there for every little thing, but we trust you and good luck.