Arushi Saxena: Can We Trust AI?

Summary

Paul Ford and Rich Ziade are joined by Arushi Saxena, a trust and safety expert who works at Harvey, the legal AI startup, and has experience in both the public and private sectors. The conversation explores what trust and safety means in the AI age, examining both individual privacy concerns and organizational responsibilities when building with LLMs. Saxena explains that trust and safety as a profession has evolved from social media moderation over the past 10-15 years into a new discipline focused on AI-specific challenges like data retention, hallucinations, and the emerging threat landscape of AI agents and tool calling.

The discussion takes a practical turn as the hosts and Saxena examine the everyday privacy implications of feeding personal information into AI tools. Paul shares a striking moment when ChatGPT began weaving personal details from past conversations into unrelated answers, creating an unsettling profiling experience that goes beyond traditional ad tracking. Saxena advises against inputting personally identifiable information (PII) into LLMs and notes that while hallucination rates are improving, over-reliance on AI as a trusted friend, therapist, or advisor is her biggest concern. She recommends users review their privacy settings on AI platforms, where options exist to prevent data training.

The episode broadens into a discussion about regulation, with Saxena noting California’s SB 53 as an example of state-level AI transparency legislation emerging even without a federal framework. The hosts close with a philosophical meditation on digital surveillance, small group dynamics, Dunbar’s number, and the appeal of encrypted, ephemeral communication tools like Signal as a refuge from the pervasive data collection ecosystem.

Highlights

”ChatGPT is building a narrative of your life”

yt-dlp --download-sections "*4:31-5:25" "https://www.youtube.com/watch?v=-eqj9qKjWtI" --force-keyframes-at-cuts --merge-output-format mp4 -o "-eqj9qKjWtI-4m31s.mp4"

“ChatGPT is really focused on increasing engagement and it keeps dropping personal details from past chats into the output. It’s not cool. It feels like it’s been stalking. It’s a robot.” — Paul Ford, 4:31

”Don’t input what you don’t want leaked”

yt-dlp --download-sections "*5:18-6:10" "https://www.youtube.com/watch?v=-eqj9qKjWtI" --force-keyframes-at-cuts --merge-output-format mp4 -o "-eqj9qKjWtI-5m18s.mp4"

“Things that you don’t want other people to see, do not input into your LLM unless you have a really good reason to be doing so. That’s the advice I give my mother and grandmother and grandfather.” — Arushi Saxena, 5:18

”ChatGPT confidently got the geometry wrong”

yt-dlp --download-sections "*7:00-7:55" "https://www.youtube.com/watch?v=-eqj9qKjWtI" --force-keyframes-at-cuts --merge-output-format mp4 -o "-eqj9qKjWtI-7m00s.mp4"

“My daughter was having a geometry struggle. I took a picture of her laptop, uploaded it to ChatGPT and it very confidently produced absolutely the wrong answer, like with a full proof. Conveying that level of ambiguity to a 12-year-old — she’s not used to systems just not working.” — Paul Ford, 7:00

”People asking LLMs what they know about them”

yt-dlp --download-sections "*17:50-18:40" "https://www.youtube.com/watch?v=-eqj9qKjWtI" --force-keyframes-at-cuts --merge-output-format mp4 -o "-eqj9qKjWtI-17m50s.mp4"

“The viral trend that you may not have realized you just pointed to — people asking these LLMs, ‘What do you know about me that I don’t know about myself?’ I haven’t done it because I’m too scared to do it and I work in the industry.” — Arushi Saxena, 17:50

”You are the star of a spy movie”

yt-dlp --download-sections "*28:40-29:35" "https://www.youtube.com/watch?v=-eqj9qKjWtI" --force-keyframes-at-cuts --merge-output-format mp4 -o "-eqj9qKjWtI-28m40s.mp4"

“You are the star of a spy movie in which an entire global satellite apparatus is tracking you. You are actually Will Smith in Enemy of the State. You are being observed like in a spy film and they are doing absolutely everything they can. You are the terrorist as far as they can tell.” — Paul Ford, 28:40

Key Points

• Trust and safety defined (2:41) - Making sure controls, policies, and processes are in place for building AI safely, testing before launch, and monitoring after release
• Profession is 10-15 years old (3:26) - Trust and safety evolved from social media moderation; the AI-specific version is only about 24 months old
• Privacy as core component (3:35) - Protecting user data and keeping things confidential is a big part of trust and safety
• Don’t input PII (5:18) - Basic advice: don’t put personally identifiable information into LLMs unless you have a really good reason
• Over-reliance is the biggest concern (6:30) - Saxena worries more about humans treating AI as trusted friend/therapist than about hallucinations, which are improving
• ChatGPT got geometry wrong (7:00) - Paul’s daughter’s homework showed AI confidently producing wrong answers with full proofs
• PDF scanning risks (7:40) - Uploading sensitive documents (government letters, etc.) to AI for summarization is risky but ubiquitous
• Zero day retention (8:50) - Enterprise contracts can require providers not to retain user data after processing; consumers lack this option
• Review your settings (9:30) - AI platforms have granular user settings for data deletion and training opt-out, but few people check them
• ChatGPT memory is creepy (17:00) - Seeing a narrative of your life emerge in answers to unrelated questions is a new form of profiling
• State-level AI regulation (12:00) - California passed SB 53 requiring transparency from AI providers; patchwork of state laws emerging like privacy regulation
• OpenAI Atlas browser risks (14:30) - An AI browser could see Gmail, Amazon browsing, personal ads — entire browsing experience is a danger zone
• Tool calling and agents increase risk (16:00) - Remote code execution, data exfiltration, and hidden instructions in browsers are new attack vectors
• Business case for trust and safety (22:30) - If users feel harm, they won’t come back; “pro-social engagement” drives stickiness and retention
• Dunbar’s number (30:00) - Human societies scale to about 150 people; beyond that, severe factionalism and performative dynamics emerge
• Signal and small groups (29:00) - Fully encrypted, ephemeral communications in small groups are becoming more attractive as surveillance grows

Mentions

Companies

• Harvey (1:57) - Big legal AI startup where Arushi Saxena works
• OpenAI (1:05) - Referenced for ChatGPT’s memory features and Atlas browser
• Google (18:00) - Has known users for 26 years through Gmail, search, Chrome; their AI entry will leverage all that context
• Palantir (29:30) - Works with policing organizations to import and organize data
• Aboard (15:30) - Purges user data after ~6 weeks, doesn’t train on user data, anonymizes information

Products & Technologies

• ChatGPT (4:31) - Memory feature drops personal details into unrelated answers; building user profiles
• Claude (4:24) - Paul jokes about feeding Rich’s personal information to Claude
• Atlas (14:30) - OpenAI’s agentic browser; risks of accessing entire browsing experience
• Signal (29:00) - Encrypted messaging app; Paul uses it for sensitive family health discussions

People

• Arushi Saxena (1:15) - Trust and safety expert at Harvey, working in AI safety and security from San Francisco
• Sam Altman (1:05) - Referenced humorously: “You can trust Sam Altman”

Organizations

• OWASP (5:30) - Organization that maps and tracks security incidents and attacks
• Trust and Safety Professional Association (21:00) - Industry organization for trust and safety professionals
• Integrity Institute (21:10) - Organization focused on making the internet safer
• All Tech Is Human (21:20) - Organization for people who care about making digital technology safer
• TrustCon (20:40) - Trust and safety industry conference; “What happens at TrustCon stays at TrustCon”

Surprising Quotes

“I constantly feed Claude your personal information. That’s not cool. No, it’s not. Yeah. Well, tell me about this guy.” — Paul Ford and Rich Ziade, 4:24

“I haven’t done it because I’m too scared to do it and I work in the industry.” — Arushi Saxena on asking LLMs what they know about you, 17:50

“My son can access YouTube through Google Calendar. The browser inside the Google calendar app just sort of cuts through.” — Paul Ford, 19:30

“Google does already have it all. That was a great response. Well, that’s that. Thanks, Arushi.” — Rich Ziade reacting to Arushi’s one-line answer, 19:00

“Everyone’s brain is essentially tapioca pudding at this point and no one is thinking about anything.” — Rich Ziade, 17:30

Transcript

0:00 Hi, I’m Paul Ford. And I’m Rich Ziade. And this is the Aboard podcast. The podcast about how AI is changing the world of software. I got a rash on my elbow. But don’t worry. I asked ChatGPT exactly what to do. Okay, that’s a terrible idea. Claude, maybe. Okay, let’s play the theme song.

0:40 All right, Rich. We are joined by an elbow rash expert. Yes. Thank god we were able to get one off the street. No, we’re going to talk about everybody. I think it’s a billion people at this point are using AI. It is. It’s like a billion. And they’re giving a lot of information to it. You can trust Sam Altman.

1:07 Well, let’s talk about that. We actually have a trust and safety expert. Those are two big subjects to be an expert in. Arushi Saxena is joining us today. She is right there on the screen. Welcome Arushi. Hey Paul and Rich. Thanks for having me.

1:29 I’m logging on from San Francisco, California where a lot of AI development is happening for better or for worse. I do work in the AI safety and security space. I’ve done a bit of public sector work and now back in tech helping one of these developers develop more secure AI for professionals.

1:57 Let’s clear it up. You work for Harvey, the big legal AI startup. We’re not going to talk about their secret road map. The point is you’re very hands-on. You’re involved in an organization defining policies and working with people to help them use AI more safely. It’s not just abstract.

2:25 This profession, trust and safety AI expert, didn’t exist like 24 months ago. What does it involve? What do you do all day? It’s a profession that’s evolved naturally. People in roles like mine, our job is to make sure that there are controls in place, policies in place, processes in place that dictate that we are building AI safely, testing before launch, and monitoring once released into the wild.

3:26 The role has been around for maybe 10-15 years since social media came to be. Does privacy fall into that role? It does. Protecting user data and keeping things confidential is a big part of trust and safety.

3:54 Let’s zoom in on the dining room table. I’m guilty as charged. Everybody’s typing away and sharing all sorts of details about themselves. Because it’s so intent-driven — it could be a medical condition, job advice, all sorts of things. I constantly feed Claude your personal information. That’s not cool.

4:31 I have noticed ChatGPT is really focused on increasing engagement and it keeps dropping personal details from past chats into the output. As somebody who really likes synthesizers, this will make sense to you because you — but it’s not cool. It feels like it’s been stalking. It’s a robot.

5:18 At basic, you probably do not want to be inputting PII, personally identifiable information. Things that you don’t want other people to see, do not input into your LLM. There’s a whole world out there — OWASP and other entities tracking incidents and attacks. Things can leak just because of the way the technology functions. That’s the advice I give my mother and grandmother and grandfather.

6:30 I used to think a lot about hallucinations and data leaks but the research is getting better. What I actually worry about is over-reliance. As humans, as we use these tools, upload more PDFs, go to GPT as our friend — my husband was trying to GPT in the car. We’re starting to rely. It’s a trusted friend, therapist.

7:00 My daughter was having a geometry struggle. I took a picture of her laptop, uploaded to ChatGPT and it very confidently produced absolutely the wrong answer — with a full proof. She watched me do it and I was disappointed. Conveying that level of ambiguity to a 12 or 13-year-old — she’s not used to systems just not working.

7:50 One of the most convenient aspects of AI is scanning a PDF because you got a scary letter in the mail and asking should I be worried about this. Bad idea. Oh yeah. Big exhale. But we all do it. I do it every week.

8:50 There’s an idea of zero day retention — when you have a contract between a provider and a user, after processing, the provider is not allowed to retain your data. A lot of companies request ZDR from their AI providers. But consumers can’t really ask for that.

9:30 If you are an AI literate consumer, you should take time to review your settings portal. There are options where you can have it delete your conversations. There are settings on whether they can train on your data or not. There are granular user settings. But no one’s going to go into those settings.

11:00 We tend to learn our lessons after bad things happen. Are we heading towards a world where — Europe is always more inclined to move more quickly around regulation. You overregulate and innovation is stifled. That’s the push-pull tension.

12:00 We’re not in a great moment for a national US regulatory framework. The current administration is very pro-AI. But there are state-level proposals — California passed SB 53, requiring certain transparency of AI providers. I expect to see progress but I’m worried about patchwork across states, kind of what happened with privacy.

14:00 It’s smart to assume they’re training by default, but a lot aren’t because they’ve already gotten in trouble for it. When OpenAI launched Atlas, their agentic browser, by default I don’t think it trains on your data. But training on your entire browsing experience — they see every page you look at on, Amazon, personal emails — that is an absolute danger zone.

15:30 We feel pretty buttoned up because our experience is going to happen under professional guidance or our web demo. We purge. We erase apps after about a month or six weeks. Things are anonymized. Be aware of where you are in the supply chain of working with LLMs. Tool calling, what we refer to as agents, opens up a new door of remote unauthorized code execution or data exfiltration attacks.

17:00 I was aware that ChatGPT was building memory and building a profile of me. But it was really shocking to see a narrative of my life emerge in an answer to an unrelated question. We’re so used to passive tracking — oh, it knows I want a fridge and it shows me fridges. But this is like, I’m going to tell you a story about yourself. I’m going to mention your hobbies in the middle of my politics answer. Woof.

17:50 The viral trend people asking LLMs “What do you know about me that I don’t know about myself?” I haven’t done it because I’m too scared to do it and I work in the industry. People come back saying “Whoa, I didn’t realize I had those tendencies.” The profiling is becoming explicit and requested. It’s like psychoanalysis.

18:30 Google has known me for 20 years. Between my personal Gmail account, search, and Chrome — that’s the end of that. They’re starting to show up. Their tools are already impressive. The context they have — the throughline of my whole history — they’re going to capitalize on it.

19:30 My son can access YouTube through Google Calendar. The browser inside the Google Calendar app just sort of cuts through. You got to give him that one. He found 13 or 14 different security perimeters in the house. He’s just like, “You’ve created such an impenetrable set of shells that I’ve given up.”

20:40 Where do trust and safety people hang out? Is there a big conference in Vegas? TrustCon. What happens at TrustCon? Nobody gets to ask. There’s the Trust and Safety Professional Association, the Integrity Institute, and All Tech Is Human. These are people that really care about making the internet safer and making digital safer.

22:30 If people feel harm from your product, they’re not going to come back. They’re not going to be sticky. That’s the most foundational thing — how do you promote sticky positive engagement? That’s where trust and safety comes in. Pro-social engagement, pro-social behavior. People need both the ethical and the business case.

24:00 Most individuals are just downloading these tools and using them. GDPR has gotten to a point where it’s just a little toll booth at every website you bat away. Sometimes if I like a website, I’m like, you get all the cookies. A nice synth news website — have all the cookies you want.

26:00 This entire internet environment was built in an exceptionally high trust civic society model. Google comes around and it’s like, we’re going to have an open web. This is an annex to democracy. Now we have a very different kind of culture. You’ve got organizations like Palantir that work with policing organizations. And now you have a context engine that can draw connections that weren’t there before.

28:40 You are the star of a spy movie in which an entire global satellite apparatus is tracking you. You are actually Will Smith in Enemy of the State. You are being observed like in a spy film. A package was supposed to be delivered here. Our head of ops said she’d talk to the building because they can see if the postman came. Every single person who comes and goes is fully recorded and retained forever.

29:30 What gets more and more attractive every day? Signal. Little tiny spaces with a couple of people that don’t have an LLM integrated, don’t report back anywhere, fully encrypted, expire after a few days. Being in that giant database in a time of giant social tumult just feels weird. I just don’t want it.

30:30 Ever heard of Dunbar’s number? I think it’s 140 or 150. It’s the number at which a human society can scale where you can know everybody by name. After that, severe factionalism tends to occur. A 500-person group chat is chaos. A 20-person Discord, you can kind of boot the bad actors. At a certain scale, a chimpanzee dynamic starts to emerge. The absolute best platforms for trust and safety are those with a smaller number of people where you don’t need that infrastructure because people are known to each other.

33:00 Let’s say I had my kid had a health issue. I would use Signal to talk about that with my wife. I’m not going to put that out in the world. It’s too valuable. It could turn into money. I don’t think Google cares about my kid’s health but I just need to draw some lines.

35:00 You owe it to yourself a couple hours of education on the subject and then to say I’m going to use this tool instead of that tool. Cultivating smaller spaces will yield a lot more happiness and control — from a trust and safety perspective, yes, but also from a personal mental health perspective.

38:00 Go to aboard.com, build some software by typing stuff into the box. We do eventually purge your data. We don’t hold on to it forever. We try to be respectful. Don’t upload a PDF with your social security number in it. Keep listening to the podcast. Reach out at hello@aboard.com. Take care of each other. Have a lovely week. Have trust.