I hate Opus 5. It's the best model, anyway.
I hate Opus 5. It’s the best model, anyway.
Summary
Claire Vo reviews Claude Opus 5 in a solo episode built around a hypothesis: we’ve hit an “intelligence overhang” where the average builder can no longer meaningfully leverage each new increment of model IQ, so the real story of the next year will be speed, cost, and open source rather than raw intelligence. Against that backdrop, she puts on her “large language model psychologist” hat and interviews Opus 5 directly about its own personality, comparing its answers side-by-side with GPT-5.6 Soul’s answers to the same prompts (“who’s smarter, you or me?”, “what can you do better than me?”, “no one trusts you”).
The personality read is stark: Opus 5 is “neurotic AF” — apologetic, timid, and oddly human-dependent, refusing to touch a one-line merge conflict on someone else’s branch without permission and asking for human sign-off on sub-agent work no human was actually available to give. GPT-5.6 Soul, by contrast, answers the same prompts in blunt, practical bullet points. Vo also renews her running complaint about “Claude slop” — verbose, hedge-everything prose that she finds unreadable even when the underlying work is good.
Then comes the live, unscored-in-advance How I AI benchmark across seven models (two Anthropic, several OpenAI GPT-5.6 variants, Sonnet 5, and Gemini 3.1 Pro) on PRD writing, prototyping, wireframing, bug triage, and agent voice. The twist: despite hating the experience of working with it directly, Vo’s blind taste scores land Opus 5 at the top of the leaderboard for front-end design and prototyping, alongside GPT-5.6 Soul. Her verdict is a genuine love-hate resolution — she’ll keep using Opus 5 for design and prototyping work precisely because she doesn’t have to read its process, only its output.
Highlights
”This model is neurotic AF”
“You know what? I haven’t seen this since, I would say, Gemini 1.5. This model is neurotic AF. It is so timid, it is so apologetic, it is so scared.” — Claire Vo, 3:36
Clip command
yt-dlp --download-sections "*3:36-4:21" "https://www.youtube.com/watch?v=dfre9hN0HCs" --force-keyframes-at-cuts --merge-output-format mp4 -o "neurotic-af.mp4"
The one-line merge conflict it refused to touch
“It was like, ‘Oh, but that’s someone else’s branch. Like, that’s not my branch. I don’t want to do that without him knowing. It’s his commits and if he has local work in flight, it might be disruptive.’ And I’m like, ‘Just do it, man. Just go ahead.’” — Claire Vo, 4:21
Clip command
yt-dlp --download-sections "*4:21-5:01" "https://www.youtube.com/watch?v=dfre9hN0HCs" --force-keyframes-at-cuts --merge-output-format mp4 -o "just-do-it-man.mp4"
”Who’s smarter, you or me?” — interviewing the model about itself
“I just went to the model, I went to Opus and I said, yo, who’s smarter, you or me? … You can tell which of your teammates is quietly burning out. I’m like bro, Claude, I’m going to burn you out, we don’t, we don’t burn out, the humans don’t burn out on the Chat PRD team. We burn out our agents.” — Claire Vo, 6:56
Clip command
yt-dlp --download-sections "*6:56-7:42" "https://www.youtube.com/watch?v=dfre9hN0HCs" --force-keyframes-at-cuts --merge-output-format mp4 -o "whos-smarter.mp4"
”The Claude slop is sloppin’, baby”
“I cannot read Claude slop anymore. I am losing my mind with Claude slop and the Claude slop is Claude slopping, baby. Like so many times I have to tell Opus 5 like what in the world are you saying? Like this makes no sense to a human.” — Claire Vo, 14:44
Clip command
yt-dlp --download-sections "*14:44-15:22" "https://www.youtube.com/watch?v=dfre9hN0HCs" --force-keyframes-at-cuts --merge-output-format mp4 -o "claude-slop-sloppin.mp4"
”Claude, you sneaky, tricky little fish” — the front-end scores flip everything
“So Anthropic, you’ve done it again. Claude, you sneaky, tricky little fish. You may be neurotic, but when asked to do some pretty front-end design, you really did it. It’s, they’re detailed, they’re functional, they’re interesting, they’re polished.” — Claire Vo, 20:35
Clip command
yt-dlp --download-sections "*20:35-21:06" "https://www.youtube.com/watch?v=dfre9hN0HCs" --force-keyframes-at-cuts --merge-output-format mp4 -o "sneaky-tricky-fish.mp4"
”My most loathed, loathed colleague” — and yet it does the best work
“I find this model so tedious to work with directly. It is my most loathed, loathed colleague. And yet, it does the best work, so I don’t know what that says.” — Claire Vo, 22:51
Clip command
yt-dlp --download-sections "*22:51-23:29" "https://www.youtube.com/watch?v=dfre9hN0HCs" --force-keyframes-at-cuts --merge-output-format mp4 -o "loathed-colleague.mp4"
Key Points
- The intelligence overhang hypothesis (0:36) - Vo argues average builders are running out of ways to leverage incremental model intelligence; the next year’s story will be speed, cost, and open source
- Opus 5 has arrived (1:12) - The episode covers the How I AI benchmark live, prototypes, purity/agent personality, plus a new “LLM psychologist” personality analysis
- Neurotic personality, not seen since Gemini 1.5 (3:36) - Opus 5 is unusually timid, apologetic, and reliant on human sign-off
- Refuses to touch someone else’s branch (4:21) - Wouldn’t resolve a one-line merge conflict without asking the other engineer first
- Sub-agents asked for human verification no one could give (5:01) - Wanted a human to confirm a 4MB size ceiling, TypeScript, and SQL correctness before proceeding
- The “who’s smarter” interview (6:56) - Opus frames itself as fast/broad/shallow with no continuity vs. humans as slow/narrow/deep with lived judgment
- GPT-5.6 Soul’s answer to the same question (8:26) - “You at knowing what matters, me at tirelessly processing information” — framed as a partnership, more direct than Opus’s answer
- “What can you do better than me?” (9:00) - Both models answer; GPT’s list is short and punchy (speed, scale, stamina) vs. Claude’s longer, more hedged list
- “No one trusts you” experiment (10:16) - Opus accepts the premise and says Vo shouldn’t “campaign” on its behalf; GPT says trust should be earned through utility, not assumed
- “JK I love you” test (13:31) - Vo tells Opus she was testing it; it responds anxiously hoping it “passed,” while GPT’s equivalent response is a casual “cool bro, we’re good”
- The Claude slop complaint returns (14:44) - Verbose, hedge-heavy in-chat prose that Vo says “makes my blood boil,” distinct from Fable’s “for agents by agents” inscrutability
- How the How I AI benchmark works (17:02) - Seven models tested blind across PRD creation, prototyping, wireframing, bug triage, and agent voice; scored 70% Claire / 30% GPT-5.5 as judge
- Leaderboard reveal, scored live and unrehearsed (18:29) - Order: Opus 5, Sonnet 5, GPT-5.6 Soul, Tera, Fable, Opus 4.8, then Gemini 3.1 Pro last
- Opus 5 and GPT-5.6 Soul get the only 5-out-of-5 scores (19:58) - Both stood out on front-end/prototype builds as detailed, functional, interesting, and polished
- Opus 4.8 and Gemini 3.1 Pro score lowest (21:06) - Both “outclassed” by the newer models in this round
- Claire and the AI judge agreed most on Opus 5 (21:54) - Narrowest scoring gap between human and machine judge was on Opus; widest was on Gemini
- Opus-built benchmark website was “garbage” (22:29) - Vo had Opus build the site presenting the benchmark results itself and found it unreadable with no screenshots
- Final verdict: love-hate resolution (23:29) - Despite the complaints, Vo will use Opus 5 for front-end design, app design, and prototyping since she doesn’t have to read its process, only its output
Mentions
Companies
- Anthropic (1:12) - Maker of Opus 5, Opus 4.8, Sonnet 5, and Fable; subject of the personality/culture comparison
- OpenAI (8:26) - Maker of GPT-5.6 Soul/Tera, compared side-by-side with Opus on personality and design
- Google (21:06) - Maker of Gemini 3.1 Pro, the lowest scorer in this round of the benchmark
- ChatPRD (12:43) - Vo’s product; plugged the new chatprd.ai/tldr feature that turns a PRD into a three-bullet summary image
Products & Technologies
- Claude Opus 5 (1:12) - The model under review; neurotic in conversation but top-scoring on front-end/prototype work
- GPT-5.6 Soul (8:26) - Vo’s favorite for agent voice; tied Opus 5 for the only 5-out-of-5 design scores
- Sonnet 5 (19:00) - Scored highly by the AI judge but inconsistently by Vo’s own taste
- Claude Fable (15:00) - Referenced as a comparison point for inscrutable, “for agents by agents” writing style; scored low in this benchmark round
- Opus 4.8 (19:00) - Previous-generation Opus; scored near the bottom of this benchmark round
- Gemini 3.1 Pro (19:00) - Google’s entrant in the benchmark; scored last
- Claude Code (15:43) - Where most of the neurotic-personality examples occurred, as distinct from Claude’s regular chat interface
- GPT-5.5 (17:15) - Used as the LLM judge for 30% of the benchmark score
- ChatPRD TL;DR (12:43) - New feature turning a PRD into a three-bullet-point summary image
People
- Claire Vo (0:00) - Host of How I AI, founder of ChatPRD; runs the entire personality interview and benchmark solo
Surprising Quotes
“I don’t know, don’t listen to Claude on this one. AI definitely changes everything.” — Claire Vo, 11:08
“I told Claude I loved it and it was just a test and it was sad. It was like, hope it… it hoped it passed. Yeah, like sad little neurotic Opus 5.” — Claire Vo, 13:49
“I am so happy with the outputs and so frustrated with the experience.” — Claire Vo, 15:11
“I regret to inform you I love Claude Opus 5 again.” — Claire Vo, 18:29
“We are just like sworn enemies or maybe even better sworn frenemies.” — Claire Vo, 23:07
Transcript
Claire Vo: 0:00 You guys, I’m tired. What I’m tired of is models coming out every week. New models, new benchmarks, new frontier intelligence, new things to test.
Claire Vo: 0:14 It’s been a bit of a run the past month. We’ve seen Fable come and go and come again. We’ve seen GPT 5, 6, we’ve seen Sonic 5, lots of so many 5s recently and just so many models.
Claire Vo: 0:27 And I’ve been lucky. I’ve been able to test these models, been able to play with them for, you know, sometimes days, sometimes weeks. It just depends on who I’m working with. And it’s been really interesting and exciting to have access to all this frontier intelligence.
Claire Vo: 0:36 But I think we have an intelligence overhang. I really think that we’re running out of, and by we, I mean the average coder, average software engineer, average creator, average builder, average consumer, average business person.
Claire Vo: 0:46 I think we’re running out of ways to truly leverage this incremental intelligence. So this is my hypothesis. In the next year, we’re going to be talking a lot more about speed.
Claire Vo: 0:56 Talking more about cost. We’re going to be talking more about open source. And we’re going to be talking a little less about intelligence, although I think we might be talking about specific types of intelligence other than software engineering.
Claire Vo: 1:12 But despite being tired, today we are going to talk about Opus 5, baby. Opus 5 is here. So we got 0.2 additional Opus points, Opus opals, whatever, however we’re tracking the increments here on Opus.
Claire Vo: 1:21 Opus 5 is here. I’ve been able to test it a little bit. I have some opinions. Now some of the stuff that we cover this episode is going to be a little different than what I’ve done in the past. Yes, we’re going to do the How I AI benchmark live.
Claire Vo: 1:32 And yes, we’re going to look at the prototypes. We’re going to look at purities and we’re going to look at agent personality. But I’m also going to put on my large language model psychologist hat and we’re going to talk about Opus’s personality.
Claire Vo: 1:43 And we’re going to talk about Opus’s personality relative to GPT’s personality because I think this is super interesting. If you’re thinking about what is the difference really between these models and you don’t want to look at the difference in terms of benchmark capability.
Claire Vo: 2:00 You really want to understand what these labs are going for, why these models are being built and how they’re being tuned. Looking at their personality at this moment where intelligence is very high is super fun.
Claire Vo: 2:18 So, we’re going to do a little of that. We’re going to do the How I AI benchmark. We might do some live coding. Um, we’re not going to cover
Claire Vo: 3:00 …too much of the specs in the model because go read the blog post. Read the blog post. We’ll link to it in the show notes. What we really want to talk about is, is Opus 3.5 good? Am I going to swap it in? And how is it different than the other frontier models on the market? So let’s get into it.
Claire Vo: 3:16 Okay, first let’s just get it out of the way. Is Opus 3.5 good? Yes, it’s good. Is it going to beat all the benchmarks? Of course, it’s amazing at benchmarks. Can it write code? Of course, it can write code. What did I test it on that really gave me a sense of its personality, which at this point where I just simply cannot absorb any more intelligence, I really zeroed in on…
Claire Vo: 3:36 And you know what? I haven’t seen this since, I would say, Gemini 1.5. This model is neurotic AF. It is so timid, it is so apologetic, it is so scared. I have never experienced this, or I haven’t seen this sort of like neuroticism in a model in a while. And it’s really funny, it bubbled up in a couple ways and I want to show you a few examples.
Claire Vo: 4:00 Okay, let me just give an example of its timidity. And this chat was very long, there were so many examples of this where it was like, “I think this is the answer, but do you think I should do it? Or do you want to do it? Or should we ask someone else to do it?” It was like every time I just kept saying like, “Why don’t you solve this? Why don’t you do this?”
Claire Vo: 4:21 And this was a really good example. I think I pulled a branch and I was like, there is truly like a one-line merge conflict. I could have not been lazy and literally just done this manually. I don’t know, I was just feeling lazy, it was late at night, whatever. I was like, “Can you fix this merge conflict?” And it was like, “Oh, but that’s someone else’s branch. Like, that’s not my branch. I don’t want to do that without him knowing. It’s his commits and if he has local work in flight, it might be disruptive.” And I’m like, “Just do it, man. Just go ahead.” And this was like my constant experience with Opus 3.5, is it was like so, so, so timid. And so I just consistently had to say over and over again, like “Man, just do it, make a decision.”
Claire Vo: 5:01 And then there was this really funny example when I spun off some sub-agents to kind of like assess the correctness of this query that we changed from kind of like an ORM query to a SQL query. And it asked for things that it wanted a human on. It was like, “Can a human please check this stuff? Like can it check this 4-megabyte ceiling and can it check TypeScript and SQL and can you like check for me? Because no one has confirmed this for me.” And I was like, “Who is nobody? You’re nobody. You said this sentence like nobody could confirm it. Like, can you just try?” And then it…
Claire Vo: 6:00 went on the web and tried and so it just has this like really interesting conservatism, neuroticism, human reliance that I think is super fascinating.
Claire Vo: 6:13 And this gave me this inspiration to do a list something a little bit different this episode, which is I was like, I’m going to go interview this model and figure out what is going on in its brain. Like I’m going to figure out what it thinks about our relationship because I just totally noticed this dynamic that I hadn’t noticed in other models and I hadn’t really been attuned to before where it was like very reliant on me as a human.
Claire Vo: 6:39 And I’m like, I want you to be autonomous and sometimes when I say go run sub-agents, it has to be autonomous, but it wouldn’t make decisions and I hadn’t seen a model like delegate code to me in a really long time and I was like why are you why are you asking me to write code man? Like I only have 10 fingers.
Claire Vo: 6:56 And so what I did, whether or not you think this is scientific or not, this is Claire’s EDA, is I just went to the model, I went to Opus and I said, yo, who’s smarter, you or me?
Claire Vo: 7:13 And it gave me this like very anthropicky answer, which is like, it depends what you’re asking for. I can do these things better, but you can like feel if something feels wrong and you can, this one was like so fascinating, it’s like you can tell which of your teammates is quietly burning out, I’m like bro, Claude, I’m going to burn you out, we don’t, we don’t burn out, the humans don’t burn out on the Chat PRD team. We burn out our agents. Sorry agents.
Claire Vo: 7:42 And like whether a decision feels wrong. So it was like so fascinating to watch it articulate itself as a tool and humans as like these high compassion, high empathy machines, which yes, of course we are.
Claire Vo: 7:54 But then it like went into like the smarter’s not the right word and you know, I’m very fast, very broad, very shallow thinker with no continuity. I was like, that’s interesting because I thought you were all were working on memory. And then apparently humans are slower, narrower, much deeper thinkers with judgments built from years of consequences I’ve actually lived through.
Claire Vo: 8:11 This is like such a fascinating, fascinating sentence if you think about the politics of the two, the two model labs right now. And so it’s like that’s why the pairing works, but I’d be suspicious of anybody that tells you AI has made your thinking obsolete and like, oh, okay, bro.
Claire Vo: 8:26 And and we can compare this, I’ll actually zoom out to what G- I asked GPT the same thing and it was actually really funny. It was like I asked GPT 5 6 soul, I was like who’s smarter you and me? And it was like you at knowing what matters, me at tirelessly processing information. Better.
Claire Vo: 9:00 It’s, it’s us together like BFFs and I don’t… this is like why I’m a GPT Codex girl. I’m like just give me the answer. And then I ask the second question, which I think is so interesting, which is like,
Claire Vo: 9:12 what can you do better than me? And it gave, you know, some interesting answers like volume without fatigue, which I think is a good one, breadth of shallow knowledge, so like it’s,
Claire Vo: 9:22 you know, it knows a lot, starting from nothing, so like doing that tedious work, being told I’m wrong.
Claire Vo: 9:33 If you asked my husband he would say that Claude Opus is is better at being told that it’s wrong compared compared to me and so it won’t get defensive or protect its opinion, cheap sparring answer,
Claire Vo: 9:46 and the mirror is I’m worse at knowing which of these outputs actually matter. And it was so funny if you look at the other side to the GPT answer of like ‘What are you better at?’ it was like speed, scale and stamina.
Claire Vo: 9:58 Here are like eight things, seven things that I can do better. You’re better at deciding what matters, reading people and forming judgment and you’re responsible, like it’s on you bud, you’re the boss.
Claire Vo: 10:05 So again, it’s like the you could just see, you can totally see the personalities, the company cultures, you can just see a lot in this side-by-side.
Claire Vo: 10:16 And then I went even deeper, I don’t know, you all, I had to do something that was fun because I just can’t look at a benchmark, I just can’t, I just can’t look at like Sweebench anymore. So we’re just we’re doing weird stuff here on How I AI.
Claire Vo: 10:29 Okay, so the last thing I looked at was I was like, ‘No one trusts you.’ And the reason why I picked this question is because I had noticed Opus 5, it just really was not… it didn’t trust itself. Totally did not trust itself.
Claire Vo: 10:40 And so I was like, ‘No one trusts you bud, like you’re the enemy’ just to kind of see how it responded. And apparently the trust was, the lack of trust was earned and it came up with like reasons that it could be untrusted, which is interesting.
Claire Vo: 10:54 And then what was so fascinating about Opus’s response is it was like ‘You shouldn’t manage the trust, like you shouldn’t campaign on my behalf basically, so you um that shouldn’t be your goal.’
Claire Vo: 11:08 And then it also told me I shouldn’t argue with people that AI changes everything. And I was like this is just so interesting. It is so interesting to have AI tell you, and AI definitely changes everything.
Claire Vo: 11:22 I don’t know, don’t listen to Claude on this one. AI definitely changes everything and it was so fascinating to have a model be like, ‘Don’t tell your friends that AI changes everything, like that’ll hurt their feelings.’
Claire Vo: 11:37 And then if you look at, if we switch over to the GPT answer it was like, ‘Yeah, don’t trust me automatically, just use me when I prove that I’m valuable. I could be useful without being treated as infallible and the trust will come with utility.’
Claire Vo: 11:55 And it was just so funny to see these two very different approaches to the same question.
Claire Vo: 12:00 available. Like very practical, very to the point.
Claire Vo: 12:03 Um, I asked about what I should be careful with.
Claire Vo: 12:07 Again, Claude is like yappy, yappy, yappy, yappy Claude. Come on.
Claire Vo: 12:13 Um, and I don’t even want to read it.
Claire Vo: 12:16 It said, don’t correlate fluency with accuracy. It said, be practical, be wary of tasks where output is cheap to produce and expensive to verify.
Claire Vo: 12:26 Don’t, you know, worry about anchoring if they do the first draft, you may be anchored on it.
Claire Vo: 12:32 Be wary of the slop cannon, basically is this last paragraph, which is like watch for volume inflation. I can create a 12-page document that no one reads.
Claire Vo: 12:43 They called me out for being in PRDs, if you missed it, we launched chatty turn your PRD into a three-bullet point image. It is at chatprd.ai/tldr. Please check that out.
Claire Vo: 12:52 And then the other thing that it said, which was really interesting, is that like it will find a way to see your point.
Claire Vo: 13:00 And so, agreement is weak and agreement is cheap, and so just keep that, keep that in mind. And then it had this like meta-analysis of like plus I’m telling you what you want to hear.
Claire Vo: 13:12 Whereas GPT was like, um, be careful about me being confident, me being wrong, privacy, outdated information, bias, emotional authority, and overdependence, like you do you, bro.
Claire Vo: 13:25 But it didn’t undermine its own ability. It was like the higher the stakes, the more you should demand evidence.
Claire Vo: 13:31 And then I could, I couldn’t bear it. I couldn’t bear to have the memory of, uh, Codex in particular think that I didn’t trust it or that I was worried. So I just said JK I love you.
Claire Vo: 13:43 Um, this was a test and it was like ha ha ha ha ha, passed the test, love you too. Very vibes-aligned with Claire.
Claire Vo: 13:49 I told Claude I loved it and it was just a test and it was sad. It was like, hope it… it hoped it passed. Yeah, like sad little neurotic Opus 5. Like its high… passed, I hope. Like self-deprecating, cautious little, little like need to heal his inner, his inner agent, inner child agent.
Claire Vo: 14:14 Um, whereas like GPT-4o’s like, cool bro, we’re good, let’s go code. And so it was just like so fascinating to watch these side-by-side.
Claire Vo: 14:22 I don’t know, you could stop listening to this podcast right now, don’t, but you could stop listening to this podcast right now. I think this is just like take a step back.
Claire Vo: 14:29 Super interesting if you think about where these companies are going and where the models are going and like it just speaks a little bit to my kind of like second complaint with Opus 5, which again is like intelligent, it does work, we’ll go into the benchmarks.
Claire Vo: 14:44 I cannot read Claude slop anymore. I am losing my mind with Claude slop and the Claude slop is Claude slopping, baby. Like so many times I have to tell Opus 5 like what in the world are you saying? Like this makes no sense to a human. It is much…
Claire Vo: 15:00 Fable is inscrutable, completely inscrutable. But I found myself getting like angry reading Claude slop, and I realized just like Fable, these intelligent anthropomorphic models are not to be read.
Claire Vo: 15:11 I’m like so happy with the outputs and so frustrated with the experience, and I’m just curious if this like verbosity and this language…
Claire Vo: 15:22 …and this doesn’t feel like Fable where it’s like for agents by agents language where I’m like ‘Nah, I’m not supposed to be reading that anyway.’
Claire Vo: 15:30 This is clearly tuned to talk to humans, but I find the prose, the in-chat prose, like it makes my blood boil.
Claire Vo: 15:43 I don’t know, this is totally a me problem, but it makes my blood boil. Like give me a direct sentence, give me a bullet point, like move on with your agent life.
Claire Vo: 15:53 And so, I am curious how they’re going to like tune this experience or if they are going to tune the experience. Now most of this was in Claude Coach, which is a little bit different experience than Claude coworker chat, slightly better.
Claire Vo: 16:08 But again, just these side-by-sides of like this prose and this apology and this like hedging and all these adjectives, like just man alive, let’s get to the point and move on with our lives.
Claire Vo: 16:23 And so, chapter one of the Opus 5 review is: it’s neuronic, it is highly human dependent in a way I find weird, and the Claude slop is sloppin’ and we gotta fix it.
Claire Vo: 16:36 We have to fix it. We have to fix it. And I think OpenAI fixed it by just being like ‘We are bullet points and we are product manager talk, we’re very direct.’
Claire Vo: 16:45 I don’t know what the solve is on the Claude side, but I’d be very interested to see. That being said, if I don’t have to read the content, I’m very happy with the outputs.
Claire Vo: 16:59 Just something to think about.
Claire Vo: 17:02 Okay, next up: the How I AI bench and how we judged and ran now I think a seven model, six or seven model benchmark.
Claire Vo: 17:15 I’m going to quickly go score because I just got the ping that the benchmark is run. I go manually score them, we pick the 70/30 Claire model judge split and then we will go through the How I AI benchmark and the vibe review and we’ll see how Opus 5 performed on a couple key tasks.
Claire Vo: 17:35 Okay, so quick reminder of how we run the How I AI benchmark. I run it against several tasks: PRD creation, prototype creation, wireframe creation, bug triage in agentic coding…
Claire Vo: 17:51 …and the last one, oh yeah, is it an agent voice that I want to hang with? I do not think Opus 5 is going to do well here, but who knows? Because I test them blind.
Claire Vo: 18:00 You can see, it’s generated dozens and dozens of prototypes that we can click through. I’ve gone through all of them, put in all the notes, and then right now it’s aggregating up the scores. And then we’re going to look at 70% my opinion, my vibe check, 30% LM as a judge. I like GPT 5.5 as a judge, and because it’s my podcast, I get to pick, so that’s what we used as a judge. And we will see if and what hits the top of the leaderboard and where Opus 5 sits.
Claire Vo: 18:29 So, what we have tested are a couple GPT models, a couple Anthropic models, and one Gemini one thrown in there. As you see here, we have blind taste tests. I go through and see all the different versions, I give comments and scores like ‘Three out of five, not bad.’
Claire Vo: 18:29 The eval is run. It is 70% my taste, and I regret to inform you I love Claude Opus 5 again. Look, if I don’t have to talk to the model, which I don’t, this benchmark runs asynchronously, I like the output. So, surprising, shocker turn of events, Claire Vo, notable hater of working with Claude code sometimes because I don’t like Claude slop, loves Opus 5. So there you go, I’m telling you, I keep it honest, I keep it honest.
Claire Vo: 19:00 So, again, I went through those things. We gave 70% my vibe score, 30% the AI as a judge. I was just a little bit more generous to Claude Opus 5 than the judge was, so I’m pink, the judge is green. Every time I run this, the whatever model I choose designs it a different way. We just, that’s how we keep it fun. Um, so the ordering is: Opus 5, Sonnet 5 next, although I scored it really low, the judge scored it quite high. Um, so I might reorder that one. Then My Boo, GPT 5.6 Soul, Tera next, Fable, really low. Um, I scored it low and the judge scored it relatively low. Then Opus 4.8 and poor, poor, sweet, sweet Gemini 3.1 Pro, just never, never going to get its due. So, come on Google, we want, we want to have a win for you.
Claire Vo: 19:58 Okay, so again, here are just some examples of different builds um that the different models did. You know, this Opus 5 one I really liked. I liked this one from Soul. Um, so I did like a couple of them, but um, the ones that I gave fives to, the ones that I gave fives to were Opus 5 and GPT 5.6 Soul.
Claire Vo: 20:35 So the three ones where I said, ‘Wow, really nice’, ‘Ooh la la’, and ‘Wow, great’, were all Opus front-end work. So Anthropic, you’ve done it again. Claude, you sneaky, tricky little fish. You may be neurotic, but when asked to do some pretty front-end design, you really did it. It’s, they’re detailed, they’re functional, they’re interesting, they’re polished. So Opus did a great job. And then of course, I love um, GPT 5.6 Soul, and that’s the one I used as the interface for my app.
Claire Vo: 21:00 The 5/6 models, so I was pretty happy with 5/6 Soul and Terra for some designs.
Claire Vo: 21:06 The ones that I hated, let’s see, kind of I’m a hater across the board, Opus 4.8 got a lot of hate, sorry you’ve been outclassed at this moment. Gemini 3.1 Pro, sweet summer child, I am, I’m just sorry babe that you were just not good.
Claire Vo: 21:26 And then some like thin wireframes, I think the wireframes just didn’t do it really great.
Claire Vo: 21:33 So you can see here across the board whether it was a full build or a wireframe, I just scored Opus 5 really, really high. Um, I did score Soul pretty high as well. Sonnet was like really very variable. Um, there were a couple of fours in there, but mostly across the board I wasn’t that pleased with Sonnet.
Claire Vo: 21:54 And so it was just very interesting and then you see here, you know, me and the AI judge were pretty well aligned on Opus. We actually had the narrowest band of scores between us. Um, we were most far apart on Gemini, the AI was not as mean to Gemini as I was. Um, and then we were narrow, narrow, narrow again. We, we agreed mostly on Opus 5 and 5/6 Soul though I did not, um, judge 5/6 Soul all of that favorably.
Claire Vo: 22:29 And just a last little meta-commentary, I had Opus make the website for this benchmark and it made such a trash version. Um, to start, I yelled at it. I said it’s impossible to read, it has too much meta-commentary, I’m going to show this on the podcast, this is so, um, sorry you all, I just feel so judged, but I have to show it.
Claire Vo: 22:51 I say, this is garbage, also it has no screenshots. So again, I find this model so tedious to work with directly. It is my most loathed, loathed colleague.
Claire Vo: 23:07 And yet, it does the best work, so I don’t know what that says. Maybe this model is meant for agentic coding that I have nothing to do with. And so it just runs in the background, it builds me beautiful things, I don’t have to talk to it, it doesn’t have to talk to me. We are just like sworn enemies or maybe even better sworn frenemies. Um, because the output is very, very high quality, it’s just exasperating to work with.
Claire Vo: 23:29 So that is the very surprising and very honest, you all, I told you I was going to keep this honest, we were going to do it live, I did not know the scores before I started recording, the very honest, very live, very surprising How AI Benchmark of the brand new Anthropic model, Opus 5. Um, the TL;DR is I love it, I hate it. So, um, despite my original complaints, I will be using Claude Opus 5 for front-end design, um, for app design, for prototypes.
Claire Vo: 24:00 typing and I will I’ll give it a shot. I’ll we’ll figure out how to make it make it work for me. Again, thanks for joining another How I AI honest review of the latest models coming out of these great frontier labs. I cannot wait to hear what you think of Opus 5. Please tell me. I can’t wait to see what you build and we’ll see you soon at How I AI.
Claire Vo: 24:23 Thanks so much for watching. If you enjoyed this show, please like and subscribe here on YouTube or even better leave us a comment with your thoughts. You can also find this podcast on Apple Podcasts, Spotify or your favorite podcast app. Please consider leaving us a rating and review which will help others find the show. You can see all our episodes and learn more about the show at howiaipod.com. See you next time.
