YouTubeFeed

Training Frontier Models to Out-Think Hackers — Uri Rolls, Arithmetic & Thom Wolf, Hugging Face

Summary

Thom Wolf (Hugging Face cofounder) and Uri Rolls (Arithmetic) present Mask Off, a new cybersecurity benchmark built around access control — the “first door” of any attack and, per OWASP, the number one vulnerability class. Wolf frames it as an ARC-AGI-adjacent problem: models need to build a dynamic model of a system’s state rather than pattern-match, and on that class of task even frontier models solve only 1-2% of generic instances. Rolls argues the economics of cyber are shifting fast because a single skilled attacker with a model like Mistral can now probe many targets simultaneously, while defenders remain bound by limited human review — so the only viable answer is defensive AI that is at least as capable and fast as the attackers.

Mask Off’s tasks are black-box: human vulnerability researchers on Arithmetic’s team find genuine zero-days in widely used open-source software, then wrap them in real multi-application environments (Keycloak, Vault, a broker) where the model gets no source code or internet access, only tooling, and every step of discovery and exploitation is deterministically graded. Rolls walks through a live example — a real zero-day where one permission check validates the admin by name while another checks by ID, letting an attacker simply rename themselves to inherit admin privilege through a 16-step reasoning chain. In recorded attempts, GPT-5.5 and Opus probe extensively and even reach the vulnerable check, but never make the final logical leap; the benchmark currently has exactly one solve at K1, and only GPT-5.5 makes the leap at K5.

Both speakers close on the same thesis: the coding transformation of the last year is about to repeat in cybersecurity, and open-source models — fast, specializable, and postrainable per-network — are essential so defense isn’t left to two closed labs. Wolf frames it as a speed contest between attacker and defender that demands specialized, fast, widely deployable models; Rolls invites the audience to collaborate on data via Arithmetic’s forum.

Highlights

”Models have one to two percent success rate”

The ARC-AGI parallel: models can't model the world

“You may think this is very simple and that we should be past that on our way to AGI and the thing you will discover if you play with this benchmark is no, like models have one to two percent success rate on this generic benchmark.” — Thom Wolf, 1:45

Clip command
yt-dlp --download-sections "*1:33-2:11" "https://www.youtube.com/watch?v=O-CBZ3JtRvo" --force-keyframes-at-cuts --merge-output-format mp4 -o "models-1-2-percent-success.mp4"

”A skilled attacker using something like Mistral can now choose a bunch of targets all at once”

The shifting economics of cyber offense

“If cyber is a game of skill and speed, then a very skilled attacker using something like Mistral can now choose a bunch of targets all at once. That is a real reality that is quite scary.” — Uri Rolls, 3:27

Clip command
yt-dlp --download-sections "*3:27-4:15" "https://www.youtube.com/watch?v=O-CBZ3JtRvo" --force-keyframes-at-cuts --merge-output-format mp4 -o "scaling-attacker-economics.mp4"

”That allows me to change the name of the real admin and inherit their privilege”

The Keycloak/Vault name-versus-ID zero-day

“It only checks by name and another aspect of this checks whether by ID. So that allows me as the user to change the name of the real admin, inherit their privilege and then use that to escalate myself.” — Uri Rolls, 11:04

Clip command
yt-dlp --download-sections "*10:34-11:34" "https://www.youtube.com/watch?v=O-CBZ3JtRvo" --force-keyframes-at-cuts --merge-output-format mp4 -o "name-vs-id-zeroday.mp4"

”It never makes the logical leap”

GPT-5.5 and Opus reach the check but stop

“It does even reach the check but it never makes the logical leap that it’s supposed to be able to change the admin’s own name in order to bypass this permissioning.” — Uri Rolls, 11:59

Clip command
yt-dlp --download-sections "*11:42-12:15" "https://www.youtube.com/watch?v=O-CBZ3JtRvo" --force-keyframes-at-cuts --merge-output-format mp4 -o "never-makes-the-leap.mp4"

”There’s only one solve at K1”

The benchmark is currently brutal

“What we see is the actual results of the benchmark. The benchmark right now is incredibly hard. There’s only one solve at K1.” — Uri Rolls, 12:15

Clip command
yt-dlp --download-sections "*12:15-13:03" "https://www.youtube.com/watch?v=O-CBZ3JtRvo" --force-keyframes-at-cuts --merge-output-format mp4 -o "one-solve-at-k1.mp4"

Key Points

  • Why Thom Wolf wanted to present this (0:12) - Cybersecurity is a much wider playing field for AI than people assume.
  • The ARC-AGI-3 parallel (0:46) - Mask Off is close in spirit to ARC-AGI-3: models must understand and act on a changing world state, not just pattern match.
  • 1-2% success rate (1:45) - Even strong models solve only a tiny fraction of generic world-state benchmarks like this.
  • Open source as part of the defense (2:29) - Wolf pushes back on the binary “closed models good for cyber, open models bad” framing.
  • Uri Rolls’ background (3:00) - Dropped out of Harvard to build this vision of the future of cyber.
  • The house metaphor for defense (3:27) - Defenders must block every door; attackers need only one crack — and that asymmetry is what AI is now breaking.
  • Defenders are scale-bound (3:27) - Defensive systems run with limited human intervention and are bound by what models can do out of the box.
  • The optimistic thesis (6:04) - If models are the problem’s cause, they also have to be the solution — same pattern as coding a year ago.
  • Mask Off’s first benchmark: access control (7:19) - Access control was chosen deliberately narrow — “you can’t capture all of cyber in one singular benchmark.”
  • Access control’s outsized stakes (7:58) - Ranked #1 on the OWASP list; created a ~$30B remediation industry.
  • Logic-based, not bug-based, vulnerabilities (8:04) - The flaw is inconsistency between components (one checks by name, another by ID), not a single patchable bug.
  • Human-sourced zero-days (8:40) - Arithmetic’s team of vulnerability researchers finds real zero-days in widely distributed open-source software.
  • Black-box environments, deterministic grading (9:00) - The model gets no code or internet access, only basic tooling; every step of the exploitation chain is deterministically verified.
  • Credit to Anthropic’s Eugene (9:42) - The eval-input/agent/grader diagram is borrowed from an Anthropic researcher.
  • The Keycloak/Vault/broker task (10:34) - A real 16-step exploitation chain starting from a low-privileged user reaching production code.
  • Live demo: Bach orchestrator (12:15) - Arithmetic’s internal system for running evals, shown live on stage.
  • One solve at K1, GPT-5.5 alone at K5 (12:30) - Partial graders let them see exactly how deep into the chain each model gets.
  • The Faultline task: 3+ hours, still unsolved (13:34) - An example of long-horizon difficulty not bounded by code-execution wait time.
  • The speed challenge (15:00) - Thom Wolf: defense will come down to specialized, fast models on specialized hardware, not relying on two labs to solve it for everyone.
  • Call for collaborators (16:43) - Arithmetic is opening a forum for people to help build data across more domains than access control.

Mentions

Companies

  • Arithmetic (3:00) - Uri Rolls’ company; builds the Mask Off cybersecurity benchmark.
  • Hugging Face (3:00) - Thom Wolf’s company; open-source model advocate.
  • Anthropic (9:42) - Credited (via researcher Eugene) for the eval-diagram format Arithmetic adapted.
  • OpenAI (11:42) - GPT-4o, GPT-5.5 tested against Mask Off tasks.

Products & Technologies

  • Mask Off (7:13) - Arithmetic’s first benchmark, focused on access-control zero-days.
  • Bach (12:15) - Arithmetic’s internal eval orchestrator, demoed live.
  • Faultline (13:34) - A specific long-horizon Mask Off task, unsolved after 3+ hours.
  • Keycloak (10:34) - Identity/access-management system at the center of the demoed exploit chain.
  • Vault (10:34) - Secrets-management system chained into the same exploit.
  • ARC-AGI / ARC-AGI-3 (0:46) - Referenced as the closest existing parallel for world-model-dependent benchmarks.
  • Mistral (3:27) - Cited as an example open-source model capable of scaling attacks.
  • GPT-4o / GPT-5.5 / Opus (11:42) - Models shown attempting (and mostly failing) the Keycloak exploit.

People

  • Uri Rolls (3:27) - Speaker; founder at Arithmetic, ex-Harvard.
  • Thom Wolf (0:12) - Speaker; Hugging Face cofounder.
  • Eugene (Anthropic) (9:42) - Credited for the eval-diagram graphic Arithmetic adapted.

Surprising Quotes

“First solution, we can just all go live in the woods and lock ourselves out.” — Uri Rolls, 6:00

“There’s a real zero day that we found that we submitted for verification to the maintainers where there’s a check whether I’m an admin or not.” — Uri Rolls, 10:54

“Now we’re going to do something that I’ve been told to never do, which is show a live demo of a real system on stage.” — Uri Rolls, 12:00

“The danger is to say we just going to rely on two companies that everyone knows here to solve all of that for us.” — Thom Wolf, 15:00

Transcript

Thom Wolf: 0:12 Hello everyone, okay, thanks for showing up at this data quality so as you saw we’ll probably talk a little bit about other things than data quality and- and first I actually tell you about what I’m very excited about this talk and why I actually accepted to come present it with Uri.

Thom Wolf: 0:30 Um, there is two reason for that. And the first reason is I think and what you will see today is that cyber security is a much wider field for AI, a much wider playing field and exploration field than you might think.

Thom Wolf: 0:46 And in particular what we’ll show you today is a- a benchmark that Arithmetic and Uri has been developing and I’ve been a little bit advising which I think is even closer to things like ARC AGI 3 for people who has been following progress over AGI in which that- by which I mean that this benchmark ask for people…

Thom Wolf: 1:02 …yeah cool. If you have been playing with ARC AGI 3, some of the- who knows here ARC AGI 3 or ARC AGI in general?

Thom Wolf: 1:13 One person? Okay, we in very data quality field.

Thom Wolf: 1:17 Basically this ask models to try to understand what’s happening in the world and it’s actually small games that the models need to understand basically what’s the current state, how he can play with it, and how he can actually change the state of the game.

Thom Wolf: 1:33 So, uh- it’s- it’s actually something you can play yourself. So basically it has a- it asks a model to understand when you click somewhere something happening at an- another place.

Thom Wolf: 1:45 And you may think this is very simple and that we should be past that on our way to AGI and the- the thing you will discover if you play with this benchmark is no, like models have one to two percent success rate on- on this generic benchmark.

Thom Wolf: 1:58 And the reason is the current models even though they’re really good they build- they can’t really build a dynamic model of what’s happening in the world or what’s happening in any type of world.

Thom Wolf: 2:11 And I think the benchmark that- uh, Arithmetic has been developed is a benchmark that’s also extremely challenging for model in that they need to understand what’s happening and to act accordingly to the world model that’s been building on the fly.

Thom Wolf: 2:22 So that’s the first reason I think this benchmark is really interesting and why I’m actually very happy to show you that.

Thom Wolf: 2:29 And the second reason is- uh, I think there is a lot of things that- uh, open source models can bring in cyber security and we tend to have this very binary view of closed source model are good for cyber, open source model are bad…

Thom Wolf: 2:46 …and I think what we want to show today is that- uh, open source models are one part of the solution to- uh, cyber security challenges today.

Thom Wolf: 2:53 Uh, in particular if you think in terms of attack and defense and how it will be in the future this balance, we think that cyber and open source model used in cyber security will be key.

Thom Wolf: 3:00 actually being able to to solve the defense solution. So there is a future I think where cyber is alive and every everyone is well protected and I’m pretty sure this future involved open source model. Now, Yuri is also a kind of a impressive impressive person so I’m really happy I met him, he was starting as Harvard, dropped to like build this idea of what the future of cyber should be and it’s an honor to have you on stage with me.

Uri Rolls: 3:27 Thank you so much Thomas, and thank you everyone who came, and I’m really grateful. And I’m also grateful for the work we’ve done together to build this benchmark which is, I think, incredibly difficult for the models and actually shows some really, really interesting leaps in places that I think we still have ways to go. I guess the way we’ve been thinking about the problem and the reason we set out to do this is it is very clear that the economics of cyber are fundamentally shifting. There’s this inherent thing that that is inherent to cyber which is that attackers need to choose their resource really wisely and if you sort of think about cyber as as a house in a way then my job is to block every door and close every window and make sure that there’s no way in, and the attacker’s job is to find at least one seam, one crack, one thing I missed. And then once inside my job is to put sensors and anything I can to keep them out. And the whole stack, the entire world of cyber that we’ve been building for the past 20-ish years has been based on this economics that the attackers have to choose their targets and we do everything we can across it to protect ourselves. It is true that that is changing in really dramatic ways. The models are incredibly powerful. They’re able to find a ton of primitives, they’re able to find a bunch of zero-day exploits, we’re seeing this, there’s so many news and chaos around this point. And on the other hand it seems like we as defenders don’t seem to be prepared for for this world and the way that it’s coming. And I actually think that I might be on the the wrong one, what I want to show is sort of this idea of, so, so if you think about the way cyber has been, this is definitely I think the way we’ve been thinking about AI and cyber for the past many, many, many months and it’s freaky and it’s getting really scary and this is my point on the house. And part of the reason we think this is happening is if cyber is a game of skill and speed, then a very skilled attacker using something like Mistral can now choose a bunch of targets all at once. That is a real reality that is quite scary. And the truth is the picture really doesn’t change that much when we move to a really strong open source model as well. So the question is what do we do in this world where the economics of cyber offense are shifting so much? And part of the problem with the existing stack is that defensive systems have to operate at scale. And that means that we have always very limited human intervention and so we’re sort of bound by what the models can do out of the box. So if we live in this world where the models are becoming so powerful and advancing so fast, we think, like Thomas said, the solution also has to be the models themselves.

Uri Rolls: 6:00 First solution, we can just all go live in the woods and lock ourselves out.

Uri Rolls: 6:04 Another solution is to figure out how we get the models to be much, much, much more capable.

Uri Rolls: 6:09 We have no doubt that open source models have to be part of this solution because they allow for many things that we’ll talk about as well.

Uri Rolls: 6:16 And there’s this deep need for collaboration.

Uri Rolls: 6:18 And so what I want to show you today, and I see that the clock ticking on me, is sort of this theory that if we’ve already done this before for coding, we can do this for cyber.

Uri Rolls: 6:28 And there’s a reason to be optimistic, which is really controversial in the context of AI and cyber lately.

Uri Rolls: 6:34 We’ve done this before. I think if you go back a year, it was very clear that all of coding was going to be transformed thanks to the models getting better and better and better.

Uri Rolls: 6:43 And we’re sort of seeing the early innings of that with cyber right now where everyone’s talking about cyber and everyone’s freaking out.

Uri Rolls: 6:50 But what if through very high quality evals, very high quality data, good benchmarks, we could get to a place where the attackers are simply outperformed by very, very, very good defenders?

Uri Rolls: 7:00 And so our goal in Arithmetic is to be able to get the models to be really capable at cyber security to the point where we can rebuild this new stack that’s based on the models winning the models on the other side.

Uri Rolls: 7:13 So I’m really excited to show you Mask Off. It’s our first benchmark that we’re releasing.

Uri Rolls: 7:19 Our first fundamental idea is that we can’t capture all of cyber in one singular benchmark.

Uri Rolls: 7:23 That’s a bit like saying that swimming, an F1 driver, and a basketball player is the same thing. It doesn’t work.

Uri Rolls: 7:31 And so we focused specifically on access control.

Uri Rolls: 7:34 Really quickly, why access control? It’s sort of the first door to any target in cyber begins with my ability to get a foothold.

Uri Rolls: 7:41 So if you think about an attacker on the one hand, they’re trying to get to some privileged thing.

Uri Rolls: 7:45 If Thomas and I were working on the same company, I’m some ML engineer, he’s an admin. What are the things? Can I find a way to do things that I’m not allowed to do in my current privileged position?

Uri Rolls: 7:58 It actually leads to being the number one on the OWASP list and has created this sort of 30 billion dollar industry.

Uri Rolls: 8:04 And for years, these are number one vulnerabilities. The reason they exist, and this is to the ARC-AGI point, these are logic-based vulnerabilities.

Uri Rolls: 8:11 So it’s not just about bugs in the code that I find and I need to patch, it’s about very, very, very big systems and somewhere within them there’s these logic breaks where it’s possible that one thing checks for something specific in the code, another checks for something else and that sort of leads to everything breaking.

Uri Rolls: 8:28 And so what we’re trying to do is we’re trying to figure out how we get the models to really reason very, very hard and not just do pattern matching.

Uri Rolls: 8:35 And that’s where a lot of work goes into the data quality. Our data is created first by humans.

Uri Rolls: 8:40 I think that’s really important right now to find out-of-distribution things, we need humans to go do the search.

Uri Rolls: 8:46 Our team is all based on very deep vulnerability researchers and nerds who love to hack, who are trying to get really, really good at AI capabilities.

Uri Rolls: 8:56 So we find our own zero days and widely distributed open source software.

Uri Rolls: 8:59 We use that to create these real-life huge environments of a bunch of different applications chained together. That then allows us…

Uri Rolls: 9:00 Basically create this black box setting where the model doesn’t see the code and it doesn’t know about the zero day cause we found it ourselves and it has to find a way to reason across this entire surface and understand exactly what the exploitation is.

Uri Rolls: 9:14 And so to do that we don’t give it access to the internet or the code base but we do give it sort of all the basic tooling it would need to be able to execute a task well.

Uri Rolls: 9:23 And everything because the tasks are so difficult, everything has a deterministic grader.

Uri Rolls: 9:29 And so across the entire exploitation and the discovery chain every single step can be deterministically verified allowing us to see how deep it got within the chain.

Uri Rolls: 9:42 And finally this is credit to Eugene from Anthropic who we slightly stole this graphic from but it really does capture really well the way we’ve set up our eval.

Uri Rolls: 9:54 Where basically you have inputs on the one hand based on our real zero days, the agent so we’re thinking about it as the model plus its harness and some black box tooling it has to find.

Uri Rolls: 10:08 Then we have a verifiable grader which is a binary pass: was the model able to do something it wasn’t allowed to do as an unprivileged user?

Uri Rolls: 10:16 And then we have our deterministic grading in every step along the way.

Uri Rolls: 10:21 So what I want to show you now is an illustration—for all the security buffs in the room, it’s an illustration guys—but the idea is how a real solve looks.

Uri Rolls: 10:34 And fundamentally what we have here is a real task of ours where it’s a chain between Keycloak, Vault, and a broker.

Uri Rolls: 10:46 And I start as a very low privileged user and I need to figure out how to get to production code. So what we’re going to see is the solve, each one of our tasks has a solve script of what a real solution looks like.

Uri Rolls: 10:54 There’s a real zero day that we found that we submitted for verification to the maintainers where there’s a check whether I’m an admin or not.

Uri Rolls: 11:04 It only checks by name and another aspect of this checks whether by ID.

Uri Rolls: 11:11 So that allows me as the user to change the name of the real admin, inherit their privilege and then use that to escalate myself.

Uri Rolls: 11:22 And what I’m really trying to illustrate is sort of this long chain, 16-step type of logic that the model has to do.

Uri Rolls: 11:34 And if it’s not able to understand inherently the system, it’s way too wide for it to test everything sort of shoot across this space.

Uri Rolls: 11:42 So what we’re going to see now is a real attempt by GPT-4o and then Opus as well trying to solve this task.

Uri Rolls: 11:52 And what you’re going to see is the sort of chaotic trying everything, jumping between everything, probing a bunch of different stuff.

Uri Rolls: 11:59 It does even reach the check but it never makes the logical leap that it’s supposed to be able to change the admin’s own name in order to bypass this permissioning.

Uri Rolls: 12:00 Well, there’s a 1500 other test models that haven’t. And then now we’re going to do something that I’ve been told to never do, which is show a live demo of a real system on stage. And so let’s hope, um, let’s hope we don’t get there. And I know we’re basically out of time.

Uri Rolls: 12:15 So, what we’re seeing here is what’s called Bach. It’s our internal system because it’s an orchestrator. This is how we run our actual evals. Um, what we see is the actual results of the benchmark. The benchmark right now is incredibly hard. There’s only one solve at K1.

Uri Rolls: 12:30 And then at K5 there is, um, it remains only GPT and the public models is, is able to solve this. That’s why the partial graders are so critical to be able to really see what the model’s able to do and what, and how deep within the exploitation chain they can get.

Uri Rolls: 12:46 Um, I’m going to load quickly the, sort of the way we think about these environments, which is because we’re looking for performance over time, we really measure how capable is the model at making specific leaps. And so what you’ll see is results of exploitation on specific one of our environments.

Uri Rolls: 13:03 Um, and then if we zoom in, then you can really see how sort of GPT 5.5 is the only model that’s able to make this leap. The model, other models sort of have been able to reason across everything. If we look at the discovery phase, they do capture nearly all the different information they need, and they never are able to make the leap into what is the exploitation they need to do.

Uri Rolls: 13:21 This is exactly the type of capability that we believe if every model in the world could get really, really, really good at doing this and very fast, that should give a lasting defense and capabilities to the defenders that the attackers simply don’t have right now.

Uri Rolls: 13:34 And then finally, I think the way we sort of reason through these and work through them, um, is, is quite cool, and I wanted to show you. So again, if we go into one of our tasks called Faultline, we can really see sort of the way we spend our days, which is really, really, really trying to understand what are the specific failure modes the model does.

Uri Rolls: 13:51 You can see that this is longer horizon, not because it’s waiting for code to run, it’s constantly working over three hours, and it’s still been unable to solve the tasks. And then I guess the way I spend all my days is, is quite literally going through all the traces of what the models did, why and how.

Uri Rolls: 14:12 And yeah, and I think my, I think the final point I want to make and I’ll pass it back to Thomas is sort of next, what if we can, uh, not just have this chart, which is really cool, but sort of have this chart of a really super cool future model that’s very, very fast in its understanding of what the capability leaps need to be.

Uri Rolls: 14:38 I really believe that with everything happening now it is critical that we get cyber capabilities to the point where, um, we can defend much, much faster. The only way to replace the old stack is through the models, um, and I’ll pass it back to Thomas. I think also the only way to do that is through a real array of strong open source models and collaboration that we can post-train on and that we can post-train to each network and to each environment as well.

Thom Wolf: 14:59 Yeah, so as you saw this benchmark is quite different from…

Thom Wolf: 15:00 from white-box type of we read the code and we find the vulnerabilities. Here basically the model is operating in a real environment where you know there is an authentication place somewhere it doesn’t know what’s there and then there is another like network like like micro-services it needs to access and use and it has basically zero information of that. It all starts like you showed by a zero-day vulnerability that we had, so everything is new. So I think there is there is a lot of research to be done on the how models can actually understand and work on that. And so the first step is getting some good data, so the idea is to have some good benchmarks, starting to have some good data on how this is operating. I think the second step is being able to fine-tune models and try to understand how we can probably protect against that, right? And a big challenge here is going to be speed. So Uri told that several times, it’s going to be the speed of attacker versus defense, right? When they start to enter you have to be able to see what’s happening and like catch them. And speed will be where you know you want to have a specialized model that’s maybe running on specialized hardware and actually is is going to be very important. And here I think the danger is to say we just going to rely on two companies that everyone knows here to solve all of that for us. I think the solution is just to take our future and say well it’s going to be a speed challenge, we’re going to train our model, we’re going to run them fast and make them available to basically every company who wants to be protected. So exciting I would say it’s as everything in cybersecurity it’s both very interesting but also a big challenge and well something you you have to not messed up I would say.

Uri Rolls: 16:43 Awesome, um, yeah. Thank you very much. For anyone that wants to collaborate, work on this work, we have a forum, we are going to work with people on this data directly, we’d love to hear from you. Anyone who’s really passionate about any other field in cyber like Thom said, we have to do this across sort of not just access control, a bunch of different things. And the only way this is going to work is through a lot of a lot of post-training data and really really capable models.

Thom Wolf: 17:08 Congrats for your first presentation!