---
title: "Everything will be Simulated"
speakers:
  - name: Cristóbal Valenzuela
    affiliation: Runway
room: "Simons Room (Metropolitan East)"
starts_at: "2026-10-14T17:06:00-04:00"
ends_at: "2026-10-14T17:24:00-04:00"
format: "keynote"
recording: "2026-10-14_1709_Runway"
duration: "17m 04s"
source: "qwen3-asr-1.7b"
---

# Everything will be Simulated

> Transcribed by qwen3-asr-1.7b with a schedule prompt. Possible prompt artifacts, names heard more often here than in the unprompted Voice Memos transcript (Qwen vs Voice Memos): Cristóbal Valenzuela (1 vs 0), Runway (5 vs 3). Check them against the audio before quoting.

### 00:00:00 — From 128-Pixel AI Films to Photorealistic Video

Please welcome Cristóbal Valenzuela, co-founder and co-CEO of Runway.

Very excited to be here. Thank you for staying for the last talk. Welcome to my talk about the matrix and how we're living through it. So you know I've been working in a number of firms that they get now. Started here in Europe, actually, maybe around ten years ago, and I want to focus on some of the learnings we've seen through, you know, building the company, building this research lab here in New York, and where do we see a lot of the work that we do in video, in pixel generation, really moving forward. And so I don't kind of exaggerated, I joke that this is might be a simulation or not.

And so for maybe for perspective, I'll walk you through a piece of history. This is like watching prehistoric ancient ruins. This is our first AI film we ever made. It was done in 2019. It's called the orange because it's going to tell very clearly. It's an orange that falls from the tree and walks through different parts of the world. This is a this is a video model and image model that we had back then that was creating pixels in 128 pixels by 128 pixels that we then forced to be upsampled to the extent of resolution. I think of this as art now. This is like looking at a film from the nineteen hundreds. You know, it's beautiful. There's a lot of history behind it, but this is two thousand and eighteen. We started the company in 2018, 2017, and so we're trying to figure out if this had any lags. We're trying to convince ourselves that if you scale this, it will naturally improve to the point where we can get somewhere close to this.

This is state of the art video today. This is all generated. None of these characters are real. None of those locations are real. I'm sure you — if everybody here, you know this by now. You've already have seen this online on Twitter. The models are just incredibly good, and the most interesting thing is that when you look at stuff like this, I still believe this will look like the orange in just a couple more years, and so I want to walk you through you know how we're thinking not only through what we're doing around you today but also in the next couple of years, and the research that's shaping that.

### 00:02:30 — Scaling Next-Frame Prediction into Physical Understanding

We are a company that's at the intersection of both art and science. We have deep roots in artistic and creative communities, and we have some of the best researchers in the world working on efficient models and large video models, and I like that combination because it allows us to you know bring the best of both worlds. We're a team of scientists and artists thinking about the problem of scaling video and world models, but in a way, that the core idea of the company was already discovered by cinema and by movie pictures, 120, 130 years ago, which is if you think about video, maybe you remember this from high school: videos and optical illusion, videos the illusion of movement, that if you do, if you speech frames fast enough, you will create movement. You will create the motion of movement. You create motion pictures. And so at Runway we're doing kind of something very similar. We're taking that idea really to the next level. How do we do next frame prediction fast enough, scale enough, but in such a way that we can hopefully do more than just pretty, pretty, and nice, pretty videos?

And so, Alitos is powered by core scaling loss. I think we believe in scaling loss, not only applied for language models, but also for video, and this has been true. We're now really ramping up the amount of compute that we put behind the models that you already have seen, and I think the interesting thing about you know seeing the model scale over time, the emergent properties that appear, the intelligence that appears when you start taking this very simple idea that cinema formed itself, figuring out 120 years ago, which is next frame prediction, and taking next frame prediction really to next level, right? So if I take any much like this, which is not real — those two people are not real, and I just keep on predicting what the next frame would look like based on the previous frames, and if I do it fast enough, and if I do it with a network that's big enough, I'll get movement. I'll get Danish movement. I'll get something that looks so real to us that we, we humans can't distinguish. I look at that video, and I've been doing this for ten years. I can't tell you if that's real or not.

And we measure that. We've actually measured trying to quantify if you look undistinguished between real and generated videos, and I think that Turing has already has — people are not able to distinguish. But the more interesting thing is here, is that really what's happening as you predict next frames is you're learning a lot more than just pretty frames. You're learning about physics. You're learning about object permanence. You're learning about gravity, you're learning about occlusion. You're learning about movements. You're learning about how the objects work in the real world. And you're doing that all through just video, through images, through next frame prediction.

So if you look at a complexing like this, a good video model for it to understand how to make this a consistent video, we'll need to naturally understand what all these subjects are, how each of them relate to one another, how a camel might probably stand when I'm all like that, what water in a container looks like. And if I do all of that together, and I do it fast enough, all get hopefully a consistent video that looks real to us. So a lot of the benchmarks that we use to evaluate our video models are not only creative — it's creative basically means do you like it or do you don't — art actually based on physics, on geometries, on how objects behave in the real world, and how those objects behave and also simulator or virtual worlds.

So you take that idea of next frame prediction, and you do it fast enough, and you get, of course, now, something that looks more like what we should expect from a video application or a game application. Again, this is just video, right? And you take you take that idea to the next level. What happens if we start — we say yes, next turn prediction, but doing the context of a 3D environment or something that looks similar to a game? You can start to be getting closer to a physics engine, a simulation system, but the simulation system has no physics laws — we didn't describe what did he do — the model just knows how to do that by pure force of scale. This kind of gets kind of interesting because you got to really think about playing roles. This is not GTA. Is someone playing someone else in a environment and a software that does access to all simulation or the simulation, which gets really meta, right? And so I'm not kidding when I said welcome to my matrix simulation.

### 00:07:00 — Real-Time Interactive Worlds & Embodied Robotics

But the most interesting thing for me is that everything you saw before was not real time, meaning that it took a couple minutes or seconds for those videos to be made. A lot of what we're seeing right now, when we think about you know the next three four years of video models and large models, is to deeply think about what happens when we start not only scaling the models and making them much, much, much bigger, which much more important these, but also what happens when you start making them interactive, right?

So, take some idea of next frame prediction. You take a frame, you predict next frame, then X, then X, then X, but then it can act at any point in time, in that prediction, which means I can simulate any possible divergence or action in that world. So in this row, I could play the video, but it can also come and touch every part, every aspect of that image, and make it interactive to affect the consequences of that world. And it works.

And of course, this is incredibly useful for a variety of different use cases, as you can imagine. We're now being an entire robotics lab in Runway. So if you come to our office in New York, half of it is what looks like a Hollywood studio. Half of it is a bunch of robots walking through and moving through the office. And so we're able to make these models, or start making these models, the brains of how autonomous and embodiment systems walk and reason about the world.

### 00:08:29 — World Interface Models Without 3D Engines

But there's been a couple of more interesting use cases that I think go beyond the two August one, which is video and robotics, which is what happens when you think about anything, anything, any pixel on a screen, as a bigger model. This is a great example. This is the video wall. What you see right now, while this is an image as well. That image could be real or not, but I could run one of our video models in real time and make this an interactive application. So let's say I wanna you know turn the lights of the car. Maybe I wanna like turn the car around. I can just flip through it. I can turn it around again. Maybe open the hood and see what's inside the car.

Now everything you just saw here has no physical 3D representation. There's no Unity. There's no Unreal. There's no Blender. It's just a bigger what. The video model has been trained to understand actions and clicks in an interface. We call this a world action model. Sorry, a world interface model. And there will interface model. Well, I'm sure you guys understand now. It's an interface, and it's a world model. Well, interface model.

I'd also do very cool stuff like this, and so here's another application, another example of how we would think about using a model like this in real time, they can take an image, a representation, move objects around and expect them to move as I was, as I was actually in that room. Again, remember, there's no 3D representation of this. This environment might actually be have been modeled before. It's just an image and the model that understands how objects look move in a space because of this very simple idea that net frame predictions scale up well enough have become a wall model or a simulation engine, so.

### 00:10:12 — Continuum & Interactive Video Portals as an OS

With the eight minutes I have left, I want to show you this idea taken to another extreme. We're thinking not only about what happens if you can control these cool applications and these cool interfaces, but what happens if you keep extrapolating? And I think, by the way, extrapolating might be one of the most important skills these days in AI in company building: what happens for the next six months, twelve months, eighteen months maybe one of the most important things to try to predict and understand.

And so we ask ourselves that question a lot at Runway. What happens? What will happen if we keep scaling the models? If we keep adding more data, if we keep making them better simulation systems, we make them universal simulation systems that can predict anything on a screen, literally editing a string. So we've been working on this idea that we call Continuum, and Continuum is the idea of real time video models or world models taken really to the next the next degree. This is an operating system. The idea is that you might open your computer at some point in the near future, and what you see streaming to your screen but be coming from a data model, so I'll show you next specific how this idea could change. This is still very early research, but I want to show you this idea of combining video models as part of your operating system as part of your daily work.

So here's an example. I was researching, you know, San Francisco and the fog in San Francisco. I wanted to learn more about when and how it happens. This is structural data. This is actually a website. This is something you can take from the internet, and now Conde has this idea that I can select any part of content in that website and make it a portal. We call this a portal, and the portal is a real-time video. It's fully interactive and fully based on the content you're experiencing reading through. I probably won't read through all of that text. It's too much text for me. So just make a video. Make a big video that is represented by the text you see there.

These portals can be visualizations. It can be any content that you're watching or interacting with — on the right, you're gonna have all the visualizations and portals you've experienced before — anything you were trying to learn. Basically, it's a it's a memory. So the desk of the imaginations or the visualizations you've seen before — so here I was researching about batteries, and so I can go back to the portal I had around batteries before — and these are real-time interactive applications. I can clip them anywhere. I can move them around. I can start in asking questions, and the videos and the content itself will change based on based on my interactions and the way I'm managing through it.

Here's another very cool example that when the first time I saw it, when the team showed it to me, was like blew my mind. Since it's a real-time video, you can make it fully interactive with anything, not only with clicks, but like resizing the obligations should also change the behavior. Right, so I can resize. I'm learning about photosynthesis here. If I resize the application, I get a different experience. And this is again full interactive applications. I can click through it, navigate through parts of the cell, or how does this work, to get a better understanding of what I'm trying to learn and do again.

The one thing that still feels to me kind of unsettling about this is, I don't know how to call this. I don't know if this is — it doesn't feel like a visualization. It doesn't feel like a video. It doesn't feel like a game. Somewhere in between, and I think we need to do the collective work to figure out how we call this kind of interaction in the first place. And I'm showing you here, kind of the other thing is that you can change the structure of the prompt, or what's actually creating that interaction in the first place.

### 00:13:49 — Visual Thinking for Coding Agents

Here is another example. I'm going to type something. And this is one of the ones that I feel has been the most underexplored yet. This is we call this visual thinking. So we've connected continuum to a coding agent, and what the coding engine can do is, it can just continue to visualize your problem. So imagine you ask your coding agent to create a 3D model for your bike, and instead of the model trying to just think through that via code, the model can spin a version of Continuum to think through the problem visually. So, if you ever think about someone, or I'm sure you have friends, or maybe a lot of people here may be visual thinkers. What a visual thinker is that things through problems and through ideas visually first, and that helps you ground and project some of the best things you want to do.

And this is somewhat very similar. Agents have the ability to do visual thinking. Agents can spin up a sort of imagination that you can tap and interact with, so not only the agent can interact with it, I can also see it myself, and I can guide the agent much better ways that I might have been able to guided just by text alone, which is for us a very powerful idea. It's a completely new paradigm. We call this idea visual thinking.

### 00:15:10 — Live Reactive Simulations & The Road Ahead

The last example I have it is how you think about taking, you know, continuum to become much more than just using or navigating interactive structural content in the way I showed before, but actually making them fully live experiences that are reactive. So in this case, I was searching for Mars. And as continuing understands the intention of what I'm trying to search from it also suggests interfaces that are specifically designed in real time just for that. So here it created for me this Mars outpost that actually should be accurate with the data about the actual like robots that are in Mars right now, or the ones that are projecting maybe 2030, and since this is fully interactive, I can interact with it. I can go to places. I can move around in the same way that I might move around, you know, in a 3D system or a 3D continuum. And it works. Just wasn't it.

So when I say everything will be simulated, I'm really trying to understand or help not only our team but the world get to a point where a lot of what we see today will become obvious. Hopefully, in a few more years or months, we look back and continue, and we'll find tools very close to the video of Yorick that we just showed you. Continuing when a lot of the things we're doing in video and in runway, when it comes to simulation, they're still very early days. We're getting to a point where it actually it actually works and it really scales. So we're really excited about is that for the next eight to twelve months, we're releasing most of what is shown here for everyone to use.

Nice to meet you. Nice to meet you. Yeah, have a good one. And we're free. As we come to the close.