Praxis-1, our first world action model, is now available for anyone to use. It is currently the leading open-weights robotics model on the RoboLab leaderboard and outperforms every other world action model on the benchmark. More than that, it’s versatile and easy to fine-tune for new hardware and tasks. We’ve tested it on 9 different robot embodiments so far, in simulation and on real hardware with multiple partners, and we’ve seen it reach good performance on a new embodiment with under 30 minutes of demonstrations. Weights, code and fine-tuning recipes are on GitHub and Hugging Face. It can run on a single RTX 5090 with NVFP4.
We started our robotics work about a year ago with the thesis that the same video pretraining we do for our creative models can be massively useful for building systems that interact with the physical world, and the benefits of transfer are now becoming clear. When I first talked to people in the field, there was a lot of skepticism about whether video models would be useful in robotics at all, and then about whether you could use them as policies. From first principles, it makes sense that if you can predict the future well in pixels, extracting an action representation should be the easy part. Reality caught up with first principles.
One of my favorite things in our technical report is the idea of self-rollouts. Because the model’s performance is bottlenecked by its visual imagination, you can quickly get an intuition for whether it will perform a new task well by rolling out the policy in the model’s imagination. That can speed up evaluation, and it opens a path to doing reinforcement learning in the model’s imagination.
There’s been a lot of discussion recently about frontier LLMs being the bitter-pilled way to solve robotics. But scaling performance from unsupervised video is an even better example of the Bitter Lesson than scaling from text. Praxis-1 is more than 10x faster and 50x cheaper than GPT-6 Astra on the manipulation tasks we tested. Frontier LLMs will keep improving rapidly on the Pareto frontier of cost and performance, but so will world models. There are also great ways of combining the strengths of the two. And we are much farther from saturating multimodal data than we are text.
Thanks to the mighty Runway Robotics team for making this happen, and to our partners for being so generous with their time helping us test the model across a wide range of hardware and tasks.
Today, we’re making Praxis-1 available to everyone. Our state-of-the-art world action model turns video pretraining into control for real robots, and it ships with open weights under Apache 2.0, ready to run and fine-tune on your own hardware.
With the @runwayml plugin in Codex and ChatGPT, you can see how a character from your prototype moves, try a different opening shot for a trailer, or turn a product photo into a video ad.
Anthony Maggio shares how Runway’s marketing team is using ChatGPT Image to produce more assets each week.
Introducing Inside View. A new series exploring the impact of AI, from the people who are shaping it.
Episode one features Anthony Maggio, Chief Product Officer at Runway, discussing what the electrification of factories teaches us about the adoption of AI.
New pod drop!! @RunwayML just turned 8 years and added $100M+ in ARR last quarter alone. I met Anastasis Germanidis (@agermanidis), co-CEO of Runway, who went from art school to building one of the most important video and world models of our generation.
We talk about:
- The Nietzsche-inspired game he tried to build at 15
- “We accidentally created one of the best models for robotics”
- 30 magic tools in 30 days
- Sora launch: a 2-hour existential crisis → a better model in 3 months
- Proof of work > credentials
- Running “a perfect simulation” of his co-CEO Cristóbal (@c_valenzuelab)
- “I’m always becoming someone else”
and so much more!
Timestamps
(00:00) Trailer
(01:21) Anastasis Germanidis, Co-Founder & Co-CEO, Runway
(02:27) Accidentally building one of the best robotics models
(07:18) How video models train robots and self-driving cars
(10:47) What a world model actually is
(16:19) Why folding clothes is still unsolved for AI
(17:37) Why AGI can never be reached
(22:03) A game built on Nietzsche's eternal recurrence
(30:35) Starting an AI company from art school
(39:47) Why staying in stealth is counterproductive
(42:03) 30 magic tools in 30 days
(55:55) Sora's launch and an hour of existential crisis
(57:02) Racing to beat Sora in three months
(1:01:56) Solaris and software generated in real time
(1:05:15) Why AI labs pay for fake versions of Amazon
(1:09:52) Why Sora struggled as a consumer app
(1:16:30) Hiring on proof of work
(1:19:44) Becoming co-CEO with Cristóbal Valenzuela (@c_valenzuelab)
(1:30:56) What he's still becoming
(1:32:52) Outro
Runway turned eight years old this week. We started at a time when generative models were a research curiosity, and had to figure out how to run an AI company from first principles. Growing this organization has been one of the biggest joys of my life.
Below is a short note that I sent the team:
Thank you for the kind words, everyone. The reality is, as convenient as it might be to portray otherwise, the Runway of today looks nothing like the Runway of 2018. And that's by design. We didn't know we'd be focusing on video, let alone pretraining video models at scale, let alone pursuing world models. It's a convention to rewrite the origin story to fit the present. But this company is all about breaking conventions, so let's do away with that one too, shall we?
So then, what is Runway about? I think Runway is a story about betting on people. By far the most gratifying moments of my time here have been watching people get an opportunity and then prove themselves far beyond what anyone expected.
The person with zero publications who became one of the most important researchers in the community. The person who barely spoke in meetings and became one of the best managers in the company. You know who you are.
And it's the three of us too: Alejandro, Cris, and me. Three people from an art school with zero connections should not have built a company that made it this far. It wasn't supposed to happen. It did because we got lucky with the people we brought on.
I desperately want the story and legacy of this company to be about how good people win. In the world we live in, the default is that the loudest people win, no matter how wrong they are. We've beaten many loud-but-wrong competitors, but we can't take that for granted. The odds favor the loud over the right, because loudness is noticed immediately and being right takes time to be noticed.
So when you know you are right, please be louder than you think you should be, even when it feels uncomfortable. The world will be better for it. And let's all make the ninth year of Runway the loudest one yet.
Introducing Runway App Studio.
Build the creative app you wish existed, just by describing it. It generates images, video and audio with Runway’s models and your Brand Kits, ready for you or your whole team. No code required.
See examples in the thread below.
From a narrow simulator of pixels to a universal simulator of reality.
This essay is the result of multiple conversations with Packy about Runway’s long-time research thesis on world models.
It's one of the most extensive overviews of our work yet, going over the applications of world models in media, robotics, interfaces, and science.
Runway has been on a tear over the past couple months.
Viral HORSE Video contest
Solaris: the world's first Interface World Model
Praxis-1: open-weight World Action Model for robotics
Project Continuum: operating system research application built around real-time interfaces
A video model. An interface model. A robotics model. An operating system. Are you getting it? These are not separate pretrains. They’re different applications, built around the same General World Model backbone, with different post-training on top.
All of them are examples of what you can do when you can simulate any pixel.
So I asked co-CEO Anastasis Germanidis to co-write an essay with me explaining how video, interfaces, operating systems, and robotics all fit together, why video was the right starting point for World Models, where the world is heading if Runway’s bets are right, and whether or not, as someone who simulates worlds, he believes we’re living in a simulation.
Welcome to the future. Enjoy!
(Video, of course, made with Runway)
Researchers at Harvard and Brigham and Women’s Hospital found that transplanted hearts shift toward the biological age of their recipients. Old hearts get younger in young bodies, and young hearts get older in older ones. Which is nuts.
Today, we're launching a partnership with our friends @runwayml to, among other things, bring one of the stories from each week's Dose to life in video. I'm not great at it yet, but not bad for a few prompts! And I'm excited to get better. I have zero talent in film/video, but I can write, which now means that I can make movies. Age of miracles.