Andy Chen

Head of Robotics @runwayml, ex-DeepMind and Meta

[NVIDIA Robotics @NVIDIARobotics]

Congrats to @runwayml on the open weight release of Praxis-1 27B world action model. 🎉.

Developers can adapt it to their own robots and tasks, running it across NVIDIA GPUs from data centers to local AI deployments on NVIDIA Jetson on the robot.

👉 Download the NVFP4 checkpoints from @huggingface: https://t.co/eeYUuoeKRY

👉 Get started with the Jetson AI Lab tutorial: https://t.co/D6bJZYXni7

👉 Post Train with NVIDIA NeMo Automodel:
https://t.co/w3eBspEX5C

[Anastasis Germanidis @agermanidis]

Praxis-1, our first world action model, is now available for anyone to use. It is currently the leading open-weights robotics model on the RoboLab leaderboard and outperforms every other world action model on the benchmark. More than that, it’s versatile and easy to fine-tune for new hardware and tasks. We’ve tested it on 9 different robot embodiments so far, in simulation and on real hardware with multiple partners, and we’ve seen it reach good performance on a new embodiment with under 30 minutes of demonstrations. Weights, code and fine-tuning recipes are on GitHub and Hugging Face. It can run on a single RTX 5090 with NVFP4.

We started our robotics work about a year ago with the thesis that the same video pretraining we do for our creative models can be massively useful for building systems that interact with the physical world, and the benefits of transfer are now becoming clear. When I first talked to people in the field, there was a lot of skepticism about whether video models would be useful in robotics at all, and then about whether you could use them as policies. From first principles, it makes sense that if you can predict the future well in pixels, extracting an action representation should be the easy part. Reality caught up with first principles.

One of my favorite things in our technical report is the idea of self-rollouts. Because the model’s performance is bottlenecked by its visual imagination, you can quickly get an intuition for whether it will perform a new task well by rolling out the policy in the model’s imagination. That can speed up evaluation, and it opens a path to doing reinforcement learning in the model’s imagination.

There’s been a lot of discussion recently about frontier LLMs being the bitter-pilled way to solve robotics. But scaling performance from unsupervised video is an even better example of the Bitter Lesson than scaling from text. Praxis-1 is more than 10x faster and 50x cheaper than GPT-6 Astra on the manipulation tasks we tested. Frontier LLMs will keep improving rapidly on the Pareto frontier of cost and performance, but so will world models. There are also great ways of combining the strengths of the two. And we are much farther from saturating multimodal data than we are text.

Thanks to the mighty Runway Robotics team for making this happen, and to our partners for being so generous with their time helping us test the model across a wide range of hardware and tasks.

[Andy Chen @AndyChenML]

Today we’re releasing Praxis-1, Runway’s state-of-the-art world action model for robotics, with open weights under Apache 2.0.

It’s the #1 world action model and the #1 open-source model on NVIDIA’s RoboLab-120 leaderboard, and it’s 15.6x faster and 51x cheaper per completed task than a frontier LLM agent driving the same robot.

Under the hood, it’s a 27B-parameter mixture-of-experts diffusion transformer that predicts video and actions together, turning video pretraining into control for real robots.

Praxis-1 is available now on @LeRobotHF and vLLM-Omni (@vllm_project), with hosted inference on @reactorworld.

Full results: https://t.co/dUfY0PjuCU

This is our first release, and we’re expanding our investment in robotics from here. If you want to put world models on real robots with us, we’re hiring!
https://t.co/XviAs6SOE9

[Runway @runwayml]

Today, we’re making Praxis-1 available to everyone. Our state-of-the-art world action model turns video pretraining into control for real robots, and it ships with open weights under Apache 2.0, ready to run and fine-tune on your own hardware.

[Ziyi Tang @tangziyi001]

Excited to share Praxis-1, our state-of-the-art, open-weight World Action Model for robotics! It ranks #1 among WAMs and all open-source robot foundation models on NVIDIA’s RoboLab leaderboard.

Praxis-1 is pretrained on general video at scale, ready to finetune on your own robot. It handles real-world tasks on single-arm, bimanual, and mobile settings and achieves competitive results across tasks and embodiments.

Checkout our technical report: https://t.co/u8qe7LHZ2v

Interested in joining our team at Runway? Please reach out!

[Anastasis Germanidis @agermanidis]

Runway turned eight years old this week. We started at a time when generative models were a research curiosity, and had to figure out how to run an AI company from first principles. Growing this organization has been one of the biggest joys of my life.

Below is a short note that I sent the team:

Thank you for the kind words, everyone. The reality is, as convenient as it might be to portray otherwise, the Runway of today looks nothing like the Runway of 2018. And that's by design. We didn't know we'd be focusing on video, let alone pretraining video models at scale, let alone pursuing world models. It's a convention to rewrite the origin story to fit the present. But this company is all about breaking conventions, so let's do away with that one too, shall we?

So then, what is Runway about? I think Runway is a story about betting on people. By far the most gratifying moments of my time here have been watching people get an opportunity and then prove themselves far beyond what anyone expected.

The person with zero publications who became one of the most important researchers in the community. The person who barely spoke in meetings and became one of the best managers in the company. You know who you are.

And it's the three of us too: Alejandro, Cris, and me. Three people from an art school with zero connections should not have built a company that made it this far. It wasn't supposed to happen. It did because we got lucky with the people we brought on.

I desperately want the story and legacy of this company to be about how good people win. In the world we live in, the default is that the loudest people win, no matter how wrong they are. We've beaten many loud-but-wrong competitors, but we can't take that for granted. The odds favor the loud over the right, because loudness is noticed immediately and being right takes time to be noticed.

So when you know you are right, please be louder than you think you should be, even when it feels uncomfortable. The world will be better for it. And let's all make the ninth year of Runway the loudest one yet.

[Andy Chen @AndyChenML]

Talking robots, sim-to-real, and what it takes to actually deploy at @encord_team’s Long Horizon Tuesday.

9:25am, Frontier Stage. Come say hi 🤖

[Encord @encord_team]

The biggest names in Physical AI are coming the Long Horizon Summit Oct 20 / 21.

1000+ VPs, Heads of and Practitioners in one room. Every conversation will be worth your day.

See the full lineup here: https://t.co/NCbPfLaF6U

[Andy Chen @AndyChenML]

Learn more: https://t.co/1HKKYZ1VKW

[Andy Chen @AndyChenML]

Two robots ran live all day at our AI Summit on an early build of Praxis-1, our new World Action Model.

No cherry-picked clips, no benchmark maxxing, just the model working in front of everyone.

[Kamil Sindi @kamilsindi]

It's really cool to see our robot packing items it hadn't observed in the training data, demonstrating zero-shot learning at our AI Summit.

[Runway @runwayml]

Runway Head of Robotics Andy Chen laid out Runway's unique approach to robotics, ending with the premiere of our open-weight world action model, Praxis-1.

Learn more about Praxis-1 and request early access: https://t.co/SA9HTPcElB

[Andy Chen @AndyChenML]

@OriolVinyalsML @OfficialLoganK 🚀🚀🚀🚀🚀🚀!!!

[Andy Chen @AndyChenML]

@sama Chat automation with humans in the loop is a start: https://t.co/CNAKXO1LYN

From the same industry that led video streaming and subscriptions—but has always been seen as "too aggressive" 🙂

[Ethan Mollick @emollick]

In all seriousness, what is the plan for college essays in the fall?

It can't be college essays as usual.

[Matthieu LC @matthieulc]

Got stable diffusion to dynamically produce a virtual desktop interface controlled via mouse inputs! No frontend code, no frontend rendering engine, pure image generation.

Blogpost: https://t.co/xfz4L4QsSA

#stablediffusion

[John Thickstun @jwthickstun]

Let's play with the Anticipatory Music Transformer: reharmonizing a MIDI arrangement of Levitating (Dua Lipa).

Blog post describing this interactive creative process:

https://t.co/IBN7K8GTB1

Colab notebook used to create this demo:

https://t.co/2sqezJzyMz

[Davis Blalock @davisblalock]

In the spirit of "Latency Numbers Every Programmer Should Know," (https://t.co/goVHF09sVE) this includes some throughput numbers every deep learning user should know. [13/16]

[Sam Altman @sama]

today we launched ChatGPT. try talking with it here:

https://t.co/uWra8LKFMN

[Andy Chen @AndyChenML]

2020 was too early for most LLM applications beyond text generation (copywriting, storytelling).

But LLM's have gotten a lot better since then, and are still improving at a rapid pace.

Looking forward to these new LLM's in the next generation of products.

[Andy Chen @AndyChenML]

An article about the amazing applications of LLM's (GPT-3 for search, Excel, codegen, SQL)

From July 2020.

In the 2+ years since this article was published, none of these applications took off.

What happened? A Thread 🧵👇

https://t.co/Dz5TgXBO7y

[Andy Chen @AndyChenML]

Some more: Chain of thought prompting (Jan 2022), increasing model size from 175B -> 540B (April 2022), training on more data with Chinchilla scaling (April 2022)

You can pile all of these techniques on top of each other and your LLM keeps getting better