Marco Pavone

Prof @Stanford, Distinguished Research Scientist and AV research lead @nvidia. PhD from @MITAeroAstro. Robotics, autonomous systems, AI. Opinions are my own.

[Marco Pavone @drmapavone]

We are hiring! The Autonomous Systems and Physical AI Research (ASPIRE: https://t.co/HZF2dnxUBf) group at @nvidia is looking for talented PhD Research Interns to join us in advancing the frontiers of #autonomous #systems and #Physical #AI.

We work across a broad range of research areas, including #reasoning models, generative simulation, #agentic AI workflows, and Physical AI #safety, with applications spanning autonomous vehicles and a broad range of Physical AI systems.

Interested in pushing the limits of what’s possible? Apply now:
https://t.co/QGQbgCl5Lu

[NVIDIA DRIVE @NVIDIADRIVE]

From Robotaxis to L4 Autonomy: A Deep Dive Into NVIDIA Alpamayo 2 Super https://t.co/qcyn8W7eU1

[Marco Pavone @drmapavone]

Join us tomorrow for a deep dive into #Alpamayo 2 Super—our most capable model yet!

We’ll unpack the key technologies behind the model, showcase hands-on notebooks to help you get started, and share the latest AlpaSim updates.

We’ll also take a closer look at our two ongoing challenges, with prizes up for grabs!

Save the date: https://t.co/PqcEMTU41X

@NVIDIADRIVE @NVIDIAAI

[Marco Pavone @drmapavone]

The @Stanford Department of Aeronautics & Astronautics has a faculty opening!

This is an incredibly exciting time for #aerospace. Rapid advances in AI, autonomy, robotics, sustainable aviation, advanced propulsion, and space technologies are reshaping how we design, build, and operate the systems that will define the next generation of flight and exploration.

Joining the Stanford AeroAstro Department is a unique opportunity to help shape these fields for decades to come — while educating and inspiring the next generation of aerospace leaders.

We are looking for outstanding faculty who want to push the boundaries of aerospace research, collaborate across disciplines, and make a lasting impact through both research and education.

📅 Application review begins: October 15, 2026
📅 Full consideration deadline: October 31, 2026

Learn more and apply: https://t.co/0cZBA5qf2N

Please share widely with exceptional candidates who may be interested!

@StanfordEng @StanfordAILab

[Marco Pavone @drmapavone]

Our latest finding: scaling self-verification can make open-weight models significantly more capable at a fraction of the cost.

With DeepSeek V4 Flash, sampling just 5 candidate solutions and using the same model to rank them with LLM-as-a-Verifier improves Terminal-Bench 2.1 accuracy from 79% → 88%—outperforming Claude Fable 5 while costing 11× less. 💰

As open-weight models become more capable, we can generate many high-quality candidate solutions and verify them at very low cost.

Try it out: https://t.co/ZNDWOl7Dmc

More on verification scaling in @jackyk02's previous post: https://t.co/ZxBfEVMbwD

@StanfordAILab @StanfordEng

[Jacky Kwok @jackyk02]

How can we extract richer signals from AI Feedback?

Introducing LLM-as-a-Verifier✨— a simple verification scaling framework that achieves SOTA on agentic benchmarks 🚀

The key idea:
- Use fine-grained scoring granularity (e.g., 1-20 instead of the standard 1-5 scale)
- Take the expectation over the full logprob distribution of score tokens
- Scale repeated evaluation and criteria decomposition

You can use these fine-grained signals for more effective test-time scaling, RL, and agent monitoring! It achieves SOTA across Terminal-Bench V2, SWE-Bench Verified, RoboRewardBench, and MedAgentBench 👑

Advised by @Azaliamirh @istoica05 @drmapavone @chelseabfinn

🧵👇

[Marco Pavone @drmapavone]

We have just released #Alpamayo 2 Super — @nvidia’s frontier open reasoning model for #autonomous #vehicles.

NVIDIA Alpamayo 2 Super is an open 34-billion-parameter #reasoning vision-language-action (#VLA) model designed to accelerate autonomous vehicle (AV) development. It combines the 32-billion-parameter NVIDIA #Cosmos 3 Super Reasoner with a 2-billion-parameter diffusion-based Action Expert and is post-trained with #reinforcement #learning.

Two aspects make Alpamayo 2 Super particularly exciting:
1. Open and commercially deployable: Alpamayo 2 Super is available on Hugging Face under OpenMDW-1.1, the Linux Foundation’s permissive license for open AI model distribution. The OpenMDW license is now being applied across the entire Alpamayo model family, enabling developers to deploy these models commercially without requiring additional permissions.

2. A multi-task foundation model for autonomous driving: Alpamayo 2 Super produces five tightly coupled outputs:
- A trajectory describing the vehicle’s planned path.
- A chain-of-causation (CoC) trace explaining the reasoning behind each driving decision, achieving benchmark-leading reasoning performance at frontier scale.
- A meta-action (e.g., yield, lane change, stop) capturing the model’s intent.
- Reasoning auto-labels that generate CoC annotations for training and validation data.
- Visual question answering responses with 2D visual grounding, linking answers to specific regions in camera images.

These multi-task capabilities enable developers to leverage a single foundation model across more of the development process, simplifying tooling and accelerating iteration.

Resources:
🔹 Interview: https://t.co/PDpu08RoZK
🔹 Video: https://t.co/lLXESgpuAn
🔹 Blog: https://t.co/cW3lb0CFtD
🔹 Technical blog: https://t.co/DCOjE2fepI
🔹 Hugging Face blog: https://t.co/iD202NTCz4
🔹 Model weights: https://t.co/xjYX2C1kT7
🔹 Inference code: https://t.co/fO9T1iNESV

As @JensenHuang has emphasized, open models help advance safety and security. We hope Alpamayo 2 Super will contribute to this vision by enabling researchers and developers around the world to build, experiment, and innovate in autonomous driving.

We are excited to see what the community builds with Alpamayo 2 Super.

@NVIDIADRIVE @NVIDIAAI

[Marco Pavone @drmapavone]

#Alpamayo 2 Super, @nvidia's frontier open reasoning model for autonomous vehicles, is now available!

[Jensen Huang @JensenHuang]

Today, we’re launching Alpamayo 2 Super, our frontier open reasoning model for autonomous vehicles.

Beyond seeing, Alpamayo understands and reasons through the complex world - thinks before it acts.

It’s a powerful backbone for robotaxis, trucks, shuttles, delivery vans, tractors and the long tail of mobile robots—billions of autonomous machines someday.

We’re releasing it for commercial use under OpenMDW-1.1 so teams can inspect it, fine-tune it and deploy it—open models advance safety and security.

The next wave of AI is robotics—and it starts with autonomous vehicles.

Great work, Alpamayo team!

https://t.co/2PYCCXWjZh

[Marco Pavone @drmapavone]

A thoughtful letter highlighting the critical importance of open frontier models — a must-read for researchers, developers, and anyone shaping the future of AI. We built the @nvidia #Alpamayo open platform (https://t.co/m9EsxzUpzl) around the same philosophy: sharing state-of-the-art #Physical #AI models, data, and tools with the community to accelerate innovation and advance the development of safer, more capable #autonomous #vehicles and #Physical #AI systems.

[Jensen Huang @JensenHuang]

For my first post, I’m sharing a letter @NVIDIA signed on why open models matter.

AI will transform every industry, power every company, and be built by every country.

Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty.

The world needs both frontier closed models and frontier open models.

https://t.co/AUKzoQ5Ikb

[Marco Pavone @drmapavone]

Verification is emerging as a new scaling axis for AI!

Scaling pre-training, post-training, and test-time compute have driven much of the recent progress in large language models. Our new work explores a fourth scaling axis: #verification —the ability to determine whether a solution is actually correct.

In LLM-as-a-Verifier, we introduce a general-purpose framework that provides fine-grained feedback across diverse modalities without additional training. We show that three simple ingredients—higher score granularity, repeated evaluation, and criteria decomposition—consistently improve verification performance.

The approach achieves state-of-the-art results across robotics, coding, and medical AI, including RoboRewardBench, Terminal-Bench V2, SWE-Bench Verified, and MedAgentBench.

I'm particularly optimistic about the implications for #Robotics and #PhysicalAI, where verification can serve as a dense reward signal for reinforcement learning, significantly improving the sample efficiency of SAC and GRPO and, in turn, enabling more capable and reliable autonomous systems.

As AI continues to scale, I believe verification will become a foundational capability for building more capable and trustworthy autonomous AI agents.

🌐 Website: https://t.co/wqe9XVhZMo
📄 Paper: https://t.co/8FZWJX7d4Z
💻 Code: https://t.co/JWXgDbkZ4k

Outstanding work led by @jackyk02, in collaboration with @shululi256, @pranav_atreya, @liu_yuejiang, @jyx_su, @chelseabfinn, @istoica05, and @Azaliamirh.

#AI #LLMs #Verification #Reasoning #AgenticAI #Robotics #PhysicalAI #ReinforcementLearning

@StanfordAILab @StanfordEng

[Marco Pavone @drmapavone]

How do we make robot policies robust to rare but high-impact failures?

Video #World #Models (WMs) are rapidly becoming a powerful tool for robotics, enabling policy evaluation and improvement by "imagining" future outcomes. But there's a catch: these imagined futures are typically nominal samples, making it easy to overlook the rare yet safety-critical events that matter most.

In our new paper, StressDream: Steering Video World Models for Robust Policy Evaluation and Improvement, we explore a simple but powerful idea:

💡 Instead of passively sampling futures, actively steer world model imaginations toward high-impact yet still plausible scenarios.

StressDream optimizes the initial diffusion noise at inference time, allowing us to generate targeted stress-test scenarios without retraining the world model. This enables:
- More robust policy evaluation by exposing failure modes that random sampling often misses.
- Improved policy optimization by training against challenging but realistic imagined futures.

As generative world models become a foundation for #Physical #AI, the ability to systematically probe their "long tail" of plausible futures will be increasingly important for building reliable and trustworthy autonomous systems.

📌 𝖯𝗋𝗈𝗃𝖾𝖼𝗍 𝖯𝖺𝗀𝖾: https://t.co/0BSM0d0X0n
📄 𝖯𝖺𝗉𝖾𝗋: https://t.co/JQ3Ij4SZWY

Work led by Junwon Seo, with a great set of collaborators: Sushant Veer, Thomas Ran Tian, Wenhao Ding, Apoorva Sharma, Karen Leung, Edward Schmerling, Andrea Bajcsy.

@NVIDIADRIVE @NVIDIAAI

#Robotics #WorldModels #PhysicalAISafety #AISafety #AutonomousSystems #RobotLearnin

[Marco Pavone @drmapavone]

Introducing ConstrainedMimic (https://t.co/v9IdS19wQP) — a control framework for #humanoid robot #safety that enables real-time constraint enforcement within #RL-based motion tracking policies by leveraging whole-body kinematics and dynamics.

Recent advances in reinforcement learning have unlocked remarkable whole-body agility for humanoid robots. However, ensuring safety and satisfying constraints—especially those introduced after training—remains a significant challenge for deploying safe and reliable systems.

ConstrainedMimic addresses this challenge by combining ideas from operational space control and control barrier functions (CBFs). The framework enables enforcement of arbitrary runtime constraints while preserving the ability of the policy to track complex motions. Importantly, constraints can be imposed on both the kinematic reference motion and the underlying robot dynamics, providing a principled approach to safer, more robust, and more controllable humanoid behavior.

As #PhysicalAI, #humanoid #robotics, and #embodied #AI systems move from research environments into the real world, the ability to guarantee safety and respect operational constraints will become increasingly important — ConstrainedMimic is a step in this direction.

📄 Paper: https://t.co/v9IdS19wQP
💻 Code: Coming soon

Great work led by @danielpmorton .

#PhysicalAI #AISafety #HumanoidRobotics #EmbodiedAI #ReinforcementLearning #Robotics

@StanfordAILab @StanfordEng

[Marco Pavone @drmapavone]

Today we're releasing @nvidia AlpaGym, our new open-source reinforcement learning (RL) framework for end-to-end autonomous driving.

A key challenge for #Physical #AI is enabling policies to learn from the consequences of their actions. While supervised learning can teach a model to imitate behavior, robust autonomy ultimately requires learning through interaction with the environment.

AlpaGym enables exactly that.

Built on top of:
- AlpaSim: our high-fidelity closed-loop autonomous driving simulator
- Cosmos-RL: NVIDIA's distributed RL training and rollout infrastructure

AlpaGym provides the glue that connects simulation, training, and driving policies into a scalable framework for post-training autonomous vehicle models in closed loop.

With AlpaGym, researchers and developers can:
✅ Train end-to-end driving policies using reinforcement learning
✅ Run large-scale closed-loop simulations
✅ Experiment with new reward functions, policy architectures, and training strategies
✅ Benchmark models on public leaderboards

📖 Learn how it works: https://t.co/9iJYY1bSMq

💻 GitHub: https://t.co/j5Zuu4af0n

🏆 Open Challenges:
- AlpaSim Closed-Loop E2E Driving Challenge: https://t.co/fR60fwcDQT
- Physical AI AV Reasoning Challenge: https://t.co/3L7TGnV4R2

Learn more about the #Alpamayo open platform:
https://t.co/Xg0LYaEd9H

#PhysicalAI #AutonomousDriving #ReinforcementLearning #Robotics #OpenSource #NVIDIA #MachineLearning

@NVIDIADRIVE @NVIDIAAI

[Marco Pavone @drmapavone]

On the power of data flywheels in Physical AI

@nvidia recently introduced Cosmos 3, a family of omnimodal world models designed to jointly process and generate language, images, video, audio, and actions within a unified mixture-of-transformers architecture.

One aspect I find particularly exciting is the data flywheel emerging between #Cosmos 3 and #Alpamayo 2.

Cosmos 3 was trained using data generated and curated in part through the Alpamayo ecosystem. In turn, the next generation of Alpamayo will build on Cosmos 3's capabilities. This creates a virtuous cycle in which better models enable better data generation, and better data leads to even stronger models.

Much of the attention in #AI is naturally focused on model architectures and benchmark results. Yet, in robotics and autonomous systems, development processes matter just as much as models themselves. Robust data flywheels are increasingly becoming a defining characteristic of state-of-the-art robot autonomy stacks.

Further reading:
- Cosmos 3: https://t.co/WB3PK9b2Iz
- Alpamayo 2: https://t.co/Xg0LYaEd9H

Join me on June 16 at 9:00 AM PT for a livestream on Alpamayo 2 Super: The Open Reasoning Model for Robotaxis: https://t.co/hyU6K4LKk4. We'll showcase brand-new elements of the open pipeline—from real-world fleet data to model training recipes to closed-loop development with simulation. If you're building toward L4 autonomy, I think you'll find it worthwhile.

I'll be joined by @iamborisi @YurongYou @yan_wang_9 @MaxiIgl

Looking forward to seeing you there.

@NVIDIADRIVE @NVIDIAAI

[Marco Pavone @drmapavone]

How much time should robots spend thinking?

Vision-Language Models are increasingly used as high-level planners for robots, and the prevailing strategy has been to scale test-time compute to boost capability. But more reasoning steps, bigger models, and longer memory all come with increased latency, tokens, and FLOPs—often with diminishing and uneven returns.

So when, and where, is test-time compute actually worth its cost? 🧐

We study three dominant scaling axes and find that each unlocks a distinct capability, showing that test-time compute is not a uniform lever:

- Chain-of-thought depth helps with tasks involving implicit semantic, physical, or spatial constraints, but its additional latency is not always necessary (on VLABench, a non-CoT model matches a CoT model on 44% of tasks).

- Model size governs the breadth of skills a planner can reliably draw upon, but its benefits appear only when those additional skills are actually required.

- Memory history improves performance on long-horizon, history-dependent tasks, but can actively hurt performance elsewhere.

Across all three axes, a consistent pattern emerges: the gap between cheap and expensive configurations is large, but highly non-uniform and task-dependent.

DIRECT (Dynamic Inference Router for Embodied Compute Tradeoffs) is a lightweight router that reads scene + instruction context and sends each task to the cheapest planner that can still solve it, allocating compute per task rather than committing to one fixed model.

👉 Takeaway: smart allocation of test-time compute can recover frontier-level planning at a fraction of the cost.

📄 Paper: https://t.co/H11V7q4zGj
🔗 Website: https://t.co/Es9RJaXE0o

Work led by @_jadelynn @milanganai

With an outstanding team of collaborators: @ajaysridhar0 @Mozhgan_nasr @katielulula Clark Barrett @jiajunwu_cs @chelseabfinn

#Robotics #VLM #EmbodiedAI #MachineLearning #TestTimeCompute

[NVIDIA DRIVE @NVIDIADRIVE]

The Alpamayo Summit at CVPR brought together AV researchers and industry leaders together under one room.

Hear from Marco Pavone (@drmapavone), Senior Director of Autonomous Vehicle Research, and other NVIDIA experts on how Alpamayo is accelerating AV development.

📺 Watch the on-demand replay: https://t.co/iE10sqlHbA

[Marco Pavone @drmapavone]

I look forward to participating in the Verification Summit (https://t.co/wVitrsgYfq) and sharing my perspective on Physical AI safety. I strongly agree that verification and validation are key frontiers for unlocking Physical AI in high-stakes, high-reliability applications, from autonomous cars to industrial robotics!

@fv_summit @khoslaventures @PramaanaLabs @boldcapfund

[Vinod Khosla @vkhosla]

I think this will turn out to be a very important area. Founders should work on things AI is not good at.

[Marco Pavone @drmapavone]

Excited to share the latest expansion of the @nvidia #Alpamayo open platform for reasoning-based autonomous vehicles.

Since its launch earlier this year, Alpamayo has seen rapid adoption across industry and academia, with its reasoning models surpassing 400,000 downloads and earning a #COMPUTEX 2026 Best Choice Award.

As announced by Jensen Huang during his #COMPUTEX keynote, we are now introducing several major additions designed to accelerate the development of next-generation AV systems (more details here: https://t.co/Xg0LYaEd9H):

🚗 Alpamayo 2 Super — a new 32B-parameter driving foundation model with:
• Full 360° surround-view perception
• Advanced reasoning capabilities and chain-of-causation outputs
• Meta-actions such as lane changes, yielding, and stopping
• Reasoning auto-labeling and visual grounding for scalable data annotation
• State-of-the-art performance across reasoning, prediction, and alignment tasks

🔄 AlpaGym — an open-source framework for closed-loop reinforcement learning, enabling AV models to learn from the consequences of their actions in simulation and helping bridge the gap between training and real-world deployment.

📊 New Open Benchmarks — including challenges for closed-loop driving and long-tail reasoning to help the community measure progress and drive innovation.

🛠️ Alpamayo Recipes — a centralized repository of end-to-end workflows covering supervised fine-tuning, reinforcement learning, quantization, and model customization.

Reasoning models and closed-loop training are becoming foundational technologies for autonomous systems. Our goal is to provide the open tools, models, infrastructure, and benchmarks needed to accelerate progress across the entire AV ecosystem.

A huge thank you to the many researchers, engineers, and community members whose feedback helped shape this release.

Resources:
• Overview of the latest Alpamayo release (note: some components will be released over the coming weeks): https://t.co/Xg0LYaEd9H
• @nvidia announcement: https://t.co/zWC2rbDK2W

#AutonomousVehicles #PhysicalAI #Robotics #AI #MachineLearning #ReinforcementLearning #OpenSource #NVIDIA #Alpamayo

@NVIDIADRIVE @NVIDIAAI