Alex Naumann

Deep learning for autonomous vehicles at @nvidia

[Jensen Huang @JensenHuang]

Today, we’re launching Alpamayo 2 Super, our frontier open reasoning model for autonomous vehicles.

Beyond seeing, Alpamayo understands and reasons through the complex world - thinks before it acts.

It’s a powerful backbone for robotaxis, trucks, shuttles, delivery vans, tractors and the long tail of mobile robots—billions of autonomous machines someday.

We’re releasing it for commercial use under OpenMDW-1.1 so teams can inspect it, fine-tune it and deploy it—open models advance safety and security.

The next wave of AI is robotics—and it starts with autonomous vehicles.

Great work, Alpamayo team!

https://t.co/2PYCCXWjZh

[Marco Pavone @drmapavone]

🚗 Imitation learning is everywhere—but is it enough?

So far, imitation learning—most commonly via behavior cloning (BC)—remains the go-to approach for training real-world autonomous vehicle (AV) driving policies. Yet BC operates in an open-loop (OL) fashion, overlooking the critical interdependence among inputs, outputs, and future states that comes with closed-loop (CL) operation. The result? The notorious—but often overlooked—OL–CL gap ⚠️

To address this challenge and encourage broader adoption of CL techniques, we’ve just published a survey (https://t.co/zLPiF17QmW) presenting a comprehensive taxonomy of closed-loop training methods for end-to-end driving. Our framework organizes approaches along three key axes:
- Action generation
- Environment response generation
- Training objectives

💡 Bottom line: enabling technologies—like neural rendering, generative world models, and scalable RL—have now matured, making closed-loop AV training ready for wide-scale adoption.

We’d love to hear your thoughts—drop a comment and join the discussion! 💬

And as a reminder, we are hiring for full-time research scientist and research engineer positions:
🔹 [Sr.] Research Scientist: https://t.co/D4Z0xLEGzv
🔹 [Sr.] Research Engineer: https://t.co/5yCpiDJCWA

@NVIDIADRIVE @NVIDIAAI @nvidia

[Marco Pavone @drmapavone]

How can we best use LLMs in an autonomy stack? An exciting prospect is to exploit their generalist experience to reason about anomalies. And one can do this in real time by leveraging their embeddings in a fast&slow decision making architecture. Work led by @RohanSinhaSU #RSS2024

[Marco Pavone @drmapavone]

Can we use NeRFs in the wild? Introducing DistillNeRF, a framework for *generalizable* 3D scene representation prediction from sparse multiview image inputs, using distillation from per-scene optimized NeRFs and visual foundation models. https://t.co/TkuwrZG2zX @NVIDIAAI

[Marco Pavone @drmapavone]

At @NVIDIAGTC, I presented my group's strategy on leveraging foundation models (FMs) to develop next-gen autonomous vehicles. Slides: https://t.co/5FPHxJGFeh Recording: https://t.co/Izu0iNwuN1 Three pillars: standing up the FMs, using them within an AV program, and AI safety.

[Maximilian Igl @MaxiIgl]

#CVPR2024 Workshop on AV Simulation!

Simulation is a crucial tool to accelerate the development of safe autonomous driving (AD).

In our full-day CVPR workshop with exciting speakers from Waymo, Tesla, Waabi, Wayve, Nuro, Princeton and many others, we’re bringing together 1/N

[Alex Naumann @AlexN4u]

@GMFarinella It's up again and everything worked fine, thanks a lot!

[OpenAI @OpenAI]

Our newest system DALL·E 2 can create realistic images and art from a description in natural language. See it here: https://t.co/Kmjko82YO5

[Alex Naumann @AlexN4u]

Thanks for the cool collaboration @paperswithcode. Looking forward to feedback and contributions :)

[Papers with Code @paperswithcode]

✨ AI Conference Deadlines gets a new look!

We also added a calendar view to easily track and discover upcoming AI conference deadlines.

Check it out here: https://t.co/TlmyQrgNlu

[Papers with Code @paperswithcode]

⏪ Papers with Code: Year in Review

We’re ending the year by taking a look back at the top trending machine learning papers, libraries and new datasets for 2021. Read on below!

https://t.co/2tUWUgrZKp

[🍉 Abubakar Abid @abidlabs]

Thanks to the more than 3000 people who have starred the Gradio GitHub repo ✨ 💫

If you haven’t already done so, consider giving us a quick star to help our free, open-source library get more visible! https://t.co/Y7aUFssobO

[GitHub @github]

Meet GitHub Copilot - your AI pair programmer. https://t.co/eWPueAXTFt

[Petar Veličković @PetarV_93]

Proud to share our 150-page "proto-book" with @mmbronstein @joanbruna @TacoCohen on geometric DL! Through the lens of symmetries and invariances, we attempt to distill "all you need to build the architectures that are all you need".

https://t.co/CBN0IG8BXR

More info below! 🧵

[PyTorch @PyTorch]

Introducing VISSL (https://t.co/iBEpmCi09R) - a library for reproducible, SOTA self-supervised learning for computer vision! Over 10 methods implemented, 60 pre-trained models, 15 benchmarks, and counting.

[Ben Recht @beenwrekt]

I’m excited to share a new textbook @mrtz and I wrote: "Patterns, Predictions, and Actions: A Story about Machine Learning." 1/5

https://t.co/3gCH2afaYM

[Sebastian Raschka @rasbt]

"Datasets for Machine Learning and Deep Learning -- Some of the Best Places to Explore" I just put together a list of all the dataset repositories we discussed on Twitter last month: https://t.co/tiwSj76kEz

[Connected Papers @ConnectedPapers]

Connected Papers partners with @arxiv!
We're excited to announce that starting today, every abstract page on https://t.co/kjItHiwGHF will link to a graph of Connected Papers.

Read more:
https://t.co/r2eD6nTyL4

[Papers with Code @paperswithcode]

🎉 Introducing... Datasets! We are now indexing 3000+ research datasets from machine learning. Find datasets by task and modality, compare usage over time, browse benchmarks, and much more! Explore the catalogue here: https://t.co/sST84xRiSd