Jonah Philion

AI Foundations @Waymo
CS PhD @UofT '25, @harvardphysics/@harvardmath '18

[Jonah Philion @PhilionJonah]

"Trajeglish: Learning the Language of Driving Scenarios" w/ @xbpeng4 @FidlerSanja

Discrete sequence modeling for controlling interactive agents in self-driving simulation

@nvidia @VectorInst @UofTCompSci @SFU

https://t.co/2hWgJmgzgi
https://t.co/VJUEM1lR5x

1/6

[Goodfire @GoodfireAI]

Neural networks might speak English, but they think in shapes.

Understanding their rich *neural geometry* is key to understanding how they work – and to debugging and controlling them with precision.

Starting today, we’re releasing a series of posts on this research agenda. 🧵

[Zan Gojcic @ZGojcic]

Data in the cornestone of everything!

We've used NCore across many internal research/product efforts and now it is finally public.

Canonical representation, easy-to-use APIs, random access, streaming, faster than WebDataset and HDF5, pip installable!

https://t.co/UQqBbGx0QF

[Zan Gojcic @ZGojcic]

A new generation in AV simulation is here!

We are announcing AlpaDreams, a real time interactive generative world model for AV simualtion! Just a year ago it took minutes to generate a few seconds of video, today it is real time and interactive!

https://t.co/FbhKu3PMqe

[Tekedra N Mawakana @TechTekedra]

A new era for Waymo. We’ve raised $16B to accelerate our mission, valuing the company at $126B. This capital is an investment in a future where more cities get a safer, more reliable way to move. Let's go. 🚀 https://t.co/UPBroOeWcR

[Dmitri Dolgov @dmitri_dolgov]

With the $16B in new funding, we’re accelerating the deployment of the @Waymo Driver - the world’s most advanced physical-world AI. With nearly 200M fully-autonomous miles under our belt and scaling exponentially, we’re just getting started… trillions of miles and safer roads ahead!

https://t.co/Uay1Ql7esy

[Yiding Jiang @yidingjiang]

Information theory often gives unintuitive conclusions when it comes to data. Many of these inconsistencies can be resolved elegantly if we limit the amount of computation the observers can use. Very happy to finally introduce our work on epiplexity! 1/🧵

[Jeff Dean @JeffDean]

Exciting expansion! @Waymo now serves the whole SF Bay Area Peninsula from SF to San Jose and is taking riders on freeways.

https://t.co/fNgqQtHB7b

[Andrej Karpathy @karpathy]

Excited to release new repo: nanochat!
(it's among the most unhinged I've written).

Unlike my earlier similar repo nanoGPT which only covered pretraining, nanochat is a minimal, from scratch, full-stack training/inference pipeline of a simple ChatGPT clone in a single, dependency-minimal codebase. You boot up a cloud GPU box, run a single script and in as little as 4 hours later you can talk to your own LLM in a ChatGPT-like web UI.

It weighs ~8,000 lines of imo quite clean code to:

- Train the tokenizer using a new Rust implementation
- Pretrain a Transformer LLM on FineWeb, evaluate CORE score across a number of metrics
- Midtrain on user-assistant conversations from SmolTalk, multiple choice questions, tool use.
- SFT, evaluate the chat model on world knowledge multiple choice (ARC-E/C, MMLU), math (GSM8K), code (HumanEval)
- RL the model optionally on GSM8K with "GRPO"
- Efficient inference the model in an Engine with KV cache, simple prefill/decode, tool use (Python interpreter in a lightweight sandbox), talk to it over CLI or ChatGPT-like WebUI.
- Write a single markdown report card, summarizing and gamifying the whole thing.

Even for as low as ~$100 in cost (~4 hours on an 8XH100 node), you can train a little ChatGPT clone that you can kind of talk to, and which can write stories/poems, answer simple questions. About ~12 hours surpasses GPT-2 CORE metric. As you further scale up towards ~$1000 (~41.6 hours of training), it quickly becomes a lot more coherent and can solve simple math/code problems and take multiple choice tests. E.g. a depth 30 model trained for 24 hours (this is about equal to FLOPs of GPT-3 Small 125M and 1/1000th of GPT-3) gets into 40s on MMLU and 70s on ARC-Easy, 20s on GSM8K, etc.

My goal is to get the full "strong baseline" stack into one cohesive, minimal, readable, hackable, maximally forkable repo. nanochat will be the capstone project of LLM101n (which is still being developed). I think it also has potential to grow into a research harness, or a benchmark, similar to nanoGPT before it. It is by no means finished, tuned or optimized (actually I think there's likely quite a bit of low-hanging fruit), but I think it's at a place where the overall skeleton is ok enough that it can go up on GitHub where all the parts of it can be improved.

Link to repo and a detailed walkthrough of the nanochat speedrun is in the reply.

[Jonah Philion @PhilionJonah]

@ZGojcic @sergioksas please feel free to try again next year

[Jonah Philion @PhilionJonah]

We'll host 2 interns next summer on @sergioksas's team at Waymo next summer, one of them hosted by myself.

Apply if you'd like to work on problems at the very frontier of ML-based planning for L4 self-driving!

https://t.co/qc7ve2zJEA
https://t.co/gMDzWBobNJ

[Jonah Philion @PhilionJonah]

@ZGojcic @sergioksas haha sorry @ZGojcic, we will not be accepting applications for these internships from senior managers at this time! ;)

[Ethan Teicher @ethanteicher]

96M miles of @Waymo safety data just dropped

[Waymo @Waymo]

All systems go at @flySFO! We’ve been approved by the airport to begin operations, and will start testing soon. More here: https://t.co/0SGUdcl3sj

[Jiahui Huang @huangjh_hjh]

[1/N] 🎥 We've made available a powerful spatial AI tool named ViPE: Video Pose Engine, to recover camera motion, intrinsics, and dense metric depth from casual videos!
Running at 3–5 FPS, ViPE handles cinematic shots, dashcams, and even 360° panoramas.
🔗 https://t.co/1mGDxwgYJt

[Chenfeng_X @Chenfeng_X]

📢 Excited to sharing a little late update (before it is no longer news): I’ll be joining @UTAustin @UTCompSci as an Assistant Professor! I'm recruiting PhD students from @UTCompSci in the Fall 2025 cycle and also looking for RAs/interns! More info see https://t.co/JPDhVplhJX

[Alexander Wei @alexwei_]

1/N I’m excited to share that our latest @OpenAI experimental reasoning LLM has achieved a longstanding grand challenge in AI: gold medal-level performance on the world’s most prestigious math competition—the International Math Olympiad (IMO).

[Zan Gojcic @ZGojcic]

Had a great time chatting with @sopharicks and the @buZZrobot community about our recent work, DiffusionRenderer, and the exciting research my team is doing at @NVIDIAAI!

DiffusionRenderer project page: https://t.co/Y4O4el21m6

[Waymo @Waymo]

We’re setting the course for what’s next. 🚙 Waymo has earned a place on TIME100 Most Influential Companies for 2025.

@TIME’s new cover features our co-CEOs @dmitri_dolgov and @techtekedra who lead our team as we pursue our mission to be the world’s most trusted driver.

Read the full story: https://t.co/rIsqRgcsNZ

Photograph by Kelsey McClellan for TIME.

[Eugene Vinitsky 🦋 @EugeneVinitsky]

We now know RL agents can zero-shot crush driving benchmarks. Can we put them on a car and replace the planning stack? We're hiring a postdoc at NYU to find out!
Email me if interested and please help us get the word out.

[Huan Ling @HuanLing6]

We are excited to share Cosmos-Drive-Dreams 🚀
A bold new synthetic data generation (SDG) pipeline powered by world foundation models—designed to synthesize rich, challenging driving scenarios at scale.

Models, Code, Dataset, Tookit are released.
Website: https://t.co/j9iQDMWMwm