Lechao Xiao

Training @OpenAI & BK. Prev @GoogleDeepMind @GoogleBrain.
Tackle scaling, along the path to AGI.

[Lechao Xiao @Locchiu]

@SavinovNikolay Welcome!

[Lechao Xiao @Locchiu]

@BlackHC -1 ; + open source option

[Lechao Xiao @Locchiu]

@Yuchenj_UW how do you define solve math ? Math is fine art though

[Lechao Xiao @Locchiu]

@_arohan_ It was a fun time. We pipelined 8 gpu (k20?) , yolo one cifar10 run, babysit for a month(!) via tensorboard. Now it feels scary, multi epoch, no skip connection , sgd, no clipping, no weight decay

[Lechao Xiao @Locchiu]

It was a record (still now?) https://t.co/uoHat6IxKp

[rohan anil @_arohan_]

Who has the deepest network of them all?

[Lechao Xiao @Locchiu]

This is going to be a super fun workshop on scaling & learning dynamics & optimization! Please consider submitting your best work there!

[Elliot Paquette @poseypaquet]

The High-Dimensional Learning Dynamics Workshop @ ICML 2026 🇰🇷, with a special focus on scaling laws, is coming up, July 10!
Submission Deadline: May 11 AoE (extended).

@Locchiu @albertobietti @JustinLin610 @inbarser @BachFrancis @ShamKakade6 @andrewgwils @blake__bordelon @_brloureiro @Qiuzihanhan

[Lechao Xiao @Locchiu]

@_arohan_ @fouriergalois @JeffDean Jax

[Lechao Xiao @Locchiu]

@JustinLin610 Best wishes for your next adventure!

[Lechao Xiao @Locchiu]

@hyhieu226 @OpenAI @xai Take care Hieu!

[Lechao Xiao @Locchiu]

@JCHaswell @TheGregYang That’s a really tough shift, especially you were in great shape before.

[Lechao Xiao @Locchiu]

@TheGregYang In my case, it correlates with exercise / sleep quite well ; exercise helps to improve both. Magnesium also boost hrv for 3-5 pts for me. No alcohol also helps 😂

[Lechao Xiao @Locchiu]

The originality and the depth of science are really impressive. High thinking, signal to flop ratio. Congrats Damien, Elliot, Courtney et al.

[Damien Ferbach @damien_ferbach]

1/10 We built ADANA, an optimizer that gets better as you scale.

It extends AdamW with log-time schedules for momentum and weight decay — same hyperparameter count, no extra engineering. Scaled from 45M to 2.6B, it saves ~40% compute vs tuned AdamW, and the gap keeps growing.🧵

[Lechao Xiao @Locchiu]

@Jianlin_S Thanks, Jianlin! Hope you have a wonderful Chinese New Year!

[trieu @thtrieu_]

Mathematicians 🤝AI researchers https://t.co/5AvHC0Stin. Our take on AI solving Erdos problems:
* Many "Open" problems are actually just obscure: many cases the AI didn't find something new, only rediscovered solutions buried in the literature. We present our systematic approach to reporting AI results on Erdos.
* The real bottleneck is still human labor, e.g. we spent lots of time filtering out technically correct but meaningless solutions (AI missed Erdos’s original intent).
* Acceleration in solving low-hanging fruits is real, but we also need to highlight the many more misses that require human auditing. Clear research directions ahead though, and we feel optimistic about drastically increasing the signal-to-noise ratio.
More to come!

[Lechao Xiao @Locchiu]

@peterjliu It feels like the beginning of vibe proof. With human in the loop as verifiers.

[Lechao Xiao @Locchiu]

@peterjliu ”My preference would still be for the final writeup for this result to be primarily human-generated in the most essential portions of the paper, though I can see a case for delegating routine proofs to some combination of AI-generated text and Lean code. But to me, the ... ”

[Lechao Xiao @Locchiu]

@roydanroy Welcome!

[Lechao Xiao @Locchiu]

@_arohan_ All-in-one shampoo is all you need

[Amr Khalifa @AmrMAlameen]

I am hiring a student researcher to work with our team in Montreal on LLMs architecture and pre-training in spring-summer 2026, if you're excited to push the frontier of research forward, join us to help keeping the TPUs warm.
fill out this form:
https://t.co/q8R3tPJkSV