Xinyang (Young) Geng

Research scientist at Google DeepMind. Opinions are my own.

[Xinyang (Young) Geng @younggeng]

Are you interested in training large models in JAX but are set back by the complicated partition specs and sharding configurations required to scale up? I've recently created scalax, a small library to help developers easily scale up JAX models. https://t.co/xqcxO1qavy

[Xinyang (Young) Geng @younggeng]

@hyhieu226 @OpenAI @xai Best wishes for you and your family! Hope you'll recover soon.

[Jacob Austin @jacobaustin132]

Today we're putting out an update to the JAX TPU book, this time on GPUs. How do GPUs work, especially compared to TPUs? How are they networked? And how does this affect LLM training? 1/n

[Jack Rae @jack_w_rae]

2.5 Flash is out!

You can now specify thinking budgets, or disable thinking entirely for lower latency.

Strong code & reasoning capabilities, cost effective, fast.

It's a great workhorse model for enterprise and developers, excited to hear your feedback.

[rdyro @rdyro128523]

Llama 4 inference in pure JAX! Expert/tensor parallelism with int8 quantization. Contributions welcome!

[Tianhe (Kevin) Yu @TianheYu]

The team has built such a smart model. The gap becomes bigger on harder problems!

[Jack Rae @jack_w_rae]

Today we are launching 2.5 Pro!

I think it's the best model in the world. State-of-the-art reasoning and great vibes (+39 ELO gap on lmsys!)

2.5 Pro improves in coding, stem, multimodal, instruction following, and lots more.

Available in AI Studio & the Gemini App!

[Xinyang (Young) Geng @younggeng]

@LiamFedus Congrats Liam!

[rdyro @rdyro128523]

Deepseek R1 inference in pure JAX! Currently on TPU, with GPU and distilled models in-progress. Features MLA-style attention, expert/tensor parallelism & int8 quantization. Contributions welcome!

[Jacob Austin @jacobaustin132]

Making LLMs run efficiently can feel scary, but scaling isn’t magic, it’s math! We wanted to demystify the “systems view” of LLMs and wrote a little textbook called “How To Scale Your Model” which we’re releasing today. 1/n

[Hieu Pham @hyhieu226]

Despite many complaints about Jax being hard to use, it has a crucial advantage over PyTorch: for distributed jobs, XLA is sufficiently good at auto-scheduling parallelism strategies, e.g., sharding, overlapping compute and comms. If PyTorch becomes good at that, it's checkmate.

[Andrej Karpathy @karpathy]

For friends of open source: imo the highest leverage thing you can do is help construct a high diversity of RL environments that help elicit LLM cognitive strategies. To build a gym of sorts. This is a highly parallelizable task, which favors a large community of collaborators.

[Jim Fan @DrJimFan]

Whether you like it or not, the future of AI will not be canned genies controlled by a "safety panel". The future of AI is democratization. Every internet rando will run not just o1, but o8, o9 on their toaster laptop. It's the tide of history that we should surf on, not swim against. Might as well start preparing now.

DeepSeek just topped Chatbot Arena, my go-to vibe checker in the wild, and two other independent benchmarks that couldn't be hacked in advance (Artificial-Analysis, HLE).

Last year, there were serious discussions about limiting OSS models by some compute threshold. Turns out it was nothing but our Silicon Valley hubris. It's a humbling wake-up call to us all that open science has no boundary. We need to embrace it, one way or another.

Many tech folks are panicking about how much DeepSeek is able to show with so little compute budget. I see it differently - with a huge smile on my face. Why are we not happy to see *improvements* in the scaling law? DeepSeek is unequivocal proof that one can produce unit intelligence gain at 10x less cost, which means we shall get 10x more powerful AI with the compute we have today and are building tomorrow. Simple math! The AI timeline just got compressed.

Here's my 2025 New Year resolution for the community:

No more AGI/ASI urban myth spreading.
No more fearmongering.
Put our heads down and grind on code.
Open source, as much as you can.

Acceleration is the only way forward.

[Andrej Karpathy @karpathy]

It’s done because it’s much easier to 1) collect, 2) evaluate, and 3) beat and make progress on. We’re going to see every task that is served neatly packaged on a platter like this improved (including those that need PhD-grade expertise). But jobs (even intern-level) that need long, multimodal, coherent, error-correcting sequences of tasks glued together for problem solving will take longer. They are unintuitively hard, in a Moravec’s Paradox sense.

Fwiw I’m ok and happy to see harder “task” evals. Calling it humanity’s last exam is a bit much, and misleading.

[Jerry Tworek @MillionInt]

Simplify.
Scale.
Resolve bottlenecks.
Repeat.

[Jack Rae @jack_w_rae]

Appreciate @aidan_mclau looking into the thinking model results. Originally scores looked weak as the response was plucked from the thought content versus output.

We are looking into ways of making thinking output less confusing for people running evals.

This is why we 🚢, to collect feedback and iterate!

[Xinyang (Young) Geng @younggeng]

Very excited to share the thinking model! It has been a lot of fun working on this.

[Jack Rae @jack_w_rae]

We released Gemini 2.0 Flash Thinking today! ⚡️🤔

It's a small step towards improved reasoning via inference-time compute, built on top of our small and mighty 2.0 Flash!

[Jerry Tworek @MillionInt]

People completely misunderstand the data wall.

It's the data slop wall. Most of the data is so bad it's a waste of a good gpu to backprop it.

[Charlie Snell @sea_snell]

Can we predict emergent capabilities in GPT-N+1🌌 using only GPT-N model checkpoints, which have random performance on the task?

We propose a method for doing exactly this in our paper “Predicting Emergent Capabilities by Finetuning”🧵

[Cristian Garcia @cgarciae88]

People learning JAX, feel free to reach out if the learning feels too steep, hopefully we can flatten it out.

Also, checkout the JAX LLM for help from the community: https://t.co/GX18E3MHvf

[Ayaka Mikazuki (Keep4o) @ayaka14732]

We finally have an official `nvidia-smi` for TPU 🎉

Simply install it with `pip install tpu-info`