Gary Wang

Speech, TTS & Gemini @google. Ex chemical Engineer (fuel cells, batteries, oil & gas). Now speech Synthesis, ASR, RL, NLP.

[Gary Wang @garygarywang]

Some fun TTS samples for the current US political climate: https://t.co/dC33BXgJtL

[Yu Zhang 🐙🌘 @yzhang_cs]

In the early morning, @nathancgy4, @Xinyu2ML, @Yulun_Du and I were preparing some showcases for the blog while watching the World Cup. The moment Argentina beat England, I felt something. I looked up, and saw the most unforgettable Beijing sunrise. So I took the photo.

I knew this was no ordinary day, and now it comes and says, "Hello, world!"

Happy hatch day, K3!

Please enjoy it, I have felt 「眩晕瘫坐」 for weeks.
It is a monster model, yet clean in architecture; elegant in optimization, beautiful in MoE desigins.
It has crossed a certain singularity, it is a year of work, in one release.

🚀🚀🚀 This is a Sputnik moment for OSS.

[Russ Salakhutdinov @rsalakhu]

Congratulations to Zhilin Yang, founder and CEO of @Kimi_Moonshot, on the latest Kimi release. What a huge win for the open-source community!

It feels like just yesterday Zhilin was graduating from my lab at CMU, jointly co-advised with William Cohen. Not only did he complete his Ph.D. in just four years, but he also made truly fundamental contributions to ML during his time at CMU.

What a spectacular career! Congrats again Zhilin, and thank you and the entire Kimi team for everything you're doing for the open-source community.

[Kimi.ai @Kimi_Moonshot]

Introducing Kimi K3: Open Frontier Intelligence

🔹 2.8 Trillion Parameters, 1 Million Context, Native Multimodal
🔹 Kimi Delta Attention enables up to 6.3x faster decoding in million-token contexts
🔹 Attention Residuals deliver ~25% higher training efficiency at <2% additional cost
🔹 Built for long-horizon agentic coding and self-evolving workflows

Kimi K3 is now live on on https://t.co/zrk6zZxZUo, Kimi Work, Kimi Code, and the Kimi API.
Open Weights by July 27, 2026.

🔗 API: https://t.co/XCrgjXAqMw
🔗 Tech blog: https://t.co/YTfiMSNM1f

[Thinking Machines @thinkymachines]

Today, we are introducing Inkling.

Inkling reasons efficiently across text, image, and audio modalities. We are making the full weights available.

https://t.co/Ghebq5mG30

Available today for fine-tuning on Tinker. Play with it in the Inkling Playground. 🧵

[Artificial Analysis @ArtificialAnlys]

Thinking Machines has released Inkling, the new leading U.S. open weights model, debuting at 41 on the Artificial Analysis Intelligence Index

@thinkymachines has previously released research previews of models and this is their first production language model release. The model is 975B total parameters, has 41B active parameters, and accepts text, image, and audio input modalities. The model is accessible via Thinking Machines’ Tinker platform API (256K context window) and weights are available on HuggingFace (1M context window).

Key results:

➤ Inkling debuts at 41 on the Artificial Analysis Intelligence Index, making it the leading open weights release from a U.S. lab. Inkling scores 3 points higher on the Intelligence Index (41) than the previous leading U.S. open weights model, Nemotron 3 Ultra (38), and also beats Gemma 4 31B (29) and gpt-oss-120b (24)

➤ Inkling stands out on agentic performance. It scores higher than both Kimi K2.6 and DeepSeek v4 Flash on both GDPval-AA v2 and 𝜏³-Banking: Inkling scores an Elo of 1238 on GDPval-AA v2, higher than Kimi K2.6 (1190) and DeepSeek v4 Flash max (1189) and scores 24% on 𝜏³-Banking, higher than Kimi K2.6 (21%) and just above DeepSeek v4 Flash max (23%)

➤ Inkling is token efficient compared to open weights leaders. Inkling averages 25K output tokens per Intelligence Index task compared to 43K, 38K and 37K by GLM-5.2 (max), Kimi K2.6 and DeepSeek v4 Pro (max) respectively

➤ Inkling natively supports image and audio multimodal inputs, a key differentiator among open weights models. Inkling accepts text, image, and audio input modalities. Images and videos are encoded via a hierarchical patch encoder and audio via discrete token encoding, with all modalities projected into a shared hidden space and processed jointly by the decoder

Additional model details:

➤ Size: 975B (41B active) parameters

➤ Input modalities: Text, image, and audio (text output)

➤ Context window: 256K tokens on Tinker, open weights model supports 1M

➤ Pricing per 1M tokens (64K context window): $1.87 input / $0.374 cached / $4.68 output

➤ Pricing per 1M tokens (256K context window): $3.74 input / $0.748 cached / $9.36 output

[Huan-ang Gao @c7wc7w]

RLVR is powerful, but repeating it for every larger target model is expensive: each target must generate its own rollouts and rediscover useful learning signals from sparse outcome rewards.

Can RL on a small, weaker model improve a stronger student—even when the student already outperforms the small model after RL?

We found that it can—but not by distilling the weak model itself.

Today, we’re excited to share Direct-OPD, joint work w/ @Shiyuan040223, @c7wc7w, @Ahydchh, @Han_lin_Wu, @zhilong_zhang26, Zheng Jiang, @HBX_hbx, Wei-Ying Ma, @yaqinzhang, @haozhou_ai, developed at SIA-Lab @hello_gensi, a joint lab of Tsinghua AIR and ByteDance Seed.

https://t.co/peZTH4QvJi

Our alternative is simple:

1. Run RL on a small model, where exploration and rollouts are cheaper.
2. Treat the model’s pre- and post-RL checkpoints as a teacher pair, whose difference captures the direction learned through RL.
3. On-policy distill this policy shift—what RL changed—using the stronger student’s own rollouts.

In one setting on AIME24:
[*] 1.5B teacher pair: pre-RL and post-RL checkpoints (Post-RL teacher score: 51.3)
[*] 7B student before transfer: 56.7
[*] 7B student + vanilla on-policy distillation: ~50
[*] 7B student + Direct-OPD: 63.1 (+6.4)

The 7B student already starts stronger than the post-RL teacher. Distilling the teacher itself makes the student worse. But distilling what the teacher pair learned through RL improves it further.

In other words, the reusable outcome of an RL run is not only the final checkpoint—it can also be the policy shift encoded by the pre- and post-RL checkpoint pair.

[Kevin Patrick Murphy @sirbayes]

@RichardSSutton is one of the few researchers tackling the problem of developing small, compute-efficient agents that learn in big worlds by inventing their own abstractions, instead of “cheating” by using human language and training on the web.

[Binfeng Xu @billxbf]

While leading RL frameworks like @slime_framework shine at RL scaling through Megatron + SGL, we find Megatron growing increasingly heavy and hard to hack on for researchers.
So we built an alternative: Molt 🦋.
Molt 🦋 is fully PyTorch-native + vLLM with minimally curated interfaces. It supports fully async, R3, TP + EP + CP, TITO, and multimodal training for MoE RL at scales of up to 1T parameters—all in ~8k lines of code (roughly 1/3 the size of Slime and 1/9 of Verl).

Start hacking! 🚀

[John Bai @johnbai]

tbh i'm not even mad, i'm genuinely impressed

[Alexander Goslin @xandurglar]

Introducing InfiniteDiffusion, my independent paper accepted to #SIGGRAPH2026!

I have one RTX 3090 Ti. No funding, advisors, or team. By day I'm a new grad SWE at Walmart.

The paper has two main contributions:
- InfiniteDiffusion: a new approach to infinite generation with diffusion models.
- Terrain Diffusion: the world’s first learned procedural terrain generator.

Here’s why this matters, and how they are connected. 🧵

[Georgi Gerganov @ggerganov]

llama.cpp at 100k stars

now that 90% of the code worldwide is being written by AI agents, I predict that within 3-6 months, 90% of all AI agents will be running locally with llama.cpp 😄

Jokes aside, I am going to use this small milestone as an opportunity to reflect a bit on the project and the state of AI from the perspective of local applications. There is a lot to say and discuss and yet it feels less and less important to try to make a point. Opinions about viability of local LLMs are strongly polarized, details are overlooked, the scientific approach is lacking. Arguments are predominantly based on vibes and hype waves.

One thing is clear though - local LLMs are used more and more. I expect this trend to continue and likely 2026 will end up being one of the most important years for the local AI movement.

I admit that I didn't expect the agentic era to come so quickly to the local LLM space. One year ago, the available models were too computationally expensive for doing long-context tasks. There wasn't an obvious path towards meaningful agentic applications. The memory and compute requirements were huge. Last summer, with the release of gpt-oss, things started to change. It was the first time we saw a glimpse of tool calling that actually works well within the resource constraints of our daily devices. Later in the year, even better models were released and by now, useful local agentic workflows are a reality.

Comparing local vs hosted capabilities at a given moment of time is pointless. To try put things into perspective:

- We don't need frontier intelligence to automate searches and sending emails
- We don't need trillion parameter models to be able to summarize articles or technical documents
- We don't need massive GPU data centers to control our home appliances or turn the lights off in the garage

I believe that there is a certain level of intelligence we as humans can comprehend and meaningfully utilize to improve our working process. Beyond that level, access to more intelligence becomes unnecessary at best and counterproductive at worst. I also believe that that level of useful artificial intelligence is completely within reach locally and it has always been just a matter of implementing the right software stack to bring it to the end user.

With llama.cpp, I am confident that we continue to be on the right track of building that software stack!

The llama.cpp project is going stronger than ever. With more than 1500 contributors, the project keeps growing steadily.

From technical point of view, I think that llama.cpp + ggml is the only solution that actually makes sense. That is, the software stack must run efficiently on every possible device, hardware and operating system. The technology is too important to be vendor-locked. It has to be developed in the open, by the community, together with the independent hardware vendors. This is the only right way to build something that will truly make a difference in the long run.

I won't try to convince you about what is currently and will be possible with local AI. We will just continue to build as usual. I am confident that after the smoke clears and we look objectively at what we have built together, the benefits will be obvious to everyone.

Big shoutout to all llama.cpp maintainers. I feel extremely lucky to be able to work together with so many talented contributors. Every day I learn something new and I feel there is so much more cool stuff that we are going to build. Also, I am really thankful that the project continues to have reliable partners to support it!

Cheers!

[Jeff Dean @JeffDean]

📢 Another exciting step forward today with the launch of Gemini 3.1 Flash Live.

It natively understands audio, making it much more capable of handling complex instructions. It leads on ComplexFuncBench, and on Scale AI’s AudioMultiChallenge, demonstrating skill in complex instruction following and long-horizon reasoning amidst the interruptions of real-world audio.

That means the model can pick up on nuances like pitch and pace, leading to much more fluid, high-fidelity voice interactions.

It’s now powering Gemini Live and Search Live globally. Huge congrats to the teams that made this possible. ✨

https://t.co/5ayNyrJak1

[Google DeepMind @GoogleDeepMind]

Say hello to Gemini 3.1 Flash Live. 🗣️

Our latest audio model delivers more natural conversations with improved function calling – making it more useful and informed. Here’s what’s new 🧵

[Demis Hassabis @demishassabis]

Gemini 3.1 Flash Live is our highest quality audio &amp; voice model yet - and a big leap towards building next-gen voice-first agents. Lower latency, better precision, more natural interactions... try it now with Gemini Live in the @GeminiApp or build with it in @GoogleAIStudio!

[Google Research @GoogleResearch]

Introducing TurboQuant: Our new compression algorithm that reduces LLM key-value cache memory by at least 6x and delivers up to 8x speedup, all with zero accuracy loss, redefining AI efficiency. Read the blog to learn how it achieves these results: https://t.co/CDSQ8HpZoc

[Antonio Orvieto @orvieto_antonio]

Optimization theory for adaptive methods actually predicts most of what we know about hyperparameter scaling in LLM pretraining, and suggests new strategies as well. We did a deep dive here.

[Cheng Lou @_chenglou]

I’m very happy to present my toy research project: Sotaku!

It's a neural net that automatically discovered the rules of sudoku and learned to solve them, achieving a new state-of-the-art score of 98.9% on one of the hardest sudoku datasets, while being agnostic to the game, and beating all other sudoku-optimized neural net architectures*

Read more for fun motivations, plus some extremely unconventional discoveries, e.g. reverse curriculum consistently beating curriculum (!), emergent reasoning-like capabilities, and the future of traditional programming

[Seungwook Han @seungwookh]

Can language models learn useful priors without ever seeing language?

We pre-pre-train transformers on neural cellular automata — fully synthetic, zero language. This improves language modeling by up to 6%, speeds up convergence by 40%, and strengthens downstream reasoning.

Surprisingly, it even beats pre-pre-training on natural text!

Blog: https://t.co/Pni0RsIcxL

(1/n)

[templar @tplr_ai]

We just completed the largest decentralised LLM pre-training run in history: Covenant-72B. Permissionless, on Bittensor subnet 3.

72B parameters. ~1.1T tokens. Commodity internet. No centralized cluster. No whitelist. Anyone with GPUs could join or leave freely.

1/n

[rohan anil @_arohan_]

Holy moly this was my dream when Gemini started, to make video tutorials on highly technical topics.

I took my old post showing Muon Shampoo relation https://t.co/dRmtS9cWWh and gave that to NoteBookLM to make a cinematic video. It did it!

Now Imagine using something like this but changing this to @karpathy giving this lecture!

🤯