When building frontier multimodal models like Muse*, we keep coming back to how to design the vision encoder. It’s an architectural decision we have to make very early.
This work finally gave us a chance to step back and study it systematically.
Check our new paper!
Labs disagree on vision encoders for MLLMs! ⚔️
Gemma 4 12B scrapped the encoder.
Kimi K3 dropped encoder pretraining.
Inkling 2 Small revived the encoder Inkling killed.
We trained 200+ models in a controlled study to settle it:
✅ Keep vision parameters, skip the pretraining. And they don't have to be an encoder! 🤔
Paper: https://t.co/e8JRIfPU2m 🧵
In Beyond Language Modeling, we studied text–image tensions in multimodal pretraining.
Now we go further: do we need a vision encoder?
At matched lifetime compute, dedicated vision capacity matters—and it can be trained jointly in early backbone layers.
how should multimodal models encode visual input?
→ pretrained encoder?
→ encoder-free?
→ before the LLM?
published models bundle these choices together. we trained 200+ models from scratch to see which of these choices matters
only one of them does
🧵 [1/n]
We’re heading to #CoRL2026 to present VoLo! 🤖 Come by our poster and say hi!
📍Poster #131 • Griffin, 2F • JW Marriott Austin
🗓️ Nov 9 • 5-6:30 PM CST
Also happy to chat or hang out-feel free to DM me! See you in Austin 👋
Might be a good time to annonce that I've joining Thinky since Jun and has been working on training systems since. 🥰Super excited to see the model I've babysitted lauching, and it's an incredible model! We have the full infrastructure setup for you on Tinker to play around.
Claim: Humans & AI are better together if we want to recursively (co-)improve to a brighter future for *humanity*
Instead of racing towards methods for AI replacing us, work on how to make AI work *with* us!
New paper that makes 4 contributions here:
1) The role of humans in directing research agents: Autobenchmark
2) Reward hacking in autoresearch agents: HackBench
3) Training agents to interact with humans: AutoInteract
4) Training agents to communicate with humans: unslopping AI
Full Paper: https://t.co/ZzD9vOGkdT
Blog posts for each section:
1) https://t.co/VYiXgK8gqt
2) https://t.co/12h1Y7rTJB
3) https://t.co/PWnLdd2rh7
4) https://t.co/XJzduS7z5z
The 3D workflows caught my attention here. Coding and computer use become much more powerful together: write code, inspect the result inside the application, and keep refining it. Excited to see Kimi pushing toward complete creative workflows.
We trained an imaginative agent! ✨
Agima: an omni agent that plans by imagining futures.
𝗦𝗙𝗧: Directly generalizes to harder tasks! 𝟮%→𝟰𝟬%, beats GPT-5.6 Sol (𝟮𝟬%).
𝗥𝗟: New ways to imagine emerge! Imagines a second plan only if the first one fails (𝟯𝟮%→𝟭𝟬𝟬%).
✨Happy to share the release of technical report for Muse Glimmer, an open-weights 30B dense LLM that we built with emphasis on local agentic use. Dig in for plots and numbers on:
🪜 Architecture and scaling ladders
📊 Scorecard across 39 (!) benchmarks
🌍 Multilingual evaluations with Muse Glimmer supporting 137 language varieties
🧱 Pretraining and mid-training, including checkpoint averaging
🎯 Post-training: SFT, on-policy distillation, and multi-objective RL
🛡️ Privacy and prompt injection robustness baked into post-training
🏭 Synthetic data generation pipeline
⚡ Inference optimizations: quantization and speculative decoding
Looking forward to the discussions and implementations this report sparks 🚀
A few months ago we set ourselves the goal of making an open model for the claw: good for agentic, fits in a 32GB gpu. Here's what went into Muse Glimmer: 1/?
https://t.co/OMFKsc6f6X
Banger paper from Meta Superintelligence Labs on on-device agent models.
(bookmark it)
If you build agents that need to run locally, this one is worth your time.
Muse Glimmer is an open 30B model, distilled from Meta's frontier model Muse Spark and designed to fit in 32 GB of VRAM. With 4-bit quantization and a speculative decoding head, it runs on a single RTX 5090.
On MCPAtlas, it scores 75.5, against 54.2 for Gemma4-31B and 62.5 for Qwen3.6-27B. On SWE-Bench Pro, it scores 51.2, against 36.9 and 50.2.
Qwen3.6-27B still leads on OSWorld-Verified and TerminalBench 2.1, so the gains are concentrated in tool use and long-horizon tasks.
The model takes text and images, covers more than a hundred languages, and is released under Apache 2.0.
https://t.co/msBemXXG3j
Chat with Paper: https://t.co/QEJMqOOtnr
The Muse Glimmer technical report is out: https://t.co/VSLVtwd1Hh
Muse Glimmer is our 30B open-weight model (Apache 2.0), designed for agentic workloads entirely on consumer hardware.
Looking forward to seeing how the community uses our model!
Metas Muse Glimmer tech report is pretty interesting.
Metas Muse Glimmer report describes training agents inside simulated digital lives, complete with 10–14 contacts, weeks of messages, calendars and documents. The agents practice tasks spanning multiple days, while training trajectories containing hallucinated results or unverified claims of success are discarded.
Meta's Muse Glimmer report cites our 2023 weight averaging paper (COLM'24). Their final training recipe also uses uniform averaging of 600 steps apart checkpoints, (what we proposed).
Great to see our work being adopted. Thanks, @AIatMeta!
If you wanna understand the science of weight averaging keep reading 🧵 (1/n).
The Muse Glimmer technical report is out today! It is our 30B open-weight model (Apache 2.0) distilled from Muse Spark, built to run agentic workloads entirely on consumer hardware — one RTX 5090, 32GB.
🚀 Honored to be a core contributor to Muse Glimmer! The model is open — and so is the homework. Our tech report lays out how we built the open-weight model from pretraining to alignment.
Today we published a technical report on Muse Glimmer, our 30B open-weight model built for agentic tasks on consumer hardware.
It runs at practical speeds on a Mac or PC with a single consumer GPU at 4-bit precision, and leads comparably-sized open models on tool use and multi-step planning.
Read about the model architecture, distillation recipe, quantization and more: https://t.co/H7n9qeDhxn