Nvidia’s Nemotron 3.5 Super VL is now the leading US open weight model, scoring 26 on the Artificial Analysis Intelligence Index
Nemotron 3.5 Super VL achieves twice the intelligence of its predecessor, Nemotron 3 Super (13), and does so using ~60% fewer output tokens per task
It launches alongside Nemotron 3.5 Bolt 74B-A9B 24GB, the first model in its new Bolt family, which scores 21 on the Intelligence Index
Congratulations to @NVIDIAAI on the releases!
Key Takeaways:
➤ Leading US open weight model: Nemotron 3.5 Super VL scores 26.5, edging out Thinking Machines' Inkling Small (25.7) and ahead of NVIDIA's Nemotron 3 Ultra (23) and Meta's Muse Glimmer (17)
➤ Bolt comes within 2 points of Ultra: Nemotron 3.5 Bolt 74B-A9B 24GB scores 21, 1.5 points behind Nemotron 3 Ultra 550B A55B (23). The 2 bit memory has about a seventh of Ultra's parameters (74B vs 550B) and a sixth of its active parameters (9B vs 55B)
➤ Super 3.5 VL near lossless in NVFP4: Nemotron 3.5 Super VL releases in NVFP4, scoring within 0.1 points of its BF16 weights on the Intelligence Index, with no component moving by more than 2.6 points
Nemotron 3.5 Super VL details
Context window: 1M tokens
Modalities: Vision and text input; text output
Parameters: 120B / 12B
Nemotron 3.5 Bolt 74B-A9B 24GB details
Context window: 256k tokens
Modalities: Vision and text input; text output
Parameters: 74B / 9B
As I’ve said for years, concentration of power, capabilities and wealth is the biggest risk in AI. The natural tendencies for it are super strong and without much much stronger open-source, we might end up in a world with more asymmetry and inequalities than ever!
Banning open weights? Not on our watch
Just picked up another pair of Sparks ☝️
Exo going to make local inference better in every way shape or form. Using morally corrupted corpo slop will soon be a thing of the past
Thanks Nvidia for making this tech accessible to the masses
Training with fewer bits is faster, but quantization error can interfere with learning.
Today on Connectionism, we detail how we trained Inkling 2 Small with a native 4-bit forward pass from the first step of pretraining through the final RL rollout.
https://t.co/CIMECSEpD0
For my first post, I’m sharing a letter @NVIDIA signed on why open models matter.
AI will transform every industry, power every company, and be built by every country.
Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty.
The world needs both frontier closed models and frontier open models.
https://t.co/AUKzoQ5Ikb
Many people get this wrong.
Supporting open source AI has nothing to do with China.
It is simply about one thing: the collective knowledge humanity has built so far should belong to humanity.
I'm terribly sad @JensenHuang is no longer leading @nvidia.
Thank you Jensen. For the tireless work, for the focus, for the wisdom, for the support, for the vision, for the chance you gave me to do my life's work @nvidia.
We will do our best to push this work forward.
📽 Five weeks of russian terror in a 57-second video.
This is what everyday life looks like for ordinary people in Ukraine today. A glimpse of what we go through, day after day.
some recent examples:
* ChatGPT was not a scaling exercise over GPT-3. they key (which moved us to the next sigmoid) was alignment/rlhf
* O1 and reasoning models were not simple scaling of assistant models (ChatGPT). It required post training innovation (RLVR)
* agentic models aren’t a simple scale up of reasoning models, training stack innovation were necessary.
and the innovations will continue
The bitter lesson 2.0 is that we are not on an exponential, but on a bunch of sigmiods stacked on top of each other. To move within one sigmoid, scaling compute is necessary but to jump from one sigmoid to another you need innovation.
I regularly hear from people that my post-training book/writing has helped them get a job at labs by focusing on material that develops intuitions for how training methods work. It's very validating, because this type of "idea/math heavy" writing doesn't really sell many books. It's rewarding the right people who put the effort into learning. I'm happy I made the time for it.