Itay Levy

Deep Learning Researcher @NVIDIA

[Itay Levy @itayoush]

@abheer Thanks, really appreciate it!

[Itay Levy @itayoush]

🧵 1/3
Our new open-source model, Nemotron-Labs-3-Puzzle-75B-A9B, is out 🎉
We compressed Nemotron-3-Super-120B-A12B into a smaller, faster derivative optimized for interactive deployment.

[Itay Levy @itayoush]

3/3
This lets the architecture search adapt as the model changes, instead of compressing everything in a single step.
Check out our tech report to learn more about heterogeneous pruning, knowledge distillation, RL, quantization, and MTP training.
https://t.co/cMkeI2GBIk

[Itay Levy @itayoush]

2/3
The result: 2x higher throughput under the same interactivity constraints, while maintaining strong accuracy across benchmarks.
At the core is Iterative Puzzle: a compression recipe that alternates heterogeneous pruning with short recovery phases.

[Pavlo Molchanov @PavloMolchanov]

We’re releasing Nemotron-Labs-Diffusion - the first Tri-mode LM family (3B/8B/14B) that switches between 1⃣Autoregressive, 2⃣Diffusion, and 3⃣Self-Speculation decoding by simply changing the attention pattern/mask.

One model Three decoding modes. No extra draft models. No architecture changes. Just significantly better efficiency across different concurrency levels.

Up to 4× higher real throughput for a single user.

🤗 HF Collection: https://t.co/1zStcCCWPi, open license
🛜 Project page: https://t.co/y6TEAvLFvD
📰 Tech report: https://t.co/NSjKxEyHnT

Details below 👇

[Jian Zhang @JianZhangCS]

Nemotron 3 Super is live! So far the most intelligent agentic reasoning model in the Nemotron family, with world leading efficiency and openness.

Super particularly marks our first infra & research milestone in agentic reinforcement learning scaling up.

Stay tuned for more infra, data and agentic generalization research we will open to the ecosystem.

🤗 Huggingface: https://t.co/WO70KiLDkF
📜 Tech Report: https://t.co/zKeHnYjEqm
🤸‍♂️NeMo-Gym (RL env data and orchestration): https://t.co/E3Q67AIA4j
🤸NeMo-RL (RL training): https://t.co/fD78eabCZv

[Soumye Singhal @soumyesinghal]

Nemotron 3 Super is here! 🚀

Big capability jump, especially on agentic benchmarks, while staying built for efficient inference. Released the @nvidia way: weights + training recipes + code + datasets.

HF: https://t.co/YzMfd5n4EW
Tech report: https://t.co/TPxIil0Xim

[Ethan He @EthanHe_42]

My last open-source project before joining xAI is just out today. Megatron Core MoE is probably the best open framework out there to seriously train mixture of experts at scale. It achieves 1233 TFLOPS/GPU for DeepSeek-V3-685B. https://t.co/QA1KRGu2Nc

[Itay Levy @itayoush]

Paper: https://t.co/rveEc6XXWw
The model will become available on HF soon

[Itay Levy @itayoush]

🧵
New paper: We compressed OpenAI’s gpt-oss-120B into a smaller, faster derivative (gpt-oss-puzzle-88B) with no accuracy loss:
⚡ Up to 1.63× higher token throughput on 8×H100
⚡ Up to 2.82× on a single H100

[Itay Levy @itayoush]

For reasoning models, tok/s isn’t sufficient because trace length can change, so we also measure request-level efficiency. On the accuracy–speed frontier, 88B outperforms 120B across all efforts, with up to 1.29× higher request rates

[Itay Levy @itayoush]

It has been an honor for us to collaborate on the Llama-Nemotron family, creating open world-class frontier models!

https://t.co/JjHdBYCVvk

[Itay Levy @itayoush]

Try Llama Nemotron Ultra 253B, the smartest open reasoning model available today!
🏆 Tops scientific reasoning, complex math and coding benchmarks
⚡️ 4x higher inference throughput over DeepSeek R1. Optimized with neural architecture search and FFN fusion

[Itay Levy @itayoush]

Distillation-based NAS (Puzzle) paper https://t.co/kg7V6nPy9o

[Itay Levy @itayoush]

Very excited about the release of the Llama Nemotron Super 49B model 🚀 #GTC25

Using distillation-based NAS (Puzzle) we achieved 5X throughput gain!

After SFT and RL, this model tops reasoning benchmarks among open 70B models

[Itay Levy @itayoush]

Hugging Face https://t.co/kz5c5PEHLC

[The AI Timeline @TheAITimeline]

Puzzle: Distillation-Based NAS for Inference-Optimized LLMs

Author's Explanation:
https://t.co/H63yEdhitH

Overview:
Puzzle introduces a distillation-based neural architecture search framework that significantly optimizes LLM inference on specific hardware, achieving a 2.17x speedup with a 98.4% retention of the original model's capabilities via blockwise local knowledge distillation and mixed-integer programming.

The framework's application to Nemotron-51B enables a single NVIDIA H100 GPU to handle large batch sizes, highlighting the efficiency of the approach with only 45B training tokens needed compared to the 15T used for the original model, thus prioritizing inference performance over parameter count for model selection.

Paper:
https://t.co/B9YPPML6Q7

[Itay Levy @itayoush]

As a highlight, we present Nemotron-51B, derived from Llama-3.1-70B-Instruct, achieving 2.17× inference throughput speedup on a single NVIDIA H100 GPU while preserving 98.4% of the original model's capabilities.
Dive into the details of how it all works in our new research paper!

[Itay Levy @itayoush]

Introducing Puzzle: Distillation-Based NAS for Inference-Optimized LLMs
🔗 https://t.co/kg7V6nPy9o
🧵

[Itay Levy @itayoush]

Puzzle accelerates LLM inference on specific hardware while preserving capabilities. Using decomposed NAS and knowledge distillation, we optimize LLMs under hardware constraints, while requiring only a fraction of the original training compute