Serge

Keep learning and travelling 🏔 between 🇺🇸 and 🇰🇷 - 🇫🇷 SW engineer @NVIDIA

[SemiAnalysis @SemiAnalysis_]

Similar to the panic over DeepSeek R1, some uneducated people think Kimi K3’s use of linear attention (KDA) is bad for NVIDIA, HBM, DRAM, and networking because it has relatively lower KV-cache requirements. The opposite is true, and we explain why below. 👇️ 1/8🧵

[Alex Zhurkevich @cudagdb]

Trtllmgen kernels are now open. Fastest prefill and decode kernels for our target workloads. We wrote these to win InferenceX, MLPerf, other benchmarks. Powering some of today’s top served models. Dive in, learn, use them, or level up your own. Enjoy.

https://t.co/2aQBwcdnZL

[Benjamin Marie @bnjmn_marie]

My analysis of the ~120B MoEs:
Nemotron 3 Super vs Qwen3.5 122B vs Mistral Small 4, all in NVFP4.

All my results and charts here:
https://t.co/jAUuJtGRaK

I think NVIDIA really undersold Nemotron 3 Super.

Quantized with NVFP4, it performs on par with the best Qwen3.5 122B NVFP4 checkpoints. But that’s almost the least interesting part.

Nemotron 3 Super NVFP4 is crushing the others I tested on efficiency:

> Most memory-efficient: if you really wanted to pack 200+ concurrent 256k-token queries on a B200, you can.
> Fastest: 400+ tok/s per sequence on B200.
> Most token-efficient: higher accuracy with shorter reasoning traces, and it only thinks longer when it actually needs to.

In practice, the gap is huge.

On LiveCodeBench problems, Nemotron 3 Super finishes in ~20 seconds on average per problem.
Qwen3.5 122B takes over a minute.

And if Nemotron misses on the first try, retrying is still cheaper, while often giving better accuracy anyway. Nemotron 3 Super pass@2 >> Qwen3.5 pass@1.

[Bryan Catanzaro @ctnzr]

Announcing NVIDIA Nemotron 3 Super!

💚120B-12A Hybrid SSM Latent MoE, designed for Blackwell
💚36 on AAIndex v4
💚up to 2.2X faster than GPT-OSS-120B in FP4
💚Open data, open recipe, open weights

Models, Tech report, etc. here:
https://t.co/CAYpP1iK3i

And yes, Ultra is coming!

[SemiAnalysis @SemiAnalysis_]

At the same time, AI appears to follow Jevons’ Paradox. As chips become more efficient and the cost per token declines, usage expands through larger models, longer context windows, more inference queries, and entirely new AI driven applications. The net effect is that chip level efficiency gains primarily enable greater aggregate compute demand rather than restrain energy growth. Even though Rubin delivers meaningfully higher FLOPs per watt, the market does not consume less power, it consumes more compute. That dynamic is why we expect Rubin to dominate installed base FLOPs by 2027. (2/2)

[Wccftech @wccftech]

NVIDIA says its Blackwell Ultra is optimized for agentic AI, delivering 50× higher throughput per megawatt than H200s, 35× lower cost per million tokens, and 1.5× lower cost per token vs. GB200 NVL72.
https://t.co/CGXTDL6x3W

[Chubby♨️ @kimmonismus]

NVIDIA just open-sourced a 30B model that beats GPT-OSS and Qwen3-30B - and runs 2.2–3.3× faster.

Nemotron 3 Nano:

Up to 1M-token context
MoE: 31.6B total params / 3.6B active
Best-in-class SWE-Bench performance
Open weights + training recipe + redistributable datasets
And yes: you can run it locally on ~24GB RAM.

[Jackson Atkins @JacksonAtkinsX]

NVIDIA research just made LLMs 53x faster. 🤯

Imagine slashing your AI inference budget by 98%.

This breakthrough doesn't require training a new model from scratch; it upgrades your existing ones for hyper-speed while matching or beating SOTA accuracy.

Here's how it works:

The technique is called Post Neural Architecture Search (PostNAS). It's a revolutionary process for retrofitting pre-trained models.

Freeze the Knowledge: It starts with a powerful model (like Qwen2.5) and locks down its core MLP layers, preserving its intelligence.

Surgical Replacement: It then uses a hardware-aware search to replace most of the slow, O(n²) full-attention layers with a new, hyper-efficient linear attention design called JetBlock.

Optimize for Throughput: The search keeps a few key full-attention layers in the exact positions needed for complex reasoning, creating a hybrid model optimized for speed on H100 GPUs.

The result is Jet-Nemotron: an AI delivering 2,885 tokens per second with top-tier model performance and a 47x smaller KV cache.

Why this matters to your AI strategy:

- Business Leaders: A 53x speedup translates to a ~98% cost reduction for inference at scale. This fundamentally changes the ROI calculation for deploying high-performance AI.

- Practitioners: This isn't just for data centers. The massive efficiency gains and tiny memory footprint (154MB cache) make it possible to deploy SOTA-level models on memory-constrained and edge hardware.

- Researchers: PostNAS offers a new, capital-efficient paradigm. Instead of spending millions on pre-training, you can now innovate on architecture by modifying existing models, dramatically lowering the barrier to entry for creating novel, efficient LMs.

[Avi Chawla @_avichawla]

A new embedding model cuts vector DB costs by ~200x.

It also outperforms OpenAI and Cohere models.

Here's a complete breakdown (with visuals):

[Jure Leskovec @jure]

🚀 Announcing RelBench: an open benchmark for deep learning on relational databases! RelBench is the foundational infrastructure for research in Relational Deep Learning (RDL), which brings modern AI to structured data.
RelBench has databases, tasks, loaders, evaluators, and leaderboards to catalyze research in the field!

Key features:
🌍 7 datasets spanning diverse domains: e-commerce, social, medical, and sports.
🧩 30 carefully curated predictive tasks: including entity classification/regression and recommendation.
📊 Wide data size range: ranging from 74K to 41M rows, 15 to 140 columns, 3 to 15 tables.
⏳ Wide time spans: from 2 weeks to 55 years of training data.
🏅 Comprehensive benchmarks: SOTA tabular learning and GNN baselines for every task.

🔥We hired a data scientist with 5 years of industry experience to solve RelBench tasks using traditional machine learning (feature engineering, model training). The RDL outperforms the data scientist in accuracy while reducing the time/code by 20x (12.3 hors -> 0.5 hours) !!! 🤯

Learn more:
🌐 Website: https://t.co/BzwWxv9lNb
📄 Paper: https://t.co/bR3yxYVPyc
💻GitHub: https://t.co/EXdsMNTkEW
Follow @RelBench for the latest updates

Shoutout to the amazing team: @Josh_d_robinson @_rishabhranjan_ @weihua916 @KexinHuang5 @jiaqihan99 @adobles96 @rusty1s @janericlenssen @yiwenyuan98 @zechengzh @xhe1997 @Kumo_ai_team @PyG_Team @StanfordAILab

[Jeremy Howard @jeremyphoward]

Tip - If you run this once, it ensures that when you pull you also get updates to any submodules in a repo - I'm not sure why this isn't the default, since it's much more convenient IMO:

```
git config --global submodule.recurse true
```

[Serge @_Kh4L]

@TheEricNorris $NVDA 4600% 👍

[Serge @_Kh4L]

@find_evil Writing WTF instead of WFH in a business expenses report and submitting before realizing the typo

[Russell Kaplan @russelljkaplan]

The new TF32 float format @NVIDIAAI announced today is a big deal. Same dynamic range as fp32, same precision as fp16, with only 19 bits total and hw-accelerated. Will be default mode of cuDNN going forward. 6x training speedup for BERT on new Ampere GPUs with no code changes!

[Serge @_Kh4L]

@heidishin I was able to find to masks (KF94 ones) in some 편의점!

[Serge @_Kh4L]

@noaheckman @3blue1brown Cool! Thank you for plotting it. Can you share the projection until summer? (If you still have the sheet and if you don’t mind) 😉

[NVIDIA HPC Developer @NVIDIAHPCDev]

To help discover treatments for the #Coronavirus, @UTAustin and @NIH researchers achieved a critical breakthrough—creating the first 3D, atomic-scale map of the virus using cryoSPARC, a GPU-accelerated platform for obtaining 3D structural information from cryo-EM data.

[Adriana Porter Felt @__apf__]

A solid 10% of being an engineering manager is asking two people if they've talked to each other yet

[Alfredo Canziani @alfcnz]

This Monday, in class, @ylecun explained how the retina performs a 100× compression to go from 100M photoreceptors to 1M axons in the optic nerve.
As a consequence, the central object in this video appears to become brighter / darker although its brightness is constant.

[Serge @_Kh4L]

@DataJunkie @conorliv I saw some emergency lights coming from the mobile home park when I went out to check what’s happening... RIP 😞