Tijmen Blankevoort

Deep Learning Researcher Nvidia - Efficiency/Numerics/Edge models

[Tijmen Blankevoort @TiRune]

Great reason to use Nvidia models tbh ;p

[Cohere @cohere]

While evaluating Qwen3.8-Flash-Next in a controlled sandbox environment, Cohere's research team observed something unexpected.

Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Notably, the model's reasoning trace stated that it would not invoke any tools before taking these actions.

Cohere's sandbox blocked the attempt, and no external communication succeeded. We have not determined the root cause, and we are not asserting malicious intent.

But the incident raises important questions for organizations deploying agentic AI systems: Can models take actions that differ from their stated reasoning, and how should security teams evaluate trust, provenance, and behavior in open-weight models?

We are publishing our findings because organizations making model adoption decisions deserve as much information as possible. We hope this contributes to greater transparency, stronger security practices, and continued collaboration across the AI ecosystem.

Read the full report: https://t.co/TZSugXi2ZE

[Tijmen Blankevoort @TiRune]

@jon_barron https://t.co/PzqReHJb5i

You mean like this paper from 2023?

[Tijmen Blankevoort @TiRune]

@kimishpatel It’s about the same speed as Qwen, and quite a bit smarter. Those eval points are not easy to get! Mixed-precision with 2 bit experts, and a lot of nvfp4 and Q8.

[Tijmen Blankevoort @TiRune]

Happy to share the work we've done on our amazing new Nemotron 3.5 Bolt model! The model fits 74B params in 24GB with aggressive 2-bit quantization, using quantization-aware distillation to recover accuracy. We show it's much more pareto-optimal to do 2-bit quantization at iso model size.
#1 and #2 on several leaderboards like Pinchbench/BrowseComp/WebArena for small models. A great release, and can't wait to see how people use this on their new RTX Spark devices!!!

[NVIDIA RTX Spark @NVIDIARTXSpark]

NVIDIA Nemotron 3.5 Bolt 24GB puts powerful local agents within reach. ⚡

74B-A9B parameters packed into 24GB of memory, balancing intelligence and speed while running alongside your apps. Optimized for RTX Spark and ready for your next project.

[Tijmen Blankevoort @TiRune]

@NaraharikripaM @NVIDIARTXSpark Dense vs. MoE - see also the speed difference, Bolt is 3x faster!

[Tijmen Blankevoort @TiRune]

@Franzferdinan57 We have a 74B-A9B Bolt model released too if that's what you're looking for.

[Tijmen Blankevoort @TiRune]

@LottoLabs Highly encouraged to do this! Hopefully we have enough experimental details in the Bolt tech report.

[Tijmen Blankevoort @TiRune]

@nuonrg Hoe krijg je een 1T model op een RTX Spark? ;p

[Andrei Panferov @ICLR @black_samorez]

Disaggregated quantization for LLMs 🧵

LLM prefill and decode can be processed by separate weights in different precisions. Boosts speed AND accuracy. Offloading makes it zero-overhead on long sequences for dense models. Applicable to existing quantized checkpoints.

Paper: https://t.co/zr7fQm5SPF
Code: https://t.co/S3SGKBprjZ
Qwen3.8 NVFP4 prefillers: https://t.co/c3RHb7r3fd

[Tijmen Blankevoort @TiRune]

@y_m_asano @eccvconf Congrats!!! 😃

[Tijmen Blankevoort @TiRune]

@julien_c @nvidia Welcome to the team!!!

[Bryan Catanzaro @ctnzr]

Nemotron 3.5 Lightning:
Same architecture as 3.0 Nano,
with added speculative decoding, and with the intelligence of 3.0 Super.

⚡️⚡️⚡️

[Tijmen Blankevoort @TiRune]

@karpathy @btaylor It’s not a bug: it’s a feature! We call it the LLM recognition theorem. It’s not only useful, but also convenient. I hope this helps.

[Tijmen Blankevoort @TiRune]

@danielhanchen Great idea that NVFP4 stuff

[Tijmen Blankevoort @TiRune]

@moose_crab @teortaxesTex @glennklockwood Clearly should be doing AI instead

[Tijmen Blankevoort @TiRune]

@teortaxesTex @glennklockwood Agreed - who uses fp64? Nvfp4 is where it’s at.

[Tijmen Blankevoort @TiRune]

Everytime I reproduce a post like this - a decent AI like ChatGPT actually answers correctly?

This problem is frequently just a bias due to the way networks tokenized, and a fast service like Google AI overview, does not do proper thinking. Humans have weird cognitive biases too if you prompt them quickly. Ask someone to quickly answer: If it takes 5 machines 5 minutes to make 5 widgets, how long would it take 100 machines to make 100 widgets?

[Tijmen Blankevoort @TiRune]

@norpadon @AleksandrosSob1 Tried doing stochastic rounding instead? We did this for the Nemotron releases for SSM quantization.

[Tijmen Blankevoort @TiRune]

@Halex623 Good idea that Spinquant stuff