3/ Long inputs: 56 / 52 / 48 / 43 tok/s at 8k / 32k / 64k / 110k, against 44 / 40 / 37 / 33 for the production setup. The gain grows with context, because attention's share of the work grows.
1/ The idea: gpt-oss's experts are already 4-bit, but attention and the output head are BF16 and read for every token. FP8 attention cuts the bytes read per token by 17%. Neither vLLM nor SGLang can quantize this model's head, so it stays BF16.
gpt-oss-20b on one DGX Spark: OpenAI's MXFP4 experts untouched, attention in FP8. On vLLM it beats a typical production setup at every load and context: +28% for one user, +10% at eight, +31% at 110k input. Same accuracy.
https://t.co/RUaV4dOjZ0
https://t.co/zA6VhQpeUy
2/ Accuracy holds: GSM8K 237/250 (production setup 232), tool calls 30/30, and every long-context retrieval test passed through 111k tokens (48 of 48). It picks the same next token as OpenAI's original 98.15% of the time.
gpt-oss-20b on one DGX Spark: OpenAI's MXFP4 experts untouched, attention in FP8. On vLLM it beats a typical production setup at every load and context: +28% for one user, +10% at eight, +31% at 110k input. Same accuracy.
https://t.co/RUaV4dOjZ0
https://t.co/zA6VhQpeUy
5/ Limits: the 131k window is capacity, not reliable recall. Every build we tested, NVIDIA's included, misses some needles at 64k and beyond. At 111k, CapyCTL's found all three on 2 fewer of 12 documents than NVIDIA's on SGLang (p=0.13). Full tables in the writeup.
1/ The checkpoint, capyctl/Gemma-4-26B-A4B-NVFP4: experts in NVFP4 with block scales searched against BF16; attention, dense MLP and output head in FP8. 17.9 GB. NVIDIA's build keeps attention and the dense MLP in BF16, so it reads more bytes a token.
Gemma 4 26B A4B on one DGX Spark: CapyCTL's recipe is the fastest we measured at every concurrency. 78.9 tok/s for one user, 302 across eight, vs 29.5 and 168 for a typical setup. Same accuracy.
Checkpoint, recipes, all numbers:
https://t.co/8o0kwdT9c5
https://t.co/MSaH8hVewM
4/ We also reran other published GB10 recipes with our checkpoint swapped in. NVIDIA's official recipe: 81.5 to 98.8 tok/s. Faster under every MTP recipe; level with RedHat's build under DSpark. A published 108 tok/s result didn't reproduce: its own checkpoint gave us 56–71.
3/ The recipe: vLLM 0.32 + Google's MTP drafter at depth 3. Against RedHat's, Unsloth's and NVIDIA's builds on that same recipe, CapyCTL leads by 13%, 4%, 1% (a tie) and 3% at 1, 2, 4 and 8 users. The cost: first tokens arrive later than in the SGLang setup without a drafter.
2/ Accuracy holds. CapyCTL's checkpoint picks the same next token as BF16 97.18% of the time (NVIDIA's build: 97.13%), scores 244/250 on GSM8K and passes 30/30 tool calls.
We actually tested this up to 32K! The biggest drop was between 16K and 32K, going from 41.6 to 29.5 tok/s. So far, 8K–16K seems to be a pretty good sweet spot.
We also tested retrieval up to 111K. Still want to compare MTP vs no-draft at longer contexts to see how much of the speedup holds up.
Here are the numbers:
https://t.co/31wbj2Wuru
Same DGX Spark, same Gemma 4 26B-A4B: 29.5 tok/s stock, 79.6 with a requant and Google's drafter in vLLM. Same math score.
Recipe matters as much as the box. Measured by @edurdias. More in today's Local AI Daily:
https://t.co/lZoi0zo9hX
@e_landeros_ The biggest gains came from our custom checkpoint (NVFP4 experts + FP8 dense layers, attention and output head) and Google’s MTP-3 speculative decoding.
Gemma 4 26B A4B on one DGX Spark: CapyCTL's recipe is the fastest we measured at every concurrency. 78.9 tok/s for one user, 302 across eight, vs 29.5 and 168 for a typical setup. Same accuracy.
Checkpoint, recipes, all numbers:
https://t.co/8o0kwdT9c5
https://t.co/MSaH8hVewM
Over the past week, I’ve been overwhelmed by the love and support of friends, industry colleagues, and people I've never met. Thank you to everyone who has shown kindness to my family and me. It means the world to me.
I came to the United States 53 years ago, and becoming an American citizen is one of the greatest gifts and proudest moments of my life. NVIDIA has been my life’s work. Building it into the company it is today alongside my colleagues has been a privilege and an honor. Nowhere but in America could a company like NVIDIA be possible. I love our country, and I love NVIDIA.
I did not live up to the standards expected of everyone at NVIDIA, and I have made serious mistakes in my personal life. I created problems for the Board that I deeply regret. I own my mistakes completely, and I am committed to cooperating with the Board’s independent investigation.
Since my departure, I’ve seen an outpouring of calls from NVIDIA’s employees, investors, business partners, and customers for me to return as CEO. Many continue to reach out to me directly as the company navigates complex technology, market, competitive, and geopolitical dynamics. NVIDIA is a once-in-a-generation company. If the Board of Directors decides that my return is in the best interest of our shareholders, our employees, the company, and America, I would be honored to answer these calls and return as CEO.
Thank you again to everyone who has shared their good wishes. I am profoundly humbled by your support. To my family: I love you, and I will spend the rest of my life being worthy of yours.
Jensen
We're teaming up to accelerate open-source Al.
OpenRouter is now integrated into Hugging Face directly. Same token, same endpoint, plus live model stats on the Hub and an inference widget we redesigned together.
The best of open, in one place. https://t.co/fNbvtJGSJB
Introducing Tess-4.5-594B-A17B, the best Tess model ever. 🎉
Inkling 2 Small, post trained with the Tess-4.5 recipe, on real engineering sessions to do what Tess does best: intellectually dense reasoning, warm and empathic tone, and hardcore engineering.
On SWE-Bench Pro v2 Hard, Tess-4.5 resolves more of the hardest public software-engineering tasks than the base model (88.2% vs 86.3%), and it does it with ~10% fewer tokens.
Model: https://t.co/BAiXWpr4cS
Impressive progress from the Qwen team! 🚀
A new MoE architecture, native vision, 1M-token context, and a strong focus on inference efficiency.
What excites me most is the upcoming smaller open-weight Qwen4 models.
📢 Meet Qwen4-Max, your new AI teammate for work, creativity, and discovery!
Qwen4-Max is built on a new MoE architecture with native vision and a 1M-token context window.
Highlights:
- Outperforms Qwen3.8-Max with up to ~50% fewer tokens, while average cost per task drops over 60% in our internal evaluation.
- Advances real-world productivity through greater proficiency with professional tools, stronger long-horizon execution, and higher-quality deliverables.
- Handles real tasks, including 3D modeling, game development, manufacturing, finance, law, healthcare, and frontier scientific discovery.
- $1.5/M tokens for input, $4.5/M tokens for output, and $0.075/M tokens for implicit caching — all lower than Qwen3.8-Max, with implicit caching down by 70%.
The API is live on QwenCloud. You can also try Qwen4-Max on Qwen Studio, QwenWork and Qoder.
Start building today! 🚀
- Blog: https://t.co/AuiWxugjE6
- API: https://t.co/aS1QEkS8nu
- Token Plan: https://t.co/YRvcGdB9Bv
More to come! The open weights of smaller Qwen4 models will be released. Stay tuned. 👀