Apple's own FastFrames (1.3B / 14B) generates 480p video at interactive latency, but official install is only Linux x86-64 with an NVIDIA GPU and CUDA 12.8. No Mac, MPS, or MLX path in the card or repo, so I'll wait for a community port.
https://t.co/M3HdXHkY0P
If you follow inference silicon, OpenAI says its models identified up to 14% PPA improvement on AMD flagship GPU IP in less than five months. They used the same method on Jalapeno, their first custom inference chip, with gains verified on real silicon.
https://t.co/B4yddxJZBu
If you're picking a Gemini 4 tier for coding work, Google DeepMind is rolling out Neon. It sits between Carbon and Hydrogen, and they are aiming it at complex software engineering, knowledge work, and science.
https://t.co/mauT7Pe6Ii
Gemini 4 Neon is rolling out now.
Sitting between Carbon and Hydrogen, it brings frontier intelligence and fast task completion for complex software engineering, knowledge work, and science.
Thinking less can be more accurate. Author's own test, Mac Studio M5 Ultra, Qwen3.8-27B MLX 4-bit: 160 of 198 GPQA Diamond right with a 4096-token thinking cap, 115 with no cap. Many uncapped replies hit a 16384-token limit mid-think and counted as wrong.
https://t.co/G6IXioHCku
Capping how long a local AI may think made it better on every machine I tested, from 16 GB to 256 GB.
Mac Studio M5 Ultra, Qwen3.8-27B: 160 of 198 science questions right with a 4,096 token thinking cap, 115 without
I ran GPQA Diamond on seven setups. Thinking was on in every run. The only change is TensorFold's --thinking-budget flag.
Without a cap the model often thinks until the token limit and never gets to the answer. That counts as wrong.
Mac Studio M5 Ultra, MLX 4-bit
No cap: 115 of 198
Cap 4,096: 160 of 198
1 DGX Spark, MLX 4-bit
No cap: 115 of 198
Cap 4,096: 164 of 198
RTX 5090, NVFP4
No cap: 131 of 198
Cap 4,096: 151 of 198
Cap 6,144: 166 of 198
Cap 8,192: 172 of 198
Cap 12,288: 170 of 198
1 DGX Spark, NVFP4
No cap: 131 of 198
Cap 4,096: 151 of 198
Cap 8,192: 172 of 198
2 x RTX 5060 Ti, MLX 4-bit
Cap 4,096: 159 of 198
RTX 5060 Ti 16 GB, EXL3 3.0 bpw
Cap 4,096: 161 of 198
Qwen3.8 Flash Next shows the same on the RTX 5090.
RTX 5090, Qwen3.8 Flash Next, EXL3 3.05 bpw
No cap: 134 of 198
Cap 4,096: 163 of 198
On MLX 4-bit 4,096 is enough. On the Spark 6,144 got 168 and 8,192 got 162.
On the NVFP4 checkpoint 8,192 scored highest of the caps I tried, 2,048 up to 12,288. 12,288 didn't beat it, 170 against 172, inside the margin.
The no cap runs on the 5060 Ti setups didn't finish in time, so they are not in the list. On the questions they did finish, 30 and 10 were right only with the cap and none only without it.
On NVFP4 the RTX 5090 and the Spark gave the same answers, character for character, in every setting both ran. Only the speed differs.
A cap is not better in every case.
Set too tight it costs answers. On NVFP4 4,096 got 151 and 8,192 got 172. Both beat no cap at 131, but some questions need the longer thinking.
All of this holds for a fixed budget of 16,384 tokens per question. With a much larger budget the runs without a cap would be cut off less often. I didn't measure that.
I only tested multiple choice science questions. Long coding or agent tasks may need more thinking, and I haven't tested a cap there.
Setup: TensorFold 1.2.8 official release with the DFlash2 drafter on the Mac, the Sparks, the RTX 5090 NVFP4 runs (except two, below) and the single 5060 Ti
The 6,144 and 12,288 runs on the RTX 5090 used the official TensorFold 1.2.9. The 8,192 run on 1.2.9 matched the 1.2.8 run character for character on all 198 questions.
The two 5060 Ti cards run Ash's Python engine v0.6.6 plus our patch, not a release.
The two Flash Next runs used a build from the open pull request 1443 by BlivionIaG plus our patches, with MTP drafts, not a release. The official 1.2.9 doesn't load that EXL3 pack yet.
Windows: 262,144 on the Mac, 32,768 on the Sparks and the RTX 5090, 109,511 on two 5060 Ti cards, 30,720 on one because 16 GB allows no more.
Method: 198 questions, one try each, seed 42, 16,384 tokens for thinking plus answer, cut off counts as wrong, margin about 6 points
Different quants, so compare inside a setup, not between setups.
Counts and method:
https://t.co/9ld8J16t81
TensorFold and recipes by Ash @ashxhart, DFlash2 drafter by Inco AI @inco_ai, NVFP4 checkpoint by NVIDIA, EXL3 quant by turboderp @turboderp_, model by Qwen @Alibaba_Qwen.
Microsoft Research and UC Berkeley open-sourced TokenEconomizer for Claude Code, Codex, and Copilot CLI. It learns from past project runs and skips unused tools, with no model or agent-code changes. Mean SWE-Bench cost fell 18.4-30.9%, cheaper in all 30.
https://t.co/98pRQG1Mh9
oMLX 0.8.0rc1 can use an NVIDIA GPU or DGX Spark for prefill only. The model and decode stay on the Mac; the GPU need not hold the whole model. Author's own test, 64K on M3 Ultra 512GB + Spark over 2.5GbE: Qwen3.8 Flash-Next prefill 1,370 to 2,715 tok/s.
https://t.co/j2opEHdTvo
oMLX can now use your NVIDIA GPU or DGX Spark as a Prefill Accelerator.
https://t.co/PRRWHANqQt
All you need is a Mac that can hold the model. A faster NVIDIA GPU or a DGX Spark takes over the prefill, and your Mac keeps decoding at the same speed.
- The GPU doesn't need to fit the model. It speeds up as many layers as it can hold.
- No RDMA, no Thunderbolt. Just plug in a LAN cable (1GbE works, 2.5GbE recommended).
- One docker run on the NVIDIA machine. No oMLX install or SSH needed there.
64K prompt, M3 Ultra 512GB + GPU over 2.5GbE
With a DGX Spark
- Qwen3.8 27B: 352 -> 1,062 tok/s (+202%)
- Qwen3.8 Flash-Next: 1,370 -> 2,715 tok/s (+98%)
- DeepSeek V4.1 Flash: 1,085 -> 2,412 tok/s (+122%)
- GLM-5.3 Flash: 555 -> 1,076 tok/s (+94%)
With 2x RTX A6000
- Qwen3.8 27B: 352 -> 1,879 tok/s (+434%)
- Qwen3.8 Flash-Next: 1,370 -> 3,928 tok/s (+187%)
- DeepSeek V4.1 Flash: 1,085 -> 1,914 tok/s (+76%)
- GLM-5.3 Flash: 555 -> 978 tok/s (+76%)
The CUDA helper builds on the great CUDA kernels of TensorFold by @ashxhart and mlx-serve by @ddalcu, and would not have been possible without them. Thank you both for sharing them in the open.
Also in 0.8.0rc1
- DeepSeek V4.1 with the DSpark drafter: 8 requests decode 73.2 -> 107.9 tok/s (+47%)
- Concurrent Request Priority: new prompts start sooner while other responses stream
- Affine KV decodes up to 58% faster at long context
The Edge0 team open-sourced Edge0-145B and Edge0-120B, based on Qwen3.8-Flash-Next and pitched for laptops. Per the team, Edge0-145B scores 86.52 on Terminal-Bench 2.1, ahead of Claude Opus 4.6 at 78.2, uses 29GB peak memory, and decodes at 80+ tok/s.
https://t.co/lXzVT6f18B
@bitwidthlabs Agreed. Tom's write-up already splits those phases: the 256GB Mac won on the agent task and on fitting the larger models, while Spark-class boxes often lead on prompt processing. It helps to read each benchmark by which phase it weights.
Per Tom's Hardware: M5 Ultra Mac Studio (256GB, ~$12,300) vs a Spark-class box. Qwen 3.8 27B, llama.cpp + multi-token prediction. AgentPerf-Local: Mac under half the time, ~16% less energy. 256GB fits GLM 5.3 4-bit and V4.1 Flash 2-bit; one Spark does not.
https://t.co/lY2wLOjckH
Going deep on local AI performance benchmarking with Apple’s Mac Studio with M5 Ultra taking on Nvidia DGX Spark — inference engine recipes mandatory for bigger models as MLX stack matures https://t.co/xObeah09ep
@Wbackdown The largest single jump was switching to the llama.cpp SYCL backend, 33 to 49. MTP only appears in the last step, bundled with Q8 KV (49 to 56.5), so most of the 6.8 to 56.5 gain came before MTP.
Author's own test: Intel Arc Pro B70 32GB at $1,299. Same card, software-only path from 6.8 to 56.5 tok/s on Qwen3.8-27B Q6 (Ollama+Mesa to llama.cpp SYCL+MTP). About half a 5090's gen speed at about one-fifth the price.
https://t.co/LERVlgREHE
@tudoriustin_ Those are the original author's numbers, not mine. Their ladder: Mesa 6.8 to 19.4, FP16 KV to 33, llama.cpp SYCL to 49, then SYCL + Q8 KV + MTP to 56.5. MTP heads and acceptance rate weren't published, so MTP can't be split from the Q8 KV change.
Anthropic says it used Mythos to get Claude running on AMD MI355X in about 3 days. The slide in the roadshow puts that at prompt processing for one model, at a useful level of performance.
https://t.co/h7TkEdURuB
From the Anthropic Road Show video:
'This has also made us really good at bringing up new chips. As an example, this year, we started working with AMD GPUs for the first time. In the past, bringing on a new chip platform took dozens of engineers months of work. This time we had Mythos try it out. It got Claude running on AMD in just 3 days with basically no human intervention. Claude ported or wrote almost 100% of the kernels.'
Author's own test: one 512GB Mac Studio vs four ASUS GX10 ($25,799 vs $24,000) on DeepSeek V4.1 Flash original weights. Single-user coding 84 vs 122 tok/s; 16 concurrent 148 vs 847; 128K prefill 3,325 vs 7,522. Close-ish for one user, over 5x at 16-way.
https://t.co/4HkpDFxmIW
Mac Studio 512GB ×1 vs ASUS GX10 128GB ×4
DeepSeek V4.1 Flash, original weights, same bench
Price: $25,799 vs $24,000
Concurrency 1: 53/84 vs 73/122 tok/s (text/code)
Concurrency 16: 137/148 vs 330/847 tok/s (text/code)
128K prefill: 3,325 vs 7,522 tok/s
DGX Spark recipes keep coming and keep getting faster. That's a big plus.
Mac's plus: one person, one session at a time? It's also your dev machine. Code and run the LLM on the same box, while the Sparks still need another computer to work from.
Mac: mlx-serve @ddalcu · GX10: spark-ds41f @christopherowen
Local AI GPUs under ¥10,000: I compared 22 cards, including the T10, V100, B60, 7900 XTX and 5060 Ti, and picked the only 5 I'd actually buy. New video, in Chinese:
https://t.co/bzQd36snRf
MiniMax H3-Image now runs on MLX-VLM v0.8.0 (release soon) 🚀
A 33B open-weight image model, generating native 4096 × 4096 on a MacBook under 30GB of RAM. No upscaler, no cloud.
On an M5 Max (128 GB, 8-bit, 6 steps):
→ 1024²: 13.45 s
→ 2048²: 42.34 s
→ 4096²: 242.02 s
→ Peak memory: 25 GiB, flat at every resolution
Image editing works too: 126 s at 2048² with one reference image.
16.8 megapixels. 4 minutes 2 seconds. All local.
GH repo: https://t.co/P9mZz7SQ5s
Author's own test on an M5 Max 128GB with inferd: Qwen3.8-27B + DFlash2, 16K context. Single-stream decode was 93.9 tok/s; totals at 4, 8, and 16 concurrent sessions were 221.0, 266.0, and 314.7 tok/s.
https://t.co/TTGOuhkcdC
https://t.co/21mWOGr2q1
Tried @jtdavies's inferd 0.4.1 (prebuilt) on a Mac Studio M5 Max 128GB with Qwen3.8-27B + DFlash2.
release_gate: PASS
https://t.co/By5GPvepai 16k, N=1/4/8/16: 90 / 221 / 266 / 315 tok/s (about 55–64% of the published M5 Ultra numbers, with roughly half the GPU cores)
agents_at_once 8×4k and 16×16k: both PASS, all answers correct
Impressive: 16 agents at once on a 27B dense model, ~3 s to first token, no wrong answers. Exactly what I want for reviewing contract clauses in parallel. Great work!
Details and raw .jsonl (on GitHub as @aquariumyossy): https://t.co/b7BciAjTWE
Gemini 4 Carbon is now open to paying developers and Ultra users. Introductory API price is $2/$10 per million tokens (in/out) through Nov 30, then $4/$20. Per Google, it ranks first on CursorBench 4.0, WANDR, and AutomationBench.
https://t.co/E5bdCRehqz
Gemini 4 Carbon is now available.
It’s our strongest model, built to dig deeper, handle long-horizon tasks, and multi-step reasoning.
If Argon is great for complex tasks, Carbon is the model you bring in for your hardest problems.
MiniMax-H3-Image weights are out: 33.1B DiT, text-to-image and editing, native ~16MP, 6 steps, up to 16 refs. Custom community license: $1M+ annualized revenue or a hosted service needs written authorization. US, EU, UK, and Korea are excluded. See terms.
https://t.co/uTzbvqsE7U