Milan Kryl

Data Scientist at @AlmaCareer_CZ
Training models for work and pleasure. 3060, 3090 and Asus Ascent GX10 @ home.

[Milan Kryl @mikr]

@0xBakeer Now running TensorFold with Qwen3.8-27B, will try TandemLLM in few days. Maybe even with ThinkingCap version from @BottleCapAI Thanks!

[AI_Pioneer @Secretof_forest]

YODAS v3 is out: 1.1 million hours of 48kHz multi-channel speech in 147 languages under CC BY 3.0. The authors call it the largest open speech dataset to date and the first at scale with stereo audio, with word-level timestamps and English translations.

[Joe @joedaroo]

Took a minute to write a few words about security & safety as someone who lived through it all at OpenAI. I hope my thoughts help someone out there. https://t.co/gJBf08vJH9

[Ondra Tomek @tomek_ondra]

Cesti investori ve vesmirnem Blue Origin Jeffa Bezose.
Poprve v nasi vlastni investorske historii se nam podarilo byt primou soucasti takto vysoce preupsaneho neverejneho upisu, kde se upsal jeden dolar z patnacti nabizenych. Byl o to takovy zajem, ze byli investori doslova vybirani organizatorem emise po boku s Jeffem Bezosem, aby mohli byt soucasti tohoto prvniho upisu ve kterem pan Bezos pustil do Blue Origin dalsi spoluinvestory. Kriteriem byla kredibilita a vyse nabizene investice. Je to neverejny upis, Blue Origin jeste neni a nejakou dobu ani nebude na burze. O to tezsi bylo dostat se tam.
Ted ale muzeme rici, ze az poleti na obeznou drahu raketa BlueOrigin a zase ponese komunikacni satelity ASTSpace mobile, bude v obou nase ceska stopa. V obou jsme nyni nezanedbatelnymi investory v souladu s nasi strategii definovanou cca 6 let zpatky = odejit postupne ze SaaS sluzeb, a silne se zamerit na Space, Data a AI. A ted drzime robustni (i na svetove rozmery) podily v Nebius, BlueOrigin, SpaceX, ASTS a dalsich firmach z techto oblasti prumyslu a pridavame dalsi. SKY IS NO LIMIT.
Amerika leta do vesmiru, po zemi tam jezdi taxiky bez ridicu a Evropa? Doplnme si sami……. https://t.co/fxFpL2oQT5

[Wësche @WescheNex1q]

TensorFold vs vLLM, Qwen3.8‑27B, one Spark, same 3 prompts, thinking off:

3× faster than vLLM on the same DGX Spark. Byte-identical output.

• TensorFold (DFlash2 drafts): 102.9 tok/s
• vLLM MTP=3 (NVFP4): 34.9 tok/s
• TensorFold serial, drafts off: 12.9 tok/s

Per prompt (TensorFold / vLLM):
sequence 126.9 / 37.0
code 75.7 / 32.7
json 106.2 / 35.0

TTFT 0.12 s. Drafted vs serial output: sha256 identical on all three. Loads in 32 s.

Not weight-matched: MLX 4‑bit g64 vs NVFP4 ModelOpt. Single pass. Long structured outputs draft well; prose will land lower.

Bonus: one Spark now beats my M4 Max on this model (oMLX MTP3: 65.8).

https://t.co/DBejw9Catm — @ashxhart

[OrcaRouter 🐳 @OrcaRouter]

Today we’re announcing OrcaSAQ-2 27B

High-fidelity mixed-precision Qwen3.8 for long-horizon agents.

55.59 → 12.06 GB — 78.3% smaller / 4.61×
3.21 bpw · 93.2% Top-1 agreement
70.0 SWE-bench Verified
58.4 Terminal-Bench 2.1
262K context

A 27B model for coding, terminal, browser, security and multi-tool agents — in a footprint you can actually deploy.

SOTA agentic capability density among similarly sized models we evaluated.
https://t.co/FqslE1kYnw

[0xSero @0xSero]

Bruh, 144 tok/s with full context and vision for qwen3.8 on a 3090

I'm going ham with Claude rn, it's actually amazing at this.

[Jaroslav Beck @JaroslavBeck]

🏢 Enterprise use
This release is trained at xhigh effort. For enterprise use, we are ready to fine-tune medium and low effort as well. Contact us at enterprise [at] https://t.co/ZC7xXiNWgR

[Jaroslav Beck @JaroslavBeck]

🚨Happy to share ThinkingCap: Qwen3.8-27B is out!
The most token-efficient Qwen3.8 27B available, this time optimized for agentic use.

🚀 Up to 65% fewer thinking tokens, 37% on average
🎯 Minimal accuracy loss
📈 Long-context retrieval actually went up 2.3pp

Side-by-side comparison video in the blogpost below.
📄 Blogpost: https://t.co/fj3oAxMpx8
🤗 Hugging Face: https://t.co/qSmYi4Xrlj
🔗 GGUF: https://t.co/zEm6m08Muc

[Wei @wei_wang]

大开眼界。

有人把 DeepSeek V4.1 Flash 的 Causal Encoder-Decoder 思路，移植到了现有的 Qwen3-8B 上。

结果相当夸张。

不修改 Qwen3-8B 本身，只在模型外面增加一个负责近似后半层 KV 的小模型，Prefill 时间就缩短了接近一半，最终输出保持一致。

这件事对我们在 DGX Spark 上部署本地模型的意���很大。

本地模型现在最折磨人的地方，经常不是逐字生成有多慢，而是第一次加载长上下文时，要等很久才能吐出第一个 token。

系统提示词、Skills、工具定义、项目代码、历史对话和检索文档一起塞进去以后，模型需要先让整段输入经过所有 Transformer 层，为每一层计算 KV Cache。

这一步就是 Prefill。

上下文越长，Prefill 越重。到了几十万 token，哪怕后面生成速度还能接受，前面那段等待也足以让本地 Agent 用起来很难受。

DeepSeek V4.1 Flash 的新结构，把模型拆成前后二十层。

前半部分像 Causal Encoder，负责处理输入。后半部分在生成阶段继续承担完整推理，但输入阶段使用由 Encoder 最终状态投影出来的共享 KV，不再让长提示词完整跑遍所有后半层。

官方模型因此只在 Prefill 阶段激活约 8B 参数，生成阶段激活约 16B。它还进一步压缩并复用 KV Cache，把全局 KV 压到大约每 token 890 字节。

原本我以为这种收益必须重新训练整套模型才能获得。

这次实验真正让我大开眼界的地方，在于研究者直接对现有 Qwen3-8B 做了近似。

模型权重不用改。

只要额外训练一个近似模块，让它根据前面得到的状态预测后半层需要的 KV，就有机会跳过一部分重复计算。

如果这个方法能在更多模型和任务上稳定复现，未来 DGX Spark 上的本地部署可能会进步得非常疯狂。

我们不一定要等下一代模型发布。

Llama、Qwen、DeepSeek 蒸馏模型，以及各种已经完成微调的垂直模型，都可能通过外挂近似模型获得一次推理升级。原来的模型继续负责生成，新增的小模型负责压缩 Prefill 成本。

这种优化非常适合本地 Agent。

Coding Agent 每次都要读取大量项目文件。Computer Use Agent 会反复携带操作历史和界面状态。Deep Research 会装入长文档。多 Agent 系统还会重复发送工具定义、记忆和任务背景。

这些工作负载都在不断支付 Prefill 成本。

如果 Prefill 真能普遍缩短一半，我们在 Spark 上打开大型代码库、加载十几万 token 的上下文、运行长时间 Agent 时，首 token 等待会明显下降。同一台机器也能更快处理并发任务。

更令人期待的是 KV Cache。

DGX Spark 有 128GB 统一内存，能让许多大型量化模型跑起来，但长上下文和多个并发 Agent 会迅速吃掉 KV Cache。模型权重勉强装下，不代表它还能给多个会话留下足够的上下文空间。

DeepSeek V4.1 Flash 已经证明，Transformer 不需要永远为每一层保存一套完整 KV。后半层状态可以投影，可以共享，也可以近似。只要近似误差受到控制，过去被认为必须保存的东西，可能有很大一部分都能省掉。

对 Spark 这种内存容量很大、带宽却远低于顶级数据中心 GPU 的设备来说，这类架构优化可能比单纯增加算力更有价值。

少读一半权重，少搬一大块 KV，减少长提示词经过的层数，带来的不只是跑分快一点。它可能直接改变哪些模型能够在单机上实用。

当然，现在还不能把这次结果理解成 Qwen3-8B 已经全面提速两倍。

作者展示的是 Prefill 时间减半，不是生成速度翻倍。测试规模、上下文长度、任务覆盖和输出一致性的评价方式也需要进一步公开。近似模块能否在代码、数学、长文本检索和工具调用中都保持可靠，还要看更完整的实验。

它也不能让 552B 参数的 DeepSeek V4.1 Flash 突然塞进一台 DGX Spark。模型权重、Engram 参数和推理框架仍然有很高的部署门槛。

可这次实验已经展示了一个非常诱人的方向。

本地模型优化以后可能不只靠量化、剪枝、推测解码和更快的推理引擎。我们还可以给已有模型增加一个轻量近似层，重新安排它处理输入和生成时使用的计算量。

模型训练完，并不代表它的架构红利已经全部用完。

未来我们在 Spark 上部署同一个模型，可能今天还要等十秒读取上下文，几个月后换一个推理框架，再外挂一个近似 Encoder，时间就只剩五秒。继续叠加 KV 量化、缓存持久化和推测解码以后，整套 Agent 的体���可能完全不同。

硬件没有换。

模型权重也没有换。

真正疯狂进步的，可能是我们终于学会了怎样少算一点，同时尽量不损失模型原本的能力。

[Milan Kryl @mikr]

@0xBakeer @FrostForger I had similar problem with pi harness, switched to qwen code, but there are timeouts after 15 mins. Tried qwen code w qwen3.8 27b w gpu and similar issues there. Need to look under the hood.

[Minseon Kim @kim__minseon]

4B parameters. ZERO distillation. 61.5% on SWE-bench Verified 🤯

Meet FrogNano 🐸: Qwen3.5-4B post-trained purely with RL on synthetic tasks from TaskPilot.
Just 5 iterations × 300 tasks.

Who said coding agents have to be huge? 🐸

[Mia @MiaAI_lab]

This is for you single DGX Spark owners 💫

You can now run Qwen3.8 Flash NVFP4 on one DGX Spark with great performance!

- Up to 1M context, 1,431,164 KV cache
- Full image & video support
- 37 decode tok/s on prose single stream
- Up to 86 tok/s on prose 4 concurrent streams
- 1500-2000 tok/s prefill on any size.
- Stress tested with 400k prefill - passed.

IMO this is the BEST model to run now on a single dgx spark. It's better than Qwen3.8-27B, and also better than the 1x dgx spark version of DeepSeek v4 Flash.

Get it here:
https://t.co/LG0I9PJKxI

[Mark Zuckerberg @finkd]

Muse Voice Transcribe is MSL's first real-time audio perception model -- rolling out today. SOTA in streaming speech-to-text, it handles speaker diarization, and endpointing natively in a single model.

[Milan Kryl @mikr]

One week with an ASUS Spark and having fun experimenting with local AI.

Tested Qwen 3.6, Qwen 3.8, DeepSeek, plus Hermes Agent, Pi and OMP.

Still exploring the ecosystem. What should I try next? 👀 #LocalAI #LLM

[kyutai @kyutai_labs]

If you're looking for a weekend project, how about training your own text-to-speech model from scratch on your own GPU, and then running it on any device's CPU?

We just open-sourced the entire Pocket TTS training stack: data pipeline, recipes, and evals.

It learns pretty damn fast:
~15k steps: babbling starts turning into words
~50k steps: it reads anything you type (WER under 1%)
~200k steps: the voice stops sounding synthetic

On a beefy consumer GPU, that's a week of training. On eight H100s: 10-20 hours. A TTS training run will cost you less than $200 if you rent your hardware, and an order of magnitude less if you just pay for power.

Some things we'd love to see people try:
- Train it in your own language (a few hundred hours of speech gets you surprisingly far).
- Add new features to Pocket TTS (Emotion tags? Make it sing?).
- Beat us at our own game: make it faster and smaller.

Show us what you build! We'll highlight the best models and new languages for the whole community to enjoy. Pocket TTS has already found many use cases, from reading for people with visual impairments to making NPCs in video games talk, and we're sure there's much more to do with it!

Here's an example of a Czech Pocket TTS. Try just asking your favorite agent to find data and apply the method, and you can have your own.

Get started: https://t.co/3EH3sbKNRU

[Qwen @Alibaba_Qwen]

⚡Meet Qwen3.8-Flash, a multimodal MoE and an early preview of the Qwen4 architecture, now open-weight!

The production version Qwen3.8-Flash will be available soon via QwenCloud API at just $ 0.16/1M input tokens and $ 0.47/1M output tokens.

125B parameters + 51B N-gram embeddings, with just 6B activated per token. Unmatched cost-efficiency.

What's new: 🥳
- Next architecture: GDN + QSA hybrid attention, Gated Residual, N-gram Embedding & Muon optimizer, serving as a precursor to the architecture used in Qwen4.
- Dramatically lower training and inference costs: trained at just 1/9 the cost of Qwen3.7-Plus, while outperforming it across the board with especially strong gains in coding and office tasks.
- Strong performance: scoring 58.7 on DeepSWE 1.1, 62.5 on SWE-bench Pro, 73.9 on CoWorkBench, 84.5 on AndroidWorld, and 95.7 on MathVision (with CI).
- 262K native context, extensible to 1M with YaRN.

We’re also releasing the weights for Qwen3.8-Flash-Next, giving the community an early look at the new architecture we’re exploring for Qwen4.🚀

We can't wait to see what you build with Qwen3.8-Flash!👀👇
- Blog: https://t.co/M5hYypFLgJ
- Technical Report: https://t.co/IF0gObIkQO
- Hugging Face: https://t.co/6ow8QVAABt
- ModelScope: https://t.co/tDOn2jNuFG

[🍂 @Lovandfear]

There is a Japanese practice of owning one very good version of something rather than several adequate ones.

One knife, kept sharp. One pen that writes exactly as you want. One bag chosen carefully and used for years.

This is not minimalism as aesthetic. This is minimalism as relationship.

You learn the object. You know its weight. You know how it responds to your hand.

The object becomes, over time, an extension of you.

That cannot happen with things you do not stay with long enough to know.

[Mia @MiaAI_lab]

A BIG moment for all DGX Spark users ⚡️

You can now run DeepSeek v4 Flash 0731 without needing a second unit, with quality high enough for reliable code generation, high context, and great speed!

Optimized for single stream session:
- EXL3 quantization
- 384k context (conservative) / ~440k kv cache (!)
- 47 tok/s single stream (structured)
- 1024 tok/s prefill
- 370k token needle test passed (super stable!)

Thanks @0xSero for this excellent EXL3 quant! Tuned for SparkInfer + DSpark speculative decoding, this is about the same quality of a Q4_K_M / Q5 GGUF!

All this is possible thanks to native NVFP4 KV cache & fixing kernel bugs in the upstream prefill path to make it work at all.

Get it here:
https://t.co/531EaR6FWJ

[Milan Kryl @mikr]

@jurbed @slushcz try dflash 2, it should help a bit
https://t.co/WeAaatSOch