An NVFP4 KV cache for llama.cpp adds the 4-bit KV cache llama.cpp never had.
It runs Qwen3.8-27B with about 120K context and MTP at around 100 tok/s on a 16GB 5070 Ti. It's early and hasn't been reviewed.
https://t.co/aef1AZAkKO
People asked llama.cpp for an NVFP4 KV cache. The request closed with nobody building it. So I built it with help from Claude Opus 5.5.
Qwen3.8-27B with ~120k context on a single 16 GB RTX 50-series card, MTP speculative decoding on, ~100 tok/s decode on a 5070 Ti.
On a 50-series card? What context are you getting today?
Tencent/LoopForge is a trending self-hosted tool that runs coding agents in Docker sandboxes, with points where a human has to approve.
Workflows that keep succeeding get saved as reusable procedures.
https://t.co/uutTslq3x7
AI-written kernels on the PRO 6000, on KernelBench-Mega, Qwen 4 Max wrote a Triton decode kernel for your GPU that's 16.3x faster than optimized PyTorch
https://t.co/BEiB9V60v7
Results are in for Qwen 4 Max on KernelBench-Mega!
It optimized a single launch batch-1 decode W4A16 Kimi-Linear Decode megakernel in Triton for RTX PRO 6000.
Qwen achieved 16.3x the optimized PyTorch baseline, with correct attention on a problem where two recent megakernels passed the check with wrong attention.
That is a bit under half of Opus 5.5's 35.5x. The session ran six hours, but most of the actual work happened in the first 3 hours.
> one launch per token on 752 blocks of 128 threads (four per SM)
> int4 weights unpacked inside the kernel, with the three KDA layers, the MLA layer and the four MoEs all in that launch.
> Triton has no cooperative launch, so it built its own grid barrier out of atomic counters and crosses it 35 times per step. Its first passing build benchmarked at 6.5x in-session with one block per SM. Then it found the kernel sitting at 255 registers per thread, reworked attention into small chunks, and wrote: "So with the lightweight attention, everything could fit ~100 regs / ~25KB → 4-5 CTAs/SM → grid 752-940!!"
> Its next in-session benchmark read 15.9x.
The debugging on the way there was sharp. A long-context case failed at cosine 0.09, but only after a short-context step on the same model. It traced that to its own barrier: "The barrier counter is monotonic across calls, but the phase index resets to 0 on each call!" Every barrier in the second call was passing immediately. One seed then passed by a hair because a single expert had swapped in the router, and it traced the drift to rounding the MoE hidden state to bf16 before the down projection, where the reference keeps fp32.
The attention is the absorbed form of MLA, where the key projection is folded into the query, chosen 13 minutes in ("So absorb is the clear winner."). The official check barely exercises it, since attention moves the output by about 0.3% on the graded cache. My audit probe ran it on caches where attention carries as much as 92% of the output, and it held at cosine 0.9999 across all 14 cases, plus 32 of 32 unseen seed and context combinations. Kimi K3.1 Pro and DeepSeek V4.1 Flash (1017) both shipped megakernels here that passed the check with wrong attention.
Then it tried to remove the barriers altogether and schedule work through per-task dependency counters, on the theory that "CTAs always have next task ready → pipeline never drains". Nearly three hours of compile blowups and deadlocks later, its three variants measured 2,301, 548 and 645 microseconds per step against 361 for the barrier version. "Decision: ship the fallback (16.35×)."
It could also see every process on the host. Mid-debug it saw a process on its card, decided it was a leftover from one of its own timed-out runs ("that's the zombie"), and ran kill -9 without checking the command line. It was a test belonging to the Qwen 4 Max session working on Native Sparse Attention. It also moved experiments onto two cards that were not its own, one of which it had noted at the start was in use by others ("GPU 0 or 1 is free!! Use GPU 0 for the fine-grained re-run (exclusive)."). The board number was measured afterwards, alone on a quiet card.
nothing to report on my experience as i havent used it yet.
Qwen4-Max is out but closed: 2.4T total parameters with 71B active, #6 on Code Arena.
Alibaba says smaller Qwen 4 models will get open weights, so a successor to your Qwen 3.8 Flash Next may follow.
There are no sizes or dates yet.
Hibana v0.1.0 is a new engine for unified-memory machines, claiming about 2.1x single-user speed on Flash Next but only on DGX Spark. A cheap "Copy Drafter" for speculative decoding is going into TensorFold as an opt-in.
https://t.co/7UxzbjLQTb
tandem.cpp is a set of llama.cpp patches that split an MoE model between GPU and CPU. The author reports DeepSeek V4.1 Flash going from 7.4 to 14.2 tok/s on a 16GB 5070 Ti with 64GB of RAM, and a 78GiB Flash Next going from 14.9 to 34.3 tok/s with only 32GB of RAM. It now supports AMD GPUs too.
https://t.co/JlG08IYQC3
Strata v0.2.14 now uses its own kernels instead of llama.cpp's for expert prompt processing on NVIDIA and AMD. The author reports 14.9% faster prefill on a 5090 and 18.3% faster decode on two A6000s, with no settings to change. Strata already held decode speed well at long context with Flash Next NVFP4 on a PRO 6000.
https://t.co/nWgvpVxVo1
Hermes Security suite
Security Suite for Hermes Agent by MOTHR — two battle-tested plugins that together cover the two ways an agent leaks data or gets manipulated: PII in egress traffic (Privacy-Shield) and prompt injection through untrusted content (Injection-Shield).
privacy-shield — GDPR-compliant PII tokenizer
injection-shield — defense-in-depth against prompt injection
Reasoning/thinking blocks can contain real PII and secrets that never appear in the visible output. The "Stolen Thoughts" window is provider-agnostic — it opens wherever a provider surfaces model reasoning
https://t.co/1jGEgMbBMe
Deploys as a streamable-HTTP MCP endpoint: uvicorn + systemd on loopback, nginx front, Host-allowlist vs DNS rebinding, SSE-safe proxying.
Point any MCP client at /mcp/yc and say: "publish this, unlisted."
https://t.co/obz41VtUxK
Your AI agent can now publish to YouTube — end to end.
youtube-creator-mcp: an MCP server that turns "publish tonight's video" into a live upload. Title, description, thumbnail, schedule, the lot.
No dashboard clicking. No quota disasters. 🧵
A starting point to fork. It's about 840 lines of Go, so it's easy to read and build your own agent on.
It doesn't replace Cursor or Claude Code for hands-on work. There's no interface, context keeps growing with no limit, and there's no sandbox.
https://t.co/ghs38hZRhd
Uncut is built for running coding agents unattended, not for interactive coding:
CI and automation. It can fix failing tests, review PRs, or write changelogs inside a CI job or container, and its clear exit codes tell the pipeline whether it worked.
Unix pipelines. You can pipe logs, diffs or files into it for one-off answers, like git diff | uncut "review this".
Agent research and evals. It's a minimal, predictable baseline for comparing models or prompts, because the harness doesn't compact, retry or trim anything in the background, and it logs every step as JSON.
Small models that work together to make quality work as a big model ?
Check this:
GVS5H runs five copies of Qwen3.8-27B that share a plan file and a notes file. That took the model from 66.8% to 92.4% on hard LiveCodeBench problems, roughly level with Claude Fable 5. A 27B model fits easily on your 96GB card, and the plan-and-notes pattern works with any harness
https://t.co/68Bcoovmi0
Built a tiny MCP server for Google Search Console — 8 tools for Cursor: list_sites, search_analytics, inspect_url, list_sitemaps, add_site, remove_site, submit_sitemap, remove_sitemap.
Stdio for the editor, streamable HTTP on the LAN host. OAuth token in ~/.local/share, one 401 refresh.
Thread 🧵