Running local LLMs + AI agents inside real trading infra. Real benchmarks, real failures. Building TransactCloud · RTX 5090 · Qwen · vLLM · Claude Code
Qwen3.8-27B, one RTX 5090 (32GB). 180K context. Production for my day job.
decode: 92 tok/s measured (median of 5, 512-token window, thinking off) — MTP on, fp8 KV, vLLM 0.24.0.
The interesting decision wasn't the model. It was the KV cache. I turned down +97% of free context.
Waymo was born too early.
The HD map was a brilliant crutch: centimeter-perfect, hand-checked, roof-lidar-dependent… and impossible to scale.
12 minutes on why the map is proof that the intelligence is missing, and why FSD drives anywhere. 🎥👇
New in Lightsprint: Console Mode 🔥
Most agent tools bury you in a chat or a board. Never both.
Console mode: flip from kanban to an IDE-style view. Agent session on one side, live preview on the other. Same task, same PR.
Built for developers who want to see the work, and for PMs who want to track the tasks on the board.
@MrMilkTrading Add 2/2: every-session control flips sign at the IS/OOS seam (t -0.13 -> +1.53), so part of the conditional edge is base-rate drift. For the paper sleeve: pre-commit the kill rule — 2022-06-13's -83 pt no-stop night is the reference case that gets rationalized after the fact.
@MrMilkTrading Receipts above the norm — DSR/PBO included. Add: "at least 3 lower closes" beats the picked cell in-sample (t 2.1 vs 1.7, 240 vs 136 trades) and holds OOS at 1.7x the n. If the mechanism is fresh-decline exhaustion, the tradeable rule is the streak band, not the exact cell.
The interesting part of running big local models isn't the hardware.
It's knowing which numbers to trust — and which "free" upgrades to refuse.
Full write-up:
https://t.co/NZub33kOjv
Now the real question: 4-bit KV.
TurboQuant works on this stack (hybrid hard block removed in 0.24.0).
- k8v4: +32% context (251K)
- 4-bit: +97% (262K native window)
Published PPL cost: +1.2% to +2.7%
That's free context. The question is whether it's actually free.
The numbers:
- APC: 100K warm turn 57.7s → 2.4s
- MTP n=3, acceptance 0.64–0.78
- 100% of APC cells correct (6/6, 100K/160K × thinking on/off)
- 78.3% of prompt tokens served from cache (first day)
The 7GiB fp8 KV pin is what makes the 188K pool and 180K context real.
Why fp8 KV: this checkpoint ships calibrated fp8 KV scales (static minmax baked in).
My previous model ran uncalibrated (scale 1.0). I didn't take it on faith: BF16-vs-FP8 harness, 30 generations, no measurable gap.
Real scales is why the dtype risk dropped.
Qwen3.8-27B, one RTX 5090 (32GB). 180K context. Production for my day job.
decode: 92 tok/s measured (median of 5, 512-token window, thinking off) — MTP on, fp8 KV, vLLM 0.24.0.
The interesting decision wasn't the model. It was the KV cache. I turned down +97% of free context.
The name says NVFP4. The weights say otherwise:
- fp8: 10.77 GiB
- 4-bit NVFP4: 6.97 GiB
- bf16: 3.21 GiB
Total: 20.95 GiB
Only a third of the checkpoint is actually 4-bit. Vendor claim vs measured weights — a lesson in itself.
We’re launching the missing team layer for AI-powered development 🚀
The SDLC was built around coding. Coding has changed, and the SDLC must be updated.
Lightsprint makes agentic development collaborative with visual plan mode, parallel cloud agents and PR preview environments.
Try now: https://t.co/U2srBAwYxq