Kevin Theo

I specify, verify and operate the AI agents that run my company. 4x DGX Spark in a switchless ring. open source: lookout, kb-build, agent-ops-kit

[Kevin Theo @kevintheo]

moved my 4 sparks from tensorfold to vllm today, @christopherowen's spark-ds41f on the 64k-page kernel. tf stays installed as fallback. ran a few tests tonight, everything in the video is replayed from the logs

[Kevin Theo @kevintheo]

@Dennis_Rye I'm curious what your camera shows. I have no ZGX Nano, so I can't compare. In a 105 min run with up to 16 requests on DeepSeek V4.1 Flash, my GPUs hit 79 °C and the clock dipped to 2,171 MHz. I cap it at 2,200 MHz.

[Kevin Theo @kevintheo]

For the quality side, I tested GLM-5.3 on its maker's hosted service, not on my own hardware. On 34 Python exercises from aider's polyglot benchmark, with hidden tests and up to 2 tries, it solved 31, my local DeepSeek V4.1 Flash 31 and Opus 5.5 all 34, from 1 run each. https://t.co/4VquqZsfTd

[0xSero @0xSero]

GLM-5.3 NVFP4 (not flash) on 4x RTX 5090

- 26 tok/s decode c1
- 1450 tok/s prefill
- 1 million token kv cache

[Kevin Theo @kevintheo]

@fharper @IronicBadger @NousResearch I like that idea

[Kevin Theo @kevintheo]

@stackdropai @rustlang Lookout already does this. Every window has a label with its state and how long it has been there. A stopped window gets its own clock, like 12 min. In the Waiting area, questions come first, then stopped, then done, and the longest wait is on top.

[Kevin Theo @kevintheo]

Lookout 4.0 is out. It lines up my Claude Code, Codex, Grok and Kimi windows in Warp or Ghostty and frames them mint when done and amber when waiting for me.

It is rewritten from Swift to Rust (@rustlang) with the same behaviour: on 41,293 real reply texts from my own sessions, the word rules give exactly the same results.

New is an optional parking lot, built for 20 to 25 workers: worker windows wait behind the active session and come to the front when done or asking something. Ctrl+Opt+4 shows all parked windows as tiles.

I designed Lookout and decided every step. Claude Code (@claudeai) wrote the code. Codex (@OpenAI), GLM (@Zai_org), Muse (@AIatMeta) and Grok (@grok) reviewed it in rounds. The last round found real bugs, like a screen change moving windows while Lookout was paused. All were fixed before the release.

Lookout is for macOS only; just the core is platform neutral.

[Kevin Theo @kevintheo]

@steddyman @christopherowen DeepSeek V4.1 Flash 1017 runs on my 4 Sparks with the official weights, no quant. On release 1107 of the vLLM build, a cold prompt of 1M tokens takes 3:02 until the first token. Will you share the performance with your Mac with us?

[Kevin Theo @kevintheo]

@Bobo_hashtag @mrru5s3ll Appreciate it, precompiled kernels for Spark will be great

[Kevin Theo @kevintheo]

@plavecmario3 @deepseek_ai Matching both arms like that would be the cleaner test

[Kevin Theo @kevintheo]

With thinking on, 34 Python exercises took 22 minutes 7 seconds instead of 83 seconds, about 16 times as long (median of 3 races each).

I used 4 DGX Sparks in a switchless ring config, DeepSeek V4.1 Flash with the official weights from @deepseek_ai and release 1107 of @christopherowen's vLLM build. 16 agents worked at once, with at most 2 attempts per exercise.

Without thinking, all 3 races solved 25 of 34. With thinking they solved 29, 27 and 30, using more than 11 times as many tokens.

Task by task over the 6 races, 20 were solved every time, 7 only with thinking, 5 always without thinking but missed at least once with it, and 2 never.

Thinking was not the only change. With it I used temperature 1.0 and 16,384 output tokens per attempt, without it greedy decoding at temperature 0 and 4,096. That is 3 races each way, 1 evening, 1 model, 1 build. Only the unit tests judged.

[Kevin Theo @kevintheo]

@jus_eng_njainam @typesafeai You are right, 40 is a small set and 40 of 40 is not proof of zero misses. There is no larger indirect only test set yet, so please treat my number as a first data point, not a guarantee.

[Kevin Theo @kevintheo]

I use a closed choice model, Jev by @typesafeai, as a prompt injection guard for my email agent. In one test on 1,102 texts it found 82.6% of attacks and 40 of 40 indirect ones with 0.7% false alarms (old keyword filter 27.3%). Curious how Gemini Decision 1 does on that job. https://t.co/h22k5SvrgI

[Sundar Pichai @sundarpichai]

Today we’re launching the Gemini Decisions API, powered by our new Gemini Decision 1 model in @GoogleAIStudio.

Instead of generating text token by token, it’s built for closed-set choices scoring up to 1,000 questions across text, images, audio, video + PDFs in a single pass. A few details:

- 3 decision types: yes/no, multiple choice (up to 255 options) + rubric scoring
- $0.10 / 1M input tokens with $0 billed for output tokens (~5x cheaper than Flash-Lite)
- leads on several benchmark evals with lowest calibration error

Google teams are already building with it, below is a look at Stitch Rapid designing a full product flow in seconds (1x speed)

[Kevin Theo @kevintheo]

@Joshua_WD You are right, my 30 invoices were the easy kind, just clean text like a PDF export, no scans and no photos. Crumpled phone photos are on the way. Prept the tests already.

[Kevin Theo @kevintheo]

On 30 made up invoices my local model on 4 Sparks read all 300 fields right, like Opus 5.5, but it was easy text from PDFs, no scans. Which office job should I test next against Claude?

[Kevin Theo @kevintheo]

@master_jpma As promised, Draft Desk is out, a Mac outbox where an agent hands over a draft with one command, and I read, edit and send it with a click. The command line can never send. After the click I have 15 seconds to undo. It is free and open source under MIT. https://t.co/zr51bNuPt4

[Kevin Theo @kevintheo]

I would not let an agent run a whole business alone for a week. What runs alone at my company are narrow jobs with fixed rules. Since 9 October, a script sends my clients the promised e-book in their language, right after Amazon confirms shipment. Since this month, a @Raspberry_Pi at my vacation apartment wakes the TVs after every checkout, logs guests out of Netflix, YouTube and Prime Video, resets the settings and switches them off. Agents draft replies for my 5 e-mail accounts and I read and send each one myself. Il created a tool where the draft is created and i can send it from there or change whatever i want... i will publish "Draft Desk" in the next days.. In my test with 30 made up emails, the best cloud model wrote 29 without breaking a brand rule and my local model managed 7 out of 30... so there are ways to implement "AI" or Automations inside your business with the help of ai.

[Captain Rational 🧮📐✏️ @noahseidman]

Pretty cool. Spin up local TensorFold, find a model already running (it'll even find remote models via ssh), or good ole Claude.

[LMSYS Org @lmsysorg]

Muse Spark Open: 517 → 1,109 tokens/s on 8× GB300 with SGLang megakernels.
One kernel launch per layer, overlapping compute, memory loads and communication. Total launches per decode step drop from 1,450 to 97.

https://t.co/YXb3pGDKH3

In our new technical post, Pranjal Shankhdhar explains what makes this work:
- Prefetching weights into shared memory and L2 before they’re needed.
- Warming the instruction cache so large kernels don’t stall fetching their own code.
- Using data itself as a readiness signal for cross-GPU communication.
- Fusing the draft model to make drafting 2.6× faster.

The post also includes measured timelines, CUDA/PTX examples, and the tradeoffs behind stopping at one megakernel per layer.

[Ash Hart @ashxhart]

🚨900+ tks decode on Tensorfold with a 5090 and Qwen3.8 27b nvfp4 🥹

[Kevin Theo @kevintheo]

@XniX Always nice when a setup just feels right 🔥

[Kevin Theo @kevintheo]

@steddyman @christopherowen I use no quant, the official FP8 and MXFP4 weights of DeepSeek V4.1 Flash on 4 Sparks, so not like 1.75 bit on 1 Spark. On release 1107, 4 streams with about 131K context each gave 137 tok/s in total on the ring. Thanks for sharing the numbers.

[Kevin Theo @kevintheo]

@plavecmario3 @deepseek_ai Not yet, I haven't run thinking at temperature 0. With thinking on I used temperature 1.0 and 16,384 output tokens per attempt, greedy at temperature 0 and 4,096. So the post mixes the effects. You are right, it would separate them.

[Kevin Theo @kevintheo]

@QwenAIFans @Bassmaster187 Wow, taking a V8 apart in the browser sounds so cool

[Kevin Theo @kevintheo]

@MiaAI_lab Let's hope the open weights follow soon, so we can make our own Tests

[Kevin Theo @kevintheo]

@niklaslenz_ai Side by side tests like this are my favorite kind of post, like always, good job Niklas!!