For the first time since Claude Code came out, I moved one of my actual work pipelines to @pidotdev & open-weight models.
And after a weekend of fighting with it, I prefer the report it made.
36 pages vs 21 from Claude, more information-dense, prose I liked more, and pennies compared to the Claude API.
It does not save on speed. 30-40 minutes instead of ~20. But it's running locally-adjacent/off resources I have so that's just fine.
Setup was one 8xB200 node split 4/4 between:
- GLM 5.2 NVFP4 (main agent/driver)
- Kimi K2.7 Code NVFP4 (retriever).
The dumbest fix throughout it: I summarized sources into briefs to save on context, then notes, losing information each time. Ended up saving all articles directly to disk so there were multiple layers of information retrieval I could work with.
You can now run @supermemory locally.
Introducing the supermemory local
- Fully self-contained. Comes with our graph engine, embedding model, etc.
- Run on any machine, with your @openclaw, hermes, claude, etc.
- SDKs to add memory to your agent, or build your company brain.
I didn't expect DeepSeek v4 PRO (not Flash) to run well on the Mac Studio M3 Ultra with 512GB of RAM. This is 2 bit quantized with the same DwarfStar recipe used for Flash. 433GB GGUF file. 130 t/s prefill, 13 t/s generation. Prefill in the video is low because small prompt.
recommended reading. strongly recommended reading.
i really like the pain avoidance angle. slots into my "paon/friction is when you learn" angle. when combined > cognitive debt.
https://t.co/clzPIgXaSU
people of pi, i hated pi. i am not a fan of those lightweight concepts
but i notice that i can make those bloated as you guys hate to do so, which i think now it is "sane"
so i created senpi, like the sane-pi + senpai
https://t.co/hLtMTUAONy
my fave point from here: the earlier you think about your agent as a system that can be measured & improved, the faster you can get a robust agent into production
This isn’t just a technical thing, it’s a human & team thing. Teams that succeed here ask questions like:
- “what do I need my agent to do to make our customers happy?”
- “What scenarios will my agent encounter in the wild and how can we recreate that in our testing”
Evals are the substrate that determines what your agent does in production. They’re training data for agents because we literally fit our agent to pass Evals via hill-climbing algorithms and human edits to pass failure modes
Once you get your agent into users hands, the eval generation loop compounds. Production data uncovers more issues, these issues turn into Evals, and the agent fits to improve over more cases that could not be captured without real user data.
In the early stages, teams dogfooding their product becomes the feedback signal
Curating evals and running experiments on different agent variants is a muscle every team develops, our goal is to create tooling so every team can create the best agents for their tasks using this foundation
Step 1: ask an agent to help you setup sqlite on a vps. Serverless/fly machines also great as long as there is persistent disk volume.
Step 2: ask an agent to help you create data backups to S3 (or wherever)
Step 3: thank your agent
Update on Ollama Pro
Specifically K2.6 limits I have ran ~56M tokens output and got 9% weekly usage
That comes out to ~392M tokens per week
At $3.5/M (typical api costs) that’s $1372 in output per week on a $20 subscription
This is running hermes essentially flat out code gen, you’ll hit the 3 hour reset window, and would likely hit weekly window after a couple days. Seems like insane value for the token though so far
Most teams measuring AI productivity are counting the wrong thing.
Lines of code. Time to first commit. Keystrokes saved.
The real wins are quieter. @coldopn wrote up 7 of them. 🧵