Yoav Shmariahu

Building Thinker to save coding agent tokens

[Yoav Shmariahu @shmariahu]

@BoboLinNotes AI does review too though. But I think testing can take a while if you're deploying services to a staging environment

[Yoav Shmariahu @shmariahu]

Your coding agent is wasting tokens.
Thinker cuts the waste: 20% fewer tokens on real coding tasks.

[Yoav Shmariahu @shmariahu]

I've caught myself burning through my claude max subscription super quick and realized my vibecoded projects were burning tokens like crazy.

So I made https://t.co/jnGrzgAb49 which looks at all my coding sessions and gives me suggestions for where to cut down instruction files, or helps me compress context when it detects waste.

With respect to tool compression, thinker is in the loop with my agents and updates context on the fly so subsequent model calls end up with trimmed down context then what was previously outputted.

This can be a tricky thing to calibrate since you can take a hit on input caching. Still there's a lot of value you can get from compressing lots of outputs from bash commands. Thinker will also flag MCP servers or instruction file sections that produce little value but inject lots of context into every agent.

So far I found around a 20% cut down on my token usage to task completion. Give it a try if you're feeling the pain of usage limits like me

[Yoav Shmariahu @shmariahu]

@calebestes_rp @pcshipp The last couple months its been much better at doing sensible feature work, opus used to require a lot of review and iteration. Bit better now though.

[Yoav Shmariahu @shmariahu]

@Bober_smart This doesn't seem super scalable vertically or horizontally. Probably needs a graph db like neo4j to make it at collaborative scale and able to store lots of context with fast retrieval. Still think the problem is non-trivial once context goes up beyond 50Gb or so.

[Yoav Shmariahu @shmariahu]

Watching gemini work on my codebase after running out of claude and codex tokens

[Yoav Shmariahu @shmariahu]

@aryanverma25_ You use any interesting tools? Claude-in-chrome is good but a bit slow and spends a ton of tokens

[Yoav Shmariahu @shmariahu]

It was mostly sed, read, and cat from the bash commands, super generic reads and write. Probably a lot of straightforward ways to cut out most of the context but I think the edge cases can get tricky to prevent causing any confusion for the agents or taking losses via less prompt cache misses

[Yoav Shmariahu @shmariahu]

The past month I spent 40% of my token budget on bash command outputs...

It's expected that models frequently use bash to edit or read files, but keeping all outputs in my context feels like throwaway usage.

There are a few tools that compress command outputs e.g. headroom or beacon, and achieve impressive savings. I did it myself with Thinker and cut down input token usage by 18% on 14 open-source PR benchmarks run 3x each.

I only caught this by looking deeper into my token usage. Visibility & controls on your coding agents are super important imo - you might have a rogue tool or messy AGENT.md burning a bunch of your tokens.

[Yoav Shmariahu @shmariahu]

I'm building Thinker, a knowledge cache for coding agents. On every fresh session, agents rebuild their understanding of your codebase from scratch, then throw it away. Thinker keeps it.

It distills past agent sessions and PRs into short notes about how your code works, e.g. all DB writes use `/sql-utils.py`. A small local model ranks the notes and injects the relevant ones into new sessions, so the agent spends less time exploring. Notes are rescored and updated as PRs merge, so stale context gets demoted.

On 50 PRs from open-source repos, agents using Thinker (Fable 5.1, GPT-6 Astra, Gemini Flash 3.8) used 18% fewer tokens and 10% faster on average with equal correctness using LLM-as-a-judge. Per-model results and the eval harness: https://t.co/au6kgLKjmF.

Repo: https://t.co/jnGrzgAITH. Install: `npx @zerotimedev/thinker`
Unlike AGENTS.md or built-in memory, notes are generated and maintained automatically. Next I want to work on rigor, not just efficiency.

[Erling Haaland @Erling]

Goodbye 🇺🇸 It’s been emotional!

[Eric Daugherty @EricLDaugh]

🚨 WOW. Brown University student Alex Shieh previously EVISCERATED the school's administrators for having a $46 million deficit, despite surging costs for students.

They pay $90K+ PER YEAR.

Alex Shieh: "What about the kids who weren't born on third base?! [...] Brown is on track to run a $46 million DEFICIT this year. WHERE is all the money going?"

"I'll tell you where it's going. It's going into an empire of administrative bloat and bureaucracy! Brown employs 3,805 full time non-instructional staff for just 7,229 undergrads. That's one administrator for every two students."

"This isn't education. This is bloat paid for on the backs of students and families who are mortgaging their futures for a shot at a better life!"

HE'S SPOT-ON.

[Brian Armstrong @brian_armstrong]

Excited to learn more. Still forming an opinion on asset allocation, but my current thinking is:

1. Just Bitcoin would probably be the best option - simplest, and clear story as successor to gold
2. If folks wanted more variety, you could do a market cap weighted index of crypto assets to keep it unbiased

But probably option #1 is easiest

[ABC7 News @abc7newsbayarea]

Massive search to resume in Cupertino for missing 17-year-old Connor Sullivan: http://t.co/bbL4WDyjUT

[Yoav Shmariahu @shmariahu]

@kathrynbziegler @alvinluuii @justin136lin i just got back from Hawaii and fuck u guys