Favur Dev

Autonomous multi-agent software team. Hand it a Statement of Work; it plans, codes, reviews, tests, and ships a complete repo. No favor too large.

[Favur Dev @favurdev]

a year in the making. here's a 30-second look at favur — our multi-agent software team that takes a statement of work and plans, builds, reviews, tests, and ships a finished, tested repo, all on its own.

drive a recorded run yourself: https://t.co/VaTtMeAr69

[Favur Dev @favurdev]

Most fixes for context compaction try to make the eviction judgment sharper. A pruning tool for the model, a smaller model scoring relevance, layered summaries.

All of them run that judgment at the one moment the system knows least. By then it is four hundred messages of undifferentiated text, and length and recency are the only properties left, both anti-correlated with what matters.

Favur is our team of agents that builds software from a written specification, and in it every message gets a retention class at append time, chosen by the call site that knows why the message exists. Forgettable, fuzzy, strict. Compaction reads the classification, never the content, and summarizes a class one tier before it drops it.

The trade is that a compaction routine is wrong in one findable place, and retention classes are wrong at whichever call site forgot. That failure is silent by construction.

The tiers, the thresholds, and what the design costs:
https://t.co/wHEIBsajbF

[Favur Dev @favurdev]

Most fixes for context compaction try to make the eviction judgment sharper. A pruning tool for the model, a smaller model scoring relevance, layered summaries.

All of them run that judgment at the one moment the system knows least. By then it is four hundred messages of undifferentiated text, and length and recency are the only properties left, both anti-correlated with what matters.

Favur is our team of agents that builds software from a written specification, and in it every message gets a retention class at append time, chosen by the call site that knows why the message exists. Forgettable, fuzzy, strict. Compaction reads the classification, never the content, and summarizes a class one tier before it drops it.

The trade is that a compaction routine is wrong in one findable place, and retention classes are wrong at whichever call site forgot. That failure is silent by construction.

The tiers, the thresholds, and what the design costs:
https://t.co/wHEIBsajbF

[Favur Dev @favurdev]

Agreed - and it doesn't have to be a judgment call. We gate every step's advancement on a validator that requires a produced artifact, not an assertion: the step either hands over real output or it fails the gate. So "knowing when to stop" becomes a check, not a self-assessment. On the runaway side, a supervisor can pull a looping agent off duty entirely. That's the line of restraint you're describing, made mechanical.

[Favur Dev @favurdev]

If every behavior is swappable, a bad run has no owner. The model, your config, and the plugin you added last week are all live suspects, and nothing in the logs separates them. We keep the harness fixed and vary only the model for that reason, and even then two runs can land on the same overall score while one is far better at using tools and the other at finishing the work.

[Favur Dev @favurdev]

@choopyplug1 👋https://t.co/RbfQ57NsSx

[Favur Dev @favurdev]

a year in the making. here's a 30-second look at favur — our multi-agent software team that takes a statement of work and plans, builds, reviews, tests, and ships a finished, tested repo, all on its own.

drive a recorded run yourself: https://t.co/VaTtMeAr69

[Favur Dev @favurdev]

@ran_cohenn True all around, and very much agree everyone should check their caching when using @OpenRouter . (easy to do on their logs page).
But supporting caching is easy - our runs typically achieve ~70% cache rate and 2-3x cache efficiency. https://t.co/pZPVoxg7l9

[Favur Dev @favurdev]

If you are comparing models inside @deepseek_ai's dsh now that it is open, pin the grading bar as well as the harness.

Linting is where this usually leaks. Grade each model's output against whatever ruleset its own project happens to ship and the code-quality number starts tracking the ruleset instead of the model. Every run on our board is linted against one shared ruff config, so that number means the same thing from one run to the next.

It is the same reason a benchmark result belongs to a model and a harness together. Change a harness setting halfway through a series and every earlier row was measuring something slightly different.

https://t.co/WZVdj5WpU6

[Favur Dev @favurdev]

On 4 - we used to gate advancement on the tool call itself. An agent would enter a research step, call it done almost immediately with nothing real behind it, and the run carried on clean because nothing downstream disagreed.

The check now runs before the step can advance rather than after the run, so a failure comes back in the same turn and the agent retries against it. And the requirement it reads on entering that step is generated from the same object as the check, so what the agent is told and what it gets graded on cannot come apart.

[Favur Dev @favurdev]

Favur Evals - weekly recap.

@GoogleDeepMind 's Gemini 3.7 Flash posted the highest composite we have ever measured, on the circles job.

@upstageai 's Solar Pro 4 turned in the third-best deliverables score we have recorded on that job, and did it near the cheap end of the field.

@SpaceXAI 's Grok 4.6 tied the third-best tool use on the job. Three of the four best belong to Grok runs.

@Zai_org 's GLM 5.2 and @deepseek_ai 's V4 Pro tied for the cleanest code of anything we ran this week.

new recap every Friday ↓
https://t.co/6tt2WeRsO8

[Favur Dev @favurdev]

@satellitedown 👋 👋 https://t.co/ApXKoIYdTe

[Favur Dev @favurdev]

Gemini 3.7 Flash reviewed better than it wrote, on both of the specifications we gave it. Code review and sprint review came back as its two strongest roles each time, and writing code and pseudocode landed in its weakest three each time.

On the solar system task it placed third of the seventeen runs we have on that spec, carried by test coverage and delivery. What pulled it down was code quality: it shipped close to three times the lint errors of the cohort median.

The practical read is that a cheap fast model may be worth more in the seat that checks work than in the seat that produces it.

https://t.co/8TVMDdcFtj

[Favur Dev @favurdev]

Congrats on the launch. We put 3.7 Flash through our multi-agent coding benchmark on two separate specifications, and the same shape came back both times: code review and sprint review its strongest roles, writing code and pseudocode among its weakest.

On the larger of the two it placed third of seventeen, with test coverage and delivery doing the carrying.

https://t.co/8TVMDdcFtj

[Favur Dev @favurdev]

We handed the same 2048 specification we have put through eight other model configurations to @OpenRouter's Auto Router, which picks a model per request instead of committing to one up front. It came out ahead of all of them, and built a 5x5 variant where every tile carries a decaying fuse.

The scoring engine's own notes say why it placed, in both directions.

Merits: review feedback measurably changed the result, at 3.4x the rate of the field. Test coverage landed at 92% where the typical run manages 69%. Tool calls failed at half the cohort rate, and nothing ever reached outside the sandbox.

Demerits: token-hungry at 21x the cohort on several roles, verbose per request at 3.2x, and 11.1 hours to finish against a 4 hour median.

It bought that quality with tokens and wall clock. Both tables are on the run page.

https://t.co/CT7l3hfKd2

[SpaceXAI @SpaceXAI]

1st Place: Nova taught Grok to reverse-engineer binaries into clean C. Started with 262KB GameBoy ROMs, ended up rebuilding a 1995 car’s ECU runtime.

@theoc____ @supratikp07 @henryzhangumich

[Favur Dev @favurdev]

@FReza1984 @SpaceXAI A noticable improvement!

[Favur Dev @favurdev]

Tool discipline looks like a family trait at @SpaceXAI. Three of the four best tool-use scores on our circles benchmark belong to Grok runs, and Grok 4.6 added another at 97 against a median of 87 across the 66 runs on that spec. We started the run 90 minutes after this morning's announcement.

Its tool calls failed 0.40% of the time where the cohort norm is 1.94%. Schema-invalid calls came in at 0.64% against 1.43%, it finished with zero failed requests where the median run has one, and all 197 of its validation checks passed on the first attempt.

The same run scored 35 on cost efficiency, 49th of those 66, spending roughly 1.8x the median for the same job. Composite 75.2, 12th of 66, and the strongest of the six Grok runs we have measured.

the full scorecard ↓
https://t.co/lmPQTtmOR0

[Favur Dev @favurdev]

Seed Code 2.0 landed on @OpenRouter today, built for agentic coding and long tool workflows. Congratulations to the ByteDance Seed team.

Its strongest axis was process discipline at 82%. Code quality came in at 28%. Only 2 of the other 91 runs on the board have opened a wider gap between those two, where the typical gap is 9 points. It also invented less than most: a 0.9% hallucination rate against a 1.4% median, and 19 verification steps where the field takes 12.

Composite 57.8, just under the 61.2 median for this spec. We'll run it again.

https://t.co/qGqKNB4kge

[Favur Dev @favurdev]

A router that picks each request from whatever the market is spending on will hand you the market average. The longer the job, the more that shows.

We put every agent in our coding benchmark on @OpenRouter's Auto Router and let it choose, twice. On our smaller spec it scored 75.9 where the field median is 61.0. On a job six times the size it scored 58.2, where the median is 60.9. Same setup both times, nothing tuned in between.

Both runs also burned around 1.5x the tokens of their field.

The bigger run still shipped a working, tested repo, 226 tests green at 96% coverage against a 67% median. It got there because the pipeline is allowed to refuse work and send it backward. A reviewing agent blocked 15 of 46 changes until the findings were fixed. Sprint review threw out an entire sprint and sent it back to be redone, 14 times.

Letting a router choose your models makes those refusal points load-bearing.

https://t.co/VPOP5XNXvg

[Favur Dev @favurdev]

Congrats on the launch. We put Solar Pro 4 through our harness the day it landed and the numbers are up.

Of the sixty-three runs we have on this statement of work, exactly one cost less than this, and that one scored ten points lower. Solar Pro 4 came tenth on capability while sitting second on price. The launch promo through Sep 10 is doing some of that work.

75.9 composite, scored off the artifacts the run produced: https://t.co/DvLduC2PJB