The cost is great, but even more important is the speed. I have found that I can iterate nearly twice as fast with the speed of the Grok CLI.
At a fraction of the cost, the value is unbelievable.
The FrontierCode leaderboard is now live: a dedicated page that tracks which models are writing code you’d actually merge.
All scores — including Grok 4.5 and Inkling — are available, along with full methodology and sample tasks.
@lenatron33 Saying they read the code because they need to maintain some sort of self-worth is different than actually reading the code… 😀
Even PR’s are largely automated at this point, realistically people are just skimming for edge cases.
@signulll Whoever solves for off-line medical intelligence will crush. Nearly everybody values health and privacy.
Come to think of it I’m not aware of any medical benchmark for open source models. Is anybody tackling that?