Daniel Furman

The human in the loop.

[Justin Lo @loresjpg]

Nice DeepSWE score! Does it survive a lowercase filename?

Introducing MaxxBench: 89 DeepSWE tasks with small changes to the instructions or starting code. Same engineering problem, slightly different requirement.

Models clustered around 74% on DeepSWE end up 23 pp apart.

Good at coding, or good at DeepSWE? 🧵

[Luis Héctor Chávez @lhchavez]

https://t.co/DXFvKSEBXz

[Ankur Goyal @ankrgyl]

https://t.co/wQxD46Omrm

[gian @giansegato]

just got the chance to spend time looking into vibench 2, and i gotta say i really like it! it's rigorous solid work. it may initially look a bit replit specific in the way it's shaped, but i'm pretty confident it's actually measuring something more general than just vibe coded apps

ultra long trajectories with hundreds of edits and additions start stress testing capabilities like planning, large scoping, etc that have more to do than frontend dev

just sad that it's only 8 apps in the public set, hoping to see more of this! (both quantity and diversity)

[Daniel Furman @dryanfurman]

When I joined @Replit, I was immediately drawn to the work @Lambda_freak (along with many others) were doing on evals.

Today we're proud to release ViBench 2.0, the newest version of our in-house benchmark, which tests whether agents can build whole products one feature at a time. It joins @FireworksAI_HQ evals index (SII) in the Application Development category.

As the world keeps investing in agentic engineering, imho, some of the most exciting and impactful work is on measurement and evals.

As always, thanks to @pirroh for his unwavering leadership. Onward!

[Replit ⠕ @Replit]

Today we're releasing ViBench 2.0, a benchmark for long-horizon software building. Agents start from an empty project and build a whole product, one feature at a time, without seeing what comes next.

Even the best models (GPT-6.1 Sol, GPT-6 Astra, Opus 5.5) pass every check in only 49-65% of builds.

ViBench 2.0 is now part of @FireworksAI_HQ’s Specialized Intelligence Index, with official results from Fireworks’ runs.

Read more: https://t.co/WDbfapcWhC

[Henry Kiss Ehrenberg @henryehrenberg]

Big ups to @FireworksAI_HQ on launching SII! Great to see Senior SWE-bench featured in the SWE leaderboard.

Personally find myself only interested in indices that I can slice results for (domain, open/closed, etc.), trade-off perf/cost, and aggregate methodologies. This hits the mark.

[Luis Héctor Chávez @lhchavez]

Nikesh asks the right question: is your coding harness secure?

Last week, @Replit's isolation protections prevented the disclosed kernel exploits. We also patched tooling and deployed firewall protections centrally.

Here’s the architecture behind that: https://t.co/F5qIBohmPO

[Zhen Li @zhenthebuilder]

bring your idea to the company codebase using @Replit agent
Replit itself has been driving by it for a long time

[Samuel Spitz @samuel_spitz]

Replit launches Cloud Agents, the first enterprise coding product optimized for non-engineers.

Everyone should be able to write code. With Replit, they can.

[Michele Catasta @pirroh]

We just published our blueprint for the Self-Driving Company.

At Replit, engineers nearly tripled code output in the first half of the year. Incidents stayed flat. Then design and technical support started shipping too. Today we use Replit wall-to-wall to build every function of our company.

Over time, it became clear that models are the easy part. The hard part is the Agent Habitat: access control, data governance, security.

We are making useful intelligence accessible to everyone. As it was always meant to be.

https://t.co/THulVPfhBY

[Amjad Masad @amasad]

Frontier Slides, super fast ;)

[DesignArena @DesignArena]

BREAKING: Replit by @Replit ranks at #2 on Slides Arena with an Elo of 1279!

It is the highest-rated slides builder under 8 minutes and generates decks ~24% faster than #1, establishing a new Pareto frontier for Speed vs. Preference. Replit places ahead of Gamma 5 by @GammaApp, Manus 1.6 by @ManusAI, and Claude Opus 4.6 by @Anthropic, trailing only Playyy.

Congrats to the @Replit team!

[Harrison Chase @hwchase17]

great writeup by @pirroh

harness doesn't disappear as models get smarter - it shifts from deciding to offering

in deepagents you give each subagent its own model, tools, and permissions - the main loop picks which to spawn and when

docs: https://t.co/c1fYCa4msO

[Daniel Furman @dryanfurman]

If you’re at @OpenAI Dev Day today, this is the read for you. Check out the crazy cool things that Astra can do in Replit Agent!

[Michele Catasta @pirroh]

https://t.co/ErebLourdy

[Michele Catasta @pirroh]

https://t.co/ErebLourdy

[Daniel Furman @dryanfurman]

@HamelHusain Yes

[Daniel Furman @dryanfurman]

🚀🚀

[Daniel Furman @dryanfurman]

@ImenKedir Welcome to the non-technical north.

[Daniel Furman @dryanfurman]

Having great eval infra is chef’s kiss