We audited SWE-Bench Pro, one of the most widely used AI coding benchmarks, and found it no longer reliably measures frontier coding capability.
We find 30% of SWE-Bench Pro tasks to be broken, and are retracting our previous recommendation that the research community use it as a leading coding eval.
https://t.co/wDdSEjBe4F
@CatOrman1 Tik Tok has the fastest share nothing async runtime for Rust, plus real-time training for its recommendation algorithm, plus a dozen other small miracles of engineering, all occurring behind the scenes so as to not ruin the carefully-constructed image of brainrot
IF YALL HAD JUST LISTENED TO @darroneggins BACK IN 2019 WHEN HE SAID STOP FLEXING BOT SUCCESS WE’D STILL BE EATING SHOE PALACE BUT NOW ITS SHOPIFY. CLIPPED BY CLOUT. GOOD JOB
devs in discord: this is so trash, garbage release, literally have 0 pies
same devs on Twitter: cooking! who needs? testing going well! so many pairs on lock 🔒
Just in case you all didn’t know... Most of those “raffles” are a front to make the process seem legit. Most stores/boutiques backdoor 90% of the pairs each week