Incredible
"What does building two Stargate Abilenes in one year look like? Let’s talk about the Colossus expansions, where Elon is already trying to buy hundreds of nearby homes to both soothe noise complaints and provide construction worker housing."
@danielrupawalla Only some of the domains (e.g. finance and legal) use binary because it fits the domain. Most of the domains use continuous scoring.
And even for the domains that use binary grading, you can see the partial scores on the leaderboard
Regarding the 0% scores, I'm assuming you're referring to legal and finance? Would recommend switching to the partial scoring view for those pages! (score dropdown at the top of the chart)
Both our finance and legal tasks use binary grading because they're high stakes domains: mess up a single key cell in an Excel workbook and the whole thing is worthless.
However, each task decomposes into lots of different criteria the model still earns partial scores for. It's just that none of the models were able to meet the threshold (e.g. 95%) to earn the binary reward. That being said, partial scores can absolutely still be useful for differentiating between models.
Was a pleasure working with you Akarsh! You + the rest of the Beautiful Machines team are cracked.
And sincere apologies to the @diodeinc team for not tagging them in the main post. That was my miss in the mad rush of publishing. The PCB layout tasks you guys collabed on are incredible.
@_rahulparikh@AndrewMegalaa Rahul + the rest of the Living Context team make the best finance data I've ever seen. No question. Was a pleasure working with you all!