Excited about CruxBench!
Folks might know our lab’s @ProphetArena which grades forecasting probabilities. CruxBench is our first effort to shift from grading probability numbers to grading reasoning INSIGHT!
Inspired by many conversations with domain experts — a constant theme I am hearing is that “I do not care about what numbers you give me and they can even be wrong, but show me your new insight that I can use to convince my VP” — that’s exactly what CRUX is for
With our amazing collaborators at @Research_FRI !
CruxBench is accepted to #NeurIPS2026! 🎉
We evaluate not the answers, but the questions worth asking: can LLMs discover the cruxes that move beliefs about the future?
🔗 https://t.co/sjruGnbSxe
with @linapiuc, @haifengxu0, and FRI team: @_elsehow, @nadja_flechner, @EzraKarger