Interesting read from Anthropic, with several parallels to our work on agent memory evaluation. Last week, we shared an approach for turning any existing benchmark into a memory evaluation by exploiting dependencies between tasks.
https://t.co/gDQ8BNHsfK
Exactly, this was my point on using "fancier" ML classifier models that are trained. They tend to overfit and not extend to certain task families; there is an art to making them generalizable. But in practice, I think before anyone would ever set this sort of loop in motion, they would need to inspect the task family closely and verify that the classifier is doing as intended. If you calibrate your classifier for a task family and make efforts to generate tasks inside that family, the cross-validation AUROCs show that it will stay highly accurate.
Can you catch an AI agent's wrong answer from one run, with no answer key, and use it to generate tasks that expose new failures? One JEV read of the trace can rank failures (even quiet failures where many runs agree) and guide hill-climbing. Code is open. https://t.co/RCHEkTZyCk
We Built the Agents Faster Than We Built the Rulers
I spent a decade doing research in theoretical astrophysics and cosmology. Compared to that field, AI benchmarks have been more vibes than science...(rant incoming)
(22/23) Rigorous work exists in AI benchmarks, but it’s just not commonplace when it should be REQUIRED... and it’s definitely not fully appreciated across industries.