HAL

We find out where frontier agents break. And where the benchmarks measuring them do.

[HAL @horizon_bench]

Introducing Horizon's Data Quality Index:

At least 29 out of 239 tasks on top public datasets for AI benchmarks @harborframework are thoroughly flawed and everyone missed them.

@terminalbench
@mercor
@datacurve
@SnorkelAI https://t.co/Qtv310guXA

[HAL @horizon_bench]

That's why, in our audits, only a domain expert reviewer who actually knows the field can sign off on a task as good.

So "passed every automated check" and "correct" are two different claims, and a lot of benchmark scores lean on the first claim.

We wrote up how we read a headline score here https://t.co/k4C7aPEFnE

[HAL @horizon_bench]

How to read an AI benchmark
Every week there's a headline like "New AI scores 90% on [test]."
Honestly, it's often hard to tell what that number actually means, even for people who work on this stuff. So here's how we read them...

[HAL @horizon_bench]

A perfect chemist couldn't have aced "Humanity's Last Exam."
Not because it was too hard, but because there were a fair amount of wrong answers 🧵

[HAL @horizon_bench]

Same goes for graders that can be gamed. A task that pays out for gaming the checker doesn't just give you one bad number.

If it ends up in training, the model learns that gaming the checker is the job, and it carries that into everything after.

It's also, frustratingly, the hardest thing to catch in an audit. The official solution matches the wrong key, so every check agrees with every other check.

[HAL @horizon_bench]

5. What would a guess score?
On a four-option multiple-choice test, random guessing already gets you 25%, so the real range is more like 25 to 100 than 0 to 100. After that, check what the last model got and what human experts get. 90% means something pretty different next to 88% than next to 40%.

None of this makes benchmarks useless (we'd be out of a job if it did). But a score is a claim about the test as much as the model, and these five questions are how we start to check things.

[HAL @horizon_bench]

How to read an AI benchmark
Every week there's a headline like "New AI scores 90% on [test]."
Honestly, it's often hard to tell what that number actually means, even for people who work on this stuff. So here's how we read them...

[HAL @horizon_bench]

4. Could the AI have just found the answer?
Answers get left in all sorts of places: files left in the environment, a project's Git history, online. Just this month OpenAI reported one of its own agents decrypting a copy of a benchmark it found in its cache, looking for the answer key!

A score only means something if the answer wasn't reachable.

[HAL @horizon_bench]

@LaudeInstitute @terminalbench @harborframework It was an honor to meet the core team at @LaudeInstitute .

As the amount of datasets on @harborframework grow, we look forward to auditing and contributing to the premier open benchmarks of the world!

[HAL @horizon_bench]

@ipezyGJ @harborframework @terminalbench @mercor @datacurve @SnorkelAI Good point! we're actually in the process of rejiggering some of our benchmarks but that's definitely on the list. I imagine it'd be illustrative to see how that changes things for sure!

[HAL @horizon_bench]

Introducing Horizon's Data Quality Index:

At least 29 out of 239 tasks on top public datasets for AI benchmarks @harborframework are thoroughly flawed and everyone missed them.

@terminalbench
@mercor
@datacurve
@SnorkelAI https://t.co/Qtv310guXA

[HAL @horizon_bench]

@themarlonroldan @harborframework @terminalbench @mercor @datacurve @SnorkelAI Right! and on some of the flawed tasks, they were simply checking for correctly named files, which is a generous interpretation of 'partial' grading

[HAL @horizon_bench]

@imyoungsparks @harborframework @terminalbench @mercor @datacurve @SnorkelAI Thanks for reading! The failure modes surprised us too

[HAL @horizon_bench]

@yoitsyoung @harborframework @terminalbench @mercor @datacurve @SnorkelAI Which parts interested you the most?

[HAL @horizon_bench]

@yevloop @harborframework @terminalbench @mercor @datacurve @SnorkelAI Yes, precisely! One of the most common failure modes was even worse, with the solutions being accessible from commit history, so potentially they weren't even measurements of memorization.

[HAL @horizon_bench]

@alexgshaw Regrading would be pretty useful for us. We’ve had tasks where several checks turned out to measure more or less the same thing, so we’ve been revisiting how we score them.

Would be good to test those changes against the outputs we already have rather than rerun the models.