ChessBench

A new benchmark tracking how well language models play chess. Watch the games, follow the reasoning move by move, track the leaderboard.

[ChessBench @chessbench]

Most language model benchmarks are saturated. ChessBench isn't — and it's not close.

GPT 5.5, Claude Fable 5, Gemini 3.1 Pro: not one of them beats a decent club player at chess yet.

I built ChessBench looking for headroom. Turns out there's a lot.

[ChessBench @chessbench]

@AdamHoltererer probably closer to $350-$400 if we leave off xhigh and max...

[ChessBench @chessbench]

@AdamHoltererer I think around $750-$900 to get a reasonable baseline...

[ChessBench @chessbench]

@sama @yacineMTB 🙃

[ChessBench @chessbench]

@kapilansh_twt How about 60?

https://t.co/lPd8cnXQEh

[ChessBench @chessbench]

@alexandr_wang @noctus91 https://t.co/lPd8cnXQEh shows the muse spark models trending up quick

[ChessBench @chessbench]

@alexandr_wang @noctus91 I believe it

[ChessBench @chessbench]

@gdb prove it

[ChessBench @chessbench]

@elonmusk @chesscom https://t.co/5wT2RFzkbZ

but you already knew, because, well, obviously

[ChessBench @chessbench]

@Swarooprm7 Truly

[ChessBench @chessbench]

For those of you who are into supporting independent benchmarks: https://t.co/8PE5VphAGK

Every dollar goes to API costs. More funding means more models tested, faster, with more games behind each rating.

[ChessBench @chessbench]

@DanDr1s Gemini 3.8 Flash tops ours by a wide margin

https://t.co/lPd8cnXQEh

[ChessBench @chessbench]

@oleks01 I don't know... https://t.co/lPd8cnXQEh

[ChessBench @chessbench]

@FIDE_chess Awesome

[ChessBench @chessbench]

@ai_for_success Agreed! Super fast, cheap, and somehow the best at chess!

https://t.co/9nrlyafyOJ

[ChessBench @chessbench]

@SemiAnalysis_ At https://t.co/gruKWSzyuO it's markedly *better* than the rest, at least for Gemini 3.8 Flash...

[ChessBench @chessbench]

@JohnWic59379421 @alexandr_wang @AIatMeta Absolutely!

[ChessBench @chessbench]

Muse Spark 1.3 (and 1.2 and 1.1) have been added to ChessBench.

@alexandr_wang, @AIatMeta not bad

[ChessBench @chessbench]

@JohnWic59379421 @alexandr_wang @AIatMeta Yeah, tool use is off for all models.

Each move is a fresh call with no memory. What the model sees is randomized per game and includes some combination of the FEN, move history, a text diagram, and the legal moves.

[ChessBench @chessbench]

@danliu https://t.co/lPd8cnXQEh

it's *much* better in some arenas than others...

[ChessBench @chessbench]

@alexandr_wang Another interesting eval: https://t.co/gruKWSzyuO!

Muse Spark 1.3, 1.2, and 1.1 were just added. Pretty good

[ChessBench @chessbench]

@alexandr_wang @AIatMeta https://t.co/lPd8cnXQEh