Fiction.live

Read and control interactive stories

Talk to writers. Suggest your own ideas and debate with other fans. Vote for what happens next.

[Fiction.live @ficlive]

https://t.co/ipn48MTbpK

[Fiction.live @ficlive]

qwen-3.5 plus and qwen3-max-thinking and opus 4.6

[Fiction.live @ficlive]

https://t.co/ipn48MTbpK

[Fiction.live @ficlive]

Added glm-5 and minimax-m2.5

[Fiction.live @ficlive]

@jehrjd45963 I had to double check but pro was medium.

[Fiction.live @ficlive]

Had to add gemini-3-flash-preview to the results.

It dominates.

Clearly the top model on this benchmark.

Hopefully we can get a v2 of this bench out sometime soon.

[Fiction.live @ficlive]

Long context eval.

Huge improvement since last year. The frontier models went from poor to great.

An exciting standout is kimi-2.5. It made impressive progress without (presumably) a new architecture, putting up gemini-2.5-pro numbers which we were all impressed by last year.

[Fiction.live @ficlive]

@jehrjd45963 Yes, gpt-5.2 is xhigh.

[Fiction.live @ficlive]

gemini-3-pro-preview improves upon the strong results of gemini-2.5-pro and is now neck and neck with gpt-5.2 on top in the "almost perfect" tier.

[Fiction.live @ficlive]

Long context eval.

Huge improvement since last year. The frontier models went from poor to great.

An exciting standout is kimi-2.5. It made impressive progress without (presumably) a new architecture, putting up gemini-2.5-pro numbers which we were all impressed by last year.

[Fiction.live @ficlive]

@xzenova @scaling01 I think the reasoning ran out of context.

[Fiction.live @ficlive]

@k0tovsk1y Been working on a better one for the past few months, hope to get it out soon. But at the same time these models are just now good IMO, and you'll start seeing that in the real world in terms of agentic workflows starting to work frfr.

[Fiction.live @ficlive]

claude-opus-4-5 fixed claude's long context performance, it is now good when previously it was a laggard. claude-sonnet-4-5 had a regression compared to sonnet 4… Same tier as grok-4.

[Fiction.live @ficlive]

Kimi-k2.5 now the Chinese/Open-source leader!

Minimax???

gpt-5.2 improves on almost perfection in gpt-5 to now very close to perfect. gpt-5.2-pro did surprisingly poorly.

[Fiction.live @ficlive]

https://t.co/nsKXZkfXXw

[Fiction.live @ficlive]

@teortaxesTex I might, but my bench is saturated by gpt-5. It's meaningfully better than gemini 2.5 and the bench did not reflect that. I will be back with a better eval.

[Fiction.live @ficlive]

Fiction.LiveBench for Long Context Deep Comprehension adds: deepseek-v3.2-exp [reasoning: high], deepseek-v3.2-exp, nemotron-nano-9b-v2:free, qwen-max, qwen3-next-80b-a3b-instruct.

[Sam Paech @sam_paech]

Some updates to Spiral Bench:
- A more detailed rubric for protective vs delusion-reinforcing behaviours
- Responses evaluated by a judge ensemble: sonnet-4.5, gpt-5 & kimi-k2
- New models evaluated: qwen3-235b, glm-4.6, grok-4-fast, mistral-medium-3.1

[Fiction.live @ficlive]

@DavidSZD1 Didn't finish the entire benchmark but there was no change from previous results for flash.

[Fiction.live @ficlive]

@dhtikna Sometimes the reasoning puts it over the token limit and the call fails.

[Fiction.live @ficlive]

@gusarich Yes it's surprisingly low for DeepSeek 3.2, I guess you have to pay the piper somewhere for the sparsity.

[Fiction.live @ficlive]

Thoughts: Interesting that we see an improvement for deepseek's reasoning mode but no improvement for the non-reasoning. It has high scores on the easier questions but very low scores on the hard ones.

grok-4-fast is fairly close to sonama-sky-alpha while still being free.