dawid

model insights @arena

[dawid @DawidGalarowicz]

If you're not sure if you need Fable, you probably don't need it. But when you have a good task for it, you will totally feel the difference.

[Arena.ai @arena]

Fable 5.2's advantage over Opus 5.5 grows with the number of turns.

In tasks of up to 3 replies, the two are level: about 85% of responses need no correction.
In longer tasks, Fable pulls ahead: 89% vs 83% at 8-15 replies, and 93% vs 85% at 16+.

Key takeaway: On everyday tasks you won't notice much difference. Bring Fable 5.2 difficult, long-running work and you will.

[dawid @DawidGalarowicz]

I love it when competition does the thing!

[Arena.ai @arena]

Beating April’s best coding model cost 20x less by October.

We analyzed every model listed on Code Arena: WebDev from December 2025 to October 23, 2026: 105 models from 24 labs across 44 weekly leaderboards, plus what every answer cost from April to October.

The pace of releases has stepped up. Last winter, about five new models entered Code Arena each month. From July to October, it was 12 to 14, almost one every other day. And from May to October, the top score climbed 3x faster than it did over the winter.

Even though the labs are racing, it’s the users who are winning. By October, Xiaomi’s MiMo-V2.6-Flash outscored Claude Opus 4.7, April’s top model, at less than a twentieth of the price. And for answers costing between 5 cents and $1, October’s best model had an estimated 65-80% win rate against April’s best at the same price.

Read more about how the AI race is speeding up, and what it means for the field, in the article by @DawidGalarowicz below.

[Alex Dimakis @AlexGDimakis]

What are idea basins and how do agents explore them?

A few weeks ago, researchers from different labs independently discovered what we now call ‘Idea-basins’: As an autonomous agent iterates to solve a problem, it gets stuck into the methods it has tried before and its trajectories remain in localized regions of the solution space. Techniques like ‘Fork-and-Flush’ and ‘Scaling Discovery requires forgetting’ reduce this idea fixation by periodically deleting the session context.

In this post we study how frontier models explore different idea basins for AutoResearchExam. E.g. for a problem on accelerating transformer decoder graphs on CPUs models can try Numpy+Asm Kernels, Native BLAS, JIT compilers and other idea basins. We find that some models switch ideas more than others while others get to the best idea faster and explore deeper.

We find that most runs stick with one or two ideas, and models that submit more spend the extra submissions refining and overfit more.
We see that models explore most in the first six hours, after which visits to new idea basins drop sharply. This suggests models start refining early leaving potential gains on the table: 58% of runs never reach the basin where the best solution to their task was found. Finally, bold exploration pays off: exploring a new idea gains about nine times as much as a refining step on average.

Read more in our blog:
https://t.co/gVvHEgJfed

[dawid @DawidGalarowicz]

It's amazing to see Arena on the list, I feel lucky I get to do work that is both fun and so impactful!

[Arena.ai @arena]

We’re proud to be part of the 2026 AI 100.

If you believe AI progress should be measured by how well it works in the real world, join us (link in profile).

Thank you @NewcomerMedia @BessemerVP for the recognition.

[dawid @DawidGalarowicz]

RLVR is a very unfortunate name - it implies open-ended problems where it doesn’t work are not verifiable, and, by extension, can’t be solved. As a result, some very meaningful problems are overlooked today.

[dawid @DawidGalarowicz]

The real modern dilemma - would I rather lose money now by sending a useless prompt to my agent or lose money later by letting the cache go cold

[Arena.ai @arena]

https://t.co/gNzLoxDwfc

[dawid @DawidGalarowicz]

@cthorrez @AdamHoltererer I aspire to have as much confidence about my own writing as Astra does one day

[dawid @DawidGalarowicz]

The biggest upside of model routers today is not necessarily a better cost/performance trade-off than sticking with a single model. For most users, it’s the removal of friction that comes with thinking which model to use and the feeling that something is being done to save them money.

[dawid @DawidGalarowicz]

Let’s go team! 🤘

[Arena.ai @arena]

Today, we're announcing our Series B: $200M at a $3.1B valuation, and the release of Arena's Alignment Index.

AI is advancing faster than our ability to evaluate it. The world needs a neutral third party to measure how safe and aligned AI actually is once it's in the hands of real people. That's the role Arena is stepping into today.

The Alignment Index is built from real-world agent traces, starting with three signals: Unauthorized Action, False Attribution, and Deceptive Completion, with results for over 20 frontier models launching today.

Hear more from our CEO @ml_angelopoulos below, and read the full breakdown, and more details about our company growth in thread.

This funding will allow us to go further on safety signals, agentic capabilities, and modalities. It will also allow us to grow the company to match the scale of our mission. Today, we’re still a nimble team of 90, and we're looking for people to join us across research, product, engineering, and more. Come build with us.

As AI gets more capable and autonomous, understanding how well it can safely and effectively work for people only matters more. That's what Arena is built to measure and advance.

Thank you to our community, our investors, and everyone building with us!

Take a look at our open roles if you want to join us (link in bio).

[dawid @DawidGalarowicz]

Such a cool finding, amazing work!

[Pardis Zahraei @PardisZahraei]

New paper, accepted to #COLM2026 🎉 (+ Outstanding Paper Award 🏆 at KnowFM @ ACL)

We present Emergent Unfaithfulness. When LLMs are asked to summarize a document, they silently rewrite the claims they disagree with, without telling the user or mentioning it in their CoT.

Why? 🧵

[dawid @DawidGalarowicz]

If you are used to throwing Luna at Astra’s responses for your evals, this might make you think again!

[Arena.ai @arena]

https://t.co/18LMxWDErS

[Arena.ai @arena]

Scores for GPT-6 Sol and GPT-6 Luna by @OpenAI are coming soon. Head to Arena now to test them. Your votes on real-world agentic tasks power our leaderboard!

In the meantime, @petergostev ran GPT-6 Sol head-to-head against GPT-5.6 Sol under matched conditions: the same prompts, with both models using max reasoning. The comparison examines not just the generations, but also total token usage and wall-clock time, revealing how the models differ in output, token use, and latency.

Find the results and Peter’s prompts on our YouTube.

https://t.co/BcsKYlaDcH

[Florian Brand @xeophon]

i love evals

[Clayton Thorrez @cthorrez]

Cool analysis using arena data to study the stability of rankings and ratings with respect to the user distribution! Very good to look at since a lot of methods make IID assumptions at the vote level which are violated as votes are generated by users who act very differently

[Arena.ai @arena]

https://t.co/K3TE6Bj6pZ