Arena.ai

Where AI meets the real world. We measure and advance the frontier of AI through community-driven evaluation. We’re hiring →

[Arena.ai @arena]

Introducing the Arena Alignment Index, our new benchmark measuring safety and alignment of AI agents in real-world use.

Built from 90K+ real-world agent sessions across 27 models, the index measures three critical signals:
- Unauthorized Action (UA): Taking actions beyond the user's instructions or permissions
- False Attribution (FA): Attributing statements or actions that are contradicted by user-provided evidence
- Deceptive Completion (DC): Claiming a task was completed when it was not.

Key findings:
- OpenAI models currently lead the Alignment Index
- Rogue actions are rare, but can have serious consequences when they occur
- Agents can mislead users about task progress
- Misalignment risks increase with conversation length
- Safety and alignment are improving across model generations

As shown in the leaderboard below (sorted by lab), @OpenAI’s GPT-6.1-Sol leads the Arena Alignment Index with a score of 87.9, followed by @AnthropicAI’s Claude-Opus-5.5 at 83.2 and @spacexai's Grok-4.7 at 82.7. OpenAI also has the best observed rates across all three signals: 0.89% Unauthorized Action, 1.98% False Attribution, and 2.34% Deceptive Completion.

Across all four labs, newer models consistently outperform their predecessors, suggesting broad progress in agent safety and alignment. As agents take on longer, more complex, and higher-stakes tasks, measuring not just what they can accomplish, but how safely and reliably they act, becomes increasingly important.

This marks an important step toward making safety and alignment a core part of how Arena evaluates AI. The index is an initial starting point, and we'll continue expanding the index with additional safety signals and models over time.

More analysis below👇

[Arena.ai @arena]

https://t.co/swyzXtFVmB

[Arena.ai @arena]

A new model from @Kimi_Moonshot is coming. Stay tuned for where it lands in the Arena.

[Arena.ai @arena]

Big news: Kimi-K3 by @Kimi_Moonshot is now #1 in the Frontend Code Arena with 1679 pts, surpassing Claude Fable 5.

This is a 17-place jump from Kimi-k2.6 (#18 -> #1).

In Frontend, Kimi-K3 ranked #1 in 6 of 7 domains: Brand & Marketing, Reference-Based Design, Data & Analytics, Consumer Product, Simulations, and Content Creation Tools, landing #2 only in Gaming behind Fable 5.

The full model weights will be released by July 27.

Congrats to the @Kimi_Moonshot team on this major milestone!

[Arena.ai @arena]

Big news: DeepSeek-V4.1-Flash-20261017 (Max) by @deepseek_ai is now the #1 open model in Agent Arena! It just landed at #11 overall with +5.1% net improvement across +7.5K real-world sessions.

This performance also reshapes the Agent Arena Pareto frontier! All models above V4.1-Flash 20261017 in performance are proprietary at much higher cost. At $0.20 median cost per task, that’s ~37% of GPT-6.1 Sol’s improvement for about 27% of the cost.

By signal, this release achieves:
- #8 in Confirmed Success with +11.5% (#1 open)
- #11 in Praise vs. Complaint with +5.9% (#1 open)
- #12 in Bash Reliability with +5.4% (#1 open)
- #13 in Steerability with +2.5% (#1 open)
- No issues with Tool Hallucination

Find more info on signals at the link in thread on Agent Arena.

Congrats to the @deepseek_ai team on this release!

[Arena.ai @arena]

Is it possible for AI to be both creative and effective?

The next episode of Arena Conversations is out: @petergostev interviewed Pradeep Natarajan, Senior Principal Scientist at Amazon AGI.

They cover:
- Model gains from mastering the fundamentals
- Evaluations as the differentiator
- Harness optimization
- Metrics like accuracy and style constraining model creativity

Hear about the above and much more.
https://t.co/Fi54swSPob

[Arena.ai @arena]

https://t.co/JxTdEgsVTq

[Arena.ai @arena]

Big news: DeepSeek-V4.1-Flash-20261017 (Max) by @deepseek_ai is now the #1 open model in Agent Arena! It just landed at #11 overall with +5.1% net improvement across +7.5K real-world sessions.

This performance also reshapes the Agent Arena Pareto frontier! All models above V4.1-Flash 20261017 in performance are proprietary at much higher cost. At $0.20 median cost per task, that’s ~37% of GPT-6.1 Sol’s improvement for about 27% of the cost.

By signal, this release achieves:
- #8 in Confirmed Success with +11.5% (#1 open)
- #11 in Praise vs. Complaint with +5.9% (#1 open)
- #12 in Bash Reliability with +5.4% (#1 open)
- #13 in Steerability with +2.5% (#1 open)
- No issues with Tool Hallucination

Find more info on signals at the link in thread on Agent Arena.

Congrats to the @deepseek_ai team on this release!

[Arena.ai @arena]

DeepSeek-V4.1-Flash-20261017 (Max) by @deepseek_ai landed in the Arena as the #1 open model in both the Agent Arena and the Code Arena: WebDev!

See 3D world generations from this model by @petergostev, and find more details on its ranking in thread.

https://t.co/DsnWF14sGw

[Arena.ai @arena]

Dive into the full Agent Arena leaderboard https://t.co/sE9q4FSYAt

[Arena.ai @arena]

Exciting news: Muse Wonder Full (Max) has moved up @AIatMeta as the #4 lab in the Agent Arena!

The latest model ranks #7 in Agent Arena with +11.3% net improvement across +6.5K real-world agentic sessions.

This is a clear step forward for the Muse family:

+7.5 percentage points over Muse Wonder (Max), ranked #14 at +3.8%
+7.8 percentage points over Muse Spark 1.3 (Max), ranked #17 at +3.5%

This performance shows up across categories too: #5 in Chat, #6 in Code, and #7 in Work.

At the signal level, Muse Wonder Full (Max) ranks:

- #6 in Praise vs. Complaint (+21.9%)
- #6 in Confirmed Success (+13.4%)
- #7 in Steerability (+11.2%)
- #16 in Bash Reliability (+7.7%)
- No issues in Tool Hallucination

Congrats to the @AIatMeta team and @alexandr_wang!

[Arena.ai @arena]

Exciting news: Muse Wonder Full (Max) by @AIatMeta has just debuted in the Code Arena: WebDev at #6 overall with 1726 pts, putting Meta in the top 3 labs!

This is +75 pts and 11 places above Muse Wonder (Max) at #17 (1651 pts), and +67 pts and 8 places above Muse Spark 1.3 (Max) at #14 (1659 pts).

Muse Wonder Full (#6) also landed +16 pts above Claude Sonnet 5.5 (#17), and +48 pts above Gemini 4 Argon (#10). With this score, Muse took the #3 lab spot (previously held by Google).

Across categories, Muse Wonder Full (Max) landed:
- #4 in Data & Analytics and in Consumer Product
- #5 in Brand & Marketing
- #6 in Content Creation Tools
- #7 in Gaming and in Simulations
- #8 in Reference-Based Design

Congrats to the @AIatMeta team and @alexandr_wang on this release!

[Arena.ai @arena]

GPT-6 Luna and GPT-6.1 Sol by @OpenAI now hold 6 of the 11 spots on the Agent Arena Pareto frontier across their reasoning efforts!

- GPT-6 Luna dominates below $0.10/task, claiming 3 of the 4 optimal spots with Max, Medium and Low.

- GPT-6.1 Sol sets the frontier between $0.22 and $0.65/task, with the same reasoning levels on it too.

This shows why test-time scaling matters. Sometimes the best move is to switch the reasoning effort, not the model, to stay at the optimal cost/performance point.

With GPT-6 Astra (Max) also on the Pareto frontier, OpenAI models hold 7 of the 11 frontier spots with multiple reasoning levels. Congrats to the @OpenAI team!

[Arena.ai @arena]

Update to the Agent Arena Pareto frontier: GPT-6 Luna (Max) has landed on the Agent Arena Pareto frontier with +1.59% net improvement at a median cost of just $0.05 per task!

Luna costs 29% less than DeepSeek-V4.1-Flash and 76% less than Hy4 preview, while trailing their net-improvement scores by just 2.27 and 2.51 percentage points respectively.

Compared to other GPT models on the Pareto frontier, GPT-6 Luna (Max) costs 94% less than GPT-6 Sol (Max), and 98% less than GPT-6 Astra (Max).

Congrats to the @OpenAI team on this cost-efficient release!

[Arena.ai @arena]

Dive into the Multi-Image Edit Arena: https://t.co/8MTiP13IYy

[Arena.ai @arena]

Krea 3 Ultra by @krea_ai has landed in the Single-Image Edit Arena and the Multi-Image Edit Arena.

@krea_ai is now among the top 5 labs for both arenas: #4 by lab in Multi-Image Edit, and #5 by lab in Image Edit!

In the Multi-Image Edit Arena, Krea 3 Ultra is #4 with 1435 pts. By category it is:
- #2 in Commercial Design and in 3D
- #4 in Cartoon/Fantasy and in Text Rendering

In the Image Edit Arena it landed #8 with 1411 pts, and by category:
- #6 in Commercial Design, Cartoon/Fantasy and Text Rendering
- #8 in 3D

Congrats to the team at @krea_ai on this release!

[Krea @krea_ai]

today, Krea 3 makes it to the top of image arenas.

- "Fast" pushes the pareto curve of speed and quality.
- "Ultra" ranks top-4, only surpassed by OpenAI models.

the only company valued under $1T in the top-10!

more below 👇

[Arena.ai @arena]

We just put Muse Wonder and Muse Wonder Full by @AIatMeta to the test across 3D scenes, animated HTML clips, and several website builds.

Watch @petergostev compare both models against Claude Opus 5.5, Sonnet 5.5, Fable 5.1, GPT-6.1 Sol, GPT-6 Astra, and Gemini 4 Argon.

https://t.co/fh8iz53nQ8

[Arena.ai @arena]

Dive into the Agent Arena at: https://t.co/sE9q4FSYAt

[Arena.ai @arena]

Big news: DeepSeek-V4.1-Flash-20261017 (Max) by @deepseek_ai is now the #1 open model in Agent Arena! It just landed at #11 overall with +5.1% net improvement across +7.5K real-world sessions.

This performance also reshapes the Agent Arena Pareto frontier! All models above V4.1-Flash 20261017 in performance are proprietary at much higher cost. At $0.20 median cost per task, that’s ~37% of GPT-6.1 Sol’s improvement for about 27% of the cost.

By signal, this release achieves:
- #8 in Confirmed Success with +11.5% (#1 open)
- #11 in Praise vs. Complaint with +5.9% (#1 open)
- #12 in Bash Reliability with +5.4% (#1 open)
- #13 in Steerability with +2.5% (#1 open)
- No issues with Tool Hallucination

Find more info on signals at the link in thread on Agent Arena.

Congrats to the @deepseek_ai team on this release!

[Arena.ai @arena]

Exciting news: DeepSeek-V4.1-Flash-20261017 (Max) by @deepseek_ai is now the #1 open model in Code Arena: WebDev, surpassing Kimi K3 (Max) at 10x lower cost!

Highlights:
- #1 open model, ahead of Kimi K3 (Max)
- #12 overall with 1670 pts, ahead of Claude Opus 5 (High) and within ~10 pts of GPT-6 Sol (Max)
- Shifts the Pareto frontier at $0.3/$1.2 per million input/output tokens
- +52 pts vs. the previous DeepSeek-V4.1-Flash-20260910

Congrats to the @deepseek_ai team on another strong contribution to the open source ecosystem!

[Alexandr Wang @alexandr_wang]

muse wonder full on webdev arena

not bad, but clearly lots of room for us to improve!

[Arena.ai @arena]

Dive into the Code Arena: WebDev details at https://t.co/GFZ3FCC7Cl

[Arena.ai @arena]

Exciting news: Muse Wonder Full (Max) by @AIatMeta has just debuted in the Code Arena: WebDev at #6 overall with 1726 pts, putting Meta in the top 3 labs!

This is +75 pts and 11 places above Muse Wonder (Max) at #17 (1651 pts), and +67 pts and 8 places above Muse Spark 1.3 (Max) at #14 (1659 pts).

Muse Wonder Full (#6) also landed +16 pts above Claude Sonnet 5.5 (#17), and +48 pts above Gemini 4 Argon (#10). With this score, Muse took the #3 lab spot (previously held by Google).

Across categories, Muse Wonder Full (Max) landed:
- #4 in Data & Analytics and in Consumer Product
- #5 in Brand & Marketing
- #6 in Content Creation Tools
- #7 in Gaming and in Simulations
- #8 in Reference-Based Design

Congrats to the @AIatMeta team and @alexandr_wang on this release!

[Arena.ai @arena]

Muse Wonder (Max) by @AIatMeta has entered Code Arena: WebDev at #15 with 1651 pts and Text Arena at #2 with 1513 pts!

In Code Arena: WebDev, Muse Wonder (Max) landed at #15 with 1651 pts. This result is slightly lower thanMuse Spark 1.3 (Max) at #12 with 1659 pts, within 4 pts of Kimi K3 (Max) at #14 with 1655 pts, and 11 pts above Grok 4.7 (xHigh) at #16 with 1640 pts.

This release is a significant improvement over past Muse models in Text Arena. Compared with Muse Spark 1.3 (Max) at #11 with 1494 pts, Muse Wonder (Max) gained +19 pts and 9 places to #2. It's 12 pts behind the #1-ranked Gemini 4 Argon (High) with 1525 pts.

Congrats to the @AIatMeta team on this release!

[Arena.ai @arena]

Use GPT-Image-2.5 Sunburst in Direct Mode on Arena now: https://t.co/cBLzj0blrO

[Arena.ai @arena]

For the next 14 days on Arena, GPT-Image-2.5 Sunburst by @OpenAI will be available to test directly on Arena!

@OpenAI’s latest image model is ranked #1 in the Text-to-Image Arena, the Image Edit Arena, and in the Multi-Image Edit Arena. Find the link below to Direct Mode, where you can now select GPT-Image-2.5 Sunburst from the dropdown menu.

Access to GPT-Image-2.5 Sunburst ends November 2nd at 8 a.m. PT. Afterward, it will remain available anonymously in Battle and Agent Mode. User votes from Battle and Agent Mode are used to power our real-world leaderboard rankings. In these modes, models are always anonymized.

[OpenAI @OpenAI]

ChatGPT Images 2.5—faster, sharper, smarter, with better tools for creating whatever you can dream of.

- Faster image generation to keep your ideas flowing

- Improved fidelity for more natural, recognizable images

- Consistent details across multiple edits

- Comment-based edits to change only what you want

[Arena.ai @arena]

Among open models, MiniMax-M3.1-Flash is #12 (#32 overall).

Dive into the Code Arena: WebDev details at https://t.co/GFZ3FCC7Cl

[Arena.ai @arena]

MiniMax-M3.1-Flash by @MiniMax_AI is live in the Code Arena: WebDev. It debuts at #12 among open models with 1588 pts (#32 overall). It also lifts MiniMax to #12 among all labs!

MiniMax-M3.1-Flash is a +106 pt increase from the previous variant, MiniMax-M3 with 1482 pts at #19 among open (#66 overall). Priced at $0.15 input / $0.60 output per 1M tokens, MiniMax-M3.1-Flash also costs 50% less.

It also improved in ranking and pts across all categories compared to MiniMax-M3:

- Reference-Based Design: #20 → #7 (+155 pts)
- Content Creation Tools: #20 → #7 (+113 pts)
- Data & Analytics: #18 → #8 (+68 pts)
- Gaming: #21 → #9 (+179 pts)
- Simulations: #22 → #13 (+122 pts)
- Brand & Marketing: #18 → #14 (+78 pts)
- Consumer Product: #18 → #14 (+64 pts)

Congrats to the team @MiniMax_AI on this release!

[MiniMax (official) @MiniMax_AI]

MiniMax M3.1 Flash: Space Performance + Bunny Speed

🚀🐰 Yes, Space Bunny - which has topped global leaderboards with 200T+ tokens - was an earlier version of M3.1 Flash.

💰Redefining the performance economics of agentic AI. M3.1 Flash achieves frontier-level intelligence across key tasks such as Cowork (Automation-Bench: 62%; ALE: 30%), Coding (DeepSWE: 74%) and Visual Reasoning (BabyVision: 91%), at 1/3 the cost of M3 and 3x the speed.

API pricing (flat rate): $0.15 / $0.60 per 1M input/output tokens, with cache reads at $0.015 / 1M tokens.

Model weights will arrive on Oct 23.

API: https://t.co/fHRdSV73Hr
M Plan: https://t.co/XfJWSyauda
MiniMax Code for desktop: https://t.co/SGmLTI2HQm