Rapidata

6K+ human judgements/minute via API for Online RLHF and Benchmarks.

Advancing SOTA GenAI research through weekly, open benchmarks →

[Rapidata @RapidataAI]

Visual realism ≠ spatial controllability. Generating convincing scenes is one thing, and being able to navigate them precisely is another. This matters for most use cases of world models.

@theworldlabs Atlas shows impressive capabilities here, ranking #1 across 600 prompts and 513K+ human judgments.

[Fei-Fei Li @drfeifei]

Great job @theworldlabs teams! 🚀 Camera control is so critical for many creative and design work. I’m very excited by Atlas’ strong results!

[Rapidata @RapidataAI]

@sample @LuminaBench The whole point of https://t.co/nsvZP9LMFB is that you can look at the outputs, and the votes behind rankings. Looking at it, astra still seems overall better than fable 5.2 at llm to 3D (you can verify it)

But fable 5.2 ranks #1 in our SVG benchmark for instance.

[Rapidata @RapidataAI]

@buddzzh @GoogleDeepMind @NewsFromGoogle @DynamicWebPaige @OfficialLoganK On https://t.co/fVVIXe7PDh if you go to the methodology notes, you’ll find some information about this

[Rapidata @RapidataAI]

🔥 Gemini 4 Carbon was released a few hours ago and ranks #1 in our LLM-to-3D Benchmark.

This benchmark evaluates how good LLMs are at constructing 3D objects; it includes 1.3M+ human judgments.

As usual, you can find all data available on Benchmark AI.

[Rapidata @RapidataAI]

@Saboo_Shubham_ You can find a benchmark of it's 3D coding abilities here, https://t.co/fVVIXe7hNJ some outputs below:

[Rapidata @RapidataAI]

@kimmonismus Benchmarked it on 3D capabilities, it ranks #1 according to 1.3M human judgements, on 200 prompts. 👀

https://t.co/fVVIXe7PDh

[Rapidata @RapidataAI]

@GoogleDeepMind Added it to our LLM to 3D benchmark, it ranks #1. You can find some matchups below.

[Rapidata @RapidataAI]

Full prompt-set, output, judgements, and methodology on: https://t.co/P8bVwxasoN

[Rapidata @RapidataAI]

After 5 edits, does the person still look the same? Ideogram 4.5 ranks #1 among 20 models for preserving facial identity! 🥇

We benchmarked character identity drift across 199 photos, applying 5 consecutive edits and evaluating only the resulting faces. This benchmark holds 322K+ human judgments.

All outputs, judgments and methodology are publicly available on Benchmark AI.

[Rapidata @RapidataAI]

Fable 5.2 enters our SVG benchmark at the #1 position.

The benchmark contains 4M human judgments across prompt alignment, coherence, and preference.
All outputs, votes, and methodology are public on https://t.co/hLcEZRLhSg.

[Rapidata @RapidataAI]

@martin_casado Potentially the most important capability a world model must master. That's a great achievement indeed.

[Rapidata @RapidataAI]

@drfeifei @theworldlabs We've added some of the benchmark's match-ups to the video below. Congratulations on the achievement!

[Fei-Fei Li @drfeifei]

Great job @theworldlabs teams! 🚀 Camera control is so critical for many creative and design work. I’m very excited by Atlas’ strong results!

[Rapidata @RapidataAI]

@KeunhongP Congrats on this milestone, @KeunhongP! Developing this capability is perhaps the most vital aspect of building an effective world model.

[Rapidata @RapidataAI]

World Labs’ Atlas takes #1 in camera movement! 🥇

We evaluated 19 world and video models across 600 prompts, collecting 513K+ human judgments to assess how accurately each model follows camera movement instructions.

All outputs, matchups, judgments, and methodology are public on Benchmark AI.

[Rapidata @RapidataAI]

Gemini Omni Flash 1.2 is #1 on both camera benchmarks: Relative Camera Movement, published yesterday (moving through the scene, "between the doors, up to the lantern"), and now Camera Movement (copying an exact move like "pan 40° left and back" from a reference clip).

Some match-ups below:

[Rapidata @RapidataAI]

@fofrAI Tested its capabilities on camera control. It ranks #1 on this benchmark that has 900K+ human judgements (dethroning Omni 1.1) Check out the results on: https://t.co/9If5sgJ9zi

[Rapidata @RapidataAI]

@_philschmid @GoogleAIStudio We benchmarked its capabilities on camera control. It ranks #1 on this benchmark that has 900K+ human judgements. You can check out the results on: https://t.co/9If5sgJ9zi

[Rapidata @RapidataAI]

@Google @GoogleDeepMind We benchmarked its capabilities on camera control. It ranks #1 on this benchmark that has 900K+ human judgements. Check out the results on: https://t.co/9If5sgJHoQ