today, we’re releasing the largest open-source human image preferences dataset, along with a $1 million data grant
- 2M+ annotations by real people
- 30 SOTA image models ranked
- 10 categories (marketing, product design, anime etc)
dataset + benchmark + grant details below:
Arrow 2 Telos by @QuiverAI had the most variance on the benchmark. While it ranks #1 for illustrations and scenes, it drops to the last spot for diagrams!
overall and category-wise rankings of 14 frontier models are available on our SVG Bench: https://t.co/aTxR90fq7H
we have billions of pairs of human preference data across every modality (audio, video, image etc) for reward modelling, benchmarking and post-training
looking to onboard more neo labs/frontier labs! DMs open if interested 👀
Another benchmark, another #1 for MolmoAct 2! 🤖🏆
Our robot policy tops @datapointai’s human-preference rankings for real-world manipulation tasks, including candy sorting & cup stacking.
Thanks to the Datapoint AI team for the evaluation & open dataset!
some of the top multimodal labs now use Datapoint as their preferred partner for human evaluation across any modality!
learn more here: https://t.co/8nQ7CDj3hL
the full dataset is free and open: https://t.co/OALdfgCRmp
overall and category-wise rankings of 9 robot policies are available on our robot bench: https://t.co/D9jO6MYuPH
Introducing Streams: online RL with live human feedback for multimodal models.
- images, audio & video
- 1B+ raters on tap
- real-time, continuous human preferences
https://t.co/SOWFVbzXN2
P-Video-2 Pro by @PrunaAI and Minimax H3 Max by @fal are the best quality–cost tradeoffs currently available, although the competition is quite fierce!
Since launch, we have granted PhDs, independent researchers and non-profits from US, Israel, India, Paris etc hundreds of thousands of dollars in data grants to accelerate their research.
We are now doubling our commitment to $2M!
https://t.co/i3GvPcMfns
today, we’re releasing the largest open-source human video preferences dataset, along with doubling our data grant to $2M
- 300K+ annotations by real people
- 15 SOTA video models ranked (Seedance 2.5, Omni 1.1, Wan 3, Flux Video 3 etc.)
- 8 categories (ads, camera movement, physics etc)
dataset + benchmark + frontier plot below:
P-Video-2 Pro by @PrunaAI and Minimax H3 Max by @fal are the best quality–cost tradeoffs currently available, although the competition is quite fierce!