CortexAI

Building the world's most diverse real-world, real-workplace, and industry-scale egocentric + robot dataset.

[CortexAI @cortexairobot]

We’re excited to share MolmoAct 2, a fully open-sourced robot foundation model for real-world deployment.

Pushing SOTA is truly a collaborative effort. We’re grateful to be @allen_ai's exclusive data collection partners on this project, contributing:
- 700+ hours of real-world bimanual data
- Five robotics policies
- Third-party benchmarks of MolmoAct 2’s real-world fine-tuning performance 🦾

[Ai2 @allen_ai]

Robotics models often struggle outside controlled environments. Ours is built to work in real ones.

Today we're launching MolmoAct 2, which can assist with a host of chores & lab tasks, plus the MolmoAct 2-Bimanual YAM dataset—the largest open robotics dataset of its kind. 🧵

[CortexAI @cortexairobot]

🤗 You can find all 700+ hours of the data we contributed to the project fully open sourced here:
https://t.co/lBhRfp44qz

[CortexAI @cortexairobot]

We provided over 700 hours of robot demo training data towards the MolmoAct2 project, performing tasks like folding a towel, scanning groceries, charging a smartphone, and table bussing.

It still blows our mind to see robot arms successfully interacting with objects they've never seen before! 🤯

[Jiafei Duan@CVPR2026 @DJiafei]

One of the coolest things we did this time around: we set up in a public space and asked folks to volunteer their own personal items for MolmoAct2 to interact with. Some of the feedback and results genuinely shocked us — like how it picked up such oddly shaped objects. We're always looking for beta testers, or you can try it yourself since everything is open-source end to end. @allen_ai

[CortexAI @cortexairobot]

@chris_j_paxton We had a blast being this third-party benchmarking firm. We believe that this helps labs across the board better evaluate their models. 🌟

[CortexAI @cortexairobot]

We're excited to hear what you think!

📝 Learn more on the blog: https://t.co/6jtJF8WD93
🤖 Model: https://t.co/SphjLnF2yP
📊Datasets: https://t.co/3FPAMpM1tZ

[CortexAI @cortexairobot]

We’re excited to share MolmoAct 2, a fully open-sourced robot foundation model for real-world deployment.

Pushing SOTA is truly a collaborative effort. We’re grateful to be @allen_ai's exclusive data collection partners on this project, contributing:
- 700+ hours of real-world bimanual data
- Five robotics policies
- Third-party benchmarks of MolmoAct 2’s real-world fine-tuning performance 🦾

[Ai2 @allen_ai]

Robotics models often struggle outside controlled environments. Ours is built to work in real ones.

Today we're launching MolmoAct 2, which can assist with a host of chores & lab tasks, plus the MolmoAct 2-Bimanual YAM dataset—the largest open robotics dataset of its kind. 🧵

[Lucas Ngoo @lucasngoo]

Big release from @allen_ai!

@cortexairobot supported Ai2 on 720h of bimanual robot data and ran the real-world evals for MolmoAct2.

Largest open dataset of its kind + strong results vs baselines. Congrats to the Ai2 team @DJiafei @hq_fang!

[Jiafei Duan@CVPR2026 @DJiafei]

1/🧵
Building more with less.
In the original MolmoAct, we introduced our first in-house dataset: 22 hours of teleoperated data on a single Franka.

Today we're releasing the MolmoAct 2 Bimanual YAM dataset, 720 hours of high-quality data collected with @cortexairobot .

Not toy tasks. Real-world tasks ready to deploy into business workflows today.

[Ai2 @allen_ai]

We retained @cortexairobot to run a third-party real-world fine-tuning benchmark.

Across trials on a broad suite of tabletop, in-the-wild, and mobile tasks, MolmoAct 2 outperformed systems including OpenVLA-OFT, π0.5, X-VLA, & Cosmos Policy.

[Ai2 @allen_ai]

Robotics models often struggle outside controlled environments. Ours is built to work in real ones.

Today we're launching MolmoAct 2, which can assist with a host of chores & lab tasks, plus the MolmoAct 2-Bimanual YAM dataset—the largest open robotics dataset of its kind. 🧵

[Jiafei Duan@CVPR2026 @DJiafei]

Excited to share Embodied Reasoning in Action at @CVPR 2026 — a workshop and open challenge on robotic manipulation.

We’re launching two challenges:
⚡ RoboSpatial — point, fit, and place from RGB-D home scenes
⚡ PointArena — language-guided pixel pointing for VLM evaluation
💰 Up to $6K in cash prizes
📍 Denver, CO
📅 June 3–4, 2026
🗓️ Challenge deadline: May 23, 2026
Huge thanks to our sponsors: @NVIDIARobotics , @BitRobotNetwork , and @cortexairobot

Learn more: https://t.co/16pPnZPuBK

[Lucas Ngoo @lucasngoo]

We’re now deploying mobile bimanual robots in-the-wild
collecting real-world data, running model evals, and capturing recovery trajectories for RL

[Lucas Ngoo @lucasngoo]

1/ They pretrain a robot world model on ~44k hours of egocentric human video.
Mostly RGB. No detailed action labels.
So the question is: how do you learn action-conditioned dynamics from unlabeled video?

2/ Their idea is “latent actions.”
They train a VAE that takes two consecutive frames (fₜ, fₜ₊₁) and compresses the transition into a small vector.
That vector represents what changed between the frames.
It becomes a proxy for the action.

3/ They use these latent actions to condition a video world model:
frameₜ + latent_actionₜ → frameₜ₊₁
So instead of passive next-frame prediction, the model learns transitions conditioned on action.
They benchmark this and show latent actions perform close to using real hand pose labels (e.g. EgoDex).

4/ After large-scale human pretraining, they post-train on real robots.
They reset the action-conditioning layer and replace latent actions with real robot controls.
Since the model already learned general physics from human video, much less robot data is needed to adapt to a new embodiment.

5/ They also show that increasing the scale and diversity of human video improves generalization to unseen objects and novel action variations. Now imagine training on 100 million hours of large scale, diverse, real world workplace data. This is the future we are excited to help power at @cortexairobot.

[Lucas Ngoo @lucasngoo]

Thanks @Fondocom for the podcast. Shared about:

> World models are the equivalent of language models for the physical world: predicting next visual frames and next robot actions.

> Scaling laws for robotics world models:

Large-scale, diverse, real-world egocentric data leads to better world models, which in turn lead to better robot action predictions from the model.

> Progress comes from real-world deployment with humans in the loop:

Human operators initially monitor and correct robot trajectories, and that recovery data feeds back into training to gradually increase autonomy.

[CortexAI @cortexairobot]

Cortex AI in Times Square.

Accelerating physical AI with large-scale, real-world robotics data.

Thanks to @brexHQ for making it happen.

[Y Combinator @ycombinator]

Cortex AI (@cortexairobot) produces the world's most diverse egocentric and robot dataset, captured in real workplace environments.

Leading research labs use Cortex AI to collect data and deploy robotics foundation models in real-world settings.