Collin Mitchell

marketing lead @midcenturyai | prev @nvidia, @amd

[Ali @robot_xyz_rpy]

Robot policy benchmarks are broken in interesting ways, they:

1️⃣ Focus on general capabilities without demonstrating reliable transfer to domain-specific tasks (A policy may excel at stacking cubes yet fail at simple tasks outside the lab).

2️⃣ Rely on metrics that do not faithfully reflect policy performance.
> Success rates often capture only the final state of the environment, overlooking execution efficiency, grasp quality, recovery from failures ......

3️⃣ Evaluate policies in overly idealized environments that fail to reflect the complexity and variability of real deployment conditions.

4️⃣ Some policies perform well in sim with high success rate, but breaks down in the real world

If you’re a researcher interested building the next era of benchmarks, you should apply to @MidcenturyAI’s 2nd residency cohort below.

[Midcentury @MidcenturyAI]

Announcing the Midcentury Research Residency.

We're bringing together a small cohort of researchers to tackle the hardest problems in physical AI, backed by:

→ $1 million in compute
→ The world's largest robotics dataset spanning 2M+ hours
→ SOTA action supervision infrastructure

Our first cohort includes an author of NVIDIA's Cosmos 3, a CoRL award winner, and researchers from CMU, UCL, and Brown, with publications at NeurIPS and ICRA.

Their research focuses on fundamental questions in robot learning and world models. We've already published MC-PhysBench, with more work coming soon.

Applications for Cohort 2 are now open. Apply here: https://t.co/olMEzcTaWG

[Collin Mitchell @micholco]

recently

[shilpi! @shilpishahh]

this was an incredibly fun and insightful project! while world models are becoming increasingly advanced in terms in terms of graphical rendering, there is still a long way to go for physical intelligence and plausibility. super grateful to @MidcenturyAI for the opportunity.

[Chetan Kulhari @ckxyz_]

we're so early on world models as simulators

we scored 17 video world models on a ball rolling down a ramp and every one breaks physics in under a second. a no-model baseline beats them in 91 of 103 results

training on pixels alone only gets you so far. modeling physics accurately takes physics-rich data

[Midcentury @MidcenturyAI]

We tested leading video world models on their ability to predict simple physics and found that most fail within a few frames.

Today we're releasing MC-PhysBench, a new kind of benchmark that measures how long a model's output stays physically valid.

15+ models. 9 scenes. 6,000+ videos scored. 7 failure checks.

[calix @calixo888]

this is exactly why we love open-source. great to see new work being built on top of of Argus

[Midcentury @MidcenturyAI]

We spent the weekend experimenting with Argus and pushed it to SOTA on WGO-Bench.

Our optimized version scores 50.2% semantic F1, up from the 29.8% out-of-the-box score, and well ahead of other leading annotation services.

All of the gains came from how the model sees the video. We gave it denser frames, a second look at each action's boundaries, and a final check on every label.

Thanks to @PantheonInc @calixo888 for open-sourcing Argus.

The full repo is coming soon. For now, you can read our tech report here: https://t.co/Ue1gHmWoBT

[Collin Mitchell @micholco]

@Baxate Having the exact same experience. Trying to stick with it because eventual Muse quality + OpenAI ecosystem is the ideal form factor but it’s just not close yet

[Archie Sengupta @archiexzzz]

Today, @MidcenturyAI introduces MC-EgoHands, our frontier 3D harness for human egocentric video.

PA-MPJPE across four public hand-pose datasets:

> ARCTIC - 3.67 mm
> HOT3D - 3.98 mm
> H2O - 5.24 mm
> FreiHAND - 4.30 mm

[Collin Mitchell @micholco]

This is the first in a long slate of very impressive research releases.

Congrats @robot_xyz_rpy @archiexzzz

[Midcentury @MidcenturyAI]

Introducing MC-EgoHands, the first all-in-one harness for turning raw ego video into training-ready data.

→ Frontier hand pose (<7mm) across 4 public benchmarks

→ Human grasps retargeted to Panda/LIBERO and Yam in one pass

→ Occluded frames flagged as interpolated or missing

→ Structured episodes in @huggingface @LeRobotHF v3

[Midcentury @MidcenturyAI]

Introducing MC-EgoHands, the first all-in-one harness for turning raw ego video into training-ready data.

→ Frontier hand pose (<7mm) across 4 public benchmarks

→ Human grasps retargeted to Panda/LIBERO and Yam in one pass

→ Occluded frames flagged as interpolated or missing

→ Structured episodes in @huggingface @LeRobotHF v3

[Midcentury @MidcenturyAI]

How do you scale 2 million hours of ego data without sacrificing quality?

Earlier this week, we revealed the world’s largest egocentric dataset. Our frontier pipeline has also turned it into the most labeled, annotated, and human QA’ed robotics ego dataset in the world.

We’ve focused on some of the hardest problems in computer vision and ego data to ensure every clip contains SOTA action supervision directly helpful for training dexterous manipulation.

[Collin Mitchell @micholco]

@archiexzzz @MidcenturyAI generational run starts now

[Collin Mitchell @micholco]

We're out of stealth!

Last month I left NVIDIA to join @MidcenturyAI in the pursuit of scaling robotics beyond the lab and into the real world.

Working on this brand has been a blast, a few examples below.

[Midcentury @MidcenturyAI]

Introducing Midcentury.

We’re building the data and simulation infra for physical AI.

Today, we’re coming out of stealth with a $15M Series Seed to scale robotics beyond polished demos.

We’re already supporting frontier labs with:

→ The world’s largest egocentric dataset: 2M+ hours, 50+ environments, 20,000+ tasks

→ Matrix: a frontier simulation platform scaled with our real-world data to evaluate and post-train policies at scale

[Chetan Kulhari @ckxyz_]

Excited to finally share what we’ve been building at @MidcenturyAI. We’re coming out of stealth with a $15M Series Seed to scale robotics beyond demos.

Physical AI is at an inflection point. EgoScale and Dyna have established a scaling law unlocked by human action data, and as pretraining scales, we’re seeing more generalist behavior and in-context learning emerge. Robotics is entering its scaling era and it’s time to embrace the bitter lesson.

But unlike LLMs, robotics can’t scrape the internet for pretraining data and real-world evaluation and RL don’t scale. That’s the gap we started Midcentury to close, and we’re already working with frontier labs on both sides of it.

Our thesis on data is simple: scale and enrichment are what matter. So we’ve built the world’s largest and most densely labeled egocentric dataset, with 2M+ hours across 20K+ tasks and SOTA action supervision.

On simulation, we’ve built Matrix, combining classical simulation, learned physics, and real-world data to close the sim-to-real gap and scale evaluation + RL.

We’ve brought together researchers and operators from the Stanford AI Lab, OpenAI, DeepMind, NVIDIA, Scale AI, and others who have built datasets, benchmarks, and models at the frontier of robotics and AI.

And we’re just getting started. We have a stacked slate of research releases coming, from SOTA 3D motion reconstruction from egocentric video to the first long-horizon physics-validity benchmark for world models.

Couldn’t be more excited to be building this with @dgmonsoon and a relentless team. Lots more coming soon!

[addeagupta @addeagupta]

We're out of stealth!! Data is the bottleneck for robotics, and the bet we're making is that human action data is how you break it. Not raw video, but millions of hours of real work enriched frame by frame with the pose, depth, and tactile signal that makes it something a robot can actually learn from. That's the machine I get to help run at Midcentury. The next era of physical AI gets built on top of it. Let's go 🚀

[Archie Sengupta @archiexzzz]

Today, @MidcenturyAI is coming out of stealth with $15M in seed funding to advance robot learning through large-scale data and simulation.

Our team brings together researchers and engineers from Stanford AI Lab, OpenAI, DeepMind, NVIDIA, and more, with experience spanning RoboNet, GDPVal, and NVIDIA Cosmos 3.

Excited to finally share what we’ve been working on.

What makes robotics such a compelling research problem is the sheer variety of the physical world. A useful robot needs to generalize across objects, environments, and tasks, and recover when things don’t go as expected.

We believe scaling that capability requires both diverse human action data and environments where robot policies can be evaluated and improved through repeated experimentation.

At Midcentury, we’re building both:

> 2M+ hours of egocentric human activity across 50+ environments and 20,000+ tasks, paired with rich action supervision to help robots learn from human behavior.

> Matrix, a simulation platform combining classical simulation, learned physics, and real-world data for policy evaluation and post-training at scale.

We’re already supporting frontier labs working on these problems. What excites me most is the research this enables: understanding how robots can learn transferable skills from human experience and turn those skills into reliable physical work.

Proud to be building this with our team @ckxyz_ @dgmonsoon @notnoahphan @sorozco0612 @micholco @addeagupta @B_S_N_Y @robot_xyz_rpy

If you’re working on robot learning, simulation, or embodied intelligence, I’d love to connect. We’re hiring.

https://t.co/pJrVpFCv1d

[Ali @robot_xyz_rpy]

Finally able to share what we’ve been building at Midcentury.

I joined for the depth of the team and the pace and ambition of the research. I lead our robotics research, focused on how robots can learn from the work people already do.

There’s a lot between recording human skill and teaching a robot to use it. We’re working across that gap: selecting useful data, reconstructing 3D motion, studying how human and robot experience fit together in training, and building better ways to evaluate robot behavior.

What excites me is following that whole path, from the data to what a robot can actually do, with a team that can tackle each part.

[Noah @notnoahphan]

What we’re working on: