Highlights from Chelsea Finn’s (@physical_int Co-Founder) talk at YC Startup School:
> RL for robotics is bottlenecked by physical rollout cost, not just algorithm quality. 1M trajectories of a 1-minute task would take ~700 robot-days, which makes direct scaling of PPO/GRPO-style methods fundamentally different from LLM post-training.
> Memory is still a major missing piece in robot foundation model: most SOTA systems have effectively zero memory, and naive video history is too expensive (10s at 50Hz across 4 cameras ≈ 500K tokens). π solves this through short-term video memory plus long-term text summaries of the prior 10–15 minutes, which enabled a fully autonomous multi-step kitchen cleaning task.
> Generalist robot models may already be beating specialist pipelines: π0.7 mixed demos, rollouts, human video, and web data, then used rich prompting + metadata to make even lower-quality data useful — and it matched/outperformed fine-tuned specialists while showing compositional transfer to new tasks/platforms.
@chelseabfinn Thank you for the great talk!!
VT-WAM tackles one of the hardest robotics problems: predicting what happens during contact-rich manipulation.
Instead of modeling only visual dynamics, it jointly learns vision + touch + actions in a unified world model.
This lets the robot imagine future interactions before acting.
For LLMs, human preference is the best guidance. The analogue for robots is human’s “understanding of physics” = sim.
This work resonates a lot. Sim is imperfect but valuable as “guidance”: constrain search space, select checkpoints, steer base policy.
things robotics engineers shouldn’t still be doing manually in 2026:
– grepping through bags/mcaps to find when a fault first appeared
– screenshotting charts into confluence to write incident reports
– being the only person who can answer “has this happened before?” because the queries live in your head
– rebuilding the same log upload → report → retro pipeline for every new robot
your best engineers should be fixing robots, not babysitting data plumbing
Robots 🤖struggle at new long-horizon tasks. One trick to improve performance is using granular subtask labels ("grab the plate" instead of just "do the dishes").
Today we're releasing our first blog 📜: using VLMs to mine subtasks at scale (19x cheaper than human annotators!)