Introducing OM-1, our first robot foundation model, zero-shot generalizing to any robot: table-top arms, industrial arms and humanoids.
- learned directly from human manipulation data
- no teleop/robot data
- close to human-level dexterity and efficiency
- multi-robot collab
Gave GPT-6 Astra a few renders of our office scan. It came back with a full Blender rebuild — 50 desks, 62 chairs, walls, blinds — aligned to the scan within 2 cm and exported to USD. Dropped it straight into Newton with a G1 walking through it. One prompt → a humanoid gym. 🤯
i gave astra a robot, a paint brush, and a camera then asked it to paint the golden gate bridge in real life!
it figured out how to control the robot, and progressively got better throughout its attempts. the timelapse is sick
Introducing S1, our new foundation model that learns from one example.
It can be taught 10-minute long tasks that it has never seen before, from one video prompt without any fine-tuning.
Watch S1 operate in real-time via in-context learning:
What if robot hands could acquire dexterity once, then reuse it to learn downstream tasks much faster?
Introducing ADEPT: a pre-training & post-training paradigm using RL entirely in sim, then zero-shot deploying visuo-tactile policies in the real world.
🔗https://t.co/jdIR1CnG4c with Dex team @NVIDIAAI
Introducing GEN-1.5, a one-shot learner.
It can learn new tasks in a few seconds. Show it what to do, and it generalizes.
This capability emerged from pretraining on physical data at scale, as a step towards our mission of building general intelligence for the physical world.
We scaled a robot model natively to 8,000 timesteps of context, 5 minutes worth of muscle memory, with constant inference cost. Robot policies used to live their lives a few frames at a time (< 0.1 sec), instantly forgetting what just happened. We pushed to 3 orders of magnitude beyond SOTA.
Introducing RoboTTT. Test-Time Training (“TTT”) carries a tiny model *inside* the model. Every incoming sensor reading triggers one gradient step on that tiny core, so the history keeps getting compressed into its weights. The hidden state has a fixed size (literally a small neural net), so the robot can “grok” arbitrarily long experience with little overhead. Learning continues indefinitely after deployment.
We can then put an entire video in context as prompt! RoboTTT enables one-shot in-context learning from human video: in circuit board assembly, a human demonstrates a never-seen configuration once, and the robot imitates it faithfully.
Humans drop things all the time, but we pick them up so fast that we don’t even notice. That reflex to fix is half of our physical competence. RoboTTT shows self-improvement on the fly: the robot is skilled at recovering from its own errors mid-episode, and each fix enters its context to inform the next move. The TTT core distills a general-purpose, failure-to-correction mapping from the training data.
One more thing. What excites me the most is a new Context Scaling Curve: from 128 to 8K timesteps, closed-loop performance hill-climbs steadily with no sign of saturation. 8K-context pretraining beats 1K by 62%. What LLM enjoys, robotics should too. Soon, even 1M context is not a fantasy.
Deep dive in thread:
Autoresearch just left the sandbox and entered the embodied world.
We are excited to introduce 𝐄𝐍𝐏𝐈𝐑𝐄: a system that drops frontier coding agents onto a fleet of real robots and hands them the entire loop:
reset the environment → search the literature → implement ideas and build the infra → train and deploy → self-verify → analyze the logs and rewrite the code → repeat, until the policy is reliable in the real world. No human in the loop.
Guided only by the robot's self-proposed, heuristic-based success signal, the agents hill-climb to 99% on dexterous real-world tasks: organizing pins into a box, seating GPUs, tying zip-ties.
We envision the bottleneck in robotics shifting — from building smarter algorithms to building the closed physical feedback loops an agent can finally turn on its own.
🔗 https://t.co/3tL2ArGo3v
From @NVIDIA@CMU_Robotics@Berkeley_AI
🧵
India’s fertility rate has fallen below replacement for the first time in the country’s history, declining from a TFR of 2.3 to 1.9 in just a decade.
Delhi’s fertility rate now sits at 1.2, lower than Finland’s.
Follow: @AFpost
We are back again :) After three weeks of quiet building.
Introducing Genesis World 1.0, our latest simulation platform, the second release in our full-stack suite. Open-sourced.
Robotics is still bottlenecked by the 1× speed of the physical world. Every model, checkpoint, and data recipe eventually needs to be tested on physical hardware, slowly, expensively, and with limited coverage.
One hour in reality can become 100 days in simulation. That is how robotics model iteration moves from a wall-clock bottleneck to a compute problem.
To make this work, simulation has to be both fast and trustworthy.
Over the past year, we rebuilt the entire stack: a GPU-accelerated cross-platform compiler, penetration-free multi-physics contact solvers, unified rigid and deformable physics, and a photo-realistic renderer purpose-built for physical AI applications.
We built Nyx, a high-performance path-traced rendering engine for robotics application.
Genesis World 1.0 achieves near realtime performance with our latest development for penetration-free IPC solver, supporting various types of deformables beyond rigid bodies. It supports contact-rich, dexterous manipulation simulation across different embodiments: unitree, sharpa, wuji, genesis hand and various types of grippers.
Under the hood is Quadrants, our effort in pushing forward cross-platform GPU-accelerated computation. Quadrants started as a fork of Taichi, and we rebuilt most of the critical parts for optimizing simulation workloads, giving 10x faster launch time and up to 4.6x runtime performance compared to the initial Genesis release.
Together, they bring us to an unprecedentedly low sim-to-real gap, enabling zero-shot real-to-sim model evaluation and much faster iteration of GENE.
All available today.
Genesis World 1.0: https://t.co/aknCM3eqws
Quadrants: https://t.co/uXqPNI4cb6
Nyx: https://t.co/R8j0djqGnV
1/ 🧠Humans are the best robot data source!
2/ 👓Human egocentric video is rich in quantity, but poor in quality.
3/ Beyond scaling data, smarter representation and architecture matter just as much.
4/ Want an open-source framework to train your own learn-from-human-data robot policy?
🚀We introduce HumanEgo: Zero-Shot Robot Learning
from Minutes of Human Egocentric Videos⬇️
✦ Zero-Shot Human-to-Robot Transfer
✦ Robot-Data-Free
✦ Just 30 min of data per task
✦ Collect by Anyone, Anytime, Anywhere
✦ Deploy on Any Robot, Any Camera, Any Environment
✦ Open-Source & Easy-to-Implement
Let's squeeze every bit of signal out of human data!
🌐 Website: https://t.co/JfsW8x6wtq
📄 Paper: https://t.co/tsaIiatmNi
💻 Code: https://t.co/jZjghCcjh2
📹 Video: https://t.co/QWmJmQ9GgQ
🧵 1/n
SONIC training code + Finetuning checkpoint + VLA data collection scripts are open-sourced.
Little easter egg on the GEAR-SONIC website too :)
https://t.co/2BgYj3tLYO
Releasing the Unfolding Robotics blog!
Time to unfold robotics: we trained a robot to fold clothes using 8 bimanual setups, 100+ hours of demonstrations, and 5k+ GPU hours.
Flashy robot demos are everywhere. But you rarely see the real story: the data, the failures, the engineering.
We’re sharing everything: code, data, and details in the blog → https://t.co/bZ8NXw0CZh
2/ I joined summer '24 to scale data operations in a small house in mountain view. 5 of us lived in there.
We all did things that don't scale:
- our Head of HW @clayhaight designed and printed gloves in the home garage
- our CTO @chichengcc installed a level 2 charger for all the 3D printers, built desks for employees, and did all interviews from his bedroom
- our Manufacturing Engineer @che_haichuan built gloves and modified disco lights so that our (s)crappy SLAM could work in low feature environments
- our Founding Designer @shyyang and I would meet craigslist strangers in the Mountain View Public Library to onboard them. We also would clean and stock the house with snacks and drinks
- our CEO @tonyzzhao ran evals and desperately tried to give me crash courses on what makes data diverse and high quality. he edited tiktok style capcut videos for our data collectors and sent them encouraging feedback on their data
- our Founding SWE @henry_yu_01 built all of our data review tools, constantly putting bandaids on original contractor code before we had time to kill/rebuild. he put a "do not disturb sign" on his chair as the house got more crowded
Meet DAS Gripper 🤖
• 5 modalities, 8 sensors
• Vision + touch + audio + IMU + magnetic encoders
• High-precision trajectory & cross-modal perception
• Plug-and-play, no base station, no PC
• Local & cloud data modes
• Fast, flexible, everywhere data.
What's remarkable to me is how well the representations align as we increase the amount of robot data. To be clear: in this video we are only increasing the amount and diversity of *robot* data, and we see human/robot images mapping to the same place in representation space.