We are back again :) After three weeks of quiet building.
Introducing Genesis World 1.0, our latest simulation platform, the second release in our full-stack suite. Open-sourced.
Robotics is still bottlenecked by the 1× speed of the physical world. Every model, checkpoint, and data recipe eventually needs to be tested on physical hardware, slowly, expensively, and with limited coverage.
One hour in reality can become 100 days in simulation. That is how robotics model iteration moves from a wall-clock bottleneck to a compute problem.
To make this work, simulation has to be both fast and trustworthy.
Over the past year, we rebuilt the entire stack: a GPU-accelerated cross-platform compiler, penetration-free multi-physics contact solvers, unified rigid and deformable physics, and a photo-realistic renderer purpose-built for physical AI applications.
We built Nyx, a high-performance path-traced rendering engine for robotics application.
Genesis World 1.0 achieves near realtime performance with our latest development for penetration-free IPC solver, supporting various types of deformables beyond rigid bodies. It supports contact-rich, dexterous manipulation simulation across different embodiments: unitree, sharpa, wuji, genesis hand and various types of grippers.
Under the hood is Quadrants, our effort in pushing forward cross-platform GPU-accelerated computation. Quadrants started as a fork of Taichi, and we rebuilt most of the critical parts for optimizing simulation workloads, giving 10x faster launch time and up to 4.6x runtime performance compared to the initial Genesis release.
Together, they bring us to an unprecedentedly low sim-to-real gap, enabling zero-shot real-to-sim model evaluation and much faster iteration of GENE.
All available today.
Genesis World 1.0: https://t.co/aknCM3eqws
Quadrants: https://t.co/uXqPNI4cb6
Nyx: https://t.co/R8j0djqGnV
We are back. After one year of quiet building.
Introducing GENE-26.5, our first robotic brain that takes a major step toward human-level capability.
For years, robotics has struggled to learn from the world’s largest and valuable data source: Humans.
Solving it means rethinking the whole stack from the ground up:
- A robotics-native foundation model.
- A 1:1 human-like robotic hand.
- A noninvasive data collection glove for motion, force, and touch.
- A simulator that turns weeks of experiments into minutes.
GENE-26.5 is trained across language, vision, proprioception, tactile, and action. We designed a set of tasks to test how far we can go with this new paradigm.
Fully autonomous, 1x speed, one model, same weights. (Enjoy with sound on)
We are approaching the endgame for robotics.
And this is just a beginning.
thanks @_akhaliq for highlighting our work! Build your own community today based on real-world 3D scenes and physically plausible simulations, and see how current AI Agents can achieve!
Check out Virtual Community! A unified platform for Embodied AI Research on Robots, Agents, and Societies, based on real-world 3D Scenes and physically plausible simulations! Build your own community today and see what current AI Agents could achieve!
https://t.co/ogwdoiC01J
World Simulator, reimagined — now alive with humans, robots, and their vibrant society unfolding in 3D real-world geospatial scenes across the globe!
🚀 One day soon, humans and robots will co-exist in the same world. To prepare, we must address:
1️⃣ How can robots cooperate or compete intelligently?
2️⃣ How do humans build social bonds and communities?
3️⃣ How can both co-exist in an open, dynamic world?
Announcing Virtual Community Project — a social-physical world simulator, where human characters and robotic agents can interact, grow, and co-evolve within open-world societies, stretching from London to New York, and beyond!
Key features include:
✅ Unified multi-agent physics simulations for rich social + physical interactions of humans and robots
✅ Massive auto-generated 3D scenes grounded with the rea-world geospatial data
✅ Agent communities populated by robots and LLM-driven human characters with rich appearances, personalities, and social ties.
🌍 Enter our Virtual Community, an open world to study embodied AI at scale— one social-physical world model at a time!
🔗 Project: https://t.co/SItesNxOvN
💻 Code: https://t.co/clgr6rP7yJ
Paper: https://t.co/VZ67DUchRg
1/n
Thanks @_akhaliq for sharing our work TesserAct: Learning 4D Embodied World Models!
Step into the TesserAct and explore — or push the boundaries of TesserAct!
Tomorrow (4.24), 15:00-17:30 at ICLR Hall 3 + Hall 2B #34,
I'll present COMBO: Compositional World Models for Embodied Multi-Agent Cooperation.
Come to talk!
Excited to introduce 3D-Mem!
Spatial Intelligence simply isn’t possible without robust 3D Scene Memory. That’s why we developed 3D-Mem, an effective framework for lifelong exploration and reasoning.
Thrilled to share that it’s been accepted to #CVPR2025 !
Everything you love about generative models — now powered by real physics!
Announcing the Genesis project — after a 24-month large-scale research collaboration involving over 20 research labs — a generative physics engine able to generate 4D dynamical worlds powered by a physics simulation platform designed for general-purpose robotics and physical AI applications.
Genesis's physics engine is developed in pure Python, while being 10-80x faster than existing GPU-accelerated stacks like Isaac Gym and MJX. It delivers a simulation speed ~430,000 faster than in real-time, and takes only 26 seconds to train a robotic locomotion policy transferrable to the real world on a single RTX4090 (see tutorial: https://t.co/bEkIlCKqdf).
The Genesis physics engine and simulation platform is fully open source at https://t.co/DhBv7NdyqH. We'll gradually roll out access to our generative framework in the near future.
Genesis implements a unified simulation framework all from scratch, integrating a wide spectrum of state-of-the-art physics solvers, allowing simulation of the whole physical world in a virtual realm with the highest realism.
We aim to build a universal data engine that leverages an upper-level generative framework to autonomously create physical worlds, together with various modes of data, including environments, camera motions, robotic task proposals, reward functions, robot policies, character motions, fully interactive 3D scenes, open-world articulated assets, and more, aiming towards fully automated data generation for robotics, physical AI and other applications.
Open Source Code: https://t.co/DhBv7NdyqH
Project webpage: https://t.co/SBNyhFB0yn
Documentation: https://t.co/3yuBoaealV
1/n
Chain of Thought reasoning prompts—like "Let's think step by step"—make large language models more performant. Including, it turns out, at spewing out toxic and biased content. In our preprint, we evaluate zero-shot CoT on harmful questions & stereotypes: https://t.co/6lzu9VVVM5
Multi-VALUE is a toolkit to evaluate and mitigate performance gaps in NLP systems for multiple English dialects.
We release scalable tools for introducing language variation, which you can use to stress test your models and increase their robustness https://t.co/WpXtpsiyT2
🧵
Demonstrations composed of RANDOM tokens can still work? YES!
In our #EMNLP2022 paper (w/@StevenyzZhang,@Diyi_Yang,@RoyZhang13), we design pathological demonstrations to investigate “Robustness of Demonstration-based Learning Under Limited Data Scenario” https://t.co/Gn32Llf1Uu
Demonstrations composed of RANDOM tokens can still work? YES!
In our #EMNLP2022 paper (w/@StevenyzZhang,@Diyi_Yang,@RoyZhang13), we design pathological demonstrations to investigate “Robustness of Demonstration-based Learning Under Limited Data Scenario” https://t.co/Gn32Llf1Uu
To examine whether this magical improvement has any connections with spurious patterns or dataset bias, we use the NRB benchmark as a case study and find that demonstrations bring no robust improvements
[6/6]