Nikita Chernyadev

Simulating Robots 🤖 #Robotics #AR #VR

[Kevin Lin @linkevin0]

Humanoids need data. Lots and lots of data.

Introducing HumanoidMimicGen: a method that automatically generates 1000s of humanoid loco-manipulation demonstrations from a single teleoperated demonstration.

[Nikita Chernyadev @nc__dev]

Glad to contribute and be a part of the team! :)

[Jim Fan @DrJimFan]

Excited to announce GR00T N1, the world’s first open foundation model for humanoid robots! We are on a mission to democratize Physical AI. The power of general robot brain, in the palm of your hand - with only 2B parameters, N1 learns from the most diverse physical action dataset ever compiled and punches above its weight:

- Real humanoid teleoperation data.
- Large-scale simulation data: we are open-sourcing 300K+ trajectories!
- Neural trajectories: we apply SOTA video generation models to “hallucinate” new synthetic data that features accurate physics in pixels. Using Jensen’s words, “systematically infinite data”!
- Latent actions: we develop novel algorithms to extract action tokens from in-the-wild human videos and neural generated videos.

GR00T N1 is a single end-to-end neural net, from photons to actions:

- Vision-Language Model (System 2) that interprets the physical world through vision and language instructions, enabling robots to reason about their environment and instructions, and plan the right actions.
- Diffusion Transformer (System 1) that “renders” smooth and precise motor actions at 120 Hz, executing the latent plan made by System 2.

We deploy N1 on GR1 robot, 1X Neo robot, and a large collection of simulation benchmarks. N1 achieves up to +30% boost in diverse manipulation tasks for household and industrial settings.

While humanoid robots are the main focus of N1, our model also supports cross-embodiment. We finetune it to work on the $110 HuggingFace LeRobot SO100 robot arm! Open robot brain runs on open hardware. Sounds just right.

Let’s solve robotics, together, one token at a time.

Links to our Whitepaper, Github repo, HuggingFace model, and open dataset page in the thread: 🧵

[Xiao Ma @yusufma555]

BiGym accepted to #CoRL2024! Big thanks to all my collaborators! Our paper and code are available. Check them out!

Project page: https://t.co/btKiEpNyvl
Simulator code: https://t.co/uDKdBLQhSu
Baselines: https://t.co/Y3mYoCB7Id

[Nikita Chernyadev @nc__dev]

@valenorf We are using AsyncVectorEnv, which is essentially python multiprocessing:
https://t.co/3M3vmGpDgD

[Nikita Chernyadev @nc__dev]

🚀 Looking for a benchmark for bi-manual mobile manipulation with nicely collected demonstrations? We are excited to release BiGym, a new benchmark with human-collected demos!

🌐 Website: https://t.co/cSNRuAZ5cV
📄 Paper: https://t.co/h8aSeUWtjM
💻 Code: https://t.co/AFYxDPnNJR

[Shole @Sholeole]

@_akhaliq The jittery legs and anxious energy made me want to add a soundtrack

[Stephen James @stepjamUK]

As we explore new opportunities and the future of this talented group, we’re grateful for all the support. Feel free to reach out—our DMs are open!

@mohito1905 @younggyoseo @iainhaughton @nc__dev @chrysalis_ai @eugene_teoh @JafarUruc @SridharSola

[Stephen James @stepjamUK]

🚨Important update from our Robot Learning Lab in London.
Following recent news, we’re moving on after a wonderful 2 years…
Today, we unveil 4 big pieces of research from our incredible team. Check out the compilation video and thread below to see our final work! 📽️👇

[Nikita Chernyadev @nc__dev]

⚙️ We believe BiGym opens wide range of research opportunities, adding more important challenges to the robotics community. We welcome the community to try out and contribute to BiGym. It’s time to roll!

[Nikita Chernyadev @nc__dev]

📊 In our preliminary experiment results, we benchmarked 7 state-of-the-art imitation learning and demo-driven reinforcement learning algorithms on all tasks, and show that existing algorithms are not yet capable of solving the challenging BiGym tasks.