Daniele Reda

Scientist @wayve_ai | PhD in RL, Physics-based character animation and Robotics at @ubc_cs

[Lily Goli @lily_goli]

🚀 🚀 🚀 Excited to share our new paper:

Remember to be Curious: Episodic Context and Persistent Worlds for 3D Exploration

What does it take for an agent to stay curious in a 3D world?
The answer is memory.

🌐 Project: https://t.co/G4SjLoFJht
📄 Paper: https://t.co/iUFwp5NvRu
💻 Code: https://t.co/KZRaQLyzyh

[Zhou Xian @zhou_xian_]

Everything you love about generative models — now powered by real physics!

Announcing the Genesis project — after a 24-month large-scale research collaboration involving over 20 research labs — a generative physics engine able to generate 4D dynamical worlds powered by a physics simulation platform designed for general-purpose robotics and physical AI applications.

Genesis's physics engine is developed in pure Python, while being 10-80x faster than existing GPU-accelerated stacks like Isaac Gym and MJX. It delivers a simulation speed ~430,000 faster than in real-time, and takes only 26 seconds to train a robotic locomotion policy transferrable to the real world on a single RTX4090 (see tutorial: https://t.co/bEkIlCKqdf).

The Genesis physics engine and simulation platform is fully open source at https://t.co/DhBv7NdyqH. We'll gradually roll out access to our generative framework in the near future.

Genesis implements a unified simulation framework all from scratch, integrating a wide spectrum of state-of-the-art physics solvers, allowing simulation of the whole physical world in a virtual realm with the highest realism.

We aim to build a universal data engine that leverages an upper-level generative framework to autonomously create physical worlds, together with various modes of data, including environments, camera motions, robotic task proposals, reward functions, robot policies, character motions, fully interactive 3D scenes, open-world articulated assets, and more, aiming towards fully automated data generation for robotics, physical AI and other applications.

Open Source Code: https://t.co/DhBv7NdyqH
Project webpage: https://t.co/SBNyhFB0yn

Documentation: https://t.co/3yuBoaealV

1/n

[Guy Tevet @GuyTvt]

[NEW Preprint] 🔔🔔 CLoSD embeds real-time Motion Diffusion into a multi-task RL agent. Performing a task is as easy as describing it with a text prompt!
Want to move to the next task? Just change the prompt on the fly😁 [1/4]🧵
https://t.co/1NvFsJdCEc

[Setareh Cohan @setarehcohan]

Flexible Motion In-betweening with Diffusion Models🧎🚶🧍Done together with @GuyTvt, @rdednl, @xbpeng4, and @Mvandepanne

📄Paper: https://t.co/4QqS7l3lsV
🌐Website: https://t.co/GfkFaTvmM2

Grateful for the insights and feedback provided by everyone!

[Daniele Reda @rdednl]

Love to see our retargeting work featured in the latest 2MinutePaper video!
For more details check the website: https://t.co/C6PAsZBWJi

[Two Minute Papers @twominutepapers]

Virtual Reality is getting closer and closer with these amazing new papers. Also, three videos this week!
https://t.co/roCHGo66gP

[Daniele Reda @rdednl]

@HelgeRhodin Congrats, Helge!

[Daniele Reda @rdednl]

@awinkler_ cool work!

[Daniele Reda @rdednl]

@fcdl94 @bcaputo_iit @massi_manc @PoliTOnews Congrats!! 💪💪

[Alexander Winkler @awinkler_]

Non-human avatars like a mouse or a dinosaur would be fun to use in VR. The challenge is how to map a users's motion, detected solely by the headset and controllers, to different creatures. In this work we use physics to fill in missing information in a natural way.
1/5👇

[Daniele Reda @rdednl]

This paper from my @RealityLabs internship shows how to control different characters and retarget the user's pose from only the sensors of the Quest using reinforcement learning.

Coauthors: @awinkler_ @Mvandepanne Jungdam Won and Yuting Ye

Website: https://t.co/C6PAsZBWJi

[AK @_akhaliq]

Physics-based Motion Retargeting from Sparse Inputs

paper page: https://t.co/aJCe011y5y

Avatars are important to create interactive and immersive experiences in virtual worlds. One challenge in animating these characters to mimic a user's motion is that commercial AR/VR products consist only of a headset and controllers, providing very limited sensor data of the user's pose. Another challenge is that an avatar might have a different skeleton structure than a human and the mapping between them is unclear. In this work we address both of these challenges. We introduce a method to retarget motions in real-time from sparse human sensor data to characters of various morphologies. Our method uses reinforcement learning to train a policy to control characters in a physics simulator. We only require human motion capture data for training, without relying on artist-generated animations for each avatar. This allows us to use large motion capture datasets to train general policies that can track unseen users from real and sparse data in real-time. We demonstrate the feasibility of our approach on three characters with different skeleton structure: a dinosaur, a mouse-like creature and a human. We show that the avatar poses often match the user surprisingly well, despite having no sensor information of the lower body available. We discuss and ablate the important components in our framework, specifically the kinematic retargeting step, the imitation, contact and action reward as well as our asymmetric actor-critic observations. We further explore the robustness of our method in a variety of settings including unbalancing, dancing and sports motions.

[Two Minute Papers @twominutepapers]

This is a super fun paper on simulating locomotion for virtual gibbons. You can even play with it - check it out! 😀 https://t.co/goDUKNJPEb

[Daniele Reda @rdednl]

@CafeRoubaix Head - MVDP
Heart - Ganna

[Daniele Reda @rdednl]

I recently discovered @notcous's NotMilk which is the best plant-based milk I've tried. They have an ML system "Giuseppe" that analyzes and proposes recipes to match foods at the molecular level and the result is amazing! It's new to me and I love when ML is used in these fields!

[Daniele Reda @rdednl]

Excited for my first SIGGRAPH conference! We’ll be presenting Learning to Brachiate at the Roundtable session on Physics-Based Character Control, Tuesday at 2.15pm, Ballroom A/B.
Video, paper, code and demo are here: https://t.co/Gf3rGKUqD1

[Tony Pann @TonyPannWBAL]

Once a year, for a rare moment at 7:15 AM EDT on July 8th, 99% of the world's population will be in the Sun. About 6.4 billion people in the daytime, while more than 1.2 billion people experience twilight. How cool is that! Source: https://t.co/5fN1IaFR3c

[Dan Deakins @DanDeakins]

Rock, paper, scissors for the stage win 😂

#TourOfSlovenia

[Patrick Mineault @patrickmineault]

PaLM can answer questions that will make an average human's head explode

[Daniele Reda @rdednl]

Have been waiting for this update since M1 was announced. Can't wait to try it!

[PyTorch @PyTorch]

We’re excited to announce support for GPU-accelerated PyTorch training on Mac! Now you can take advantage of Apple silicon GPUs to perform ML workflows like prototyping and fine-tuning. Learn more: https://t.co/8VmtnhfrZy

[Daniele Reda @rdednl]

Our most recent work is now available!
Check out the web demo at https://t.co/Ym8OqkD2RY

[Jia-Bin Huang @jbhuang0604]

Excited to see many SIGGRAPH 2022 go public!

https://t.co/NUM5EMfYrN

It's amazing that Ke-Sen single-handedly provides the best indexing of graphics papers on the web for *15 years*!