I created my own daily automatic newsletter which lists me relevant (in reation to libreyolo) arxiv papers and r/computervision threads from the previous day. For example I discovered this today, amazing:
https://t.co/5ZKdm0JC1Z
6M param CNN for real time depth estimation. (video from their website)
Better data for robot hands! 🤲🏼
Embodied reasoning is the bottleneck now. It’s the data we are missing.
Specifically, understanding what the robot's hands are doing throughout a task.
@perceptroninc new Egocentric offering tracks both hands through an entire video instead of guessing from sampled frames.
That matters because a lot happens between frames. Hand positions shift. Contacts change. The robot adjusts grip. You miss all that if you're sampling.
0.280 semantic end-to-end F1 score on WGO-Bench vs 0.158 from the best pipelines built on Gemini Robotics-ER 1.6.
Training a robot to manipulate objects requires understanding every moment of hand-object interaction.
If your training data has gaps: missing frames, missed contacts, unclear hand positions, the policy learns from incomplete information.
You end up with robots that work in demos but fail in real tasks.
Continuous hand tracking gives you the full picture. The policy learns from cleaner, more complete data. That compounds into better real-world performance.
~~
♻️ Join the weekly robotics newsletter, and never miss any news → https://t.co/GoA3ZuwoPB
Why should collecting robot demonstrations require a robot at all? Introducing RynnWorld-Teleop -- a new paradigm we call Digital Teleoperation. Instead of teleoperating physical robots, operators control a robot-centric world model that generates synchronized robot videos and actions for imitation learning.
⚡ 40+ FPS real-time generation
🤖 Effective zero-shot Sim2Real
🌍 Hardware-agnostic, infinitely scalable robot data collection.
It is also a joint work with Haoyu Zhao & Damo Academy. Paper & code👉https://t.co/aimHEOcPWx
We’re launching the first in a series of Embodied Reasoning offerings: Perceptron Egocentric. It achieves SOTA over the best robotic annotation pipelines built on Gemini 3.5 Flash and Gemini Robotics-ER 1.6. Early access is available to partners.
👋AnyHand introduces a large synthetic RGB-D dataset for 3D hand pose estimation, showing that carefully designed synthetic supervision can substantially improve downstream performance.
An exciting resource for research in 3D hand understanding.
All sources are available now
putting final touches on our POV Stereo + Multi-View Dome data pipeline: zero sim2real gap, semantic+instance 4D segmentation, perfect temporal stability and trajectory tracking, near-perfect depth
For anyone looking to use egocentric data for training their robot policies, EgoVerse is a great starting point. It's open, permissible, and the dataset includes 1,362 hours of demonstration data!
The Github is easy to install and use for pulling curated datasets.
I would also be remiss if I didn't mention that @LightwheelAI joined to consortium to grow the project.
You can get started at https://t.co/yQeONlzxm6
#Robotics #PhysicalAI
Our book "Generative AI and Stochastic Thermodynamics: A Tale of Free Energies" is out this month. With @sirui_lu97 and @wellingmax I'm posting about topics it covers. Yesterday: heat and work in variational EM.
Today: the variational free energy, and why the ELBO is one.
Happy to share EgoWAM, led by @BaoyuLi6!
Naive co-training on in-the-wild human data hurts policy performance. The reason is simple: humans move very differently from robots.
EgoWAM bridges this gap through world representations: humans and robots may move differently, but achieve similar effects. We find that motion-centric representations and strong visual pretraining solve different parts of the embodiment gap.
Our @UnitreeRobotics G1 now knows what its arms can reach before it ever moves
We sampled 2 million poses per arm and built a reachability map and the planning setup went from ~100 ms to ~62 microseconds. 4.9% false positives, verified against IK
Fully open source
Auxiliary prediction targets, such as DINO feature and 3D flows, enforce the policy to learn better task semantics, and transfer better in the real world.
Great work by my former labmates @BaoyuLi6 and @XinchenYinYXC !
This is a massive year-long effort made possible by academic partners from Georgia Tech, Stanford, UC San Diego, ETH Zurich, and industry partners from @MeckaAI, @scale_AI@ScaleAILabs , and @RealityLabs@meta_aria .
Website: https://t.co/sltdIhoUEI
Code & Data: https://t.co/0OG5SFkT2S
Data Viewer / App: https://t.co/bK9c7CJfPn
EgoVerse data is curated for robot learning, with
- Large-FoV egocentric videos
- Accurate hand and camera tracking
- Dense natural language annotations