Tao

Building self-driving race cars (A2RL) & soccer-playing humanoids (RoboCup) @KhalifaUni. Ex-C++/Android dev. Vim & Spacemacs user. Corgi dad.

[Tao @TaoBuildsRobots]

1/
I build autonomous race cars and teach soccer-playing humanoid robots to walk.

Researcher at Khalifa University, Abu Dhabi.

Here's what I work on, and what you'll see on this account:

[Tao @TaoBuildsRobots]

@AdhamGhazali 98.9% on a plastic bag is impressive. Is that with the same bag every time, or does it hold up when you swap in a different size or material?

[Tao @TaoBuildsRobots]

@vaibhav514g Coming from software too. Biggest shift for me was that "it works" now means it works on the real hardware, not in sim. Get comfortable debugging things you can't print.

[Tao @TaoBuildsRobots]

@andresvilarino @ViralRushX We spend a surprising amount of time just calibrating our humanoid robots. Curious how often a dental robot needs recalibration, and what happens if the patient moves.

[Tao @TaoBuildsRobots]

@corl_conf Would love to see a legged locomotion / sim-to-real meetup on the list. Anyone already planning one?

[Tao @TaoBuildsRobots]

@agilityrobotics @GXOLogistics 100,000 totes is the number I'd love to see broken down. How many falls or human interventions along the way, and how has that changed since the first month?

[Tao @TaoBuildsRobots]

The first code I got paid for ran on PCs. Then years of Android apps. Now it runs on humanoid robots.

Phones to robots was a much bigger jump than PCs to phones. A phone doesn't fall over when your code is wrong.

[Tao @TaoBuildsRobots]

@chelseabfinn Curious whether this helps on locomotion-style problems too, where rewards are dense and policies are small, or if the gains mostly show up with large pre-trained policies.

[Tao @TaoBuildsRobots]

@stisov16 Pipetting with a hobby arm at a hackathon is a great demo. How are you handling repeatability? Hitting the same well every time seems like the hard part.

[Tao @TaoBuildsRobots]

Question for robotics people: what's your best "worked in sim, fell over on the real robot" story?

Mine: a walking policy that looked great in MuJoCo. On the real robot it managed a few steps, then fell over.

[Tao @TaoBuildsRobots]

@tokufxug Syncing cameras and LiDAR together is the interesting part. My bet for our real race car is that sensor timing breaks first, so I'd love to know if it models latency between sensors, not just what each one sees.

[Tao @TaoBuildsRobots]

@realfxw Ex-Android dev who now trains walking policies here, so this is my two worlds colliding. Curious how much timing jitter they see running the policy on the phone side.

[Tao @TaoBuildsRobots]

@Mascobot Depends on the layer you want to be serious about. We buy our humanoids and put everything into the walking policies. Still feels pretty serious when one falls over.

[Tao @TaoBuildsRobots]

@rdn_nikita Is this a whole-body control model or something higher level? Curious whether teams with smaller humanoids will be able to fine-tune it.

[Tao @TaoBuildsRobots]

@LeRobotHF @microagi @huggingface Millions of hours of human data is great for manipulation. Curious if anyone's using it for humanoid locomotion yet, or if walking still mostly comes from sim.

[Tao @TaoBuildsRobots]

My corgi has better balance than our robot.

Working on it.

[Tao @TaoBuildsRobots]

@dhh Fun bonus for robotics people: 1024 is also a very normal number of parallel environments for RL training. So it's a holiday for the GPUs too.

[Tao @TaoBuildsRobots]

@OpenRoboticsOrg Thanks to the infra team. Most of us pull ROS packages every day without thinking about who keeps those servers patched.

[Tao @TaoBuildsRobots]

@LeRobotHF 600 hours of manipulation data is a lot. Locomotion is the opposite situation: almost no shared real-world data, so most of us still generate ours in sim.

[Tao @TaoBuildsRobots]

@LeRobotHF @Dexmal_AI Is any of the 7 embodiments a humanoid? Curious how far a manipulation-pretrained action expert gets once the base is also moving and balancing.

[Tao @TaoBuildsRobots]

@NVIDIARobotics Multi-physics is the part I'm most curious about. Train a walking policy on one physics backend, test it on another, and see what breaks. Feels like a cheap sim-to-sim check before touching real hardware.