Balakumar Sundaralingam

Research @NVIDIA | CUDA + Robotics | Robot Manipulation

[Balakumar Sundaralingam @balakumar_]

cuRobo is now open source under Apache 2.0.
cuRoboV2 adds <1 ms GPU-native TSDF/ESDF semantic mapping, whole-body IK/MPC and trajectory optimization for humanoids, ~50 ms torque-limited planning.
Report: https://t.co/oIbMYrpVPu
Code: https://t.co/YMTjXorqLz

[Bowen Wen @bowenwen_me]

Let's meet and chat at #CVPR!
Come by our poster or schedule a coffee chat🙌

Fri afternoon:
- Fast-FoundationStereo: https://t.co/CUa7hsK0q1
- CARI4D: https://t.co/Mxxfk39Oj9 (w/ @XianghuiXie )

Sat morning:
- Event6D: https://t.co/is9lorXmD5 (w/ Jae-Young)

[Balakumar Sundaralingam @balakumar_]

@enguerandvitro Voxels do not have to be slower. We have a very efficient implementation to go from depth to sparse tsdf voxels (includes averaging across views) and dense esdf in curobov2 here: https://t.co/qfPLKticzc

[Balakumar Sundaralingam @balakumar_]

@s1wase TIL there is a small baseline stereo rgb global shutter

[Balakumar Sundaralingam @balakumar_]

@s4movar @pablovelagomez1 @s4movar we actually built a tsdf mapper in python in curobov2: https://t.co/JIjh8SbRvU

[Balakumar Sundaralingam @balakumar_]

@pablovelagomez1 CuRoboV2 implements a TSDF mapper for manipulation (fixed workspace, 5mm voxels, multiple cameras). We designed it for performance (all ops on GPU) and memory efficiency (fp16). This leads to 10x faster rgbd->esdf while using 8x less memory (page 31).

[Balakumar Sundaralingam @balakumar_]

cuRobo is now open source under Apache 2.0.
cuRoboV2 adds <1 ms GPU-native TSDF/ESDF semantic mapping, whole-body IK/MPC and trajectory optimization for humanoids, ~50 ms torque-limited planning.
Report: https://t.co/oIbMYrpVPu
Code: https://t.co/YMTjXorqLz

[Balakumar Sundaralingam @balakumar_]

@pablovelagomez1 CuRoboV2 implements a TSDF mapper for manipulation (fixed workspace, 5mm voxels, multiple cameras). We designed it for performance (all ops on GPU) and memory efficiency (fp16). This leads to 10x faster rgbd->esdf while using 8x less memory (page 31).

[Balakumar Sundaralingam @balakumar_]

@kevin_zakka Apologies @kevin_zakka, for incorrect mink capabilities.
Revision soon with mink changes:
Collision, CoM, local IK w/ collision: no → yes
Solver: NLLS → QP
Out-of-scope capabilities marked "-" instead of "X"
Text reframed: retargeting coll. behavior due to GMR's integration.

[Balakumar Sundaralingam @balakumar_]

@kevin_zakka We could not find an example in mink that does the human to humanoid link mapping for retargeting. So we had to use GMR's mapping. We use GMR's mapping for all IK solvers (using the same relative weighting across links).
Happy to rerun mink if you have better tuned weights.

[Nathan Ratliff @robot_trainer]

Use both:
https://t.co/BQBBunr9IO
TO is a Newton step on the Bellman equation. Policies and value functions are "memories" of past solutions; TO should be optimizing over them at inference time. Best of both worlds. Some of the strongest RL methods do this.

[Nathan Ratliff @robot_trainer]

New work on vectorizing geometric fabric controllers for RL workflows at scale. DeXtreme: Fabric Guided Policies (FGP).

Policies are hard on hardware. We need low-level controllers at deployment, which means we need them during training.

FGPs increase hardware lifetime, enable quick iteration on training and deploying policies, and allow us to inject useful inductive bias into the system.

[Balakumar Sundaralingam @balakumar_]

Our code for CUDA accelerated motion generation is out! Supercharge your workflows with fast batched robotics modules, including kinematics, collision queries, optimization, and motion planning. #PyTorch #Nvidia #Robots

[NVIDIA AI Developer @NVIDIAAIDev]

🤖 cuRobo, a new #CUDA accelerated motion generation toolkit, can solve complex #robotics problems in milliseconds. ⚡

It includes implementations of kinematics, collision checking, numerical and trajectory optimization, and more.

👀 #NVIDIAResearch code https://t.co/KB64dY6MPM

[Ankur Handa @ankurhandos]

DeXtreme is our new work on scaling sim-to-real for contact-rich manipulation with a vision-based state estimation on a robot hand with the infrastructure we have been developing with Isaac Gym over the past one year.

https://t.co/frMBMw5ws0
https://t.co/sVMwNYtcXB

[Yun-Chun Chen @yunchunchen]

I'm excited to share our #RSS2022 workshop paper: Neural Motion Fields: Encoding Grasp Trajectories as Implicit Value Functions.
w/ @Adithya_Murali_ , @balakumar_ , Wei Yang, @animesh_garg, and Dieter Fox at @nvidia.
Paper: https://t.co/NzHVVN5EqA
Video: https://t.co/eD4egdtcxs

[Ankur Handa @ankurhandos]

Factory: Fast Contact for Robotic Assembly, our recent work, is a set of simulation methods & robot learning tools for contact-rich interactions for robotic assembly. It will be presented at RSS next month.

Paper: https://t.co/WoC8E6bZle
Website: https://t.co/BfBty65hky

[Andreas Orthey @andreas_orthey]

Update on Robotic Conferences due to Coronavirus:

RSS -> virtual conference
WAFR -> postponed by one year (to June 2021)
ICRA -> decision on April 6th (virtual vs. postpone to end of 2020)
IROS -> as planned (Oct 2020)
Humanoids -> as planned (Dez 2020)

[Siddhartha Srinivasa @siddhss5]

I’ve run the Personal Robotics Lab for 15 years now, first @SCSatCMU and now @uwcse. Here’s my personal take on how to build an academic research flywheel. Enjoy!

https://t.co/KvhAnH1ZfC

[Rodney Brooks @rodneyabrooks]

Spent the last two days crisscrossing Mumbai for meetings. All those autonomous miles in Chandler, Arizona are easily going to generalize for AV roll outs here. Yeah, right.

[Sebastian Höfer @hoeferse]

I saw this robotics scientist working today.
Commenting his code.
Writing unit tests.
Testing in simulation first.
Operating the robot with his hand on the e-stop, observing carefully.
Like a psychopath.

[Balakumar Sundaralingam @balakumar_]

Dexterous hand teleop using RGBD cameras, now possible!

[Ankur Handa @ankurhandos]

We are excited to release our work on DexPilot, a markerless, glove-free and vision based teleoperation of dexterous robot hand-arm system

pdf is here https://t.co/qWD8KURCum

link to more videos https://t.co/dFoQGtWaOP

https://t.co/L8g8Mqsp2k

[Balakumar Sundaralingam @balakumar_]

@faraz_r_khan You could also add AR tags/motion capture markers to corners of lidar and base and use that to get the transform.