han qi

[Heng Yang @hankyang94]

Cannot be more excited about my PhD student @ShuchengK's new work after spending his first year diving into moment and sums of squares relaxations.

Fast and Certifiable Trajectory Optimization

We have seen so many pendulum swing-ups, but I bet you haven't seen this one:

- The nonconvex trajectory optimization is solved to certifiable global optimality by a semidefinite programming (SDP) relaxation with 49,500 variables and 47,351 constraints

- How fast do we solve it? On average 0.66 seconds!

- How did we do it? A fast sGS-ADMM solver directly implemented in CUDA and runs in GPUs + data-driven warmstarts.

The method generalizes to more complicated dynamical systems:
- Cart-pole
- Car back-in (with obstacle avoidance)
- Vehicle landing
- Flying robot

Solving large-scale SDPs far beyond the reach of existing solvers.

Check out project STROM:
https://t.co/RKPAuQj6FY

[Heng Yang @hankyang94]

Robots in our group are finally moving!
Excited to share the first #RobotLearning project from my group led by Han Qi and @hcy1n:

Control-oriented Clustering of Visual Latent Representation

In classical robotics, "perception" and "control" are separated with an explicit state representation.

In end-to-end vision-based control, what's going on in the visual latent representation space?

- Focusing on the planar pushing task, we found an elegant geometric structure in the visual latent space: visual features are clustered according to "control-oriented" goals. Moreover, such clustering has a rigid structure known as "neural collapse" (NC, first found in the image classification literature).

- How does this understanding help? We show that pretraining the vision encoder with control-oriented clustering significantly increases test-time performance in the low-data regime (e.g., 50 or 100 demos). For example, with 100 demos, baseline succeeds 5 out of 10, but NC-pretrained policy succeeds 8 out of 10.

Check out more about the project:
https://t.co/b9UeCSoPmf