Jonghae Park

Reinforcement Learning & Robotics | Ph.D. Student @SeoulNatlUni, Visiting Student @Princeton

[Jonghae Park @jonghae717]

No prior data. No rewards. No pretrained models.

Can an RL agent learn to solve extremely long-horizon tasks (e.g., 5-cube stacking) from scratch?

We show it can! C-Slerp learns hyperspherical contrastive representations and infers waypoints from them in closed form 🧵👇

[Alireza Modirshanechi @modirshanechi]

Super happy to announce that we have got an Oral at #NeurIPS! 🥳

Together with amazing @ben_eysenbach, Peter Dayan, & @cpilab, we proposed a notion of controllability to derive bounds linking unsupervised skill learning to downstream goal-reaching.

https://t.co/B5BiCwGcLl
1/8

[Alexander Swerdlow @alexjswerdlow]

In language models, turning a generative model into a critic is simple: pass in the output and prompt for a score. Robot policies can't do this - they output physical actions & there's no built-in vocabulary to predict scores.

So when it comes to RL on robot policies, we typically train a separate critic from sparse rewards - blind to everything the policy learned from the actions in the pretraining data.

We think this has it backwards - can a critic learn from those actions the way the policy does? (1/n)

[Ben Eysenbach @ben_eysenbach]

Very excited to share @jonghae717's new paper on goal-conditioned RL!

I've enjoyed working on this over the last 9 months because there are many mental models that can be used for understanding the same (simple) approach 🧵

[Jonghae Park @jonghae717]

Videos, visualizations, and code 👇

🌐 Project page: https://t.co/ijFUnOQFe9
📄 Paper: https://t.co/yJ59wjN9k1
💻 Code: https://t.co/THJ7fGOXDh

Joint work with my amazing collaborators @SungilSon11 , @Jonghun_Shin_ , @Greg3566Lee , H. Jin Kim, and @ben_eysenbach 🙌

[Jonghae Park @jonghae717]

No prior data. No rewards. No pretrained models.

Can an RL agent learn to solve extremely long-horizon tasks (e.g., 5-cube stacking) from scratch?

We show it can! C-Slerp learns hyperspherical contrastive representations and infers waypoints from them in closed form 🧵👇

[Jonghae Park @jonghae717]

Does adaptivity matter? 🤔

Fixing the number of waypoints substantially degrades performance, and adding more alone does not help.

C-Slerp also selects which waypoint to pursue: the farthest one the agent can likely reach now. Fixing that choice also degrades performance.

[Jonghae Park @jonghae717]

Videos, visualizations, and code 👇

🌐 Project page: https://t.co/ijFUnOQFe9
📄 Paper: https://t.co/yJ59wjN9k1
💻 Code: https://t.co/THJ7fGOXDh

Joint work with my amazing collaborators @SungilSon11 , @Jonghun_Shin_ , @Greg3566Lee , H. Jin Kim, and @ben_eysenbach 🙌

[Jonghae Park @jonghae717]

Does adaptivity matter? 🤔

Fixing the number of waypoints substantially degrades performance, and adding more alone does not help.

C-Slerp also selects which waypoint to pursue: the farthest one the agent can likely reach now. Fixing that choice also degrades performance.

[Jonghae Park @jonghae717]

Videos, visualizations, and code 👇

🌐 Project page: https://t.co/ijFUnOQFe9
📄 Paper: https://t.co/yJ59wjN9k1
💻 Code: https://t.co/THJ7fGOXDh

Joint work with my amazing collaborators @SungilSon11 , @Jonghun_Shin_ , @Greg3566Lee , H. Jin Kim, and @ben_eysenbach 🙌

[Jonghae Park @jonghae717]

We evaluate C-Slerp on 12 tasks: 6 manipulation (BuilderBench) and 6 locomotion (JaxGCRL), up to 8-cube towers and extreme mazes.

C-Slerp achieves the highest success rates on most tasks and is the only method to succeed on every cube-stacking task with a gripper. 📈

[Jonghae Park @jonghae717]

Does adaptivity matter? 🤔

Fixing the number of waypoints substantially degrades performance, and adding more alone does not help.

C-Slerp also selects which waypoint to pursue: the farthest one the agent can likely reach now. Fixing that choice also degrades performance.

[Jonghae Park @jonghae717]

💡 Key idea: we use contrastive representations on a hypersphere as reusable "building blocks" for planning.

Composing them decides how many waypoints each state-goal pair needs; spherical interpolation (SLERP) infers them in closed form.

No high-level policy. No graph search.