Shaofei Cai

CS PhD student @ Peking University. Interested in foundation model for decision making and generalist agent in open world.

[AK @_akhaliq]

ROCKET-1

Master Open-World Interaction with Visual-Temporal Context Prompting

[𝚐𝔪𝟾𝚡𝚡𝟾 @gm8xx8]

ROCKET-1: Master Open-World Interaction with Visual-Temporal Context Prompting

paper: https://t.co/fMGQI16TsJ
project page: https://t.co/DwoqYb6Clf

ROCKET-1 enhances interaction in open-world environments by using visual-temporal context prompting. It combines object segmentation with temporal observations to improve decision-making, particularly in tasks that require spatial awareness. This method integrates models for reasoning and object interaction to handle complex, dynamic tasks, as shown in a Minecraft environment.

[Shaofei Cai @ShaofeiCai]

@jeasinema Thanks Xiaojian. We're grateful for the excellent STEVE-1 @SheilaMcIlraith @Shalev_lif, ensuring JARVIS-1 interacts flawlessly with its surroundings. Kudos also to VPT @jeffclune and Minedojo @DrJimFan @guanzhi_wang @YifanJiang17 @yukez for their outstanding foundation models.