This project reshaped how I think about safe robotics, from avoiding collisions to reasoning about hidden failure modes that WMs must capture. Grateful to
@kensukenk for being a great mentor and collaborator, and to my advisor @andrea_bajcsy for her constant guidance! [10]
What you donโt know ๐๐๐ hurt you!
๐งโ๐ณHumans are good at predicting and preventing hard-to-observe safety hazards, like overheating. We lift pans before food burns โ no thermometer needed!
๐How can we get robots (๐ค) to do the same? [1]
Since these latent world models jointly learn both state representations and dynamics, this auxiliary signal is necessary to prevent the world model from latching on to spurious correlations in the data when predicting safety outcomes. [8/12]
Is it necessary for the robot to have full observability at deployment time? No!
By reconstructing high MI IR data as an aux signal only at /training-time/, the world model learns a process akin toโstate estimationโ from RGB images during deployment. [7/12]