Meta coming in hot with SAM
Segment Anything Model (SAM) is a promptable segmentation system. It can "cut out" any object, in any image, with a single click.
Masks can be tracked in videos, enable image editing apps, and even be lifted to 3D
🧵Quick tour and test
📢Panoptic Lifting for 3D Scene Understanding with Neural Fields #CVPR23 highlight!
Given only posed RGB images of a scene, we optimize a panoptic radiance field representing color, depth, semantics, and instances at any point in space.
Vid: https://t.co/vFpb02zsZQ
@yawarnihal
"near real-time method for 6-DoF tracking of an unknown object from a monocular RGBD video sequence [...] a Neural Object Field that is learned concurrently with a pose graph optimization process in order to robustly accumulate information into a consistent 3D representation"
#CVPR2023 paper by R. A. Rosu and S. Behnke: "PermutoSDF: Fast Multi-View Reconstruction with Implicit Surfaces using Permutohedral Lattices"
https://t.co/27hQJeUdWr