Youwang Kim @youwangk - Twitter Profile

Pinned Tweet

7 months ago

𝗘𝗟𝗜𝗧𝗘, 𝗼𝘂𝗿 𝗿𝗲𝗰𝗲𝗻𝘁 𝗽𝗿𝗼𝗷𝗲𝗰𝘁 𝗼𝗻 𝗵𝗶𝗴𝗵-𝗳𝗶𝗱𝗲𝗹𝗶𝘁𝘆 𝟯𝗗 𝗚𝗮𝘂𝘀𝘀𝗶𝗮𝗻 𝗮𝘃𝗮𝘁𝗮𝗿 𝘀𝘆𝗻𝘁𝗵𝗲𝘀𝗶𝘀 𝗵𝗮𝘀 𝗯𝗲𝗲𝗻 𝗮𝗰𝗰𝗲𝗽𝘁𝗲𝗱 𝘁𝗼 𝗖𝗩𝗣𝗥 𝟮𝟬𝟮𝟲! 💡𝗧𝗟;𝗗𝗥 We study a mutually reinforcing synergy of 2D & 3D face priors (generative & data priors) to tackle the longstanding challenge in monocular synthesis of animatable, photorealistic 3D face avatar — Balancing between in-the-wild generalization and efficient synthesis. 𝗘𝗟𝗜𝗧𝗘: 𝗘𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝘁 𝗚𝗮𝘂𝘀𝘀𝗶𝗮𝗻 𝗛𝗲𝗮𝗱 𝗔𝘃𝗮𝘁𝗮𝗿 𝗳𝗿𝗼𝗺 𝗮 𝗠𝗼𝗻𝗼𝗰𝘂𝗹𝗮𝗿 𝗩𝗶𝗱𝗲𝗼 𝘃𝗶𝗮 𝗟𝗲𝗮𝗿𝗻𝗲𝗱 𝗜𝗻𝗶𝘁𝗶𝗮𝗹𝗶𝘇𝗮𝘁𝗶𝗼𝗻 𝗮𝗻𝗱 𝗧𝗘𝘀𝘁-𝘁𝗶𝗺𝗲 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝘃𝗲 𝗔𝗱𝗮𝗽𝘁𝗮𝘁𝗶𝗼𝗻 🔗 𝗣𝗿𝗼𝗷𝗲𝗰𝘁 𝗽𝗮𝗴𝗲: https://t.co/gsbkL5lYeC 📄 𝗣𝗮𝗽𝗲𝗿: https://t.co/sJGGGHRQ7p ELITE was a project that I've worked on with all my heart, and thanks to my collaborators' tremendous efforts and their patience. Huge congrats to my coauthors: Lee Hyoseok, Subin Park, @GerardPonsMoll1 , @Tae_Hyun_Oh Now that we have building blocks for "𝙥𝙝𝙤𝙩𝙤𝙧𝙚𝙖𝙡𝙞𝙨𝙩𝙞𝙘" and "𝙨𝙤𝙘𝙞𝙖𝙡" world simulations. What'll be the next steps? 🧐

1

16

3

8

958

youwangk retweeted

Jitendra MALIK

@JitendraMalikCV

2 days ago

3D/4D reconstruction is one of the great achievements of computer vision. With roots in photogrammetry, multi-view geometry was much studied (e.g. Hartley & Zisserman). In the last decade, it was successfully combined with deep learning. Shape priors which come from recognition got incorporated (e.g. for humans, in HMR) and recent systems like VGGT and SAM 3D can do amazing stuff. This progress is directly relevant for robotics, which (duh) operates in the 3D world. Many robotics papers in the learning era weren't exploiting 3D structure, which IMHO is just wasting valuable signal. But there has been a strong real2sim2real tradition for years e.g. here are two projects from my group (and there are many others, I am not claiming exclusivity). I am glad that people have rediscovered this capability with Astra. But please do give credit to human researchers, not just AI models. https://t.co/OeBSOfO6Sg , https://t.co/7ZIrP8SPU1

8

658

68

423

126K

youwangk retweeted

Keenan Crane

@keenanisalive

3 days ago

For 3D learning, the “Bitter Lesson” doesn’t properly respect Kolmogorov complexity. The colloquial version of the bitter lesson is that, in a nutshell, just scaling up data and compute always beats problem-specific modeling (note that this is not actually what Sutton said; more on that later). But look at the current situation with LLMs and agents generating 3D content: the really impressive examples depend on configuring an API, language, or software package (Three.js, OpenSCAD, Blender, etc.). Each representation impacts the description size for a given object—and for the classes of object that can be faithfully represented via one of these procedural descriptions, the encoding is vastly more efficient than an explicit description (points, splats, polygons, etc.) or even an implicit description (SDF, NeRF, etc.). Moreover, the smaller you can make the procedural description, the better the LLM works: it has to predict fewer tokens to describe the same object, because each token does a better job of compressing geometric information. E.g., the words “unit sphere” are far more compact than a list of points or a collection of voxels. So, in the procedural setting, building models for predicting shape becomes an exercise in bounding Kolmogorov complexity: what’s the smallest “program” I can use to describe a given object? (Or more precisely: which language minimizes description length *and* the entropy of the description, for a model trained on that language.) That’s why current tools lean on carefully crafted abstractions (again, Blender, OpenSCAD, Three.js, etc.), for which many examples are available. Astra doesn’t run off and build you a giant neural network, hoping that the “right representation” will somehow be learned deep within the network weights. None of this, by the way, contradicts what Rich Sutton actually said: the input/output encoding matters, even if the rest is just scaling up search and learning. And the reason we see examples in Three.js and not a more esoteric language is that there is a large corpus of examples to learn from. But the idea that the network will just “learn the best representation” by adjusting its weights is a misguided fiction. Instead, it really seems like we are entering an era of “code as geometry” counterposed to WorldLabs’ “geometry as code”, where design of domain-specific languages is a key task. (And of course, search and learning will inevitably play a role in this design…) Computer Graphics also plays a central role, because it’s the mechanism for decompressing the procedural description into an explicit representation (building a cylinder, tessellating a curve, etc.). That’s the tool calling part of the agentic process. And the better these graphics algorithms get, the more effective the language becomes. People are worried about the death of Computer Vision—but it seems Graphics is coming back from the grave. 🧟‍♂️ (And, might I add, that differential geometry is a very expressive language that few are taking advantage of in this setting. 😉)

18

370

44

266

21K

Youwang Kim

@youwangk

Who to follow

Last Seen Users on Sotwe

Trends for you

Most Popular Users