Jin Cao

CS PhD @UTAustin. Prev @Stanford, @ZJU_China
representation & generation
Let's digitalize everything

[Jin Cao @xjincaox]

@giohuh_ very interesting

[Jin Cao @xjincaox]

@Zeyu_Shen_yo Wow so Opus5.5 is actually better than Fable?🤔

[Dimitris Papailiopoulos @DimitrisPapail]

I think it’s a rough time to be doing basic research but also extremely important. My advice is to go after crazy hard and/or way out of the box problems and increase your ambition to levels that would seem hubristic a year ago.

[Jin Cao @xjincaox]

@DimitrisPapail thanks, needed to hear this

[Jin Cao @xjincaox]

@ychngji6 Totally agree. Less hand-crafted inductive bias, more scalable learning.

[Jin Cao @xjincaox]

@ronnie8327 Just do it!

[Jin Cao @xjincaox]

CS academia is dead. So where does that leave AI PhDs like me?

Did my ICLR reviewer bidding today. Skimmed a few abstracts, and the methods are the exact same recipe I learned when I got into 3DV two years ago. Swap in a newer video gen base model and boom, new paper 🥲 Makes you wonder how many of the 60k submissions were actually thought up by AI. Meanwhile, World Labs' Atlas has basically solved 4D scenes. Academia is so far behind it's not even funny.

I still remember the day GPT-6 Astra dropped. My feed was flooded with GPT + Blender doing inverse graphics and GPT driving robot arms through manipulation tasks. The results were so good I literally had to sit down. A year ago, I was dead sure LLMs could never have spatial intelligence. A year later, Astra slapped that belief right out of me 💀

There's no doubt Astra was post-trained on tons of 3D and manipulation data, and it's only going to get bigger and faster. To me, that means any domain that can be represented symbolically, with clean benchmarks for RL, is going to get swallowed by LLMs. Next to real LLM intelligence, most academic papers that add a bit of inductive bias and tune their way to SOTA are just roadkill waiting to happen.

Sadly, these papers keep piling up and flooding every conference. It's inertia, plain and simple. We've been chasing SOTA for so long that it's hard to stop overnight. But make no mistake: the paradigms in a lot of areas converged long ago. What used to be research is now pure engineering, and the room for academics to add inductive biases is only going to shrink.

So as a PhD student, I see two paths left. One: if you can't beat them, join them. Clean data, build infra, then go to industry and train foundation models. Two: go back to real science. Stop caring about squeezing out another 0.1% on a benchmark, and start asking why this works and that doesn't, with foundation models themselves as the object of study. As a friend put it: AI research might end up looking more and more like biology, except the organisms are silicon-based 🤣 Beyond these two, it's hard to see anything that won't get eaten by LLMs.

Still, I'm pretty pessimistic. I can feel the value of human knowledge being eroded bit by bit. Everyone will get their own AlphaGo moment. What will academia even look like after this... 😭

[Jin Cao @xjincaox]

@smykx Thanks.

[Jin Cao @xjincaox]

@omarslop Still important for interactive media like games

[Jin Cao @xjincaox]

CS academia is dead. So where does that leave AI PhDs like me?

Did my ICLR reviewer bidding today. Skimmed a few abstracts, and the methods are the exact same recipe I learned when I got into 3DV two years ago. Swap in a newer video gen base model and boom, new paper 🥲 Makes you wonder how many of the 60k submissions were actually thought up by AI. Meanwhile, World Labs' Atlas has basically solved 4D scenes. Academia is so far behind it's not even funny.

I still remember the day GPT-6 Astra dropped. My feed was flooded with GPT + Blender doing inverse graphics and GPT driving robot arms through manipulation tasks. The results were so good I literally had to sit down. A year ago, I was dead sure LLMs could never have spatial intelligence. A year later, Astra slapped that belief right out of me 💀

There's no doubt Astra was post-trained on tons of 3D and manipulation data, and it's only going to get bigger and faster. To me, that means any domain that can be represented symbolically, with clean benchmarks for RL, is going to get swallowed by LLMs. Next to real LLM intelligence, most academic papers that add a bit of inductive bias and tune their way to SOTA are just roadkill waiting to happen.

Sadly, these papers keep piling up and flooding every conference. It's inertia, plain and simple. We've been chasing SOTA for so long that it's hard to stop overnight. But make no mistake: the paradigms in a lot of areas converged long ago. What used to be research is now pure engineering, and the room for academics to add inductive biases is only going to shrink.

So as a PhD student, I see two paths left. One: if you can't beat them, join them. Clean data, build infra, then go to industry and train foundation models. Two: go back to real science. Stop caring about squeezing out another 0.1% on a benchmark, and start asking why this works and that doesn't, with foundation models themselves as the object of study. As a friend put it: AI research might end up looking more and more like biology, except the organisms are silicon-based 🤣 Beyond these two, it's hard to see anything that won't get eaten by LLMs.

Still, I'm pretty pessimistic. I can feel the value of human knowledge being eroded bit by bit. Everyone will get their own AlphaGo moment. What will academia even look like after this... 😭

[Jin Cao @xjincaox]

@ryoroke785 確かに、だからは怖いね。

[Jin Cao @xjincaox]

@senseterna lol ok so the silicon biologists are already here

[Jin Cao @xjincaox]

@acrosson That's the optimistic read. But most of what PhD students do today is the engineering layer. If that gets absorbed, the science left over is more interesting, but it has room for far fewer people.

[Jin Cao @xjincaox]

@GenAI_is_real 太夸张了😆这就是system 1智能的发文章速度

[Hong-Xing (Koven) Yu @Koven_Yu]

🤩 3D world models are cool, but it is cooler to have multi-scale worlds! Introducing WonderZoom: Now you can create a 3D world from a single image, and then infinitely zoom-in into it 🔍, from a city view to the feather details of a bird in the city! 🧵 1/5

[Jin Cao @xjincaox]

@kfountou You're right on both counts. That's what scares me...😇

[Jin Cao @xjincaox]

@YuanLiu41955461 Very cool!!!

[Jin Cao @xjincaox]

@AlexHe00880585 Sad story for academia😭

[Jin Cao @xjincaox]

@tongzhou_mu Yeah.. As a first year cs phd student, I don't know what to do in academia..

[Jin Cao @xjincaox]

Isn't this a bug? My understanding is that the model allows you to move freely in 4D (x, y, z, time). If we hold time fixed, I don't think the shadow should move. I suspect this comes from the training data, where camera motion is often entangled with the motion of the physical camera or drone.