Kyle Hsu

AI @Tesla_Optimus, PhD @StanfordAILab

[Kyle Hsu @kylehkhsu]

@Joe__Black__ yes, please bear with us! doing things right takes time :)

[Kyle Hsu @kylehkhsu]

the team is cooking

[Tesla Optimus @Tesla_Optimus]

Just set a new PR in the lab

[Kyle Hsu @kylehkhsu]

that's my labmate 🥳

insanely impressive capabilities from NO ROBOT DATA

[Tony Zhao @tonyzzhao]

Today, we present a step-change in robotic AI @sundayrobotics.

Introducing ACT-1: A frontier robot foundation model trained on zero robot data.

- Ultra long-horizon tasks
- Zero-shot generalization
- Advanced dexterity

🧵->

[Kyle Hsu @kylehkhsu]

@YunzhuLiYZ @XuanlinLi2 @Jiayuan_Gu @jiajunwu_cs @haosu_twitr @QuanVng @xiao_ted Superb work @kaiwynd @shashuo0104 @YunzhuLiYZ and team!

[Kyle Hsu @kylehkhsu]

Teleop: OFF. AI: ON

[Elon Musk @elonmusk]

Tesla Optimus learning Kung Fu

[Kyle Hsu @kylehkhsu]

@roydanroy Congrats!

[Kyle Hsu @kylehkhsu]

Congrats Chuning and collaborators! Really nice to see work on doing joint generative modelling over all (or at least more) of our available robot data.

On a personal note, I remember pushing for this high-level approach to be articulated in the robotics section of the foundation models paper, so a small pat on the back for past me (and maybe a cookie).

[Chuning Zhu @chuning_zhu]

Scaling imitation learning has been bottlenecked by the need for high-quality robot data, which are expensive to collect. But are we utilizing existing data to the fullest extent? A thread (1/11)

[Kyle Hsu @kylehkhsu]

I was very surprised when (other) Kyle obtained SOTA ImageNet reconstruction using relatively modest resources (8xH100), all credit to him!

[Kyle Hsu @kylehkhsu]

FlowMo is a SOTA image tokenizer based on diffusion autoencoders -- no convolutions, adversarial training, or distillation! Personally, I expect to see similar approaches from frontier labs soon: diffusion is much more scalable than convnet GANs.

[Kyle Sargent @KyleSargentAI]

Modern generative models of images and videos rely on tokenizers. Can we build a state-of-the-art discrete image tokenizer with a diffusion autoencoder? Yes! I’m excited to share FlowMo, with @kylehkhsu, @jcjohnss, @drfeifei, @jiajunwu_cs. A thread 🧵:

[Kyle Hsu @kylehkhsu]

Check out our work on LLM personalization led by Anikait and Sheryl!

Personally (hehe), it's really cool to see how ideas in automated few-shot task construction I explored way back before my PhD remain applicable in the era of foundation models.

[Anikait Singh @Anikait_Singh_]

Personalization in LLMs is crucial for meeting diverse user needs, yet collecting real-world preferences at scale remains a significant challenge. Introducing FSPO, a simple framework leveraging synthetic preference data to adapt new users with meta-learning for open-ended QA! 🧵

[Kyle Hsu @kylehkhsu]

@init_malachi Hi, may I ask what domain you tried it on? Feel free to DM.

[Kyle Hsu @kylehkhsu]

Incredibly saddened to hear of @FelixHill84's passing. His work and talk on "Environmental drivers of systematicity and generalization in a situated agent" (https://t.co/uBxU2FNsRZ) directly inspired my first successful PhD project.

Rest in peace, Felix.

[Kyle Hsu @kylehkhsu]

@mtschannen Awesome work, is there a plan to release JetFormer code and checkpoints?

[Sumeet Batra @SumeetBt]

Is it possible to obtain zero-shot generalization of vision-based RL agents without data augmentation, task-centric representations, etc? Yes! by disentangling the latent space and, you might’ve heard about this given the latest news, associative memory i.e. Hopfield models!🧵

[Kyle Hsu @kylehkhsu]

@SumeetBt Super cool! Glad to see work building on QLAE for RL.

[Kyle Hsu @kylehkhsu]

@sunfanyun Learning about 西遊記 and thinking about how the devs riffed off it for this game has been super engaging.

[Kyle Hsu @kylehkhsu]

[n/n] For more details, including implementation, fully-fledged background, derivations, decoded latent intervention visualizations, etc., see

code: https://t.co/DEdz8gjQYe
paper: https://t.co/jZQ9q53Jup

[Kyle Hsu @kylehkhsu]

🚨 NEW state-of-the-art model for unsupervised disentanglement 🚨

[1/n] Tripod melds three complementary methods for disentangled representation learning that each target a separate component of an autoencoder.

[Kyle Hsu @kylehkhsu]

[14/n] Jubayer and I will be presenting this work at ICML this Thursday, 11:30 am to 1 pm local time, Hall C 4-9 #409. Come chat about disentanglement!