Jiachen Zhu

Making robots see better and reason further at @SkildAI. | Past: @meta (FAIR) @NYU_Courant

[Skild AI @SkildAI]

We have acquired Zebra Technologies’ robotics arm (formerly Fetch Robotics).

This is what happens when orchestration meets intelligence -- a major step toward fully autonomous warehouses.

More robots. More environments. One unified brain.

[Skild AI @SkildAI]

Nearly every system today, from energy to chips to food, is bottlenecked by scarce human capital.

We are changing that by building AI-powered industries of the future.

Check out Skild Brain robustly assembling GPU racks, a highly precise task, live at #NvidiaGTC.

[Skild AI @SkildAI]

Robotics is a data problem.

Today, we’re partnering with @ABBRobotics, @Universal_Robot, and @NVIDIARobotics to deploy the Skild Brain across real-world industries from manufacturing to factory lines.

This will help us build the world’s biggest data flywheel for physical AI.

[Yann LeCun @ylecun]

@RichardSSutton 🙏
Every cake needs its cherry 😁

[Boyang Zheng @boyangzheng_]

We've been RAE and Scale-RAE were both trained on TPUs, generously supported by the Google TRC program. As a small return, we wrote a blog post sharing what we learned — why TPUs can be both tricky and great to work with. We hope it’s useful for new onboarders!

Check it out: https://t.co/zdeEH07Ahx

[Deepak Pathak @pathak2206]

@promptChatGpt @ylecun It doesn't contradict @ylecun; rather, our approach is highly aligned with his proposal

[Peter Tong @TongPetersb]

With RAE, visual understanding and generation operate in the same shared representation space.
We show that generative training doesn't hurt understanding, and crucially, this shared space enables the LLM to perform Test-Time Scaling directly in the latent space.

[Peter Tong @TongPetersb]

Huge thanks to my amazing collaborators @boyangzheng_ Austin Wang @tangbingda @ma_nanye @_ellisbrown @jihanyang13 and advisors @rob_fergus @ylecun @sainingxie for making this happen.
We are releasing code, data, and checkpoints:
Website: https://t.co/5TfAn7QsFs
Code: https://t.co/YUveQQzdnR
Data: https://t.co/Bw6HKtW690
Checkpoints: https://t.co/JrGIyNHlFk

[Peter Tong @TongPetersb]

Last October, we introduced Representation Autoencoders (RAE), showing that training diffusion on frozen semantic representations works and outperforms VAEs on ImageNet.
We received many questions: Can this scale to complex settings like T2I? Do the advantages hold?
The answer is YES. 🧵

[Saining Xie @sainingxie]

love this teaser lol (and it is real)

academia boxed us in sooo tightly that we nearly broke, but we clawed our way out and found a whole new universe on the other side😅
thank you to Google for supporting the gpu-poor rebels and pulling us into this ride, helping us build what I believe is one of the best tpu/gcp infrastructure teams outside of google

[Peter Tong @TongPetersb]

We have been training with TPUs in academia for two years now (huge thanks to Google TRC!). Works like Cambrian-1, Cambrian-S, RAE, and Scale-RAE would not have been possible without TPUs.

We wrote a blog post sharing our experiences, optimizations, and lessons learned: https://t.co/vHTQXQyqqJ

We hope this can help more people having a smoother experience working with TPUs, they are very powerful!

[Boyang Zheng @boyangzheng_]

TL;DR: RAE scales to text-to-image and still maintains its convergence advantage over FLUX-VAE across many settings.

Scaling this up in practice was a lot of fun, and it’s genuinely exciting to train a model that uses the exact same visual encoder for both visual generation and understanding. Huge shout-out to @TongPetersb, Austin Wang, and many others for making this possible!

[Saining Xie @sainingxie]

> "rae can’t scale"
> "rae can’t generalize past imagenet"
> "rae can’t do details"

> instead of arguing online
> students put heads down
> try it at real t2i scale
> results come back
> look extremely bullish

> shoutout to peter, boyang, austin
> and everyone who shipped

> code, model, data
> all open-sourced 👇

[Skild AI @SkildAI]

Let robots clean in peace 🧺🤖✨

[Skild AI @SkildAI]

Humans learn by watching. Robots should too.

[Zhuang Liu @liuzhuang1234]

Stronger Normalization-Free Transformers – new paper.

We introduce Derf (Dynamic erf), a simple point-wise layer that lets norm-free Transformers not only work, but actually outperform their normalized counterparts.

[Deepak Pathak @pathak2206]

.@SkildAI + @NVIDIA = 🔥

[NVIDIA Robotics @NVIDIARobotics]

Imagine one brain for all robots. 🤖

@SkildAI is building a universal robot brain trained across arms, humanoids, and quadrupeds, powered by @NVIDIAOmniverse and NVIDIA Isaac.

With foundation models, robots learn faster, cost less, and generalize better.

Learn more. ➡️ https://t.co/Dcp1IWx5A8

[Skild AI @SkildAI]

Proud to partner with NVIDIA and Foxconn to automate @NVIDIA's Houston GPU Factory and drive the next era of American manufacturing!

[Deepak Pathak @pathak2206]

Jensen at #NVIDIAGTC: "The factory is essentially a robot that's orchestrating robots to build things that are robotic." This is just the beginning, exciting times ahead!