On the zeroth order lm training stuff:
It's cool that people try stuff instead of only reading and dooming. But i feel the need to curb the enthusiasm on this one a little. Two things:
(1) methods in the genetic-algo and fwd-diff categories, which imo these fall in, is established that they are fine for smaller parameter counts but fall off hard as parameter space increases. It's not necessarily raw param count that matters here but "intrinsic" size. This is the whole point of backprop: few outputs many params.
(2) loss 4.2 for LM is very high even for a 30M parameter model. Go a bit higher and you are at learning token unigram distribution. This means for this score not much interesting/difficult learning has to happen yet.
Again it's fine to rediscover things like this from first principles. In fact it's the very best way to learn and i recommend it; i spent a lot of time learning like this myself! But now with social media hype amplification and crazy vcs and journalist attention that our field gets, i do feel the need to be a bit more cautious with communication and there to be "old pessimist" voice like mine in the room to.
(But unlike some other old pessimist voices, if i turn out to be wrong I'll happily admit it and cheer on =)
Launching a new humanoid learning course at @CMU_Robotics in Spring 2027!
Every team will get its own (small) humanoid robot for hands-on labs.
Will open-source everything (lecture slides, codes). Stay tuned!
My brother made a fun website: https://t.co/kWSoI21aQJ
Enter your job and it tells you when you'll be deprecated, and exactly how. For me it's... pretty soon and pretty accurate!
He's a very creative person (artist turned frontend dev to pay bills, now obv worried by AI) and this is full of little nuggets and easter eggs. It's an eerie mix of fun and jokes and real worries. Check it out!
This is one of my favorite projects Iโve been a part of this year. Itโs ostensibly about robotics, but I feel that I learned something very fundamental about how AI models should represent all modalities that (superficially!) seem like they are best suited to being represented in continuous-valued (e.g. float) vectors. TL;DR discrete parametrizations are awesome, learn geometry from data, but arenโt constrained by it, and can perform better as a result. Exactly what the bitter lesson tells us we should look for.
Interaction with the real world is the major bottleneck in robot learning. So what would robot RL look like if we didnโt need to limit compute per interaction? Our latest work, Off-Policy Generative Policy Optimization (OGPO, accepted to ICML26) embarks on answering this question (spoiler alert: when done correctly, it helps massively!).
๐งต(1/N)
As a PhD student, I was told not to work on deep RL - too full of hacks and alchemy" But after a year or two of working in this area, Iโve come to (deeply?) appreciate all of the thoughtful research thatโs gone into understanding what/why things work and how to make them better.
My lab (joint with @abhishekunique7) took what weโve learned by reading this body of literature to answer the question: what are the actual best-practices for finetuning a diffusion/flow/generative robot policy (for now, in sim)? Under one set of constraints - ample compute but limited time on your robot - @servo97 paper gives a pretty compelling answer.
Diffusion (or flow) makes for excellent policies, but training them with RL is notoriously hard: BPTT is unstable, RL over diffusion blows up the horizon. In our new paper, we show how we can optimize flow matching actors by using "one weird trick" -- "approximate" the Jacobian of the flow denoising process with the identity matrix. ๐
๐ HPC-AI is heading to #CVPR2026! Find us at Booth 237 from June 3โ7! Whether you're building AI Agents or hunting for top-tier compute, weโve got you covered.
๐ The Ultimate CVPR Swag & Perks:
1๏ธโฃ Limited Swag: Grab custom caps, canvas bags ๐งข ๐๏ธ
2๏ธโฃ Top-up Reward: ๐ฐ$15 = GPU Hour + Model APIs + Summer mini-fans / Bluetooth speaker (Only 20 set)๐ฅ
3๏ธโฃ Accepted Author Special: Top up $40 Get $20 voucher ๐ฐ
4๏ธโฃ Survey Gift: Just do a quick survey, choose your refrigerator magnets ๐งฒ
Stop by, talk tech, and claim your rewards! ๐ค
๐ Booth 237 | June 3โ7 | Colorado Convention Center
#CVPR2026 #HPCAI #AI #MachineLearning #GPU #ModelAPIs