Max Simchowitz

Assistant Professor @mldcmu. Formerly: Postdoc @MITEECS, PhD @Berkeley_EECS, Math Undergrad @Princeton. New to Twitter.

[Max Simchowitz @max_simchowitz]

Personal Update: I'll be splitting my responsibilities between CMU and Amazon FAR over the next year. I'm convinced we need a unified understanding of how the axes of scale, data, optimization and control interact for robot learning, and am excited for my lab + FAR to push towards uniting the "science of robot learning" with the "science of deep learning" at scale.

Please reach out if you are looking for a full-time position building powerful multimodal AI models for deployment on hardware, and if you are intern looking to tinker and do good science on architectures, optimizers, data, tokenization, VLAs, WAMs and more!

[Jon Barron @jon_barron]

Hi all, I have a new result I'm excited to share. It turns out that *all existing implementations* of softmax and cross-entropy (that's JAX, PyTorch, textbooks, etc) have a subtle bug --- two bugs, actually. A simple rewrite fixes both, and eliminates "slingshots" when grokking:

[Max Simchowitz @max_simchowitz]

Taken together, these results suggest (but do not yet prove!) a broader principle in AI: that discrete representations are really really really powerful, and enable greater policy expressivity. Stay turned for some more ambitious work coming out that will exploit these insights in VLA design and RL ;)

[Max Simchowitz @max_simchowitz]

This is one of my favorite projects I’ve been a part of this year. It’s ostensibly about robotics, but I feel that I learned something very fundamental about how AI models should represent all modalities that (superficially!) seem like they are best suited to being represented in continuous-valued (e.g. float) vectors. TL;DR discrete parametrizations are awesome, learn geometry from data, but aren’t constrained by it, and can perform better as a result. Exactly what the bitter lesson tells us we should look for.

[zeji yi @ZejiYi]

Common belief: Robot policies need continuous action heads; discrete actions are 𝐥𝐞𝐬𝐬 𝐧𝐚𝐭𝐢𝐯𝐞 𝐚𝐧𝐝 𝐥𝐞𝐬𝐬 𝐩𝐫𝐞𝐜𝐢𝐬𝐞.

❗We find policies w/ the simplest discrete representations 𝐦𝐚𝐭𝐜𝐡 𝐨𝐫 𝐛𝐞𝐚𝐭 continuous ones—in sim & real, in single-step & diffusion/flow settings.

Here's why 🧵

[Max Simchowitz @max_simchowitz]

Ok, but aren’t you throwing away geometry with binned representations? No How does the policy know that nearby bins are “similar”?  Discrete policies learn geometry in their embeddings, rather than having them imposed from the continuous-vector representation of actions. This is peak bitter lesson.

[Max Simchowitz @max_simchowitz]

@yuchen010807 super cool work! definitely updated my priors. Thanks @jayden_teoh_ for pointing this out

[Max Simchowitz @max_simchowitz]

There seems to be fear reasoning models will trivialize research on robot learning architectures, algorithms, etc...

I disagree. Innovation comes from constraints, and robotics imposes a ton of then. Our new paper, led by @DashoraNitish , suggests a really promising new direction for meeting the constraints of real robot deployment whilst capturing the "intelligence" of agentic reasoning models. Check it out!

[Nitish Dashora @DashoraNitish]

There’s lots of buzz around agentic harnesses for robots, particularly for long horizon tasks that require complex reasoning and memory. But what will it really take to turn a reasoning agent into a reactive, reliable and low-latency robot policy?

In our new paper, workspace models, we design a new memory architecture that acts as a latent harness for stronger reasoning models. Here’s why we think this might be the way forward:

(1/9)

[Max Simchowitz @max_simchowitz]

@Shalev_lif well done!! v timely.

[Max Simchowitz @max_simchowitz]

@ana_dodik thanks ill read it!!

[Max Simchowitz @max_simchowitz]

everyone talks about AI model alignment, but what about AI oligopoly alignment?

Suppose a super powerful A̵I̵ ̵m̵o̵d̵e̵l̵ AI company thought it was acting in the public interest but was actually undermining it. How would we stop i̵t̵ them?

[Max Simchowitz @max_simchowitz]

@andrewgwils why?

[Max Simchowitz @max_simchowitz]

@A_niche_ shoot me an email!

[Max Simchowitz @max_simchowitz]

Personal Update: I'll be splitting my responsibilities between CMU and Amazon FAR over the next year. I'm convinced we need a unified understanding of how the axes of scale, data, optimization and control interact for robot learning, and am excited for my lab + FAR to push towards uniting the "science of robot learning" with the "science of deep learning" at scale.

Please reach out if you are looking for a full-time position building powerful multimodal AI models for deployment on hardware, and if you are intern looking to tinker and do good science on architectures, optimizers, data, tokenization, VLAs, WAMs and more!

[Max Simchowitz @max_simchowitz]

@Kangwook_Lee backprop through the whole model, and find a way to scale more flops (BoN, GRPO, advantage subtraction) per update

[Max Simchowitz @max_simchowitz]

thoughts on robot/generative RL: the best regularization is more expressive updates, not less. What we've learned from the LLM community is that data and model scale lead to the most informative and stable gradients, and we should do the same in robotics.

[Dimitris Papailiopoulos @DimitrisPapail]

A mathematical breakthrough used to effectively be defined as solving a problem that many smart people failed at for N years. Now a subset of those problems fall to LLMs with effectively zero human input and each gets announced as a breakthrough. Fair enough, by the old standard it is one.

But the difficulty by which a breakthrough was defined, was measured against human effort. If a magic wand now solves them, how do we redefine difficulty in terms of the magic wand? A hard problem is one that resists the wand. Which leaves us with the actual question:

What is now considered a breakthrough?