Language models process each token through a single residual stream (blue arrows): reading / writing memory and applying MLPs at each layer, and finally decoding the next token.
Can we scale up this single stream computation?
Introducing Latent Lattice Language Models (LLLMs): augmenting this vertical stream with a horizontal axis of computation—the Lattice of Latents (orange arrows).
RNN 🤝 Transformers 🧵
Can use just 10 images to teach a model what a sea anemone looks like?
In our #neurips2026 paper, "Dataset Distillation via Drifting," we distill ImageNet's 1,300 images per class down to just 10, optimized from noise.
https://t.co/NBnFsVpkJ8
More info in thread! ⬇️
1/N
New preprint! 🐳🦁🐸
Many animals vocalize to communicate, but when we capture them in the field we get noisy recordings. We create a new, lightweight biodenoising model, CallCleaner, that you can run >90x realtime on a laptop CPU! (1/3)
@kar_sheifer In the paper, we compared the plausibility of our reconstructed videos with ground truth MRI and X-ray data. The main challenge here is that this data is very rare! The scarcity of articulatory data, especially for languages other than English, is a challenge for eval
NeurIPS 2026 Spotlight! 🌟 Physics simulators like MuJoCo MJX are changing robotics. But for studying the sensorimotor learning of speech, there's no equivalent. We built diffTLM, a differentiable vocal tract simulator, to help close that gap. (1/6)
Excited to share LightShop (UIST 2026)! We created an interactive, computational system that enables the creation of high-precision light paintings.
What makes it really neat is that it combines light painting's physically accurate lighting with computational precision.
@pablogimenezz@heybackchannel@vincesitzmann Yes this was one of the original motivations for this project! Voice coaching is so unintuitive since we can't see how our tongue position affects the sound we create
@Mingxuan_Cai_ Thanks! There is certainly still a sim2real gap. One assumption of most vocal tract models, ours included, is that speech is a source-filter process. But in reality, the vocal tract can act as more than just a filter and also affect the vocal source in nonlinear ways.
My friends often ask why I got into speech. Well I think there's a missing perspective in studying language as a control problem.
We now often equate language modeling to (artificial) intelligence, but learning to speak is also an act of embodied intelligence.
NeurIPS 2026 Spotlight! 🌟 Physics simulators like MuJoCo MJX are changing robotics. But for studying the sensorimotor learning of speech, there's no equivalent. We built diffTLM, a differentiable vocal tract simulator, to help close that gap. (1/6)
I hope that this work can make speech a useful model problem for embodied intelligence more broadly. Speech combines aspects of control, reasoning and perception all together.
This work would not have been possible without my collaborators Jin Woo Lee and @vincesitzmann!
I will be at NeurIPS in Sydney this December.
Check out our project page and paper below! (6/6)
Project: https://t.co/IJWmuMVoYP
Paper: https://t.co/kmeWzGvHU7
We also made a WebGPU demo by porting our Python simulation to Javascript with coding agents! Fit a speech recording, watch the vocal tract move, and listen to the reconstruction, all in your browser. Try the example or record your own voice: