leafs falling apart. raptors losing the same way every night. vladdy traded in winter ball. tim hortons stealing my personal info when im just trying to enjoy a farmers wrap in the morning. it’s dark at 5pm. kawhi please.
What is science, really, and why do we do it?
In my new post I discuss the plurality of perspectives on these questions. As we automate the scientific process with AI, it is important to hold onto the deeply human origins of this collective practice!
https://t.co/N1cM32mdbE
Thinking about things like "notary public," "body politic," phrases where English puts the modifier after the noun like a Romance language. There must be some other examples
1/4) We're hiring in my lab, **please share**! 🚀
There are two postdoc opportunities in cog neuro and AI.
This is for a new project supported by a Schmidt Polymath Award. This project aims to build an engaging video game to collect data for neurofoundation models. 🧠🤖
Couldn't be happier to announce my new paper on defining compositional structure — and generating it! I see this project as the culmination of my PhD, as it unifies all my interests: compositionality, compression, structure, and open-endedness. More below! https://t.co/G2T6sOQDGc
Before Thomas moved onto better things, he was first my roomie in 2024/25 – a period where a) I snorted when he first showed me Claude Code and b) a beer or two (or more) were shared. A million things have changed since, but it makes me smile that these two interests have not :)
New blog post - Chasing the Karpathy Peak (and Falling Off Slop Cliff): Confessions from Slopaholics Anonymous
My story of becoming a slopaholic, who’s most at risk, and how to avoid it (or recover)
https://t.co/9RHumEXfMM
Most papers treat the privileged information in OPSD as a fixed parameter and ablate other things around it. When OPSD fails they call the algorithm doomed or a waste of time, without actually touching the privileged information once.
It was pretty clear form our initial work on this that the quality of privileged information is the most important factor to make OPSD work (https://t.co/afnryCexEG).
@zkjzou did the most comprehensive analysis on this figuring out what acc makes OPSD tick and why it may not be doomed.
This is the first work that convinced me OPSD when done right can actually consistently outperform GRPO.
Your neo-lab wants to pre-train a frontier model, but no provider will sell you enough co-located chips?
We got you.
Our new work shows that SOAP & SPlus achieve strong worker scaling to 96+ DiLoCo workers (e.g., 96 providers) by whitening the update’s covariance. 🧵1/9
Mila's annual supervision request process is now open to receive MSc and PhD applications for Fall 2027 admission! For more information, visit https://t.co/r01eLcXtZw
Today, we introduce Priming Models, a new paradigm for training one-step generative models from scratch.
No diffusion teacher. No flow map. No CFG. No JVP.
We believe this work will challenge how we think about building one-step models.
Joint work with @sitanch, @du_yilun, @parkwbin03, Bohyung Han.
🔥 1.19 FID one-step on ImageNet 256×256 while using substantially less compute, e.g., 12X less than Drifting-B, 6X less than Drifting-L
🎨 Scales to text-to-image generation (GenEval 0.82)
🤖 World modeling on RT-1 / Bridge-V2
Priming Model is built on one core message: [Reconstruction pretraining --> Distributional post-training].
@neuralink Really excited to see this progress! We’ve been exploring some of these ideas in our work on neuro-foundation models. Glad to see these models enabling live BCI use.
This new post from @neuralink combines POYO-style spike tokenisation with a Mamba backbone, akin to our POSSM architecture, and explores self-supervised learning for these models, as we did recently with MOJO. Exciting to see these ideas supporting real-time BCI use! 🧵