Ekdeep Singh Lubana

Member of Technical Staff @GoodfireAI; Previously: Postdoc / PhD at Center for Brain Science, Harvard and University of Michigan

[Ekdeep Singh Lubana @EkdeepL]

Catch my bro and get an insider look into our upcoming work 👀

[Thomas Fel @thomas_fel_]

At CVPR this week for a talk on neural geometry of large vision models. If you’re interested in interpretability or joining @GoodfireAI, come say hi. 🤠

[Naomi Saphra @nsaphra]

My big takeaway from our new work: saturation is the underrated key to learning. Always think about what concepts are saturating, because that’s when you get to learn the next one.

[Tomek Korbak @tomekkorbak]

such a good twitter thread!

[Ekdeep Singh Lubana @EkdeepL]

@tomekkorbak Possibly my biggest learning from this project was that @ChrisGPotts is a master of tweets :p

[Ekdeep Singh Lubana @EkdeepL]

@ericjmichaud_ Thanks! FWIW, the toy task and theory are super simple and arguably a special case of some of the prior works (as we noted in related work discussion). However, the fact that the predictions of this account generalize to large scale models was very exciting to me. 😁

[Ekdeep Singh Lubana @EkdeepL]

Very excited to have this paper out! We show by having more parameters, larger models see reduced interference between updates. This allows them to retain memories of rarely observed samples of a task, eventually allowing them to learn even the tail-end of the distribution. (1/3)

[Christopher Potts @ChrisGPotts]

We take for granted that larger models are better than smaller ones, but why is this so? Our new paper, led by Jing Huang and @EkdeepL, traces this to a data-induced competition for resources (neurons), using formal analysis, idealized tasks, and real pretraining.

[Ekdeep Singh Lubana @EkdeepL]

@solacebellamy @ChrisGPotts FWIW, one could argue such an experiment has happened before. GPT-3 was 175B params and is easily outperformed by models of 7--8B scale, because these models were trained on more and better data.

[Ekdeep Singh Lubana @EkdeepL]

@solacebellamy @ChrisGPotts Your inference is correct, but the suggestion requires us knowing what to train on. We'll have to change the data mixture in a manner that allows us to get, from a standard training pipeline, the performance of a larger model. (1/2)

[Ekdeep Singh Lubana @EkdeepL]

@jiaxinwen22 @ChrisGPotts Had not! Damn, this literature is insane haha.

[Ekdeep Singh Lubana @EkdeepL]

@jiaxinwen22 @ChrisGPotts Definitely! Let me email you.

[Ekdeep Singh Lubana @EkdeepL]

@jiaxinwen22 @ChrisGPotts Hmm, curious to hear more. For context, the above is Olmo pretraining data and the task is just comparing numbers. Comparison for ordered concepts can be expected to be present in general pretraining data, and if you can isolate such data, I expect you'll see curves like this.

[Ekdeep Singh Lubana @EkdeepL]

@jiaxinwen22 @ChrisGPotts Yeah agree with this. We made an intentional choice early on in the project to not bother with shared task structures just yet, since we didn't know what the dynamics without shared structure looked like. However, we fully intend to follows up soon. :)

[Ekdeep Singh Lubana @EkdeepL]

@mirandrom Totally missed this; will read and respond later, but will also add a citation!

[Goodfire @GoodfireAI]

New research from Goodfire and collaborators: why do larger models learn more tasks?

(spoiler: it’s bottlenecked by data)

[Ekdeep Singh Lubana @EkdeepL]

Check out @AndrewLampinen's post and the shoutout to our work: I really loved the emphasis on inability to learn, more than the ability to forget! Gotta ask the right questions. :)

[Christopher Potts @ChrisGPotts]

Speaking of blog posts, our coauthor @AndrewLampinen just did a post that, among other things, relates our results to themes of continual learning and catastrophic interference: https://t.co/YRHEhRJs6d

[Ekdeep Singh Lubana @EkdeepL]

This was such a collab work: Jing + @ChrisGPotts built the analysis recipe, finding our toy setup generalized very well to LMs. @danielwurgaft @rach_it_ @elmelis @nsaphra helped concretize this setup, and the core hypothesis came from chats with @LauraRuis and @AndrewLampinen!

[Ekdeep Singh Lubana @EkdeepL]

We've done a lot of work in the past on understanding how scaling enables learning of new abilities (see https://t.co/CNC5eXLNdk), but this is the first time we attacked param scaling, finding that for a given training process, having more params offers a genuine advantage. (2/3)

[Goodfire @GoodfireAI]

The most popular way to interpret AI is missing the bigger picture.

Models think in curved shapes. But sparse autoencoders (SAEs) work with straight lines.

Can they still capture models’ curved neural geometry? Yes, but not how you might think! (1/7)

[Ekdeep Singh Lubana @EkdeepL]

@thomas_fel_ @sarahwiegreffe @GoodfireAI Look who's talking :)

[Ekdeep Singh Lubana @EkdeepL]

@sarahwiegreffe @GoodfireAI We're gonna try our best to make sure that list keeps growing 😁