Dashiell Stander

machine learning haruspex

[Dashiell Stander @dashstander]

Excited to share our #ICML2024 paper "Grokking Group Multiplication with Cosets" with @BlancheMinerva, @qinan_yu and @Void13950782! We reverse engineered neural networks that perfectly learned to multiply elements of the symmetric groups S5 & S6. 🧵 on our key findings below

[Sebastian Majstorovic @storytracer]

Common Pile v0.1 is only the beginning. At @AiEleuther we will publish open datasets on a regular basis from now on, using the toolkit we launched together with @mozilla to extract high-quality data from openly licensed content: https://t.co/C8TbLwgAJD. Stay tuned for more!

[Dashiell Stander @dashstander]

@livgorton I would genuinely love to be asked questions like this instead of general leetcode

[Dashiell Stander @dashstander]

@NeelNanda5 Yep!

[Dashiell Stander @dashstander]

@volokuleshov And even more I want the technique to work so they can be used for cool combinatorial optimization things, so I'll definitely be watching for any publications as well

[Dashiell Stander @dashstander]

@volokuleshov Great to hear. I've wanted discrete diffusion LMs to work for a long time and can't wait to play around with some

[Dashiell Stander @dashstander]

@moultano the answer is to unfollow them

[Dashiell Stander @dashstander]

@_onionesque I think some of the work done by the Italian school of algebraic geometry would provide examples. According to an answer to this question, the Severi proved a bound in 1946 that wasn't disproved till 1986
https://t.co/4iQnLJdBvw

[Dashiell Stander @dashstander]

@liambai21 I have high hopes that eventually we'll think of massive ANNs as akin to microscopes or telescopes for peering into massively high dimensional spaces

[Dashiell Stander @dashstander]

@ykilcher Wait, what, how did I miss this?

[Dashiell Stander @dashstander]

@lastpositivist I heard through the grapevine that there's a single dude who has blocked any research supporting any notion of a "historical" King Arthur for a long time (decades maybe?). I have no opinions about King Arthur, but that seems bad to me

[Dashiell Stander @dashstander]

@_onionesque a minor point, but I also always thought that the "Hopfield Networks are All You Need" paper was a really classy way of laying out the connection between transformers and the prior literature
https://t.co/nlL8LIGakK

[Dashiell Stander @dashstander]

@Noahpinion Is there any academic writing responding to his work?

[Dashiell Stander @dashstander]

long-ish term I'd like to expand this to work on all sorts of (finite) groups, hence the ambitious library name. But I'm not sure when exactly that will happen

[Dashiell Stander @dashstander]

I finally got around to fixing up the code from my last paper: a pure-PyTorch FFT over the symmetric group
https://t.co/UY1B5HMYxc

[Dashiell Stander @dashstander]

@typedfemale I spend a lot of time focusing my eyes on something right in front of me -- text on paper or a screen -- so it's honestly quite nice to consume some media in a way that lets me rest my eyes

[Dashiell Stander @dashstander]

@keriwarr @livgorton I've heard that Aluffi is good as an intro to category theory / the categorical perspective of algebra, is that also your impression of it?

[Dashiell Stander @dashstander]

@livgorton This might not be at all what you're looking for, but I've found _Group representations in probability and statistics_ by Persi Diaconis to be amazing showing you how to actually apply algebra to real-world stuff.

https://t.co/b5uLYaDvLN

[Dashiell Stander @dashstander]

@livgorton More like it's not pedagogic. There's no time spent on motivation or exploring typical examples. It is, on the other hand, extremely complete. So it might actually work well paired with a few other books.