Oops

[Horace He @cHHillee]

Presenting my grand unified theory of ML researcher impact: Your impact is directly proportional to how much pain you cause to infra.

Fundamentally, you can only inflict pain upon infra if your approach actually works. And the better your approach works the more pain infra is forced to endure.

So, to give some examples:
- MoE's add a ton of data-dependent computation => pain (shazeer++)
- GDN/KDA are the most complex architecture I've been forced to care about and a very annoying matrix inversion => pain (sonta++)
- Muon is much more annoying than Adam and causes annoying restrictions on parallelism => pain (keller/jeremy++)
- RL scaling forced many researchers to care about LLM inference and RL infra as a category => pain (tworek++)

Even papers like Attention Is All You Need have lead to significant pain! Before transformers were invented everyone was running small jobs and I never needed to think about kv-caches or 6D parallelism.

[Jack Morris @jxmnop]

today i found out that this one australian guy has been toiling away making incredibly detailed Neural Circuit Diagrams with the vibe of a 1950s issue of Popular Mechanics, but content fit for the 2020s

behold. the Transformer

[Oops @Fox_the_Old]

This is really nice!

[Oops @Fox_the_Old]

Don't forget to mint!

[Oops @Fox_the_Old]

Thanks, SpaceID

[Oops @Fox_the_Old]

I love this

[Oops @Fox_the_Old]

About to mint

[Oops @Fox_the_Old]

Don't forget to mint!

[Oops @Fox_the_Old]

I love this

[Oops @Fox_the_Old]

I love SPACEID!

[Oops @Fox_the_Old]

Spaceid is on fire

[Oops @Fox_the_Old]

Thanks, SpaceID

[Oops @Fox_the_Old]

I'm going to join rn!!

[Oops @Fox_the_Old]

Thanks, SpaceID

[Oops @Fox_the_Old]

This is really nice!

[Oops @Fox_the_Old]

I love SPACEID!

[Oops @Fox_the_Old]

Just minted!

[Oops @Fox_the_Old]

Spaceid is on fire

[Oops @Fox_the_Old]

Don't forget to mint!

[Oops @Fox_the_Old]

I'm going to join rn!!