celeste

computer hacker, alignment @flockcap, part-time independent journalist

opinions my own

🤍 @computerphile

[celeste @vmfunc]

excited to announce a proposal from @flockcap and Wakaru: ontologize

mixture-of-experts interpretability where the features are the parameters.

instead of an SAE's flat bag of directions, you get stacks of contrastive classifiers you can read from weights and poke directly 🧵

[Flock Capital @flockcap]

Our updated research paper about Ontologize, a proposal for MOE-Based interpretability, is now live on our research repository.

Link in Replies

[celeste @vmfunc]

excited to announce a proposal from @flockcap and Wakaru: ontologize

mixture-of-experts interpretability where the features are the parameters.

instead of an SAE's flat bag of directions, you get stacks of contrastive classifiers you can read from weights and poke directly 🧵

[celeste @vmfunc]

@AnotherSpanish 🩶🩶

[celeste @vmfunc]

doing better by the way 🩶

[celeste @vmfunc]

hospital :))

[Flock Capital @flockcap]

the flock capital research department would like to announce that it has been doing interpretability

with Wakaru. paper soon. we also printed the plates ourselves

[celeste @vmfunc]

massive thanks to wakaru for building the thing together and letting us break it

code's MIT. go poke it. paper soon!

[celeste @vmfunc]

excited to announce a proposal from @flockcap and Wakaru: ontologize

mixture-of-experts interpretability where the features are the parameters.

instead of an SAE's flat bag of directions, you get stacks of contrastive classifiers you can read from weights and poke directly 🧵

[celeste @vmfunc]

→ we found what was freezing heads late in training and killed it
→ some things very much did not work (an HSIC bottleneck does not want to be your friend), and those go in the paper too negative results are results

[celeste @vmfunc]

@transkatgirl i can already tell who said that

[celeste @vmfunc]

@iamaheron_ i also got the notification and was surprised

[celeste @vmfunc]

@parafactual hey celeste i applied awhile ago with no answer on partiful could u approve me

[celeste @vmfunc]

@rogesterone do you have signal, i can't message you on here

[celeste @vmfunc]

@justinsunyt ZRH is pretty on par too

[celeste @vmfunc]

@senseterna this this this this!

[celeste @vmfunc]

i think human-level intelligence was always a suspiciously anthropocentric unit of measurement

[celeste @vmfunc]

@sama this is not a safety issue

[celeste @vmfunc]

@pangram @max_spero_ can you help me get in per chance >.< i don't use linkedin