celeste

computer hacker, alignment & safety @flockcap, part-time independent journalist

opinions my own

💍 @computerphile

[celeste @vmfunc]

we made sparse autoencoders compete with tiny softmax classifiers and the classifiers won on bits 🐐

a pure discrete code (no coefficients!) beats a top-k SAE at matched bits on SONAR, and an 800-bit hard code recovers 97.6% of gpt-2's loss

paper out now! (link in replies)

[celeste @vmfunc]

if you want the boring version with 77 pages of ablations:

https://t.co/3imMTON7Ga

[celeste @vmfunc]

fun demo! this runs on the ontologizer work we’ve been doing.

sparse autoencoders can’t make this chart. they give you a bag of 16k latents per person. an ontologizer gives every attendee a discrete code word, which is why you get a typology instead of a feature dump.

[&. @amplifiedamp]

At @WakaruLabs, we needed to decide who to prioritize talking to at EA Global. I used a truebase's residual stream activations and our library to make a color-coded ontologizer of conference attendees

[&. @amplifiedamp]

At @WakaruLabs, we needed to decide who to prioritize talking to at EA Global. I used a truebase's residual stream activations and our library to make a color-coded ontologizer of conference attendees

[Michael Tsai @thedataroom]

Temple

She loves Factorio

Crack for autistic systems thinkers and engineers hyperoptimizers

Eventually she figured out in the lategame that the ultimate bottleneck is your computer’s memory and processing power to “hold state” on the entire map and gigabase, and you’d have to optimize for that

Every bottleneck you solve…creates the next bottleneck somewhere else

[celeste @vmfunc]

@TetraspaceWest good luck

[celeste @vmfunc]

@oggyxe it's over

[celeste @vmfunc]

@r_marked oh damn it worked for other items..? i missed out..

[Steve Weis @sweis]

CLAUDE FOUND THE SEEDS
https://t.co/m8N6OPQoCQ

[celeste @vmfunc]

@repligate huh...? watermarking maybe..?

[celeste @vmfunc]

@transkatgirl h i

[celeste @vmfunc]

@vxfemboy seattle!! but i was in LA recently

[celeste @vmfunc]

@kennethnym @vivariumsf i gave you an hug and hour ago dude how did you forget where you were 💀💀💀

[celeste @vmfunc]

lately it feels like we've been moving like "we're so back", as the kids seem to call it

[celeste @vmfunc]

@potterlab "nobody tell zoox" *tweets about it*

[celeste @vmfunc]

happy to announce @flockcap has signed its first contract in AI hardware.

can't say who yet, but we're really excited about the product. more soon!

[celeste @vmfunc]

went side questin' @NotionHQ yesterday :)

[celeste @vmfunc]

another day at the "building the optimizer that tiles the lightcone" factory

[celeste @vmfunc]

@rogesterone illegal here

[celeste @vmfunc]

@nikunj great message and very on point but why did you have to use AI for all of it :(