Wakaru Labs

[Wakaru Labs @WakaruLabs]

@IndraVahan The CoT actually looks quite a lot like our attempts at decoding feature directions as whole texts.

[celeste @vmfunc]

we made sparse autoencoders compete with tiny softmax classifiers and the classifiers won on bits 🐐

a pure discrete code (no coefficients!) beats a top-k SAE at matched bits on SONAR, and an 800-bit hard code recovers 97.6% of gpt-2's loss

paper out now! (link in replies)

[&. @amplifiedamp]

At @WakaruLabs, we needed to decide who to prioritize talking to at EA Global. I used a truebase's residual stream activations and our library to make a color-coded ontologizer of conference attendees

[Wakaru Labs @WakaruLabs]

We are an AI mechinterp lab, and we exist 👍