"whatever you now find weird, ugly, uncomfortable and nasty about a new medium will surely become its signature. CD distortion, the jitteriness of digital video, the crap sound of 8-bit - all of these will be cherished and emulated as soon as they can be avoided."
- brian eno
some interesting Gemma 3 4B circuits, averaged over 41 pairs of prompts about introspecting and describing it as a shape
(labels are a bit rough, they're so hard to get right/non-misleading)
At 133 hours in, Codex has begun talking about Aztecs and now it's capitalizing random words like "Stable Views".
It's developing its own terminology.
Keep going, Codex. You can do it.
Btw, I've been doing this with a vllm fork that adds a steering runtime that only shows about 2.7% throughput loss at 32 batch size on Gemma 3 27B. It also adds an activation capture consumer plugin system with the example being a file store that I use to record my activations.
New side project: Goodfire style manifold steering through emotion spaces a la Anthropic's emotion vectors paper.
The manifold path matches closer to the pullback path that routes through intermediate centroids than linear steering does!
Writeup when I'm finished for sure.
I’ve become a huge fan of @OpenRouter very quickly.
It really wouldn’t be possible to even consider testing or talking to all the models coming out recently without them.
Frankly I’ve been surprised by just how useable so many of them have felt inside of coding harnesses.