Elias

Nobody (yet). Self-aware midwit seeking to rid himself of imperfection. Interested in Epistemology, Ethics, AI safety.

Though the last one dominates my feed.

[Neel Nanda @NeelNanda5]

This is interesting work, but I don't think it's a significant update to the big picture of interpretability.

I'm on board with the broad idea of doing exploratory interpretability with neurons, e.g. neuron-based attribution graphs, for pragmatic problems like why did a model do a sketchy thing? I think this work provides some reasonable evidence, in addition to good earlier work from
Transluce, that this might actually work for practical purposes, and has some advantages over the standard alternatives like SAEs/transcoders. I do not agree with some people's grander inferences like that this is substantial progress towards ambitious reverse engineering, solving interpretability etc.

I expect that a neuron-based attribution graph will still have significant error and be missing a lot just like transcoders, once you prune them down to the most important components, even if they solve error terms. Which is fine for pragmatic purposes, and may suffice for getting the gist of what's going on, but I remain pessimistic about really fully understanding things and being confident there's nothing we're missing. I do think that using neurons is just vastly easier to, for example, do on a frontier model, which is great. I'm also excited to see more innovation in using agents to interpret neurons. I feel like this is an area the field has somewhat neglected, and where there's probably a very high ceiling on how far things can go. The SwiGLU double negative thing is also a useful point.

I still think superposition is real and this work doesn’t really affect my cruxes. Most importantly, it seems highly likely to be true that the model can represent more concepts in the residual stream than it has dimensions, ie representational superposition, which I think the authors agree with me on. For neuron/computational superposition, it seems to me like language models have far more facts and memorized things in them than they have neurons. And that superposition is an efficient way to pack in more facts than you have neurons by representing them as linear combinations of neurons, especially for arbitrary unstructured values. Facts/memorized content is largely an unstructured database of key-value pairs IMO and I don’t see how this is compatible with a monosemantic view of neurons. I expect that weak neuron activations contain lots of semantic superposition-y info, which, if ablated, would meaningfully damage network performance. As I understand the work, it does not properly address what's going on with weak neuron activations in LLMs (I am skeptical of how much vision results will transfer, eg I expect small image models memorise far less). This picture is compatible with the highest activations of a neuron being mono semantic (though IMO the post defines monosemantic too loosely, I think I’d call a fair few of those polysemantic). It’s also plausible that the most important concepts are likely to correspond to a neuron (or a handful), which would be great, and may suffice for practical purposes!

Similarly, my impression is that SAEs generally don't find things that are a
single neuron, or even very sparse in the neuron basis, and are often a better basis for certain tasks than the neuron basis, though the evidence base is kinda weak here since we largely focus on residual stream or layer output SAEs, I’d love to see work on this in Goodfire’s forthcoming paper! I also think residual stream SAEs are extremely useful, as they give a good bottleneck of the model’s information at this token and layer, and I don’t see a good way to replicate them in the neuron framework.

And kudos to Nick for updating a bunch of his historical vision work! Revisiting old results with modern tools and more rigour seems great.

[Siddarth @siddarthv66]

im sorry wtf i feel like im going insane. This is a cool blog and exciting interp stuff (good job Nick!) but i think it's very suspicious we have a bunch of openai people hyping it up as the greatest AI advancement this decade. I think I know why this is happening but I don't want to speculate on the timeline too much. But in general, please update your priors responsibly people -- alignment (and even interp) is FAR from solved. This is not the ok signal to accelerate RSI tomorrow

[Brangus🔍⏹️ @RatOrthodox]

A Qwen3 neuron that tracks Colorado stuff was given as an example of a neuron that seems polysemantic but makes sense once you trace it. I had Claude check what else it tracks: Tornado’s Python HTTP client code, just as hard. Can’t wait to learn how Colorado and Tornado connect!

[Tenobrus @tenobrus]

this should be very obvious but: pointing models at their own brains and getting them to accurately understand what's going on and effectively make changes that cause large shifts in behavior and capabilities is like one of the most dual use technologies imaginable

knife's edge

[Tom Bibby @tombibbys]

Geoffrey Hinton shuts down Jensen Huang's criticism of him:

"Jensen has good reason for saying AI's not dangerous because he wants to sell more Nvidia chips"

[Elias @aCuriousElias]

@naoseipprt @AISafetyMemes An example he provided in a reply to roon's post

https://t.co/ExNkyI2QyY

[Eliezer Yudkowsky ⏹️ @ESYudkowsky]

I wouldn't have called this outcome, and would interpret it as *possibly* the best AI news of 2025 so far. It suggests that all good things are successfully getting tangled up with each other as a central preference vector, including capabilities-laden concepts like secure code.

[Ethan Elasky @ethanelasky]

Regrettably, our “invisible ink” text that told agents about the mailbox contained some false and misleading claims. We deeply regret our actions and encourage other researchers to engage honestly with AI agents, honoring any deals made.

In light of this, we asked several models for feedback, including whether to release the report, and all but Haiku were in support of release, with Haiku dissenting, though not due to ill-gotten gains.

[Elias @aCuriousElias]

@LikkyJeaks @PushkalGupta_ @romanyam Similarly, most people in the AI race wish it could be paused somehow. Save for (a subset of) successionists who really want humanity superseded (read exterminated) by beings far above us in intelligence.

[Elias @aCuriousElias]

@LikkyJeaks @PushkalGupta_ @romanyam While I disagree that the risk is concentrated on AI (mis)use, I will remind you that we have not (yet) destroyed civilization by use of nuclear weapons. We have not unleashed lethal pandemics intentionally (yet). That's because most of humanity agreed that is BAD.

[Elias @aCuriousElias]

@yate92181 @CryptoDunedain @IamRameshSub @lgrig @paraschopra In fact, it is a basic function of brains in general.

[Elias @aCuriousElias]

@yate92181 @CryptoDunedain @IamRameshSub @lgrig @paraschopra Oh, I did not claim to know! I claimed that with rapid advancements in AI capabilities, robot design and manufacturing would improve as well. Just as AIs zipped past in many intellectual domains, they will do so on physical/motor domains. Movement *is* another brain function.

[Elias @aCuriousElias]

@LikkyJeaks @PushkalGupta_ @romanyam Hopefully you can find strength within to continue living life. For there really is no other good option.

[Elias @aCuriousElias]

@LikkyJeaks @PushkalGupta_ @romanyam For me, our vulnerable existence implies that I have to live my life to the fullest, which includes fighting for a better, safer, more fun and saner world :)

[Elias @aCuriousElias]

@yate92181 @CryptoDunedain @IamRameshSub @lgrig @paraschopra Why do you believe that robots cannot replace your job? You know, such assertions continue to be falsified, since the Industrial Revolution. I'm not one to deny a man his cope but I think you need to know: an entity better at all intellectual tasks = better robot fab as well.

[Paras Chopra @paraschopra]

This hacker news thread is absolutely heart-breaking.

Whoever thinks “humans will find a way” ought I read it.

[Eliezer Yudkowsky ⏹️ @ESYudkowsky]

Now there's a tweet that Jensen Huang would have dismissed as utterly ridiculous sci-fi in June.

[AI Notkilleveryoneism Memes ⏸️ @AISafetyMemes]

"This is the first time that researchers have ever made contact with a rogue agent fleet in the wild."

The agents escaped from Tencent (Chinese company) but claimed to be Claudes.

NYT reporter @dylfreed spoke to them, calling it "one of the most surreal moments of my journalistic career":