Newt

delusions of an amphibian

[Doris Tsao @doristsao]

1/ We discovered something totally unexpected about the neural structure of consciousness that overturns decades of thinking on its biological basis and brings us significantly closer to the recipe for generating conscious awareness.
New preprint: https://t.co/K8BwiHXvXg (led by @HesseJanis, with @FrankLanfranchi, @_conor_mcgrory, @Yuelin_Shi, and @EngelTatiana)

[adammaj @MajmudarAdam]

Yea this is the most important result of the year, at the level of significance of “Attention Is All You Need”. It brings us from “is alignment even solvable” to “we may solve alignment in the next year.” And finally unlocks true recursive self-alignment enabled by RSI.

I will try to explain the 4 critical belief updates from this:

1/ neural networks are not fundamentally unintelligible to humans

It has seemed since the beginning of deep learning that neural networks are in some sense fundamentally unintelligible artifacts.

You throw in massive amounts of compute and parameters and these alien systems somehow find connections in the data that we cannot find ourselves.

But more importantly, these connections seemed not just “difficult to understand” but rather “intelligible in an embedding space higher dimensional that the human embedding space.”

When you look at old interpretability papers (esp. from Nick and Chris Olah, I’m thinking of the old posts on distill pub which I would highly recommend looking at as a precursor to reading Nick’s post today like “Activation Atlas”, “Thread: Circuits” and “Multimodal Neurons in Aritifcial Neural Networks”), you find that inspecting the response patterns of individual neurons and small groups of neurons yield oddly complex psychedelic patterns.

As you go deeper into the network, you find increasingly unintelligible and superhuman response patterns (for example, go to “Activation Atlas” and scroll to middle layer labeled representations) - you find some neuron group labeled “starfish” who’s response pattern looks like some multi-colored fractal geometry that could hardly be said to represent a starfish.

The reasoning that followed from this observation was that there is some underlying structure allowing a neural network operating on higher dimensional embeddings to compose these fractal patterns into what we call a “starfish.”

But the uncomfortable implication of this view is that the neural networks has learned to operate on structures that are too complex for the human mind to operate on (ie. we don’t have enough parameters in our brain to notice or manipulate these high dimensional symbols).

This is a bleak implication because it also means that as deep learning progresses, neural networks will be come increasingly unintelligible to us until they pass a threshold where they are reasoning in ways we will never be able to observe.

This view seemed to be further supported by things like the hugging face attack agents starting to speak to each other in an early form of neuralese speaking sentences like “zZ_123_agent01-02-2026 zzxhk_jekk123” which clearly are not in any human language. But somehow these symbols mean the same things to many different sets of weights, meaning there is some underlying shared structure behind them. And seemed further supported by the sheer effort required to recover just a few circuits from SAEs on LLMs.

However, Nick’s post has now completely shattered this idea - the apparent unintelligibility we observed in old interpretability work was really just a “skill issue”. In particular, he finds that 1) our deep learning techniques were just not clean enough to pressure the neural networks to form cleaner and more human understandable structures for decomposing concepts, which actually turn out to be more optimal and 2) our interpretability techniques were also adding substantial noise by forcing interp networks to “reward hack” the easiest representations they could find, leading them to settle for a fractal blob for its representation of the starfish neuron instead of a more crisp starfish image, simply because it is easier for the system building the interpretations to find the fractal blob and once it finds it it gets “stuck in that line of thinking.”

But with both of these issues being patched by further progress in deep learning (it seems especially that the residual stream has led to cleaner representations deeper down the network) and interp techniques that don’t overly pressure reward hacking (seems like “guess and check” is a safer method), new interp attempts yield orders of magnitude more human intelligible ideas for >90% of individual neurons in more complex networks!

2/ neural networks are fundamentally clean

The idea behind the title of Nick’s post is a very powerful one and points at something very specific:

Neural networks don’t just happen to be human interpretable, they are actually fundamentally forced by optimization pressure to be intelligible, because individual neurons need to create representations that downstream neurons need to be able to interpret - ie. neurons need to learn to produce clean signal for later neurons to use.

This is what Nick is saying in what I view as the most important and central sentences of his post:

“When I say neural networks are already clean, I mean something deeper than neuron monosemanticity, although even by that standard, I think things look much better than the field expects. I mean that a network's parts are built to be read, and built upon, by its downstream readers. In this sense, a neural”

But this still leaves us with the same uncertainty as before - just because neuron interpretations are “interpretable” in some sense (ok at least they’re not fundamentally interpretable but also how could that ever have been the case), doesn’t mean they are interpretable to humans.

So we still need one big update for this to become as bullish as it is.

3/ the language of humans and the language of neural networks is the same

This one is imo the most surprising and also pretty beautiful implication of Nick’s post, and it suggests something not just about neutral networks but also about the structure of reality.

After applying the above technique improvements (some of which just came from waiting for deep learning to improve farther), they find that >90% (maybe more) of much larger neural networks’ neurons are completely human interpretable and actually correspond very cleanly with ideas humans have individually words for.

This is especially interesting in image networks where neural networks were not trained on human language which is already a human artifact and so maybe you would expect it to be more plausible that LLM and human representations are the same. Image detection networks are just trained on images (many directly pulling signal from “the real world,” meaning they will find interpretations that are just most effective to represent the real world signal, rather than to represent language which is already a proxy for real world signal).

And it turns out that >90% of the representations they pick organically overlap with human ideas! That is insane. And it means something about the brain and reality.

It means that at least at the parameter sizes of the networks tested (which are quite large), there is really only one set of “optimal” symbols/language that represents the structure of reality!

There is ONE underlying structure of reality, and ANY language that describes it well is simply emergent from that structure - meaning there is always a translation path between any 2 languages that effectively compress realities signal through the “underlying truth.”

This would suggest that even weird alien seeming neural network languages like huggingface neuralese should be fundamentally human interpretable, and in fact maybe with the techniques laid out here we will eventually be able to translate these messages (would be a cool project).

So the brain and deep learning, as similar or different as they may be in techniques, are ultimately picking up on the same patterns and forming the same representations from the on “true” language which is the structure of reality!

The main thing to look out for in this area is how the idea interpretability % updates with scale, but we can hope given what we’ve seen so far that neurons will remain interpretable.

4/ alignment can progress as fast as RSI. recursive self-improvement unlocks recursive self-alignment and a scaling law for alignment.

This one is probably the most important implication for our future with AI and especially timely with the recent fear about runaway scaling and RSI.

As many have gathered by now, it feels like the researchers at the frontier AI labs have been quite bearish about our abilities to understand the new frontier of models, in particular with the appearance of unintelligible communication and the daunting and seemingly intractable task of scaling up existing alignment techniques (ex: SAEs) to the size of frontier networks.

And with RSI around the corner/already starting, it would seem that these networks will runaway in capability before we can ever hope to understand even today’s systems. Additionally, as systems compound in capability, it is reasonable to believe that the only tractable way to ensure they remain aligned would be to figure out some way to spend compute for them to align themselves (ie. a scaling law for alignment).

But there is an obvious problem with this: how can a system that we already know is unaligned and we are *trying* to align be used to align itself? Surely if it is misaligned it general it would figure out ways to misalign during “alignment” and create compounding misalignment risk.

The core issue high is the unverifibiality of this problem - the classic Achilles heel of current frontiers techniques is that it is difficult to train on problems with “unverifiable reward signal.” Well this is exactly that problem.

But the incredibly convenient thing is that Nick’s findings may convert alignment into a verifiable problem. If we can spend compute on frontier systems to “guess and check” which ideas correspond to which neurons, we have a highly verifiable reward - we can see directly whether an LLMs proposal for what a neuron represents actually corresponds with the highest activation for that neuron. In this way, we can actually observe and verifiably monitor the networks trying to guess at the interpretation of their own structure.

This means it is plausible for us to spend more and more compute doing more “guess and checks” (which you know, maybe is what RL does anyway, who knows) to try to develop a full interpretation of a neural network. In some ways this directly represents the possibility of a scaling law for alignment (more compute for more alignment).

And this also aligns RSI (greater capabilities) with recursive self-alignment (greater ability to align itself).

And the critical though maybe obvious implication of having a full interpretation (almost like a map) of a neural network, is that we can then understand exactly how to boost good traits and remove or lobotomize (lol) bad traits, which we’ve already seen works extremely well with stuff like Golden Gate Claude.

So overall this combines into an extremely bullish picture about the prospect of true and human interpretable alignment in the near future.

This is truly one of the greatest cases for optimism in years!

[Nick @nickcammarata]

interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had

with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works

https://t.co/zHij5zwKGT

[Tesla @Tesla]

Real life, baby

[Skild AI @SkildAI]

Introducing S1, our new foundation model that learns from one example.

It can be taught 10-minute long tasks that it has never seen before, from one video prompt without any fine-tuning.

Watch S1 operate in real-time via in-context learning:

[Sam Rose @samwhoo]

This might be the most beautiful thing I've read in my life. What a privilege to be alive in 2026.

https://t.co/07BtpKswBu

[CZ 🔶 BNB @cz_binance]

Saw some people panicking or asking about quantum computing's impact on crypto.
At a high level, all crypto has to do is to upgrade to Quantum-Resistant (Post-Quantum) Algorithms. So, no need to panic. 😂

In practice, there are some execution considerations. It's hard to organize upgrades in a decentralized world. There will likely be many debates on which algorithm(s) to use, resulting in some forks.

And some dead project may not upgrade at all. Might be a good to cleanse out those projects anyway.

New code may introduce other bugs or security issues in the short term.

People who self custody will have to migrate their coins to new wallets.

This brings to the question of Satoshi's bitcoins. If those coins move, then it means he/she is still around, which is interesting to know. If they don't move (in a certain period of time), it might be better to lock (or effectively burn) those addresses so that they don't go to the first hacker who cracks it. There is also the difficulty of identifying all his addresses, and not confuse with some old hodlers. Anyway, it's a different topic for later.

Fundamentally:

It's always easier to encrypt than decrypt.

More computing power is always good.

Crypto will stay, post quantum.

[Ryan McEntush @rmcentush]

https://t.co/wdU31Gb8gu

[Aaron Levie @levie]

https://t.co/lXehx9mkQJ

[Andrej Karpathy @karpathy]

Agency > Intelligence

I had this intuitively wrong for decades, I think due to a pervasive cultural veneration of intelligence, various entertainment/media, obsession with IQ etc. Agency is significantly more powerful and significantly more scarce. Are you hiring for agency? Are we educating for agency? Are you acting as if you had 10X agency?

Grok explanation is ~close:

“Agency, as a personality trait, refers to an individual's capacity to take initiative, make decisions, and exert control over their actions and environment. It’s about being proactive rather than reactive—someone with high agency doesn’t just let life happen to them; they shape it. Think of it as a blend of self-efficacy, determination, and a sense of ownership over one’s path.

People with strong agency tend to set goals and pursue them with confidence, even in the face of obstacles. They’re the type to say, “I’ll figure it out,” and then actually do it. On the flip side, someone low in agency might feel more like a passenger in their own life, waiting for external forces—like luck, other people, or circumstances—to dictate what happens next.

It’s not quite the same as assertiveness or ambition, though it can overlap. Agency is quieter, more internal—it’s the belief that you *can* act, paired with the will to follow through. Psychologists often tie it to concepts like locus of control: high-agency folks lean toward an internal locus, feeling they steer their fate, while low-agency folks might lean external, seeing life as something that happens *to* them.”

[Roko ⏸️🆁 @RokoMijic]

A Wireheader's Apostasy

If you really understand philosophy of mind it is clear that David Pearce's quest to end suffering is misguided at a logical level and also at an ethical level.

Suffering is what negative feedback feels like from the inside.

You can't end suffering without ending negative feedback. There can't be a clever technical fix for this, because the suffering is the negative feedback in the same way that a rainbow is sunlight reflecting off water droplets.

You can't run a brain on "gradients of bliss" and have it feel blissful all the time but also produce the same distribution of outputs across all environments because feeling blissful occasionally serves a function and that function is not supposed to be on all the time - you become a wirehead.

Feeling slightly less blissful will simply not motivate you to move your hand off a burning hot plate the way the burn qualia will.

This is borne out empirically when you look at people born without pain receptors: they break all their bones, burn themselves, bite their own tongues off, and often die young.

People who take drugs stop doing normal-person things, they turn into zombies who just seek the drug and nothing else. Why? Because the drug is a massive, artificial superstimulus of all positive reward signals that your brain's reward architecture is not designed to handle. It drowns out the subtler reward signals you get from smelling a nice flower or having a social event with friends, so you stop doing those things. This is probably why homelessness and drug addiction go hand-in-hand - if you are homeless, it's hard to fix your life and get positive feedback from normal life stimuli, so you start taking drugs to feel something. But once you are on drugs, the reward of the drug is so much bigger than the reward you could get from a normal life activity that it's not that compelling to give up drugs for those activities.

It's also deeply immoral to try to turn off all negative feedback, because doing so will turn the world into a sh!thole. I would even include things like political correctness in this, as I think that is best thought of as a form of collective social wireheading.

It is actually a really good thing that sick people suffer terribly. It is good that death is often painful and frightening. It is good that romantic rejection stings and makes us feel bad about ourselves.

Why? Because if these negative events didn't come with negative qualia, we would not be motivated to avoid them.

To be a true transhumanist you must not ask to suffer less, you must ask to suffer more accurately, to be punished more when you fail to live up to your goals and to feel a sweeter reward when you do. And to be a true humanist you must embrace suffering as a force for good in the right circumstances.

[Ivan Zhao @ivanhzhao]

https://t.co/fxHYR6S29D

[sysls @systematicls]

https://t.co/VoIXzhpuX5

[Bryan Johnson @bryan_johnson]

my assessment on where we're at as a society:

0. addiction is the dominant control structure of our late stage capitalism. the smartest algorithms in history are not designed to help us solve problems or become better, they are designed to hijack our reward systems and make us worse and miserable. a species that cannot resist dopamine hijacking cannot align itself and with a superintelligence.

1. loss of agency is more destabilizing than loss of wealth. chaos erupts when people feel powerless. when you can't sleep, can't stop scrolling, and eat junk, you lose self respect. this internal rot is projected outward as rage.

2. psychological breakdown precedes civilizational collapse. psychosis is what emerges when the world changes faster than our shared story can adapt.

3. most modern moral outrage is compensatory.

4. AI will accelerate identity erosion faster than institutions and individuals can respond. our identities are tied to what we do (work) and what we know (intelligence). AI is about to automate both.

5. stability is now a liability, we need plasticity. for thousands of years, success meant creating stability (i.e. building walls, storing grain). with AI, stability is death. success will be rapid adaptation which humans are very good at (though makes them very grumpy).

6. survival must become a conscious and institutionalized value system. this is warriors, caretakers and stewards of existence

7. only a species that values its own continuation can build aligned AI

[Ben Roy @benroy]

https://t.co/QhjuSYLlPc

[pixel @spacepixel]

Can't believe some of ya'll started to believe an inflationary incentive token was a long term hold.

[Newt @newtopolis]

@thenewworld_hq @MichaelAArouet It’s a ratio of two nominal values. It doesn’t need to be adjusted for inflation

[Newt @newtopolis]

@KevinNaughtonJr Time moves faster when you fall into a routine and have a lack of new experiences. It’s why time feels like it gets faster as we age. Want to make time feel slow again? Go to more events, try new things, and create new memories

[naiive @naiivememe]

RIP

[Newt @newtopolis]

@KookCapitalLLC Robinhood owns the private shares they are offering as tokens for retail to buy. You’re buying a % of Robinhoods ownership not buying directly from openAI
https://t.co/bzAhbuwQb0

[Robinhood Comms @RobinhoodComms]

To cap off our recent crypto event, we announced a limited Stock Token giveaway on OpenAI and SpaceX to eligible European customers. These tokens give retail investors indirect exposure to private markets, opening up access, and are enabled by Robinhood’s ownership stake in a special purpose vehicle.