Jon

[Jon @jonnyhaks]

@Allen_Schmaltz @boazbaraktcs misnagd

[Ely Hahami @ElyHahami]

🚨New Paper: Training Small LLMs to Introspect!🚨

We create two rigorous evals of LLM introspection and:

- we show introspection emerges with scale
- we *train-in* introspection (even on llama 1B!) by SFT'ing on perturbed forward passes, and performance generalizes! 🧵(1/5)

[Ely Hahami @ElyHahami]

IFT generalizes to increasing accuracy on strength detection tasks: detecting magnitude differences in steering vector injections. It was not trained to do this.

So the model may be learning a function of its internal activations 🤔!

Paper: https://t.co/za9gIr7EfT (5/5).

[Sar Haribhakti @sarthakgh]

@BrianNorgard Text is not dead. Yet. Chatbots are counting on it not being dead. I think Chatbots are transitional step to what's next-voice

[Jon @jonnyhaks]

@mwilcox his name is andrew desantis

[Jon @jonnyhaks]

@kyliu99 and @sesfonstein's pantheon really is something, huh

[Jon @jonnyhaks]

@KobeissiLetter bullish @SorellaLabs

[Ely Hahami @ElyHahami]

it is obvious that
every user
will have model weights
specific to them

[Rohan Paul @rohanpaul_ai]

New Harvard paper shows LLMs can sense forced internal "thought" strength, but usually fail to name its source.

Means the researchers artificially push the model’s hidden activations in a chosen direction, and the model can often tell how strong that push was.

But even when the model notices something changed inside, it often cannot correctly say what concept was injected, like “betrayal” or “satellites”.

So the model can go, “something is strongly influencing me right now,” but it struggles to go, “it is specifically the betrayal concept,” unless the prompt is set up in a very particular way.

That is why the paper calls it partial introspection, it can read a simple signal (strength) from its internal state, but it cannot reliably explain what that internal state means.

On Llama 3.1 8B Instruct, it labels injection strength as weak to very strong with 70% accuracy, far above the 25% chance level.

This matters because some safety ideas ask a model whether a harmful internal state is active.

Inside an LLM, each token produces a large list of numbers that acts like its internal state for the next word.

The authors create a concept direction, a pattern in those numbers, by comparing examples against a contrasting set.

They then nudge the internal state at a chosen layer, meaning a single processing step, with different strengths, and ask questions.

With Anthropic’s multi-turn prompt, the model names the injected concept about 20% of the time, and rephrases break it.

That is partial introspection, the model senses the push strength but cannot reliably say what concept it came from.

The practical message is that LLMs can read some internal signals, but their explanations of what those signals mean are not dependable.

----

Paper Link – arxiv. org/abs/2512.12411

Paper Title: "Feeling the Strength but Not the Source: Partial Introspection in LLMs"

[Ely Hahami @ElyHahami]

Paper link: https://t.co/fXaRqkQmwH
This might suggest that LLMs have some notion of their baseline activations and effectively compute functions of their internals during introspection, as Anthropic claimed.

Work done during for @boazbaraktcs's amazing AI Safety Course!

[Ely Hahami @ElyHahami]

New paper! https://t.co/5JNKMno5LD

The problem: LLM weights need to update to new information.

Traditional fine-tuning often fails at "reversal curse" style questions: if model learns "A→B", it struggles with "B→A".

Then, data augmentation is needed.

[Jon @jonnyhaks]

@stoichog23 Top clear pisser in Colorado

[Ely Hahami @ElyHahami]

The Internet Contraceptive. In 2 hours, I built what people are calling protection against the algorithm™

A chrome extension using LLMs to filter out the slop on your feed.

Soon I'll add more features, go multimodal, and improve latency. Comment 'cherri' for the source code.

[Jon @jonnyhaks]

@ElyHahami cherri!

[Ely Hahami @ElyHahami]

Humans learn continually throughout their lifetime, but how can we get AI systems to do so?

Introducing Memory Embedded in Gated LLMs (MEGa), our method for continual learning and injecting novel knowledge directly into LLM *weights*.

Paper link: https://t.co/hj41kUNSHh
🧵

[Jon @jonnyhaks]

@deepfates Jonas Von Essen, two-time world memory champion, can encrypt every memory task to a specific place in his childhood home, allowing him to simply fetch them when needed

[Jon @jonnyhaks]

@johncoogan @HipCityReg are you responsible for this?

[Jon @jonnyhaks]

@0xvanbeethoven @angstromxyz They can jack your equations, but never the swag

[Jon @jonnyhaks]

@jsngr To put something back