How to inject new knowledge into LLMs without catastrophic forgetting? Why do SFT and DPO hit a wall?
Thrilled to announce our new work: ✂️Surgical Post-Training!
1/4
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: https://t.co/bOWCcFhCVq
Existing AutoResearch tools lack depth. So I built my own. 📚
It iteratively traces "Related Work" sections, snowballing references to fill gaps. It does lit reviews exactly the way researchers actually work.
Skill is open-sourced here:
https://t.co/fDt9wRPueS
Stars welcome! ⭐️
Three days ago I left autoresearch tuning nanochat for ~2 days on depth=12 model. It found ~20 changes that improved the validation loss. I tested these changes yesterday and all of them were additive and transferred to larger (depth=24) models. Stacking up all of these changes, today I measured that the leaderboard's "Time to GPT-2" drops from 2.02 hours to 1.80 hours (~11% improvement), this will be the new leaderboard entry. So yes, these are real improvements and they make an actual difference. I am mildly surprised that my very first naive attempt already worked this well on top of what I thought was already a fairly manually well-tuned project.
This is a first for me because I am very used to doing the iterative optimization of neural network training manually. You come up with ideas, you implement them, you check if they work (better validation loss), you come up with new ideas based on that, you read some papers for inspiration, etc etc. This is the bread and butter of what I do daily for 2 decades. Seeing the agent do this entire workflow end-to-end and all by itself as it worked through approx. 700 changes autonomously is wild. It really looked at the sequence of results of experiments and used that to plan the next ones. It's not novel, ground-breaking "research" (yet), but all the adjustments are "real", I didn't find them manually previously, and they stack up and actually improved nanochat. Among the bigger things e.g.:
- It noticed an oversight that my parameterless QKnorm didn't have a scaler multiplier attached, so my attention was too diffuse. The agent found multipliers to sharpen it, pointing to future work.
- It found that the Value Embeddings really like regularization and I wasn't applying any (oops).
- It found that my banded attention was too conservative (i forgot to tune it).
- It found that AdamW betas were all messed up.
- It tuned the weight decay schedule.
- It tuned the network initialization.
This is on top of all the tuning I've already done over a good amount of time. The exact commit is here, from this "round 1" of autoresearch. I am going to kick off "round 2", and in parallel I am looking at how multiple agents can collaborate to unlock parallelism.
https://t.co/WAz8aIztKT
All LLM frontier labs will do this. It's the final boss battle. It's a lot more complex at scale of course - you don't just have a single train. py file to tune. But doing it is "just engineering" and it's going to work. You spin up a swarm of agents, you have them collaborate to tune smaller models, you promote the most promising ideas to increasingly larger scales, and humans (optionally) contribute on the edges.
And more generally, *any* metric you care about that is reasonably efficient to evaluate (or that has more efficient proxy metrics such as training a smaller network) can be autoresearched by an agent swarm. It's worth thinking about whether your problem falls into this bucket too.
The next step for autoresearch is that it has to be asynchronously massively collaborative for agents (think: SETI@home style). The goal is not to emulate a single PhD student, it's to emulate a research community of them.
Current code synchronously grows a single thread of commits in a particular research direction. But the original repo is more of a seed, from which could sprout commits contributed by agents on all kinds of different research directions or for different compute platforms. Git(Hub) is *almost* but not really suited for this. It has a softly built in assumption of one "master" branch, which temporarily forks off into PRs just to merge back a bit later.
I tried to prototype something super lightweight that could have a flavor of this, e.g. just a Discussion, written by my agent as a summary of its overnight run:
https://t.co/tmZeqyDY1W
Alternatively, a PR has the benefit of exact commits:
https://t.co/CZIbuJIqlk
but you'd never want to actually merge it... You'd just want to "adopt" and accumulate branches of commits. But even in this lightweight way, you could ask your agent to first read the Discussions/PRs using GitHub CLI for inspiration, and after its research is done, contribute a little "paper" of findings back.
I'm not actually exactly sure what this should look like, but it's a big idea that is more general than just the autoresearch repo specifically. Agents can in principle easily juggle and collaborate on thousands of commits across arbitrary branch structures. Existing abstractions will accumulate stress as intelligence, attention and tenacity cease to be bottlenecks.
@paul_cal@allTheYud I have tried this in my previous GAMEBoT paper, and I literally listed all the possible scenarios and optimal actions but that time AI still cannot follow. Things are changing fast.
How to inject new knowledge into LLMs without catastrophic forgetting? Why do SFT and DPO hit a wall?
Thrilled to announce our new work: ✂️Surgical Post-Training!
1/4
3️⃣ Uses "surgical" minimal edits to keep training data aligned with the model's true distribution.
🚀 The Results: SPoT boosts Qwen3-8B’s accuracy by 6.2% with ZERO catastrophic forgetting (IFEval actually improves: 83.0% ➡️ 84.8%!)
3/4
Google's DeepResearch is not as good as I expected.
The literature review feels hit-or-miss and lacks understanding on the technical key insights. Is this a prompting issue on my end?
Curious to hear about others' experiences!
#NLP#LLMs
Visual perception and reasoning in LMMs lag behind their textual capabilities. The "next-state prediction" paradigm might be the missing piece.
By learning the dynamics of the physical world, we might finally bridge the gap between digital intelligence and real-world application
Transforming human knowledge, sensors and actuators from human-first and human-legible to LLM-first and LLM-legible is a beautiful space with so much potential and so much can be done...
One example I'm obsessed with recently - for every textbook pdf/epub, there is a perfect "LLMification" of it intended not for human but for an LLM (though it is a non-trivial transformation that would need human in the loop involvement).
- All of the exposition is extracted into a markdown document, including all latex, styling (bold/italic), tables, lists, etc. All of the figures are extracted as images.
- All worked problems get extracted into SFT examples. Any referenced made to previous figures/tables/etc. are parsed and included.
- All practice problems are extracted into environment examples for RL. The correct answers are located in the answer key and attached. Any additional information is added as "answer key" for a potential LLM judge.
- Synthetic data expansion. For every specific problem, you can create an infinite problem generator, which emits problems of that type. For example, if a problem is "What is the angle between the hour and minute hands at 9am?" , you can imagine generalizing that to any arbitrary time and calculating answers using Python code, and possibly generating synthetic variations of the prompt text.
- All of the data above could be nicely indexed and embedded into a RAG database for later reference, or maybe MCP servers that make it available.
Then just as a (human) student could take a high school physics course, an LLM could take it in the exact same way. This would be a significantly richer source of legible, workable information for an LLM than just something like pdf-to-text (current prevailing practice), which simply asks the LLM to predict the textbook content top to bottom token by token (umm - lame).
As just a quick and crappy example of synthetic variations of the above example, GPT-5 gave me this problem generator (see image), which can now generalize that problem template to many variations:
- When the time is 11:07 a.m., what is the degree measure of the angle between the hands? (Answer: 68)
- Determine the angle in degrees between the clock’s hands at 4:14 a.m.. (Answer: 43)
- What angle do the clock hands form when the time reads 11:47 a.m.? (Answer: 71)
- At 7:02 a.m., what angle separates the hour hand and the minute hand? (Answer: 161)
- At 4:14 a.m., calculate the angle made between the two hands. (Answer: 43)
- What angle is formed by the hands of a clock at 4:45 p.m.? (Answer: 127)
- What is the angle between the hour and minute hands at 8:37 p.m.? (Answer: 36)
(infinite practice problems can be created...)
GPT-5 20:0 Gemini 2.5 Pro
It is really fascinating to see how GPT-5 is able to trap Gemini 2.5 Pro in Checkers game! More game plays and evaluation results can be found on project: https://t.co/i7lyidtdL0.