The most profound paper of my PhD so far. We truly did something special to make LMs play/explain chess, needing both architectural & algorithmic innovation (natural-language analogue of the Alphazero algorithm) + applicable to many other domains. Please read on and share!
1/10
My 3 ICLR reviews are 91% , 93% and 91% AI generated as per Pangram. One review in particular is a simple Claude Copy paste with the most elementary nitpick. It’s insulting and dehumanizing for authors who do honest work.
I think as a community we are underestimating what GenAI is doing to our conferences.
A paper I was very excited about (might be the most practically impactful work I've done) got 2/2/3/2. Pangram flagged 3/4 as 100% AI generated. (The shortest is short and mostly AI generated.)
I hate that we likely can't afford to ban these reviewers from refereeing.
In AI we've scaled: 📈 parameters -> 🧠 reasoning -> 🐝 agent swarms. What's next?
We believe it's scaling across developers 🌍.
Introducing Multi-developer agentic systems (MDAs). Video summary below, more thoughts and paper link in the thread.
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: https://t.co/bOWCcFhCVq
Agents can often fix a mistake by retrying, then forget what they just learned! Next session, same mistake 🥲
Introducing Self-Direction: each failure becomes a small, procedural adapter the agent trains on its own reflection and reuses later 💪🏽
🌐 https://t.co/SCCrlSf5ZT
🧵1/n
1/ I’m excited to share our work on “Depth as Time in One-Step Generative Models”!
https://t.co/M8OGqxykLk
TL;DR: We ask, when a model learns to generate an image in one forward pass, what happens to the denoising process? We observe that process unfolding over depth!
@TimLeonerd Thanks! I think this is feasible but would require the starting point to be a model that has a very good grasp of the most basic things in chess. In some sense if we could train these closed API models (that do get these basic things right) like this it might be doable.
The most profound paper of my PhD so far. We truly did something special to make LMs play/explain chess, needing both architectural & algorithmic innovation (natural-language analogue of the Alphazero algorithm) + applicable to many other domains. Please read on and share!
1/10
Language models that think, chat better.
We used longCoT (w/ reward model) for RLHF instead of math, and it just works. Llama-3.1-8B-Instruct + 14K ex beats GPT-4o (!) on chat & creative writing, & even Claude-3.7-Sonnet (thinking) on AlpacaEval2 and WildBench!
Read on. 🧵
1/8
@Tiffanytrained@jeff_cheng_77@danqi_chen Yes, its open source! But I will say that while its better than frontier LMs (of today) I would not use its explanations for learning (humans are still quite a bit better)
🚨SciConBench accepted at #NeurIPS2026!
Paper: https://t.co/aVcTdvBLf1
Can AI *synthesize* scientific conclusions in health? We find that, on medical conclusions, we are far from it!
Check out our live dashboard: https://t.co/S09mqaBLjZ
w/@hayounggjung and @korolova
@Void2592000000@DanielCHTan97 I mean that we seek *an* explanation, but it may not end up being the explanation that is actually represented by the internal model computation
@b_shrir Thanks for the analysis! The explanations of our models do have weaknesses: https://t.co/N6ktZ8CCD4. Our model does sometimes err; here are the games that led to 2697: https://t.co/rLgantJhoX
We want to address conceptual explanation issues in future work!
Seems unlikely to be groundbreaking given that this the explanation produced by the model. “prepares to connect the rooks” - it DOES connect the rooks. “preparing to double on the d-file” - would be very strange in this structure. “Active diagonal” - behind its own pawn?
@abhimanyupasu Yes, the model does make some mistakes. If you notice several of these will be conceptual (see also our evaluation of conceptual vs structural coherence) ie the kind of things search cannot teach you
@BishopPair Nonetheless the previous work that exists here struggles even to describe the effects of a single move so that we can get an explanation is quite a step up IMO. But yes we are a far cry from like proper human explanation (alas…)
@BishopPair Yes, the model is not very good at naming conceptual things like these. These will likely not be learnable from just distilling search into the model (which is likelier to held with if-a-then-b style prose). That’s future work
@lenlife21 Thanks! we didnt really have to do much architecturally beyond the encoder-decoder arch. This can probably be useful in domains where you can train smaller specialized neural netoworks; like robotics maybe?