SFT gives the model "knowledge," but RL gives it the "skill" to use it. In Continual Learning, how do we get both efficiently?
Introducing PaST: A method to inject pre-learned reasoning skills into SFT models via simple vector arithmetic. 🧠⚡️
Blog: 👉 https://t.co/BDoKBd2HGt
The paper and accompanying artifacts are now released — including 500+ RLVR checkpoints for studying training dynamics and extrapolation! 🥳🥳
📚 Paper: https://t.co/olkSYHFAHb
📝 Blog: https://t.co/H9xWxD6dlZ
💻 Code: https://t.co/0ZF1WBlfAr
🤗 Checkpoints: https://t.co/Uj4OrbpoQl
Can we build true lifelong AI simply by relying on longer context, external docs, and complex harnesses?
In our latest position paper, we argue no. ❌
Future AI demands more than in-context learning (ICL)—it requires continuous in-parameter learning (IPL). 🧵👇
Stop paying the equivariance tax — canonicalize instead!
We propose canonical diffusion: map each molecule to a canonical pose/order, train a plain (non-equivariant) diffusion/flow on the canonical slice, then Haar-randomize at sampling to restore invariance.
CanonFlow reaches SOTA on GEOM-DRUG, with a large edge in few-step generation.
arXiv: 2602.15022
https://t.co/4tYpqjiFSm
We often focus on new ML algorithms, but the objectives we optimize for matter far more. Check out Fahim’s excellent thread on our maximum likelihood approach to reinforcement learning.
@cwolferesearch Looking forward to your blog!
We have also confirmed the critical role of RL in Continual Learning. In our work, we inject RL skills into knowledge-adapted models through vector arithmetic.
Welcome to view our blog post if you are interested!
https://t.co/kuQB5JxTBW
Huge thanks to my amazing collaborators for making this work happen! 🙌@tngpngzh111992@muhanzhang93
More implementation details,
check out the full paper:
📄 https://t.co/9ygJrMHiQk
We’d love to hear your feedback!
SFT gives the model "knowledge," but RL gives it the "skill" to use it. In Continual Learning, how do we get both efficiently?
Introducing PaST: A method to inject pre-learned reasoning skills into SFT models via simple vector arithmetic. 🧠⚡️
Blog: 👉 https://t.co/BDoKBd2HGt
Some Key Takeaways:
1. Skill Transfer might be more important than refining SFT data.
2. RL Skills can be knowledge-agnostic.
3. More training compute, better Skill Vectors.
Humans search adaptively—only when uncertain. Do agents know when to search?
Our evaluation says no!
🤩We introduce AdaSearch, a simple RL framework that disentangles and optimizes problem solving & search decisions using outcome reward only!
🧵[1/n]
Unveiling the "Black Box" of Reasoning Models (o1/R1)! 🧠 New Survey by PKU, Tsinghua & NTU on LRM Mechanisms. We dissect:
1️⃣ Training Dynamics (SFT, RL)
2️⃣ Inference Behaviors
3️⃣ Failures (Overthinking/Faithfulness/Hallucination/Safety)
🌟 https://t.co/KhxtF1NDDo
🔥🔥Check out our new paper: What Affects the Effective Depth of LLMs?
The scaling of LLMs emphasizes increasing depth, yet they fail to fully utilize the layers. We study how effective depth varies with model scale, training type, and task difficulty.
https://t.co/e99UYV6bPP
Compared to the model depth, the actual effective depth is more important. Check out our new paper led by @AheadOFpotato on the factors that affect the effective depth of LLMs! Link: https://t.co/a7mWpsyJpI
🤔Want a principled way to RL your diffusion model?
Check Data-regularized Reinforcement Learning (DDRL)! Post-train @nvidia#Cosmos World Foundation models with a million GPU hours! 🤯
Novel formulation ➡️ Theoretically integrates SFT into RL ➡️ Robust to Reward Hacking 🛑
Details: https://t.co/1A9q8ho2xb
#DDRL #Diffusion #RL #NVIDIA #Cosmos
Multi-Agent Evolve is now fully open-source 🚀
With our codebase, you can pick your favorite LLM checkpoint and let it self-evolve, WITHOUT external supervision
💻Code:
https://t.co/GSetfobMwE
🤗Model Checkpoints:
https://t.co/Bz583Rg1s0
Feedback and contributions are welcome!
I am also willing to meet up with old/new friends and discuss everything about [LLMs/DLMs/RL/Agents/PEFT/Continual Learning/Social Application/…]! Don’t hesitate to reach out!☺️
Will be presenting our paper HD-PiSSA at EMNLP 2025!
We challenge LoRA’s hypothesis that updates can be approximated by low-rank adapters and provide a high-rank alternative.
Nov. 5, 11:00 Low-resource Methods for NLP, A303
Paper: https://t.co/5BYWbulblo
Leveraging the distributed training nature (e.g., DP), we initialize orthogonal adapters on different devices and aggregate their gradient updates, resulting in high expressivity and superior performance.
Code: https://t.co/DVAXDLUlxs