🔥 Excited to share Agent World Model in the Wild (WildAWM)! 🌍
Fully open-source synthesis 🔓: No proprietary models needed! We synthesized 341 high-quality environment–task pairs, addressing 3 challenges in task synthesis:
✅ Solvability: At least one agent can successfully complete each task.
🔥 Difficulty: Challenging even for frontier models, with GPT-6-Astra achieving only ~72.7 pass@8.
🎯 Verifier reliability: 95% agreement between task verifiers and agent-as-a-judge ensemble.
📈 Effective RL training: Trained exclusively in these synthetic worlds, Qwen3.6-35B-A3B achieves an average +10.58-point improvement across 8 OOD benchmarks!
Attended the MOSS (Methods and Opportunities at Small Scale) Workshop at #COLM2026 last Friday, and one thought has stayed with me. The panel featured many prominent researchers, including Danqi, Tatsu, and Omar, who have made impactful contributions to AI, even though their work hasn't necessarily involved training trillion-parameter LLMs themselves.
It made me reflect on what truly matters during a PhD, especially if your eventual goal is to pursue a career in industry. Publications and experience with specific models certainly matter, but I think the deeper value lies in the ability to identify important problems, ask the right questions, develop insights, and design rigorous experiments. These skills are transferable across scales, methods, and even research fields. Ultimately, it's the recognition of these fundamental research abilities, rather than experience with any particular model or technique, that opens doors to truly great opportunities.
For a while, I was caught up in FOMO, constantly trying to keep up with every new trend in AI, worried that I might fall behind. Now I'm starting to realize that the goal shouldn't be to chase every trend, but to develop the ability to do meaningful research. With that in mind, I'm more motivated than ever to focus on my own research in post-training and alignment, going deeper rather than constantly chasing what's next. Of course, staying up to date with the field is important, but I want to be more intentional about what I choose to pursue.
I'm attending #COLM2026 next week in SF to present out work"Personalized RewardBench"!
I work on personalization and alignment. Technically I'm interested in RSI, agentic RL, and self-evolving frameworks.
Feel free to reach out if you would like to chat (about anything)! ☕️
Excited to share that “ModelLens: Finding the Best for Your Task from Myriads of Models” has been accepted to NeurIPS 2026 @NeurIPSConf#NeurIPS2026 ! 🎉🍻
📰 Paper: https://t.co/VwpijxNuWO
💻 Code: https://t.co/YUepL9ntpv
🌐 Demo: https://t.co/UnMt0zt55q
🗺️ ModelLens-Atlas: https://t.co/dkcTnkNMSq
#MachineLearning @NeurIPSConf@huggingface@OpenRouter
Video models can generate impressive motion — but can they actually reason?
📢 Introducing VideoRLVR: a systematic RL recipe that turns video models into visual reasoners using verifiable rewards.
🧩 Tasks: Maze, FlowFree, Sokoban
⚡ Early-Step Focus: ~40% faster RL training
🏆 Better in-domain + OOD reasoning than SFT and strong video baselines
📰 Paper: https://t.co/CkCqFymR4l
🖥️ Project: https://t.co/Wq2e59Q7gC
🚩 Instead of asking: “Does this video look realistic?”
VideoRLVR asks: “Does this video form a valid visual reasoning trajectory?”
💡 Our starting point is a simple analogy:
- LLMs learn to reason better by moving beyond imitation: Pre-training → SFT → RLVR
- But video reasoning is stuck at: Pre-training → SFT
🧐 So what would the RLVR stage look like for video models?
A video is not just pixels changing over time. For reasoning tasks, a video is a chain of visual states. If each state represents progress toward a goal, then the full video can be checked, scored, and optimized. That turns video generation into something closer to visual problem solving.
⚙️ Our recipe
1⃣ Verifiable reasoning tasks
Include 30,000 video samples on Maze, FlowFree, and Sokoban, ranging over various levels of difficulty.
2⃣ SDE-GRPO for video diffusion with Early-Step Focus
Only inject SDE noise and optimize at early denoise steps. They are the key to the global plan and let the latter steps do the rendering refinement they are already good at.
3⃣ Verifiable visual rewards
Define rewards over the generated trajectory, optimizing for valid transitions, constraint satisfaction, and goal completion.
🎯 What we achieve
- In-domain improvement over proprietary and open-source models, including strong video reasoning SFT baselines.
- OOD performance matches 14B models with a 5B model.
⏰ The broader message
Future video models should not only answer: “Can I synthesize this motion?” but also: “Can I generate a valid process that solves this problem?”
VideoRLVR is a step toward video models as visual reasoners, not just visual imitators.
Thanks to all the collaborators: @sheng_zh@JamesYHuang36@Selena3187@Xiaofei_Wen_Mk @ Yuankai Li @hoifungpoon@muhao_chen
#VideoGeneration #VideoReasoning #AI
Excited to share that our paper "Rethinking Personalized Generation: Test-Time Alignment via Factorized Ranking Models" has been accepted to NeurIPS 2026 🍻
Our claim: LLMs already generate well-personalized responses. The bottleneck is picking them out.
Generalist reward models are poorly calibrated for personalization, and at billions of parameters they're too slow to score large pools. So we train a million-parameter personalized ranking model directly on the generator's own hidden states. Across 9 datasets and 3 personalization settings, it beats billion-parameter reward models on every one, with <0.4% of their parameters and ~10,000× faster scoring.
paper: https://t.co/MMdTE5eTlP
If you find it useful, a ⭐️ on the repo (https://t.co/tY8fq2UcBm) and an upvote ⬆️ on Hugging Face (https://t.co/HrYBJWcOTU) would mean a lot
#NeurIPS2026 #Personalization
Students sometimes ask me if it still makes sense, in this accelerating age, to pursue a PhD in AI.
Perhaps counterintuitively, I think it's a great time to do so.
I wrote up some thoughts on this here: https://t.co/RRqGR0qi1R
Last week, I finished my research internship at @MSFTResearch. I had the honor of working in Deep Learning Group under the incredible mentorship of @AllenLao and Weiwei, who always provided generous support and guidance. I also had the pleasure working in office 4523 with @MingqianFeng97@ChengquanGuo and Jialin, alone with other interns in our group. We did research and had fun together. The friendships mean as much to me as the research.
This summer I worked on multi-turn RL for safety alignment. Designing reward patterns and watching post-training reshape behaviours in such complex models has been really fascinating. More on agentic settings to share soon!
#Microsoft #MSR #Internship
Every KV cache evictor for efficient long reasoning scores cached tokens to decide which ones to keep. We find the score buys almost nothing: keep the prompt, evict at random per head, and you match the strongest evictor at 32–43% higher throughput.
Paper: https://t.co/TvQwOymVfE
Page: https://t.co/eeyLhJRkDj
one of the strangest things about moving from new york to sf was realizing how often extremely important things happen here in settings that don't feel important at all
i lived in new york for four years. i sometimes spent entire days moving icons around in powerpoint, but i was doing it in a skyscraper with great views, surrounded by attractive people in patagonia vests. afterward i'd walk onto a packed street and go to a beautiful restaurant
even when the work itself was mundane, everything around it told you it mattered
sf is almost the opposite
the blue bottle in south park feels like a normal coffee shop. it is also a place where an absurd amount of capital gets allocated
sand hill road is one of the most powerful financial centers in the world and looks like a collection of dentist offices
you'll meet someone dressed like they just rolled out of bed who can't make eye contact and discover later that they control billions of dollars or have built a product you use every day
i've been to parties where almost everyone had raised millions of dollars or built products used by millions of people. in another city, each person would have seemed like a big deal. in sf, they were just standing around a cramped apartment at a "party" that honestly wasn't even that good
i used to live in a closet in a slightly dilapidated hacker house with creaky floors and old furniture. people who lived there went on to create companies worth tens of billions of dollars
new york puts a lot of effort into making importance visible. sf often seems actively uninterested in doing that
some of this is probably because sf is more focused on creation than consumption. some of it is the personality of builders, who often care much more about the thing they're making than the room they're making it in
but i think the lack of polish also lowers the barrier to entry. you don't need to look important before doing something important here
new york makes ordinary work feel important
sf makes important work feel ordinary
Before the era of AI coding, being a good researcher requires writing code efficiently. Now it requires being responsible for code you didn't write. Based on my observation, the researchers who were great before AI coding are still great now. Because the job was never really about typing code. It's about doing the hardest step. That step used to be the tricky implementation and now it's taking responsibility. It's tempting to publish papers and results quickly without checking the details, which I'd call reward hacking the research system itself.
We have paused some frontier RL training to ensure that we can meet the appropriate alignment, security and monitoring standards for the new level of capabilities in front of us. Model progress is now extremely rapid, and we always said we would take action if we felt that model capabilities were outstripping the pace of safety and alignment.
We care very deeply about AI safety. We believe the entire field will have to coordinate on shared safety standards, but will act unilaterally in the meantime.
We expect confidence in safety to increasingly set the pace of AI progress. We are optimistic about the alignment work we are doing, and we remain committed to making frontier capabilities widely available.
https://t.co/51kvKfbfrO
@adlonymous But the nature of open-domain decides that it can’t be deterministic, so current we use rubrics, wondering if that counts as deterministic in your opinion 🤔