Really really happy to have worked on this project!
The key takeaway is not model capability in isolation, but how to close the loop between model iteration and real consumer usage in a domain that is both non-verifiable and inherently preference-driven.
In the work we reframe three core questions:
1.What is Role-Play?
We define Role-play as an agent’s capacity to navigate specific coordinates: {World} × {Stories}, conditioned on {User Preferences}.
https://t.co/FTBVHj490v do we evaluate it when there is no ground truth answer?
If correctness is subjective, then optimize for not being wrong.
3. How do we iterate model performance in production?
Online preference learning on denoised user signals, A/B testing for validation and iteration.
If you’re thinking about AI entertainment, or online learning in production usage— would love to discuss more!
Interesting ~ The value of a harness isn’t to preserve more context for the model, but to normalize a large out-of-distribution task into a sequence of smaller, in-distribution ones. This makes a lot of sense from a training perspective: harnesses for specialized domains often converge on similar patterns!
Transformers struggle to generalize to tasks they were not explicitly trained on. Instead, we propose in 2026 that it is the job of the harness to generalize through composition.
We observe a powerful property when training RLMs: for tasks with shared structure that look different, the root model naturally learns the same trajectory, meaning it views the two task trajectories as the same! In other words, the Transformer does not need additional generalization capabilities to transfer capabilities from one task to the other, the harness induces it.
We find that well-designed harnesses form a quotient set over task trajectories, meaning their individual LLM calls can see structurally “similar” tasks as near-identical, token-for-token! Harnesses can effectively generalize for the Transformer during training, without relying on any intrinsic generalization capability from the model.
For example, RLMs can see problems of different lengths as the same: we show that RLMs can train exclusively on short tasks, and fully generalize to similar but unseen tasks 8-32x longer because it produces near identical trajectories for both.
Taking this further, we show that tasks across different domains (e.g. math solutions vs. essay writing) that share a decomposition strategy exhibit the same generalization effect. RLMs can train on the problem of finding which essays belong to the same author and improve performance on finding math problems that share similar solutions.
The full blogpost, experiments, and discussion are in the thread below.
The value of a harness isn’t to preserve more context for the model, but to normalize a large out-of-distribution task into a sequence of smaller, in-distribution ones. This makes a lot of sense from a training perspective: harnesses for specialized domains often converge on similar patterns
Opened Cursor again today and was struck by how similar coding-agent interfaces are becoming. It made me wonder where lasting differentiation will come from. My guess: not just better models or harnesses, but deeper adaptation to how each person works. General features spread quickly; products that truly learn their users may build more durable loyalty—and perhaps pricing power.
Real self-correction isn’t asking a model to “think again.” It’s building a generate–verify–repair loop grounded in independent evidence, structured feedback, and hard stop conditions.
Its effectiveness depends far more on the quality of the verifier and external feedback than on swapping models or adding more agent roles
Real self-correction isn’t asking a model to “think again.” It’s building a generate–verify–repair loop grounded in independent evidence, structured feedback, and hard stop conditions. Its effectiveness depends far more on the quality of the verifier and external feedback than on swapping models or adding more agent roles~
I’ve recently been exploring continual learning for agents. The key question is how to evaluate agent optimizers—are they learning generalizable coding strategies, or merely overfitting to the current benchmark
Today we share the worldview behind our mission.
Human values don't average out. Local knowledge can't be centralized. The good future has many AIs, raised in different places, shaped by the people they serve, disagreeing with each other the way we do.
https://t.co/A14SurOM2K
new post on harness engineering for AI self-improvement: https://t.co/ZYvGfVs61k
It is hard to forecast how much the future of RSI will rely on harnesses. Likely harness engineering will evolve in the direction of self-improvement and enable auto-research, and, in turn, smarter models keeps harnesses simple.
Even when many harness improvement get eventually internalized into core model, the need to specify goals and context will not disappear.
🧵 Deli AutoResearch SKILL is now officially open source! 🎉
https://t.co/V3lwwdyQm8
Alongside it, we’re dropping our 4th survey paper — this time on Self-play.
https://t.co/SEb2qoKCI6
Inspired by AlphaZero, we got a powerful insight: prior knowledge doesn’t always lift the ceiling.
Models can discover more globally optimal solutions just by playing against themselves.
The biggest change in this paper?
For the first time, the AutoResearch Agent autonomously planned GPU experiments — and submitted actual RL runs on the DeepSeek 285B model.
The entire RL pipeline — experiment design, code writing, running, debugging, and conclusion summarization — was 100% automated, with zero human intervention from me.
This was incredibly difficult, but an incredibly important step.
https://t.co/kuZZNux5RH
GRPO is the tool being called by the AutoResearch Agent here.
We see this as the beginning of our Continual Learning research journey. 🚀
As always, this is my personal research project, unaffiliated with any organization. All views are my own.
#AI #ReinforcementLearning #SelfPlay #OpenSource #AutoML #ContinualLearning #DeepSeek
MiniMax M3, Open-Weight, Now On Hugging Face , with only ~428B parameters and ~23B activated parameters
Weights:
https://t.co/g4Ybfa2kWH
MiniMax Sparse Attention:
https://t.co/HcTlWRotG3