SFT as a training technique is generally very underrated. People have gotten this idea that you can’t SFT without catastrophically degrading a model. This is completely untrue. Done well, you can often come close to RL performance with an order of magnitude less data (1/n)
@Damianships@zebassembly They routed some people's GPT-6-Astra to a GPT-4o like model. Those pictures are my Codex GPT-6-Astra's great paintings, not even close to real GPT-6-Astra.
Same task, different use:
roll it out → training data;
train against it interactively → RL task;
freeze it for evaluation → benchmark.
The difference is mostly in how the task is packaged and used. (2/2)
With enough environments + LLM compute, a good Task Generator—or a human acting as one xD—can keep producing training data, RL tasks, and non-creative benchmarks, mostly bounded by the underlying raw data.
The key idea: they’re all fundamentally tasks. (1/2)