@theanakin87 But watch out: "Overtraining SFT or training on shorter/simpler examples boosts SFT scores fastest but often produces the worst RL learners"
https://t.co/46cB9IsCxq
HT @maximelabonne from LI https://t.co/yCN8WRwSQe
PRd a small fix to Prime-RL: https://t.co/0ciyj5TTEM
Training unaffected, but if anyone is manually using solve_all/all or reward/all/mean as stopping signal or to make decisions about hyperparameters mid-run those decisions could be wrong
@samsja19@jannik_stra@johannes_hage
@ypatil125 How quickly do you think this problem compounds wrt context window length? DS r1 was only 128k, but 1M+ becoming table stakes (DS v4 for example) cc @lindensli
@msfeldstein@MountainsGuy1 “Likely telemarketer” ≠ “flagged as spam”. You want the silence unknown callers option checked. That’s what actually hides them. Still shows the missed call notification but 🤷
@alexgshaw@cl571128 Not “Cheating” (implies malice intent).
+1 requiring auditable trajectories, but are trajectories being audited, inc when submitted by staff?
@togethercompute’s DSGym has read-only volumes. Reduced attack surface + faster rollouts https://t.co/8ULjaK7GMn @Ameen_ml your team?
My second fav example is @cursor_ai ‘s Cloud Agents setting stage for RL for Composer 2 rollout infra
“When a pod fork is requested, we attempt to first schedule the fork onto the same node” 👌
@vmg@ellev3n11@srush_nlp@EvanHub@aditjain1980
Whoever at @AnthropicAI‘s idea for forked subagents was, what a nice convergence of UX, technical efficiency (no compression loss), and likely Stealth RL Hack: every forked agent is a potential super rich env/task pair way deep in the weeds in something that has to be figured out
Anthropic just introduced forked subagents in their latest update.
Unlike regular subagents, forked subagents can inherit the same context as the main agent. This looks convenient for cases where richer context matters more.
This is just what I needed!
@stefanopopoulos@aditjain1980@vmg@ellev3n11 can you say whether the training environment was production-with-limits or simulated-well-enough? I’m about to start building simulated mcps into procedural env gen at @InvTechInc and am curious if anyone else is going the mock-mcp route, or how far that’ll get me
@stefanopopoulos@aditjain1980 Or similar enough to it.
That was the thing in the Composer technical paper that blew me away. It was pitched as a “simulated” backend but sounds like maybe it’s easier to limit the agents auth and have it hit production itself than build a faithful simulation of it
"Hardening" that we think of as the realm of cyber security is actually one way to prevent whole classes of reward hacking, and to run faster and more cost-efficient rollouts https://t.co/RgiAc61VjH
@heatherandlace_ No longer true. https://t.co/oFfhG1Y2Gw filters noisy job-search emails using ai so you see interviews, and nothing else. https://t.co/QoglbgTMnh