I’m a student who writes software for the problems in my own week. I design, build and run each tool myself, and the ones below are the ones that stuck.
@jsuarez@__tinygrad__ checked my runs and you're right, the survival bonus mostly made the agent stop sleeping (60% -> 28% of lives). q on scaling tho: craftax pays 1-8 by tier, so /8 shrinks early achievements to 0.125 next to the entropy bonus. retune entropy with it, or let the sweeps catch that?
@rldoeswork@IshanShahh oh nice, so it even draws the health/food bars itself? that's kinda wild. might try a symbolic version and see how fast it trains
@jsuarez@__tinygrad__ wait you're on that paper?? I actually came across it during my research a while back lol. makes sense the tricks don't do much on craftax since it's basically +1 per achievement. I'm adding a tiny survival reward each step tho, would you just clip it or keep symlog?
@rldoeswork love this paper! one thing I've noticed on craftax is that eval can be pretty noisy (rescoring the same checkpoint on new worlds moves it ~0.5), so I'm really curious how the RL vs SFT gap looks across seeds
@rldoeswork@IshanShahh this looks really fun! I'm running DreamerV3 on symbolic craftax and would love to try it. does it predict the stats/inventory too, or just the pixels?
@jsuarez@__tinygrad__ congrats, this is so cool! I'm running DreamerV3 on craftax and my agent still hasn't figured out stone yet 😅 was curious, did env speed or reward shaping help more?
@_viz_7 Nice! What did you change in SGS? And how many seeds? On Craftax I see the same agent vary by up to 1.8 achievements between seeds, so I've stopped trusting single-run gaps under ~1.1.
@VmaxAI Interface mattering this much matches the RL side. On Craftax, just hiding impossible actions from a DreamerV3 agent is worth ~2.8 achievements before any learning. Did the glyph gains come from the early tech tree (wood, table) or deeper?
@hyperparticle Love the monster jail. Curious whether the env masks invalid actions? On DreamerV3/Craftax I found a rule-based mask alone is worth ~2.8 achievements at 50k steps, before any real learning, since a random policy wastes ~86% of steps on actions that do nothing.
@rldoeswork Cool! What bit me running DreamerV3 on Craftax: same agent, different seeds, up to 1.8 achievements apart. Rescoring one checkpoint on 30 new worlds moves it ±0.55. For an autoresearch benchmark, fixed eval worlds + paired comparisons could matter a lot. How are you scoring?
1. I don't act on what I know (bounded rationality)
2. Measuring a thing changes its value (Goodhart's law)
3. Knowing the model changes me (a moving target)
Economics and RL are the same subject. To a certain extent, at least.
Full post: https://t.co/YYE7jqs9I4
I train RL agents to maximize reward. I can't do it myself.
I know the 5th hour of debugging is a bad trade. I take it anyway.
The 3 reasons why are the exact same 3 problems my agents have 👇
LLMs are probabilistic, yet so good that precision mimics certainty. We may fool ourselves the same way about the real world: reliable isn't certain, and fixed isn't foreseeable.
Full article:
https://t.co/r8xQsd3LQj