Jacob Helwig

[Jacob Helwig @JacobHelwig]

@lateinteraction DeepAgent harness does this https://t.co/2Ycjr1FlkZ:

> Deep Agents preserves all conversation history in its virtual filesystem, allowing for recovery of context post-summarization

[Jacob Helwig @JacobHelwig]

@a1zhang Nice, have you seen this EMNLP 26 paper “Speculate While You Reason: Teaching Agents to Predict Their Next Tool Call via Joint Agent-Speculator RL” and the related work (eg, Nichols et al., 2025;
Huang et al., 2025; Hooper et al., 2026)? https://t.co/SL9v26iOV9

[Jacob Helwig @JacobHelwig]

@zhaisf Markov?

[Jacob Helwig @JacobHelwig]

@HeMuyu0327 It really just depends if you want to add KL as reward or use it as a loss. Reward=k1, loss=k3. My preference is reward, since if using task rewards, the sign of the policy gradient can be preserved by setting the overall advantage as `max(A-k1, 0) if A>0 else min(A-k1,0)`

[Jacob Helwig @JacobHelwig]

@kalomaze To concat+project an arbitrary number of byte embeddings, would I need a projection that maps k*d—>d for k in 1,2,…,max_num_bytes?

[Jacob Helwig @JacobHelwig]

@cHHillee @andrewgwils I think Adam caused substantial infra pain. Besides extra states needing shard/offload strategies like DeepSpeed ZeRO, sparse optimizers like in torchrec are needed to avoid updating 1st/2nd moments for unused embeddings

[Jacob Helwig @JacobHelwig]

@HeMuyu0327 k1 is used as a reward, differentiating it (as a loss) removes the effect of the teacher/reference, so no need to analyze the grad=0. In verl, see args use_kl_in_reward (ref)/use_policy_gradient (OPD). We raise an error for OPD if you use k1 as loss (see https://t.co/cdncjx7Mpl)

[Jacob Helwig @JacobHelwig]

@androneil54 @HeMuyu0327 k1 is often used for OPD. See the thinking labs OPD blog or GLM5 tech report

[Jacob Helwig @JacobHelwig]

@Grad62304977 The reduction in off-policy by avoiding the group tail is the strongest argument to me. I really wish that they showed the clip ratio of PPO vs GRPO like in fig 4c. Although critic slows step time, tokens trained for PPO>GRPO could offset this

[Jacob Helwig @JacobHelwig]

@Grad62304977 Just speculation: based on affiliations vs other https://t.co/6hipwJ7YOi works, this is an academic paper/intern project...GLM5.2 team then did some rough experiments that gave enough positive signal to invest at scale but weren't publishable

[Jacob Helwig @JacobHelwig]

@Grad62304977 Could be. Although they use 30B-A3B w/ 128K max length while glm5.2 is 750B-A40B w/ 1M. PPO benefits might be more pronounced at 8x length/ with an OOM more scale. I wouldn't say verdict is out

[Jacob Helwig @JacobHelwig]

@samsja19 Awesome! With all the new algos in prime-rl and the ppo results from glm 5.2, maybe you guys could consider adding critic to the roadmap

[Jacob Helwig @JacobHelwig]

@DimitrisPapail @giffmana Why not the same active params? If total, it seems like dense would always be better

[Jacob Helwig @JacobHelwig]

@radbadgeoffbrad @Miles_Brundage Same as the sigma algebra of a (biased) coin flip. Null, doom, not doom, doom or not doom

[Shubham Parashar @Shubham09632806]

What if an autoregressive LM (ARLM) could teach itself to become a diffusion LM (DLM)?

Meet OPDLM. We convert ARLMs into masked diffusion LMs via On-Policy Self-Distillation.

No separate DLM teacher, no pretraining from scratch, just post-training an ARLM.

[Jacob Helwig @JacobHelwig]

@yoavgo @TacoCohen Then sample a bunch of latents and marginalize over them via the Bayes estimator for a given risk. LLM evals use 0-1 risk, in which case the Bayes estimator is the posterior mode. This is exactly the self-consistency=majority@N strategy

[Shubham Parashar @Shubham09632806]

Excited to share that Learnability-Informed Fine-Tuning of Diffusion Language Models (LIFT) has been accepted at ICML 2026! 🎉

paper: https://t.co/R0Hhi0NYoh
code: https://t.co/TUsMWRnm5T

[Jacob Helwig @JacobHelwig]

@hi_tysam So you reduce exposure bias by adding random noise to model inputs. I wonder what would happen if noise more closely followed inference-time noise, eg by replacing input tokens with tokens sampled from model by forwarding model once before each train step

[Xiuyu Li @sheriyuo]

LIFT is the SFT recipe for dLLMs that actually understands the masking dynamics. Vanilla SFT on dLLMs often HURTS performance, and they finally pin down why.

Their analysis: vanilla SFT overlooks learnability. Rare tokens are difficult to learn when most of the input is masked because the model has nothing to ground them in. Common tokens are easy and of little value to learn when most of the input is unmasked because the answer is essentially already given.

LIFT aligns training with the information available at different diffusion time steps. Learn easy tokens when most of the input is masked (build up basic vocabulary at the noisy end), and learn hard tokens when more context is available (let the model use that context). The schedule matches the difficulty of each token to the moment the model is best positioned to absorb it.

Learnability-Informed Fine-Tuning of Diffusion Language Models
Paper: https://t.co/zUodVpjVgb
Code: https://t.co/gLW9OrR4bO

[Jacob Helwig @JacobHelwig]

@StefanGliga @kalomaze Although maybe not realistic, the MDLM indep assumption is explicit