Dilxat Muhtar

Enthusiast in AI, focusing on CV & NLP.

[Dilxat Muhtar @pumpkinnnnne]

@Shiwei_Liu66 @icmlconf 🎉🎉🎉

[Dilxat Muhtar @pumpkinnnnne]

🥳🥳🥳

[Jason Liu @JasonLiu106968]

#Agentic #OpenClaw #RL #Code_agent #LLM
🚀 **The secret for breaking the bottleneck of agentic RL training efficiency? From self-evolution to co-evolution!** 🧠

Excellent project led by @pumpkinnnnne and supported by Alibaba ROLL. It's my honor to collaborate with them!

[Dilxat Muhtar @pumpkinnnnne]

@Shiwei_Liu66 Thank you for the post🥳🥳🥳

[Shiwei Liu @Shiwei_Liu66]

Residual connections and pre-norm are not the whole story behind depth utilization.

Our new paper shows that many seemingly different design choices — MoE, grouped-query attention, weight decay, and longer sequence length — can be understood through one unifying lens: sparsity.

These components induce different forms of sparsity, which reduce output variance and in turn preserve healthier gradient flow across depth. Strikingly, these techniques also complement each other remarkably well: when combined, they lead to substantial improvements in depth utilization and notable gains in downstream accuracy.

Paper page: https://t.co/ZL3U3E3Fms
Arxiv: https://t.co/3iUm4UVUMG
Leading by @pumpkinnnnne