1/
We propose MIRA, a framework for mid-training data selection.
Core idea: ๐ learn source-specific quality rubrics instead of using one global scorer ๐ distill them into scalable scoring models
This makes data selection adaptive to data type, not just sample-level ranking.
2/ Why this matters:
Mid-training data is highly heterogeneous:
code / QA / agents / docs
A single quality function fails to capture:
๐ different formats
๐ different capability signals
๐ different โgoodnessโ definitions
The Call for Papers for #INLG2026 is out!
๐๏ธ Submit by July 15 (AoE)
๐ ARR commit by August 5
๐ Squibs welcomed (raising an issue without needing to solve it)
๐ Non-archival track for WIP
๐Utrecht, NL โ Oct 17โ21, just before EMNLP
https://t.co/eLnNE3Fqrc
#NLProc#INLG
Maybe Claude looped many times ๐ถ, but our research shows a 7B model needs Only Loop Once to rival giantsโbecause the 2nd pass is the ceiling and the 3rd is a cliff dive ๐๐.
LoopCoder-v2 is out ๐
Loop Transformers reuse the same block for recurrent hidden-state refinement โ letting models โthinkโ more without simply stacking more layers.
We study how many loops are actually worth it in Parallel Loop Transformers.
๐https://t.co/6iKMPelqQ2
๐ From simple code completion to autonomous software engineering agents โ what changed in the past 5 years?
We wrote the playbook ๐ "๐ ๐ซ๐จ๐ฆ ๐๐จ๐๐ ๐ ๐จ๐ฎ๐ง๐๐๐ญ๐ข๐จ๐ง ๐๐จ๐๐๐ฅ๐ฌ ๐ญ๐จ ๐๐ ๐๐ง๐ญ๐ฌ" โ 300 pages covering exact recipes ๐งช, scaling laws ๐ & RL techniques ๐ฏ for state-of-the-art Code LLMs.
What's inside:
โจ Full lifecycle: Data โ Pre-training โ SFT โ RL specifically for Code LLMs.
๐งช Empirical Training Recipes: We reveal language-specific scaling laws (Python vs. Java), where Python benefits massively from scale, but C# and Java are "easier" to learn and saturate faster.
๐ค SWE Agents Taxonomy: A detailed look at agents that handle the real tasks, including Environment, Dev, Testing, and Maintenance โ moving beyond simple generation to full workflow automation.
This work was led by Jian Yang (@jian_yang96 ) at Beihang University, alongside a stellar collaboration of researchers from Alibaba, ByteDance, and other affiliations. And please find more details in the paper!
๐ https://t.co/sJwikNRw7H