Song Yang

Computational Linguistics & Natural Language Processing & Machine Learning & Information Extraction & Coreference Resolution

[ModelScope @ModelScope2022]

🤖Meet Nanbeige4-3B from Boss Zhipin—a 3B-parameter LLM that outperforms Qwen3-32B on math (AIME), science (GPQA), and tool calling (BFCL-V4), while matching Qwen3-30B-A3B on human preference alignment (Arena-Hard-V2).

How?
✅ 23T tokens of ultra-curated data
✅ Fine-grained WSD scheduler
✅ 30M+ high-quality SFT instructions
✅ Multi-stage RL + innovative distillation (DPD)
✅ Chain-of-thought reconstruction & deliberative generation

It even ranks top 15 on WritingBench & EQ-Bench3—beating models 100x larger like GLM-4.5 and Deepseek-R1.

All models + tech report now open-source:
🔗 Weights: https://t.co/80t5t6nHeS
📄 Paper: https://t.co/hxjstlfo0G

Proof that with smart data + smarter training, small can be mighty. 💡

[DAIR.AI @dair_ai]

A 3B model outperforms models 10x its size on reasoning benchmarks.

Small language models (SLMs) are often dismissed as fundamentally limited. The belief is that more parameters mean more capability, and that's it.

More recent research indicates that the real ceiling isn't parameter count. It's the training methodology.

This technical report introduces Nanbeige4-3B, a family of SLMs trained on 23 trillion high-quality tokens and finetuned on over 30 million diverse instructions.

The results challenge assumptions about model scaling. On AIME 2024, Nanbeige4-3B-Thinking scores 90.4% versus Qwen3-32B's 81.4%. On GPQA-Diamond, it achieves 82.2% versus Qwen3-14B's 64.0%.

This shows that the 3B model consistently outperforms models 4-10x larger.

Here's how they did it:

Fine-Grained WSD scheduler: Rather than uniform data sampling, they split training into stages with progressively refined data mixtures. High-quality data is concentrated in later stages. On a 1B test model, this improved GSM8K from 27.1% to 34.3% versus vanilla scheduling.

Solution refinement with CoT reconstruction: They refine answer quality through iterative critique cycles, then reconstruct a chain-of-thought that logically leads to the improved solution. This yields SFT examples far better than rejection sampling.

Dual Preference Distillation: The student model simultaneously learns to mimic teacher output distributions while distinguishing high-quality from low-quality responses. Token-level distillation combined with sequence-level preference optimization.

Multi-stage RL: Rather than mixed-corpus training, each RL stage targets a specific domain. STEM reasoning with agentic verifiers. Coding with synthetic test functions. Human preference alignment with pairwise reward models.

On the WritingBench leaderboard, Nanbeige4-3B-Thinking (79.03) approaches GPT-5 (83.87) and outperforms DeepSeek-R1 (78.92), Grok-4 (74.65), and O4-mini (72.90).

The report demonstrates that carefully engineered small models can match or exceed much larger models when training methodology is optimized at every stage.

Paper: https://t.co/bFPJOZycji

Learn to build with LLMs and AI Agents in our academy: https://t.co/zQXQt0PMbG

[OpenCompass @OpenCompassX]

🔥🔥 Exciting update - the latest OpenCompass Leaderboard result is here!

- We added new LLMs such as GPT-4o-20240513 @OpenAI , Doubao-pro-4k, Qwen-Max-0428 @Alibaba_Qwen , Yi-Large @01AI_Yi and Nanbeige2 @nanbeige to the leaderboard.
- Two new subtasks, "algorithm interview" and "algorithm competition" are added under the code benchmark of objective comprehensive evaluation.
- We increased the subjective comprehensive evaluation of Code and Conversation capabilities, using GPT-4o as evaluation model.

👀 For an in-depth review of each model’s performance and more insights, visit our detailed rankings page!
🔗https://t.co/rEwbNsKwfg

#opencompass #LLMs

[Nanbeige @nanbeige]

We published our model Nanbeige2-16B-Chat(https://t.co/uXpSOq834R) with MT-Bench 8.6, AlpacaEval2.0 LC WinRate 43% and AlignBench 7.62. And a new open source model with the context window of 1 million tokens is on the road. Enjoy :-)

[Song Yang @magicsongyang]

@bingyuicejade 阴阳怪气的可真多。。。在中国它的名字叫抖音。。。如果你不知道活着没用过，建议可以下载一个体验体验

[Song Yang @magicsongyang]

@wangdan1989 现在用哪个id呢？

[Song Yang @magicsongyang]

@youwillwin 这是我见过的关于该片最给力的影评。

[Song Yang @magicsongyang]

@xlvector 恭喜恭喜

[Song Yang @magicsongyang]

@youwillwin 方校长都混成总理的待遇了。。。

[Song Yang @magicsongyang]

@wangdan1989 悲催啊

[Song Yang @magicsongyang]

@communicating you are interested on coreference resolution?

[Song Yang @magicsongyang]

@youwillwin 可以看看这个，里面提到一些参考文献http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.99.6217&rep=rep1&type=pdf

[Song Yang @magicsongyang]

http://mloss.org/software/

[Song Yang @magicsongyang]

回归twitter！

[Song Yang @magicsongyang]

@benhetwit 恭喜恭喜！你是帝都的人了！

[段叔.AI @duanzi]

Via @rincos: 儿子发短信给老爸：尽管翻开我的钱包，没有钱；但是翻开我的人生，有你，我亲爱的老爸。父亲收到很高兴回短信勉励。过了一会儿子回短信：关键是第一句…… @duanzi #段子

[Song Yang @magicsongyang]

有一个人，爷爷是皇帝，父亲是皇帝，儿子是皇帝，孙子是皇帝，他自己却不是。。。

[Song Yang @magicsongyang]

Hardy每次做船的时候，总是怕沉了。克服这个东西的一个方法是，每次不得不坐船航行的时候，他会给同事发个电报或者明信片什么的，说已经搞定了Riemann猜想回来之后会给出细节的。他的逻辑是，上帝不会允许他被淹死，否则这又将是第二个类似于Fermat大定理的事情。

[Song Yang @magicsongyang]

我将勇敢的去捍卫真正的科学，将其开拓，为之添彩；既不为厚禄所驱，也不为虚名所赶，只求上帝真理的神辉普照大地，发扬光大。

[Song Yang @magicsongyang]

蒋介石为了娶宋美龄，恳求陈洁如离开去美国深造，发了这样的毒誓：“我答应自今年起的5年之后，与洁如恢复夫妻关系。如我违背誓言，任凭佛祖罚我和我的南京政府；如我在10年或20年内不履行我对她的义务，任佛祖毁灭我的政府，并将我永远放逐国外。”