Yang Liu

#LLM Researcher @Microsoft; PhD @EdinburghNLP

[Yang Liu @nlpyang]

Excited to introduce our MAI Code model at Microsoft Build. As shared in the session, this is a MoE (5B active / 137B total) initialized from an MAI pretrained model and trained for real user scenarios with product harnesses. I’m proud to have served as the research lead for this effort, and even prouder of what the team has achieved. It’s a beast for its size. Stay tuned — a larger model could come :)

[Microsoft AI @MicrosoftAI]

MAI-Code-1-Flash is here! Built and optimized for GitHub Copilot. From quick fixes to complex engineering challenges, write better code with more return on token. Rolling out to GitHub Copilot individual users in Visual Studio Code in the model picker and under the default auto picker now.

[Yang Liu @nlpyang]

More details here https://t.co/Orj8uXGjxI

[Yang Liu @nlpyang]

Excited to introduce our MAI Code model at Microsoft Build. As shared in the session, this is a MoE (5B active / 137B total) initialized from an MAI pretrained model and trained for real user scenarios with product harnesses. I’m proud to have served as the research lead for this effort, and even prouder of what the team has achieved. It’s a beast for its size. Stay tuned — a larger model could come :)

[Microsoft AI @MicrosoftAI]

MAI-Code-1-Flash is here! Built and optimized for GitHub Copilot. From quick fixes to complex engineering challenges, write better code with more return on token. Rolling out to GitHub Copilot individual users in Visual Studio Code in the model picker and under the default auto picker now.

[Liliang Ren @liliang_ren]

Introducing HyperP, a scaling framework gives you better compute efficiency & transferable stability. At 6e21 FLOPs, HyperP reaches 1.58× compute efficiency over a strong Muon baseline; +MoE further gets 3.38× over dense. Gains even grow with scale🤯

📖: https://t.co/gZRvX9qAIi

[Zhangchen Xu @zhangchen_xu]

Missing coding data in your R1? Introducing KodCode 🐱: a diverse, challenging, and verifiable synthetic dataset for LLM coding!

With 447K verified question-solution-test triplets, KodCode is designed for supervised fine-tuning (SFT) and reinforcement learning (RL).

💡Key Features

✨Diverse & Challenging: 5 synthesis methods, 12 subsets covering multiple domains (algorithms to package-specific knowledge) and difficulty levels (basic exercises to competitive programming tasks).

✨Verifiable Correctness: Question-solution-test triplets are systematically validated via a self-verification process with GPT-4o.

✨ Supports RL & SFT: Unit tests enable RL tuning, plus verified CoT responses generated by DeepSeek-R1 🐳 via reject sampling to support SFT.

---------------
>> Project Page: https://t.co/9IL4gEofmH
>> KodCode-V1 (for RL): https://t.co/CWE7SRBZXA
>> KodCode-V1-SFT-R1 (for SFT): https://t.co/4kK5eAhpCs
>> Paper: https://t.co/9J8uLMy3CU
>> Codebase for creating this dataset: https://t.co/EwKHpbdMsM

Thanks to my great mentor @nlpyang for invaluable guidance and support on this project! 🤩

[1/5]

[Yang Liu @nlpyang]

Great work from @zhangchen_xu

[Yang Liu @nlpyang]

Missing coding data in your R1?

🔥 Introducing KodCode—the largest verified synthetic coding dataset for Code LLM training!

• 447K question–solution–test triplets
• 12 diverse subsets
• 10-trial solution verification for rock-solid correctness

https://t.co/HILpz70PTT

[Yang Liu @nlpyang]

I don't really understand why people think RL takes less compute than pretrain.

[Yang Liu @nlpyang]

Check our paper so you can really challenge an LLM

[Yulong Chen @Yulongchen1010]

Evaluating LLMs usually requires sophisticated human designs and with the continuous improvement of LLMs, it is difficult for humans to find their limitations.

Can LLMs find their own limitations by proposing questions to themselves? Check our new paper: https://t.co/lDkBCQsIVR

[Yang Liu @nlpyang]

@kdcreer Ah, thank you for saving this post

[Yang Liu @nlpyang]

Microsoft GenAI is looking for a summer intern to work on Sparse LLMs, if you are interested, please DM me or send a resume to yaliu10 at microsoft dot com

[Yang Liu @nlpyang]

If you want to know our recent research efforts, check the great Samba model https://t.co/JaUeaiPcnh

[Liliang Ren @liliang_ren]

Introducing Samba 3.8B, a simple Mamba+Sliding Window Attention architecture that outperforms Phi3-mini on major benchmarks (e.g., MMLU, GSM8K and HumanEval) by a large margin.😮 And it has an infinite context length with linear complexity.🤯

Paper: https://t.co/KwMpeyaDxc

(1/6)

[Yang Liu @nlpyang]

I truly believe now it is the time that you change all your local attentions to SSMs :)

[Liliang Ren @liliang_ren]

Introducing Samba 3.8B, a simple Mamba+Sliding Window Attention architecture that outperforms Phi3-mini on major benchmarks (e.g., MMLU, GSM8K and HumanEval) by a large margin.😮 And it has an infinite context length with linear complexity.🤯

Paper: https://t.co/KwMpeyaDxc

(1/6)

[Yang Liu @nlpyang]

not multilingual enough :)

[Atty Eleti @athyuttamre]

GPT-4o is also natively multilingual. We've improved our tokenizer for various languages, resulting in, for example, a 3-4x speed improvement for Indian languages.

[Yang Liu @nlpyang]

@BorisMPower So proud we did this❤️

[Yang Liu @nlpyang]

If you are at NeurIPS, you could check out our poster about Efficient Transformer this evening.

[Yang Liu @nlpyang]

spent some time looking for Gemini's "uncertainty routed" prompt. but still no clue

[Boris Power @BorisMPower]

The top line number for MMLU is a bit gamed - Gemini is actually worse than GPT-4 when compared on normal few shot or chain of thought

[Yang Liu @nlpyang]

Wow, google really cares a lot about MMLU 🧐

[Yang Liu @nlpyang]

I have to admit, writing ICLR meta reviews is much more fun than writing ARR ones. What happened here?

[Yang Liu @nlpyang]

After 1 year birthday of ChatGPT, you think it is a good thing or a bad thing for NLP research?