Fangkai Jiao

Third-year PhD candidate @ntunlpsg, working on NLP, reasoning and large language models.

[Shafiq Joty @JotyShafiq]

Meet SFR-VibeTrain 🚂 — built at Salesforce AI Research: a Slack-native training agent powered by our large-scale RL training stack, SFR-RL.

From chat → code → training → analysis, VibeTrain turns conversational workflows into end-to-end RL experimentation. 🧠⚡

[Salesforce AI Research @SFResearch]

[11/11] Authors: Srijan Bansal @SrijanBansal1, Jiao Fangkai @FangkaiJiao, Yilun Zhou @YilunZhou, Austin Xu @austinsxu, Shafiq Joty @JotyShafiq, Semih Yavuz @semih__yavuz

📄 Paper: https://t.co/GFyP6OhLup
💻 Code: https://t.co/ipr6Gs9UnG
📊 Dataset: https://t.co/2fwLxEVbWd
📝 Blog: https://t.co/3CbR8RMIGD

[Quanquan Gu @QuanquanGu]

Seed 2.0 is finally out! 🔥

[Salesforce AI Research @SFResearch]

🚨 New Survey Alert! 🚨
🧠”A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems”

📘 Paper: https://t.co/9YIfksgnkA
🧠 Project Page: https://t.co/jvh1dLmTpm
🧵 Researcher's thread: 👇

(1/6) Reasoning is the key to unlocking true AI intelligence.🔑

Two factors that affect the reasoning capabilities are:

1⃣ Regime: how and at what stage is reasoning achieved?
2⃣ Architecture: what components are involved in the reasoning process

⚡️We present a comprehensive survey along these two dimensions, summarizing recent progress and covering:

Regimes, from inference scaling (e.g., OpenAI o1) to learning to reason (e.g., DeepSeek-R1), also including learning algorithms for both the reasoner and the verifier;

Architectures, ranging from standalone LLMs to agentic systems (e.g., OpenAI’s deep research).

We also unify techniques from input and output perspectives, clarifying what must be customized or designed when building reasoning systems.

[Caiming Xiong @CaimingXiong]

As large language models increasingly serve as judges for evaluating other models during both development and deployment, most existing benchmarks still focus on non-contextual tasks like chat completions or logical reasoning. Our team present ⚖️ ContextualJudgeBench ⚖️ — a benchmark designed to evaluate LLM-as-judge capabilities in context-rich scenarios such as RAG-based QA and summarization.

In real-world business applications, context is everything—AI systems must provide accurate and timely information. But assessing their performance is challenging: judge models need to interpret both the context and the question, then evaluate accuracy, justification for refusals, and overall quality, including completeness and conciseness.
📘 Paper: https://t.co/7uH3DQA6q7
📊 Data: https://t.co/h5xyNW1zOl
💻 Code: https://t.co/K1G2Y6zTlw

[Caiming Xiong @CaimingXiong]

Testing LLMs' reasoning skills is tough—human evaluations are expensive, data contamination is common, and LLM judges can be biased. We propose StructTest, the first benchmark that checks how well LLMs follow complex instructions and create structured outputs. It uses a rule-based evaluator that’s easy to adapt to new tasks. StructTest is unbiased, cheap, hard to cheat and highly scalable.

By testing structured outputs in areas like Summarization, Code, HTML, and Math—and evaluating 17 top LLMs—StructTest proves to be a challenge even for models like Deepseek-V3/R1 and GPT-4o. It’s also highly correlated with ChatBot Arena (~93%) and MMLU (>96%), making it a solid way to measure reasoning skills.

Code & Data: https://t.co/5urKBaXJLT
Paper🔗: https://t.co/ASLGIuSR0F

[Ross Taylor @rosstaylor90]

🎉 Excited to release General Reasoning: a new community resource for building open reasoning models.

We’re looking to make personal, open reasoners a reality. Starting with a small step in that direction today!

Read the thread in the quote tweet for details, or my personal analysis below!

[Fangkai Jiao @FangkaiJiao]

@JotyShafiq Code: https://t.co/83kJNIpsXt
Data: https://t.co/nxzTmYn2vU

[Fangkai Jiao @FangkaiJiao]

Code: https://t.co/83kJNIpsXt

[Fangkai Jiao @FangkaiJiao]

The data can be found at:
https://t.co/nxzTmYn2vU

[Shafiq Joty @JotyShafiq]

The success of #deepseek-r1 largely hinges on the availability of reasoning data with programmatically verifiable answers. Such data is often scarce. In our #ICLR2025 paper, we introduce a method to generate pseudo feedback from test cases, to enhance #LLM reasoning.

[Fangkai Jiao @FangkaiJiao]

@agihippo We have just finished one work on scaling pseudo feedback:
https://t.co/zAFGYCvdjX

Thanks!

[Zhihong Shao @zhs05232838]

Here comes the official release of DeepSeek-V3. We also share a lot in the tech report. Check it out!

[Jiao Sun @sunjiao123sun_]

Mitigating racial bias from LLMs is a lot easier than removing it from humans!

Can’t believe this happened at the best AI conference @NeurIPSConf

We have ethical reviews for authors, but missed it for invited speakers? 😡

[Xidong Feng @Xidong_Feng]

Happy to share our new exploration "Natural Language Reinforcement Learning" (NLRL), the last dance of my PhD 🛎️(1/n):

Paper: https://t.co/k94IQxs8eC
Code: https://t.co/LlfuTA2y53 (released soon)

NLRL reframes core RL concepts—policy, value function, Bellman equation, MC, TD, and policy iteration—into natural language, enabling the training of chain-of-thought language policies and language value function (known as generative value) solely from environment feedback, without human or stronger model's labels.

[Shafiq Joty @JotyShafiq]

We got an outstanding paper award today at EMNLP for the work: "Learning Planning-based Reasoning by Trajectories Collection and Process Reward Synthesizing (https://t.co/x5QJ6ZKh8D)". Thanks to my awesome collaborators!

[Fangkai Jiao @FangkaiJiao]

@sbmaruf Thank you Maruf!

[Thomas Wolf @Thom_Wolf]

There was a super impressive AI competition that happened last week that many people missed in the noise of AI world. I happen to know several participants so let me tell you a bit of this story as a Sunday morning coffee time.

You probably know the Millennium Prize Problems where the Clay Institute pledged a US$1 million prize for the first correct solution to each of 7 deep math problems. To this date only one of these, the Poincaré conjecture, has been solved by Grigori Perelman who famously declined the award (go check Grigori out if you haven't the guy has a totally based life).

So this new competition, the Artificial Intelligence Math Olympiad (AIMO) also came with a US$1M prize but was only open to AI model (so the human get the price for the work of the AI...). It tackle also very challenging but still simpler problems, namely problems at the International Math Olympiad gold level. Not yet the frontier of math knowledge but definitely above what most people, me included, can solve today.

The organizing committee of the AIMO is kind-of-a who-is-who of highly respected mathematicians in the world, for instance Terence Tao widely famous math prodigy widely regarded as one of the greatest living mathematicians.

Enter our team, Jia Li, Yann Fleuret, and Hélène Evain. After a successful exit in a previous startup (that I happen to have know well when I was an IP lawyer in a previous life but that's for another story) they decided to co-found Numina as a non-profit to do open AI4Math.

Numina wanted to act as a counterpoint to AI math efforts like DeepMind's but in a much more open way with the goal to advance the use of AI in mathematics and make progress on hard, open problems. Along the way, they managed to recruit the help of some very impressive names in the AI+math world like Guillaume Lample, co-founder of Mistral or Stanislas Polu, formerly pushing math models at OpenAI.

As Jia was participating in the code-model BigCode collaboration with some Hugging Face folks, came the idea to collaborate and explore how well code models could be used for formal mathematics.

For context, olympiad math problems are extremely hard and the core of the issue is in the battle plan you draft to tackle each problem. A first focus of Numina was thus on creating high quality instruction Chain-of-Thought (CoT) data for competition-level mathematics. This CoT data has already been used to train models like DeepSeek Math, but is very rarely released so this dataset became an unvaluated ressource to tackle the challenges.

BigCode's lead Leandro put Jia in touch with the team that trained the Zephyr models at Hugging Face, namely, Lewis, Ed, Costa and Kashif with additional help from Roman and Ben and the goal became to have a go at training some strong models on the math and code data to tackle the first progress prize of AIMO.

And the trainings started:

Jia being an olympiad coach, was intimately familiar with the difficulty level of these competitions and able to curate an very strong internal validation set to enable model selection (Kaggle submissions are blind). While iterating on dataset construction, Lewis and Ed from Hugging Face focused on training the models and building the inference pipeline for the Kaggle submissions.

As often in competition it was an intense journey with Eureka and Aha moments pushing everyone further.
Lewis told me about a couple of them which totally blow my mind. A tech report is coming so this is just some "along the way" nuggets that will be soon gathered in a much more comprehensive recipe and report.

Learning to code: The submission of the team relied on self-consistency decoding (aka majority voting) to generate N candidates per problem and pick the most common solution. But initial models trained on the Numina data only scored around 13/50... they needed a better approach. They then saw the MuMath-Code paper (https://t.co/9KGmjGJvT7) which showed you can combine CoT data with code data to get strong models. Jia was able to generate great code execution data from GPT-4 to enable the training of the initial models and get to impressive boost in performance.

Taming the variance: Another Ahah moment came at some point when a Kaggle member shared a notebook showing how DeepSeek models worked super well with code execution (the model breaks down the problem into steps and each step is run in Python to reason about the next one).

However, when the team tried this notebook they found this method had huge variance (the scores on Kaggle varied from 16/50 to 23/50).

When meeting in Paris for a hackathon to improve this issue (like the HF team often does) Ed had the idea to frame the majority voting as a "tree of thoughts" where you'd progressively grow and prune a tree of candidate solutions (https://t.co/dkKtBMrIPm).

This had an impressive impact on the variance and enabled them to be much more confident in their submissions (which showed in how the model ended up performing extremely well on the test set versus the validation set)

Overcoming compute constraints: the Kaggle submissions had to run on 2xT4s in under 9h which is really hard because FA2 doesn't work and you can't use bfloat16 either. The team explored quantization methods like AWQ and GPTQ, finding that 8-bit quantization of a 7B model with GPTQ was best

Looking at the data: a large part of the focus was also on checking the GPT-4 datasets for quality (and fixing them) as they quickly discovered that GPT-4 was prone to hallucinations and failing to correctly interpret the code output. Fixing data issues in the final week led to a significant boost in performance.

Final push: The result were really amazing and the model climbed to the 1 place. And even more, while tying up for first place on the public, validation leaderboard (28 solved challenges versus 27 for the second place), it really shined when tested on the private, test leaderboard where it took a wide margin solving 29 challenges versus 22 for the second team.

As Terence Tao himself set it up, this is "higher than expected"

Maybe what's even more impressive about this competition, beside the level of math these models are already capable of is how ressource contraint the participants were actually, having to run inference in a short amont of time on T4 which only let us imagine how powerful these models will become in the coming months.

Time seem to be ripe for GenAI to have some impact in science and it's probably one of the most exciting thing AI will bring us in the coming 1-2 year. Accelerating human development and tackling all the real world problems science is able to tackle.

[Caiming Xiong @CaimingXiong]

Are your LLMs highly accurate, or simply contaminated?
As the race to build the best LLM intensifies, clean evaluation is becoming more important than ever, yet contaminated LLMs and benchmarks obfuscate the real performance of models.
Checkout our new work (comprehensive survey + library) at NTU-NLP lab & Salesforce Research on the critical issue of contamination detection in LLMs, cc @ntunlp @MatRavox @D_Boss001 @HailinChen3 @XingxuanLi @RuochenZhao3 @FangkaiJiao @qcwntu @CaimingXiong @JotyShafiq
Paper:
https://t.co/16jXQ1hNXk
Library:
https://t.co/vNgXrto1XJ

[Thomas Wolf @Thom_Wolf]

Llama3 was trained on 15 trillion tokens of public data. But where can you find such datasets and recipes??

Here comes the first release of 🍷Fineweb. A high quality large scale filtered web dataset out-performing all current datasets of its scale. We trained 200+ ablation models to craft this dataset carefully parsing and filtering Common Crawl.

All recipes, data, ablations models, hyper-parameters are open-source and we plan to improve Fineweb over time so stay tuned for future versions.

Finally we made a number of surprising observations along the way (all Common crawl years are not equal, the influence of ChatGPT in latest webdata, etc) that we’re compiling in a longer tech blog post to be released in the coming days for the fine data lovers.

Enjoy 😊

[Xiao Liu (Shaw) @ShawLiu12]

Thanks @arankomatsuzaki for sharing our work☺️

To achieve both improvement of language and math is crucial for reaching GPT-4/* performant LLMs.

Related new MathUserEval dataset and code will be released at https://t.co/xg1ryZ187J