Meng (Caden) Cao

Final-year Ph.D. student in Mila @Mila_Quebec and McGill University. | Intern @google | NLP, RL, LLMs

[codila @0xCodila]

https://t.co/UJL7o2l9oR

[Ziling Cheng @ziling_cheng]

1/N Why do LLMs fail at math word problems without CoT?
🔍Final-answer accuracy alone doesn’t tell the full story.
💥It’s not that they can’t understand the problem, the major bottleneck is doing the calculations correctly.
📌Read our EMNLP main paper: https://t.co/GOqPRqZ1y4

[Ziling Cheng @ziling_cheng]

Our paper on reasoning × interpretability × evaluation has been accepted to EMNLP main!

Excited because this marks the start of a new research direction I’m diving into.

Huge thanks to @Meng_0209, @yanshuaicao, Leila, and Jackie!

📌https://t.co/rCpXYj7SqN

[Meng (Caden) Cao @Meng_0209]

@gregd_nlp Congratulations!

[Ziling Cheng @ziling_cheng]

Do LLMs hallucinate randomly? Not quite. Our #ACL2025 (Main) paper shows that hallucinations under irrelevant contexts follow a systematic failure mode — revealing how LLMs generalize using abstract classes + context cues, albeit unreliably.

📎 Paper: https://t.co/YEK4TaI7pq 1/n

[Greg Durrett @gregd_nlp]

Revoking visas to Chinese PhD students is economically shortsighted and inhumane.

Most Chinese PhD students stay in the U.S. after graduation (first image, stats from 2022). They're staying and building technology in the U.S., not taking it to China.

Immigrant students create startup companies that employ Americans (second image, stats from 2018). I couldn't find stats for Chinese students specifically but anecdotally they are a significant force powering GenAI and LLM companies. These are the next $1B+ companies that are going to hire thousands of people even as other jobs are automated.

There are no real "secrets" in academic GenAI. Everything is open-source. What is the security risk? It's a much higher risk to give DeepSeek and others an advantage in hiring the most talented young researchers in these areas by preventing them from coming to the U.S.

Revoking visas for these students destroys American innovation for no benefit.

It's also unfair and inhumane. In my experience, Chinese PhD students are hardworking young researchers interested in their fields of study and being part of the cutting edge of academic and industrial innovation. They aren't geopolitical pawns. Stop implementing policies that treat them that way.

[Alex Dimakis @AlexGDimakis]

"RL with only one training example" and "Test-Time RL" are two recent papers that I found fascinating.

In the "One Training example" paper
the authors find one question and ask the model to solve it again and again. Every time, the model tries 8 times (the Group in GRPO), and a gradient step is performed, to increase the reward which is a very simple verification of the correct answers, repeated thousands of times on the same problem.

The shocking finding is that the model does not overfit to this one question: RL on one example, makes the model better in MATH500 and other benchmarks.
(If instead you did SFT repeating one training question-solution finetuning, the model would quickly memorize this answer and overfit). But with RL, the model has to solve the problem itself, since it only sees the question, not the answer. Every time it produces different answers, and this seems to prevent overfitting. The other papers are relying on the same phenomenon: you can have a small number of training questions and re-solve them thousands of times. You can do this for the test set (as test-time RL does) and still not overfit. We also independently saw this by doing RL training on half the test set and seeing benefits in the other half for BFCL agents.

My thought now is that this shows our RL learning algorithm must be extremely inefficient. When a human is learning by solving a math puzzle, they immediately learn what they can learn by solving it once (or twice). No further benefit would come by assigning the same homework problem to students a tenth time. But in RL, we keep asking the model to re-solve the same question thousands of times, and the model slowly gets better. We should be able to have much better RL learning algorithms since the information is there. (1/2)

[Rohan Paul @rohanpaul_ai]

Automated detection of LLM hallucinations using only correct examples is fundamentally difficult.

This paper shows detection is equivalent to the hard task of language identification, but providing detectors with both correct and explicitly incorrect examples makes reliable detection theoretically achievable.

📌 Equating detection with language identification formalizes the inherent difficulty of finding LLM errors automatically.

📌 The work strongly supports needing explicit negative examples (like human feedback) for effective hallucination detection.

----------

Methods Explored in this Paper 🔧:
→ The paper models hallucination detection using a formal framework inspired by classical language identification theory.

→ It proves an equivalence: algorithms solving language identification can enable hallucination detection, and vice-versa.

→ This equivalence reveals inherent difficulty when detectors learn only from correct statements (positive examples).

→ Adding explicitly labeled incorrect statements (negative examples) fundamentally changes the problem, enabling successful detection algorithms for all countable language collections.

----------------------------

Paper - arxiv. org/abs/2504.17004v1

Paper Title: "(Im)possibility of Automated Hallucination Detection in LLMs"

[Lei Yu @jade_lei_yu]

🤔How do LLMs perform reasoning and recall memorized knowledge? How similar are their underlying mechanisms? We reveal their inherent distinction within LLMs' representations, and identify linear features that mediate model switch between genuine reasoning and memory recall.

[Lewis Tunstall @_lewtun]

New R1-Zero experiments with GRPO:

1. Mask the loss from completions that don't terminate in an EOS token (DAPO). Significantly improves stability when doing importance sampling with μ>0. Coming soon to TRL!

2. Use a "soft" format reward function to elicit the <think> and <answer> tags from the model, and then refine with a strict format. Inspired by @willccbb's famous GRPO training script and makes all the difference between never learning the format to rapidly getting it 100% correct

Log book: https://t.co/wNO56LyaQ6

[Charlie Snell @sea_snell]

R1-zero is such a striking example of a discovery that’s blatantly obvious in retrospect, yet eluded so many for such a long time

[Junxian He @junxian_he]

We replicated the DeepSeek-R1-Zero and DeepSeek-R1 training on 7B model with only 8K examples, the results are surprisingly strong.

🚀 Starting from Qwen2.5-Math-7B (base model), we perform RL on it directly. No SFT, no reward model, just 8K MATH examples for verification, the resultant model achieves (pass@1) 33.3% on AIME, 62.5% on AMC, and 77.2% on MATH, outperforming Qwen2.5-math-7B-instruct and being comparable to PRIME and rStar-MATH that use >50x more data and more complicated components.
🚀 Increased CoT length and self-reflection emerge

We share the details and our findings in the blog:
https://t.co/DoupDsB4hA
Training code and implementation details here: https://t.co/6TRqSr2eBG

[Meng (Caden) Cao @Meng_0209]

@mohitban47 @RealAAAI Congratulations!!!

[Wenhu Chen @WenhuChen]

I spent the weekend reading some recent great math+reasoning papers:
1. AceMath (https://t.co/AB0y6X21u8)
2. rStar-Math (https://t.co/5aLdzcYgqV)
3. PRIME (https://t.co/p4kSH2Ig31)
Here are some of my naive thoughts! It could be wrong.

All of these papers are showing possible ways to reach o1. The secret source is pretty much the same thing:
**high-quality/difficult prompt with verifiable answer**

1. AceMath takes a simple approach (rejection-fine-tuning -> RFT) to scale up all the SFT dataset to massive size based on the verifiable answer matching. No RM is necessary, but you can still use outcome RM to help boost the performance.
2. rStarMath uses self-evolving SFT approach to gradually boost the data quality and process preference model (PPM) performance. rStarMath is still an RFT, where the samples are coming from MCTS guided with PPM. Still, it requires strong supervision from the verifiable reward in the end. rStarMath also scales up inference compute by utilizing the PPM at each step.
3. PRIME takes a very different angle! PRIME actually uses PPO to train the model, but the major contribution is on how to assign the outcome's reward to each intermediate steps. It also relies heavily on using the verifiable answer to obtain the "correct" on-policy model outputs.

The results are quite interesting. It seems that all these approaches are reaching similar results. Eurus-2 might seem weaker due to its smaller training set size. These results are all somewhat on par with o1-mini already/ Given some leakage that o1-mini is ~20B, it basically says that on there is no gap with o1 at least on math problems now. However, o1-mini might win significantly in other broader reasoning tasks, like physics, puzzles, etc.

These results might reveal that reaching o1 is more of a data or infra problem than an algorithm problem. As we find great ways to scale up the (good and difficult prompt, verifiable answer) pairs from different domains, the actual algorithm might not influence too much. Some algorithms are more data efficient than the others, but many of them will take us to o1 or even o3.

[Amirhossein Kazemnejad @a_kazemnejad]

From OpenAI’s PPO, people start simplify it by removing its mechanisms, especially credit assignment, without performance loss.

This contradicts the DeepRL belief that credit assignment is crucial.

Find how we address this contradiction at MATHAI workshop on 11AM &amp; 4PM.

[Jing-Jing Li @drjingjing2026]

1/3 Today, an anecdote shared by an invited speaker at #NeurIPS2024 left many Chinese scholars, myself included, feeling uncomfortable. As a community, I believe we should take a moment to reflect on why such remarks in public discourse can be offensive and harmful.

[Philipp Schmid @_philschmid]

Not Llama 3 405B, but Nemotron 4 340B! @nvidia just released 340B dense LLM matching the original @OpenAI GPT-4 performance for chat applications and synthetic data generation. 🤯 NVIDIA does not claim ownership of any outputs generated. 💚

TL;DR:
🧮 340B Paramters with 4k context window
3️⃣ Base, Reward Model and Instruct Model released
🔢  Trained on 9 trillion tokens with 2 phases
🌎 Trained on English, 50+ languages and 40+ programming languages
🧠 Requires 16x H100 in bf16 and ~8x H100 in int4
🥇 Base Model achieves 81.1 MMLU; 90.53 HellaSwag; 85.44 BHH
🧬 Used SFT, DPO, and RPO for post-training
🔓 Commercially useable, but custom license
🤗 Available on @huggingface

Model: https://t.co/Ba3dD7BOB5
Technical Report: https://t.co/q58tND603e

[Tianle Cai @tianle_cai]

Just wrote a script to further investigate how the corpus used to train the gpt4o tokenizer is polluted by Internet scams. The results are quite interesting... 🤦‍♂️🤦‍♂️🤦‍♂️

https://t.co/Fc2T4rSHix

[Meng (Caden) Cao @Meng_0209]

@shuyanzhxyc @DukeU @dukecompsci Congrats!

[Yu Lu Liu 🦋@ liuyulu.bsky.social @liu_yu_lu]

How and when, and with which issues, does the text summarization community engage with responsible AI? 🤔 In our new #EMNLP2023 Findings paper, we examine reporting and research practices across 300 summarization papers published between 2020-2022 🧵