Jane Pan

CS PhD at @nyuniversity, @NSF GRFP, @Deepmind Fellowship, @SiebelScholars | @Princeton @Princeton_nlp '23 | @Columbia '21.

[He He @hhexiy]

https://t.co/H3TAsaThYQ

[John (Yueh-Han) Chen @jcyhc_ai]

Can LLMs control their chains of thought (CoT)? If so, they could evade CoT monitors 🚨
We introduce the CoT Controllability eval suite to find out.
Our results leave us cautiously optimistic that today’s models struggle to obfuscate their CoT in ways that undermine monitorability.

In this thread, I explain additional findings that I find interesting

Joint work w @OpenAI

[Jane Pan @JanePan_]

Excited to see NeuNeu out — learned scaling laws from open-source LM trajectories!

[Michael Hu @michahu8]

if you truly believe in the bitter lesson, then why hand design scaling laws?

introducing: neural neural scaling laws (NeuNeu), a neural network
- trained on open-source LM trajectories
- that predicts LMs' future downstream task performance

🧵👇

[Vishakh Padmakumar @vishakh_pk]

Our work on LLM novelty as the frontier of original and high-quality output was accepted to #ICLR26! Come talk to us about how model scale, SFT, and RL affect this trade-off! See you in Brazil!🇧🇷h/t to my awesome collaborators @hhexiy @valeriechen_ @JanePan_ @jcyhc_ai

[Richard Pang @yzpang_]

🚨Prompt Curriculum Learning (PCL)
- Efficient LLM RL training algo!
- We investigate factors that affect convergence: bsz, # prompt, # gen, prompt selection
- We propose PCL: lightweight algo that *dynamically selects intermediate-difficulty prompts* using a learned value model

[Jane Pan @JanePan_]

Bored of seeing pristine, perfect posters? Come see me at Hall X5, Board 105 at 6pm to witness my masterpiece, featuring bonus Sharpie scribbles and a QR code that betrayed me at the last moment 😤

[Jane Pan @JanePan_]

I'll be at ACL Vienna 🇦🇹 next week presenting this work! If you're around, come say hi on Monday (7/28) from 18:00–19:30 in Hall 4/5. Would love to chat about code model benchmarks 🧠, simulating user interactions 🤝, and human-centered NLP in general!

[Jane Pan @JanePan_]

I'll be at ACL Vienna 🇦🇹 next week presenting this work! If you're around, come say hi on Monday (7/28) from 18:00–19:30 in Hall 4/5. Would love to chat about code model benchmarks 🧠, simulating user interactions 🤝, and human-centered NLP in general!

[Jane Pan @JanePan_]

When benchmarks talk, do LLMs listen?

Our new paper shows that evaluating that code LLMs with interactive feedback significantly affects model performance compared to standard static benchmarks!

Work w/ @RyanShar01, @jacob_pfau, @atalwalkar, @hhexiy, and @valeriechen_!

[1/6]

[Vishakh Padmakumar @vishakh_pk]

What does it mean for #LLM output to be novel?
In work w/ @jcyhc_ai, @JanePan_, @valeriechen_, @hhexiy we argue it needs to be both original and high quality. While prompting tricks trade one for the other, better models (scaling/post-training) can shift the novelty frontier 🧵

[Yulin Chen @YulinChen99]

We're excited to receive wide attention from the community—thank you for your support!
We release code, trained probes, and the generated CoT data👇
https://t.co/Rkw6LJtAyj
We have labeled answer data on its way. Stay tuned!

[Jane Pan @JanePan_]

Do reasoning models know when their answers are right?🤔

Really excited about this work led by Anqi and @YulinChen99. Check out this thread below!

[Yulin Chen @YulinChen99]

Reasoning models overthink, generating multiple answers during reasoning. Is it because they can’t tell which ones are right?

No! We find while reasoning models encode strong correctness signals during chain-of-thought, they may not use them optimally.

🧵 below

[Jane Pan @JanePan_]

Our work bridges the gap between existing static benchmarks and real-world usage, and we hope to inspire future work on scalable methods for evaluating models in a collaborative setting.

Read our preprint at https://t.co/gwPhIJDmqR!

[6/6]

[Jane Pan @JanePan_]

When benchmarks talk, do LLMs listen?

Our new paper shows that evaluating that code LLMs with interactive feedback significantly affects model performance compared to standard static benchmarks!

Work w/ @RyanShar01, @jacob_pfau, @atalwalkar, @hhexiy, and @valeriechen_!

[1/6]

[Jane Pan @JanePan_]

We also investigate how much a code model adjusts its solution in response to feedback. Weaker models tend to make many surface-level changes that do not greatly change code behavior; stronger models may make relatively small edits that highly affect code behavior.

[5/6]

[Jacob Andreas @jacobandreas]

@tallinzen @akyurekekin @yoavartzi @NeelNanda5 I really like the paper from Jane Pan (w @danqi_chen) abt this: https://t.co/rnbu4Mnfab. ICL in big models is clearly a mix of task recognition and "real learning" (you're not learning to translate from 3 examples, but you're not getting an arbitrary label mapping from the prior)

[Jane Pan @JanePan_]

Our pre-print is available on arXiv here:
https://t.co/cYUa7vUBOB

[7/7]

[Jane Pan @JanePan_]

Do LLMs exploit imperfect proxies of human preference in context? Yes!

In fact, they do it so severely that iterative refinement can make outputs worse when judged by actual humans. In other words, reward hacking can occur even without gradient updates!

w/ @hhexiy, @sleepinyourhat, @ihsgnef
[1/7]

[Jane Pan @JanePan_]

We follow the canonical definition of reward hacking, observing a divergence between the ground-truth reward (human expert judgment) and its proxy (an LLM judge following the same scoring criteria as the humans).

Our results complement recent work on output degradation via iterative refinement when measured with secondary objectives (https://t.co/x4WpZ8sky4) or with reference-based metrics (https://t.co/31ugQmUcZq).

[6/7]