Very excited about our work on "Predicting Memorization Before Fine-Tuning Language Models" getting accepted to EACL 2027 Main Conference! Joint work w/ amazing @niloofar_mire & David A. Smith!
More details coming soon 👀
The wonderful “Don’t just Fix it in Post” paper identifies predicting memorization as a key open problem.
A progress report: we can now accurately predict verbatim memorization using just three well-understood variables. Experiments by Jing Huang, using the Hubble model suite:
On the job market this year! Open to faculty as well as RS/MTS positions and primarily interested in the science of language models. Please reach out if you are hiring and you think I would be a good fit.
Some thoughts on reviewing: https://t.co/cSo5bLgtzr
First for @satml_conf where we trialed an early reject phase which cut submissions by ~40%.
Then more generally about how not everything deserves peer-review, and why we should reward writing things that people actually read.
Excited to share our new paper!
“Forecasting Downstream Performance of LLMs With Proxy Metrics”
w/ my amazing advisors @sivareddyg, @mariusmosbach, @DBahdanau
Cross-entropy loss is a poor predictor of how models perform on downstream tasks (esp. reasoning). We propose something better: proxy metrics computed over expert reasoning traces.
🧵 Thread below 👇
Happy to be heading to #COLM2026 in SF to present this as an ✨ oral ✨! You can catch me at:
📢 Oral Session 3 (Wed morning)
🪧 Poster session 3 (Wed morning), I-3-24
Look forward to interesting discussions! I am also on both academic/industry job markets (more info coming)
We can finally talk about it:
We found a way to extract hidden reasoning of frontier models using a vulnerability in the APIs of every frontier AI company.
We verified that our reasoning token count matches billed API thinking tokens 1:1 for most of the prompts we queried.
Excited to be heading to #ICML2026 to present 3 papers! ✈️
Happy to talk about developmental LM science, continual learning, or LM creativity! See more details below, and send me a message via email if you'd like to chat! (Twitter chat isn't working for me). Also excited for shopping 👜
NLAs look like they "work"---but compared to what? How can we know that they actually tell us anything about model internals?
In my new blogpost (and paper, to be presented at ICML!), we investigate whether activation verbalizers (like NLAs) produce faithful explanations.
We don’t always know what problems are hard for LLMs. So devs evaluate on tasks HUMANS find hard or on broad benchmarks. What if we could instead anticipate which scenarios a model will fail on—all without evaluating specific input examples?
🧵NEW PAPER by @jenniferlumeng &al
really excited to head home for icml:) and attending the co-located @farairesearch's alignment workshop (for the first time)! would love to meet others interested in training & interpretability
Benchmarks can be superficial, but model explanations and evaluations are fundamentally intertwined. What if we used interpretability as principled, scientific evaluation? If it met scientific standards?
https://t.co/vsI1jgKlQF
coming to @evaluatingevals at ACL as oral 🧵
1/6
work w/ @_emliu@cathy__jiao@BrihiJ@DaniYogatama@FazlBarez@m2saxon
since am headed home for icml, it'll be presented by the amazing @BrihiJ!
this was my first time writing a position paper, which turned into a grant, which i'm turning into multiple projects 🙂 stay tuned
6/6