We’ve all offloaded the writing to LLMs, but have you considered REWRITING with LLMs? 👀
Rewriting your private data with an LLM before finetuning gives a better empirical privacy vs. capability trade-off than PII scrubbing or DP-SGD!
To test this, we generated 5,000 synthetic medical records with private information. Then, we finetuned Qwen3.5-4B and gpt-oss-20b with several methods to mitigate privacy leakage and asked two things: does the model learn from the medical records, and does it leak private information?
1/n 🧵👇
🤖 AI is now writing its own research papers, resolving decades-old Erdős problems, and even producing a claimed proof of Navier–Stokes. But we only ever see the final results, never the ideation behind them.
How does an LLM actually search its way to a final idea, and where does it get stuck during ideation?
We turn this into a game of Twenty Questions: an LLM asks yes/no questions to rediscover the core idea of a published paper.
Introducing 𝗥𝗲-𝗦𝗲𝗮𝗿𝗰𝗵, which makes every step of ideation in LLM autoresearch observable and measurable.
Paper: https://t.co/Xt59XlkxSS
@aleinboii Thanks for reading! Scrubbing is good for capability, held-out perpelxity is only 2-3% worse than plain finetuning, since the scrubber only removes identifiers and leaves the clinical content the model learns from. Happy to chat more if it's useful!
We’ve all offloaded the writing to LLMs, but have you considered REWRITING with LLMs? 👀
Rewriting your private data with an LLM before finetuning gives a better empirical privacy vs. capability trade-off than PII scrubbing or DP-SGD!
To test this, we generated 5,000 synthetic medical records with private information. Then, we finetuned Qwen3.5-4B and gpt-oss-20b with several methods to mitigate privacy leakage and asked two things: does the model learn from the medical records, and does it leak private information?
1/n 🧵👇
"The Privacy-Hallucination Tradeoff in Differentially Private Language Models" was accepted to EMNLP 2026 Findings!
We find and investigate why training models with DP causes them to hallucinate more, instead of simply abstaining from generating information they fail to learn.
A lot of people are using AI interviewers, but like most conversational AI, they’re hard to evaluate. That’s why we created InterviewPlayground, a simulation environment for evaluating AI interviewers. And we do what a lot of simulation benchmarks don’t…
I will present at EMNLP on ✨Oct 27 (9 AM) ✨about our work "Exploring Sparse Autoencoders in Text-Based Causal Confounding Adjustment". Many causal questions hinge on texts. For example, does the type of media coverage of a bill affect the number of legislators who vote for it?
Tomorrow I’ll be presenting our work “What Makes a Good Response? An Empirical Analysis of Quality in Qualitative Interviews" at EMNLP 2026. My talk will be at 4pm in Oral Session 4, Room G7-8. Come check it out, or send me a DM and we can grab coffee!
Check out @ZihaoZhao1's thread on our new paper!
We compare various techniques to mitigate privacy leakage, finding that synthetic data and LLM rewriting offers a particularly strong privacy/capability trade-off 📈📈
Can an LLM judge pick your synthetic data for you? Only partially.
LLM-judged privacy scores are nicely correlated with privacy leakage metrics (ρ = 0.89). But its judged quality scores are even negatively correlated with capability metrics (ρ = −0.64), so don’t rely on it to judge capability.
6/n
Labs and regulators need reliable, scalable ways to measure how much harm frontier models can actually enable for catastrophic AI misuse.
@KamacheeMax and I believe this is an urgent and underexplored area where better research could help prevent serious incidents before they happen, and we're leading a SPAR stream this fall to carry out these impactful projects. Through the fellowship, we want to improve uplift measurement, propose new ways of measuring harm uplift, and build defenses that providers can use before these attacks scale.
Apply by August 18:
https://t.co/bdhKf3smIi
New #ICML2026 paper!
A popular claim says machine-text detection is hopeless since detectors get defeated by cheap attacks (paraphrasing, prompting, DPO). We show the picture is more subtle. 👇
📄 https://t.co/yLCOaMwxNr
💻 https://t.co/F33JaFQVFK
Join us on Wednesday, June 3 for a lecture with Johns Hopkins computer scientist Anjalie Field on the Practical Guidance for the Ethical Use of AI in Research. Presented in collaboration with the BEAD Core and @HopkinsDSAI, this event is free and open to all. https://t.co/Z3tPQKsJMJ
🤖 We often talk about “context rot”: LLMs get worse as context grows.
But once distracting information enters, is it just “a bit more noise → a bit worse performance”?
Our #ICML2026 paper finds: no! 🤯 Instead, we reveal a striking "First Drop of Ink" effect: the first very few hard distractors do almost all of the damage, exactly like how one drop of ink clouding clear water.
Paper link: https://t.co/s3zg49imUr
Real-world agents juggle instructions from skill files, tools, other agents, ... each with different trust levels. When these conflict, can models reliably prioritize the most trusted one?
Our ManyIH-Bench🪜 finds that even frontier models like GPT-5.4 only get ~40% accuracy! 👇