Somnath Basu Roy Chowdhury

Research Scientist at Google Research

[Somnath Basu Roy Chowdhury @SomnathBrc]

𝐇𝐨𝐰 𝐜𝐚𝐧 𝐰𝐞 𝐩𝐞𝐫𝐟𝐞𝐜𝐭𝐥𝐲 𝐞𝐫𝐚𝐬𝐞 𝐜𝐨𝐧𝐜𝐞𝐩𝐭𝐬 𝐟𝐫𝐨𝐦 𝐋𝐋𝐌𝐬?

Our method, Perfect Erasure Functions (PEF), erases concepts from LLM representations w/o parameter estimation, achieving pareto optimal erasure-utility tradeoff w/ guarantees. #AISTATS2025 🧵

[Somnath Basu Roy Chowdhury @SomnathBrc]

@prateeky2806 @AIatMeta Congratulations 😀

[Snigdha Chaturvedi @snigdhac25]

@SomnathBrc is at #ICLR2025 presenting our work on perfect unlearning. Find him!

[Somnath Basu Roy Chowdhury @SomnathBrc]

@snigdhac25 (9/n) I’m attending ICLR in person and presenting our poster on 25th April in Poster session 3 between 10AM-1230PM. Please feel free to stop by our poster if you’re interested.

I’m also happy to chat about unlearning or AI safety in general.

cc: @uncnlp @unccs

[Somnath Basu Roy Chowdhury @SomnathBrc]

(4/n) To prevent this, we propose training the model on multiple sequences (or permutations) of the same data. While functioning within a budget, we generate diverse permutations using cyclic rotation when no prior deletion information is available.

[Somnath Basu Roy Chowdhury @SomnathBrc]

(8/n) Finally, I would like to thank all my amazing co-authors: Krzysztof, Arijit, Avinava, and @snigdhac25.

Code: https://t.co/LWoy5cT3ZL
Paper link: https://t.co/0q02GTPT1B

[Somnath Basu Roy Chowdhury @SomnathBrc]

(3/n) We can easily unlearn data using S³T by switching off LoRA layers below the affected module. However, when the topmost module is affected in the rare scenario, we may need to retrain from scratch.

[Somnath Basu Roy Chowdhury @SomnathBrc]

(2/n) We propose a sequential fine-tuning strategy that trains individual PEFT layers using different data subsets. This helps convert LLMs into a modular system that is helpful in executing exact unlearning.

[Somnath Basu Roy Chowdhury @SomnathBrc]

𝐇𝐨𝐰 𝐜𝐚𝐧 𝐰𝐞 𝐩𝐞𝐫𝐟𝐞𝐜𝐭𝐥𝐲 𝐮𝐧𝐥𝐞𝐚𝐫𝐧 𝐝𝐚𝐭𝐚 𝐟𝐫𝐨𝐦 𝐋𝐋𝐌𝐬 𝐰𝐡𝐢𝐥𝐞 𝐩𝐫𝐨𝐯𝐢𝐝𝐢𝐧𝐠 𝐠𝐮𝐚𝐫𝐚𝐧𝐭𝐞𝐞𝐬?

We present S³T, a scalable unlearning framework that guarantees data deletion from LLMs by leveraging parameter-efficient fine-tuning. #ICLR2025 🧵

[Ahmad Beirami @abeirami]

Finally, if you are also going to #AISTATS2025, @SomnathBrc will be presenting 𝐩𝐞𝐫𝐟𝐞𝐜𝐭 𝐜𝐨𝐧𝐜𝐞𝐩𝐭 𝐞𝐫𝐚𝐬𝐮𝐫𝐞.

Somnath will be at ICLR 2025. Please catch him and talk to him if you are interested in concept erasure / unlearning / etc.
https://t.co/CF47Jd4EuT

[Somnath Basu Roy Chowdhury @SomnathBrc]

(9/n) Finally, I would like to thank all my amazing co-authors: Avinava, @abeirami, Rahul, @nicholasmonath, Amr, @snigdhac25.

cc: @uncnlp @unccs

[Somnath Basu Roy Chowdhury @SomnathBrc]

𝐇𝐨𝐰 𝐜𝐚𝐧 𝐰𝐞 𝐩𝐞𝐫𝐟𝐞𝐜𝐭𝐥𝐲 𝐞𝐫𝐚𝐬𝐞 𝐜𝐨𝐧𝐜𝐞𝐩𝐭𝐬 𝐟𝐫𝐨𝐦 𝐋𝐋𝐌𝐬?

Our method, Perfect Erasure Functions (PEF), erases concepts from LLM representations w/o parameter estimation, achieving pareto optimal erasure-utility tradeoff w/ guarantees. #AISTATS2025 🧵

[Somnath Basu Roy Chowdhury @SomnathBrc]

(8/n) Here is a blog post with a simplified overview of our work: https://t.co/iYcljXaAZP

Code: https://t.co/owoZEhEHCY
Paper link: https://t.co/fiJtpXJKjo