Mansi Sakarvadia

Bluesky: @mansisakarvadia.bsky.social

Computer Science/Machine Learning Ph.D. Student @UChicago & @globus. @doecsgf Computational Science Graduate Fellow.

[FORTUNE @FortuneMagazine]

"Without long-term, foundational, and high-risk federal research investments, the seeds of innovation cannot take root," Rebecca Willett and Henry Hoffman write in a new commentary piece. https://t.co/n8lscVfZid

[Mansi Sakarvadia @Mansi__S]

Check out a recent interview in which I discuss the recent Nobel Prizes and some thoughts on the impact on both the domain sciences and ML communities.

[Science in Parallel @scienceinparall]

New 🎙️episode: @Mansi__S of @UChicago and @JoshVermaas of @MSUDOEPlantLab discuss the #AI #Nobels and their impact on computing and research: https://t.co/PJOhOHC4oE Both are part of the @doecsgf community. #HPC

[Mansi Sakarvadia @Mansi__S]

Reflecting on my 2024 PhD journey: passed my qualifying exam, spent the summer at Berkeley, mentored undergrad students, and tackled the fast pace of AI/ML research. It’s been a year of milestones and growth! Read more here: https://t.co/8LiZaZd8Rn #PhDJourney #AIResearch

[Mansi Sakarvadia @Mansi__S]

Congrats to Jordan for winning 1st place at SC24 student poster competition! It was super fun to mentor him this summer on his project "Mind Your Manners: Detoxifying Language Models via Attention Head Intervention".

[Nathaniel Hudson @Nchudson95]

I am super proud of Jordan Pettyjohn, an undergraduate student I had the privilege of working with this past summer, for winning the Student Research Competition at the @Supercomputing conference! 🏆

His work studied ablation strategies for toxicity in #LLMs.

[𝚐𝔪𝟾𝚡𝚡𝟾 @gm8xx8]

Towards Interpreting Language Models: A Case Study in Multi-Hop Reasoning

paper: https://t.co/03JhOvpZep

This method improves multi-hop reasoning in language models by injecting “memories” into key attention heads, increasing accuracy in complex tasks. An open-source tool, Attention Lens, interprets attention outputs, helping trace the model’s reasoning and pinpoint issues like bias or harmful content.

[Globus Labs @LabsGlobus]

Excited to share our latest work: "SoK: On Finding Common Ground in Loss Landscapes Using Deep Model Merging Techniques"! 🧠
https://t.co/XAG79rS3ij

By @arhamk1216, @toddknife, @Nchudson95, @Mansi__S, @dcgrzenda, @aswathy__ajith, Jordan Pettyjohn, @chard_kyle, and @ianfoster.

[Mansi Sakarvadia @Mansi__S]

7/ 🙏 Special Thanks:
A huge shoutout to my incredible co-authors from multiple institutions for their contributions to this work: @aswathy__ajith, Arham Khan, @Nchudson95, @calebgeniesse, @nsfzyzz, @chard_kyle, @ianfoster, Michael Mahoney

[Mansi Sakarvadia @Mansi__S]

1/🧵New Research on Language Models!
Language models (LMs) often "memorize" data, leading to privacy risks. This paper explores ways to reduce that!
Paper: https://t.co/OBtYz9mJON
Code: https://t.co/x0C5I77CG3
Blog: https://t.co/nA6AH5rnXV

[Mansi Sakarvadia @Mansi__S]

6/ 🌍 Scalable Impact:
Our methods aren’t just for small models! We show that they scale effectively to larger LMs, providing robust memorization mitigation without compromising performance across different sizes of models. Exciting progress for real-world applications!

[Globus Labs @LabsGlobus]

Language models can memorize sensitive data! 🔒 Our new research by the team (@Mansi__S, @Nchudson95, and others) with TinyMem shows unlearning methods like BalancedSubnet effectively mitigate memorization while keeping performance high. #AI #Privacy
https://t.co/usBb1wLSID

[Globus Labs @LabsGlobus]

Jordan Pettyjohn, @Nchudson95, @Mansi__S, @aswathy__ajith, and @chard_kyle just published new work demonstrating detoxification strategies on Language Model outputs at @BlackboxNLP!

""Mind Your Manners: Detoxifying Language Models via Attention Head Intervention"

Congrats All!

[Mansi Sakarvadia @Mansi__S]

@doctagert Thanks, Adam!

[Mansi Sakarvadia @Mansi__S]

🎉 I successfully defended my Master's dissertation in the area of interpretable Language Modeling!

Check out my work's applications in better understanding multi-hop reasoning, bias localization, and malicious prompt detection in my talk: https://t.co/DGEvsfzawK

[Globus Labs @LabsGlobus]

@Mansi__S presented her Master's thesis on "Memory Injections: Correcting Multi-Hop Reasoning Failures during Inference in Transformer-Based Language Models". Watch the recording here: https://t.co/4tgQUgFDTT

[Ben Blaiszik @BenBlaiszik]

Interested in understanding how #LLM s work, why they often fail to reason, and how to improve performance? One tool to boost multi-hop reasoning is with targeted memory injections. This improves desired token probability by up to 424%!

🎥 Watch the talk by @Mansi__S now: https://t.co/ek1HIvDPXk

[Ben Blaiszik @BenBlaiszik]

✨Trillion Parameter Models in Science✨
We present an initial vision for a shared ecosystem to take the next step in large language models for scientific research – Trillion Parameter Models (TPMs). #LLM are becoming more powerful by the day. But, there is still work done to enable discovery of new therapeutics, materials, and physics with these tools. 🔬

📜 https://t.co/5Onyx4jTnD