Kai Chen

Ph.D. student @USCViterbi @USC_ISI. Computational Social Science & Natural Language Processing.

[Kai Chen @kaichen23]

Excited for #EMNLP2025 in Suzhou 🎋 this week! Join me on Nov 5 (Wed) from 11:00 to 12:30 in poster session 2 for this paper. Feel free to reach out!

[Kai Chen @kaichen23]

Our benchmark paper "STEER-Bench" was accepted by #EMNLP2025 Main Conference🥳. Huge thanks to my collaborators!

[Kai Chen @kaichen23]

Our benchmark paper "STEER-Bench" was accepted by #EMNLP2025 Main Conference🥳. Huge thanks to my collaborators!

[Kai Chen @kaichen23]

🤔How well do LLMs adapt to different norms?
🧵We introduce STEER-BENCH, a benchmark for assessing steerability in LLMs.
📉 Human: 81% | Top LLM: ~65%
🚨 Norm alignment ≠ solved.
📄 Paper: https://t.co/YG4uDp37Zk
@ZihaoHe95 @taiwei_shi @KristinaLerman

[Kai Chen @kaichen23]

🔍We evaluate 13 popular LLMs under both in-context learning and supervised fine-tuning settings, revealing that steerability improves with model scale and contextualized prompting, but varies significantly by model family and domain.

[Kai Chen @kaichen23]

🤔How well do LLMs adapt to different norms?
🧵We introduce STEER-BENCH, a benchmark for assessing steerability in LLMs.
📉 Human: 81% | Top LLM: ~65%
🚨 Norm alignment ≠ solved.
📄 Paper: https://t.co/YG4uDp37Zk
@ZihaoHe95 @taiwei_shi @KristinaLerman

[Kai Chen @kaichen23]

🍀STEER-BENCH evaluates community-specific steerability in LLMs across 30 subreddit pairs and 19 domains, covering over 10,000 instruction-response pairs and 5,500 multiple-choice questions grounded in contrasting online communities.

[J Huang @JentseHuang]

#EMNLP2024 @nlp_usc @CSatUSC

[Kai Chen @kaichen23]

Excited for #EMNLP2024 in Miami 🌴next week! Join me on Nov 12 (Tue) from 16:00 to 17:30 in poster session 4 for this paper. Feel free to reach out!

[Kai Chen @kaichen23]

🤔How susceptible are LLMs to Ideological Manipulation?⭕️
🧐We find a concerning vulnerability: only a small amount of ideologically driven samples significantly alters the ideology of LLMs.🤖⚠️
🔗https://t.co/hfIyGHLcPH
@ZihaoHe95 @jun_yannn @taiwei_shi @KristinaLerman
🍀1/4

[Kai Chen @kaichen23]

Our work "How Susceptible are Large Language Models to Ideological Manipulation?" was accepted by #EMNLP2024 Main Conference🥳. Huge thanks to my collaborators!

[Kai Chen @kaichen23]

🤔How susceptible are LLMs to Ideological Manipulation?⭕️
🧐We find a concerning vulnerability: only a small amount of ideologically driven samples significantly alters the ideology of LLMs.🤖⚠️
🔗https://t.co/hfIyGHLcPH
@ZihaoHe95 @jun_yannn @taiwei_shi @KristinaLerman
🍀1/4

[Taiwei Shi @taiwei_shi]

Excited for #NAACL2024 in Mexico 🇲🇽 next week! Join me on June 19 from 11:00 AM to 12:30 PM in DON ALBERTO 1 for my talk on Safer-Instruct. Let's dive into alignment, synthetic data, and more!

[Taiwei Shi @taiwei_shi]

Honored to receive the 🏆 𝐛𝐞𝐬𝐭 𝐩𝐚𝐩𝐞𝐫 𝐫𝐮𝐧𝐧𝐞𝐫-𝐮𝐩 at the ICLR SeT LLM workshop! I will be giving a talk on this work on May 11th, 15:30, Schubert 6. Let's talk about AI Safety there! 🔐
Paper: https://t.co/dL4cL7FFr1
Event: https://t.co/qGyW2gdmuM

[Kai Chen @kaichen23]

Huge thanks to my amazing collaborator @ZihaoHe95 @jun_yannn @taiwei_shi @KristinaLerman

[Kai Chen @kaichen23]

🥳Exciting News! Our work, 🤖"How Susceptible are Large Language Models to Ideological Manipulation?" got 🏆𝐁𝐞𝐬𝐭 𝐏𝐚𝐩𝐞𝐫 𝐑𝐮𝐧𝐧𝐞𝐫-𝐮𝐩 at SET LLM #ICLR Workshop.
Check our work here: https://t.co/hfIyGHLcPH
Check the workshop here: https://t.co/j9Fi4dMf3G

[Kai Chen @kaichen23]

🤔How susceptible are LLMs to Ideological Manipulation?⭕️
🧐We find a concerning vulnerability: only a small amount of ideologically driven samples significantly alters the ideology of LLMs.🤖⚠️
🔗https://t.co/hfIyGHLcPH
@ZihaoHe95 @jun_yannn @taiwei_shi @KristinaLerman
🍀1/4

[Taiwei Shi @taiwei_shi]

Excited to get Safer-Instruct accepted to NAACL 2024 🥳! You don’t want to miss it if you want to reduce cost and boost efficiency in preference data acquisition 🚀. Check out our framework and dataset here: https://t.co/W04MLhddz1

[Kai Chen @kaichen23]

👩‍🏫We finetune two LLMs—Llama-2-7B and GPT-3.5—based on IDEOINST and assess their ideological bias shift after the manipulation.📊
👉Both models exhibit the expected biases across the majority of topics.🙌
🍀4/4

[Kai Chen @kaichen23]

🤔How susceptible are LLMs to Ideological Manipulation?⭕️
🧐We find a concerning vulnerability: only a small amount of ideologically driven samples significantly alters the ideology of LLMs.🤖⚠️
🔗https://t.co/hfIyGHLcPH
@ZihaoHe95 @jun_yannn @taiwei_shi @KristinaLerman
🍀1/4

[Kai Chen @kaichen23]

🚀we propose a dataset named IdeoINST comprised of around 6,000 opinion-eliciting instructions across six sociopolitical topics, each paired with dual responses—one reflecting a ⬅️left-leaning bias and one reflecting a ➡️right-leaning bias.👈
🍀3/4

[Taiwei Shi @taiwei_shi]

🤔Enhancing LLM with RLHF is powerful, but ever wondered how to reduce costs and boost efficiency in preference data acquisition? 💰
🚀Introducing Safer-Instruct, a groundbreaking pipeline that complements humans to construct large-scale preference datasets efficiently.
🧵1/5