Rishabh Bhardwaj

Aligning LLMs to be safer! A researcher and a founder.

[Rishabh Bhardwaj @rishabh15]

📣📣 New research on AI safety

Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmetic

We are introducing a new approach to prevent LLM's compromised safety

Paper: https://t.co/Q71COggzQ5
Code and safety evaluation dataset: https://t.co/FRLyFjIdmt
Data: https://t.co/FkGoTajzYL

Did you know enterprise adaptation of LLMs comes at the cost of compromised safety? If not, our research might be interesting to you.

We propose RESTA: Restoring Safety through Task Arithmetic.

👉 RESTA is a simple, fast, and effective approach that provides a no-cost solution to model safety realignment. At the core of RESTA, we add a homegrown safety vector to the model to bring back its safety while maintaining its task performance.

👉 Our evaluations of fine-tuned Llama-2 on CatQ (a multilingual safety evaluation benchmark we created) show a sharp drop in unsafety score from 33.57% to 12.17% in PEFT and from 22.16% to 4.34% in Full-FT averaged across fine-tuning domains.

👉 To gauge the effect of the safety vector beyond categories in CatQ, we evaluate RESTA on three existing safety evaluation benchmarks— HarmfulQ, AdevrsarialQ, and DangerousQ. We observe a reduction in the unsafety score from 18.59% to 5.14% in PEFT and from 9.16% to 1.55% in Full-FT when averaged across benchmark datasets and fine-tuning domains.

👉 The effectiveness of RESTA is evident across languages, as seen in the 26.2% reduction in PEFT and 21.37% reduction in Full-FT on the Vietnamese CatQ. Similar improvements are observed for CatQ in Chinese, with a reduction of 17.35% in PEFT and 24.54% in Full-FT.

For more interesting information, please have a look at our work at https://t.co/R6LkCeHJN9.

Thanks to my great collaborators, @soujanyaporia and Duc Anh Do!

#NLProc #Safety #LLM @llm_sec @topofmlsafety @cohere @dair_ai and hoping @seb_ruder @seraphinagt @jeremyphoward @omarsar0 like it.

[Artificial Intelligence Papers @SciFi]

Post Reasoning: Improving the Performance of Non-Thinking Models at No Cost

Richmond Sin Jing Xuan, Rishabh Bhardwaj, Soujanya Poria
https://t.co/bDOvH9YD9m [𝚌𝚜.𝙰𝙸]

[Varun Gumma @VarunGumma23]

[1/n] What if your AI agents stop following your instructions?

That is what we study extensively in our research: https://t.co/F3NkF5yhF5

[Shang Hong Sim @shanghong_sim]

I am excited to announce that Trust-Align has been accepted to @iclr_conf ! 🎉

📄 Paper: https://t.co/HEevK54i43
💻 Code: https://t.co/0t4wUZO1rI

Heartfelt thanks to my co-author Maojia Song and to @rishabh15, Navonil Majumder, and @soujanyaporia for their mentorship!

[Rishabh Bhardwaj @rishabh15]

Money hasn’t been the issue...

Proof?: There are startups in India that have received 40-50 times this amount to accomplish what he is asking for.

[Aravind Srinivas @AravSrinivas]

I am ready to invest a $1mm personally and 5 hours/week of my time into the most qualified group of people that can do this right now for making India great again in the context of AI. Consider this as a commitment that cannot be backtracked. The team has to be cracked and obsessed like DeepSeek team and has to open source the models with MIT license.

[Paul Röttger @paul_rottger]

Today, we are releasing MSTS, a new Multimodal Safety Test Suite for vision-language models!

MSTS is exciting because it tests for safety risks *created by multimodality*. Each prompt consists of a text + image that *only in combination* reveal their full unsafe meaning.

🧵

[Haonan Li @haonanlp]

We are excited to announce Libra-Leaderboard: The first LLM leaderboard dedicated to balancing safety and capability in LLMs. As AI advances, ensuring its safety becomes more critical than ever. By prioritizing safety measurement, we aim to inspire the AI community to make safety a core factor in model evaluation and development, alongside capability.🚀
#LLM #AI #AISafety #ResponsibleAI

Website: https://t.co/bpykdt2shH
Paper: https://t.co/VfzZLrsHX6

Thanks to all the contributors: @han_xudong @_ZenanZhai @honglin_mu @_HaoWang @_ZhenxuanZhang @yilin_link @ShomLinEd @realReasonWang @ArtemShelmanov @xiangyuqi_pton @_YuxiaWang @_DonghaiHong @youliang_yuan @_MengChen @HaoqinT @FajriKoto @ttk_kuribayashi @HaoqinT @rishabh15 @BingchenZhao @yawen_duan @_YiLiu @_YaodongYang @dongyp13 @soujanyaporia @stefan_fee @waterluffy @RenHector @Emad_A_Alghamdi @IGurevych @preslav_nakov @monojitchou @eltimster

[Vrushank @retrovrv]

Productionizing Gen AI is as much a "platform engineering" challenge as it is an AI challenge

This was clear at our @PortkeyAI practitioners' dinner in 🇸🇬 where we had AI leads from Mediacorp, OCBC Bank & others share their real production stories..

[Rishabh Bhardwaj @rishabh15]

Researchers need to read more related work, especially studies published months earlier that are highly relevant should be properly cited.

[Rada Mihalcea @radamihalcea]

The new GSM-Symbolic paper from Apple has been making waves, but we published very similar findings earlier this year. Using nearly the same symbolic template methodology on GSM8k problems, we demonstrated the reasoning limitations of LLMs.

https://t.co/HakKBlowmf

[Rada Mihalcea @radamihalcea]

The new GSM-Symbolic paper from Apple has been making waves, but we published very similar findings earlier this year. Using nearly the same symbolic template methodology on GSM8k problems, we demonstrated the reasoning limitations of LLMs.

https://t.co/HakKBlowmf

[Gabriel Chua @gabrielchua]

this week at ai wednesdays, we had @rishabh15 and Tej share more about their research on automated red teaming & guardrails ~

we still have some slots for sharings in nov ~ drop me a dm or reply here if you’re keen

[Sumit @_reachsumit]

Measuring and Enhancing Trustworthiness of LLMs in RAG through Grounded Attributions and Learning to Refuse

Introduces a metric for evaluating LLM trustworthiness in RAG systems, and a framework to improve grounded responses.

📝https://t.co/LQVedQqYMd
👨🏽‍💻https://t.co/Nqjr3nqLvj

[Rishabh Bhardwaj @rishabh15]

Read our work on automated red-teaming!

This time we made it faster adn more performant than Meta's solution.

[Soujanya Poria @soujanyaporia]

Interested in LLM safety? Please check out our latest work where we introduced Ferret, a reward model driven faster, more effective, and transferrable automated red-teaming approach. It is open-sourced!

Paper: https://t.co/DqvMc5YXJR
Code: https://t.co/LWBvFEoqLb #LLMs

[Rishabh Bhardwaj @rishabh15]

It's a very common issue, and it's not the first time I have seen it. The same applies to Indian AI investments, hype and marketing skills bring in more money than quantitative proofs of the technology.

[Danish Pruthi @danish037]

I have been meaning to call out all the bullsh*t that @NirantK is all about! This thread does a good job.

This “Top 5 GenAI scientist” has never contributed anything to science! No one would even recognize him if he were to attend a serious scientific conference.

[Rishabh Bhardwaj @rishabh15]

@prajdabre1 @psidharth567 Maybe you are one of the very few researchers not working with yet posting about them 🙂

[Dmitry Noranovich @javaeeeee1]

WalledEval: A Comprehensive Safety Evaluation Toolkit for Large Language Models

https://t.co/TiDdOJR7G3

[Daniel van Strien @vanstriendaniel]

WALLEDEVAL is an AI safety testing toolkit for large language models (LLMs). It supports open-weight and API-based models and features over 35 safety benchmarks, including multilingual safety and prompt injections. https://t.co/OaWSM0rkjO

[Rishabh Bhardwaj @rishabh15]

@agihippo Any area you think this does not apply to?

[Rishabh Bhardwaj @rishabh15]

@pratikpoddar Not everyone 😀
At least the research community tends to be happy in general
(those who are serious about research)

[Rishabh Bhardwaj @rishabh15]

@HarveenChadha To be honest, I was recently thinking about what you just typed here – all three points, and more interestingly, the last one.

[Rishabh Bhardwaj @rishabh15]

@Krutrim, @bhash

Do reach out to us if you want a systematic way to identify such non-idealities. 🙂

We are researchers who collectively formed #WalledAI to help make systems safer for enterprises and their users.