Ian Tenney (@[email protected])

Staff Research Scientist, People + AI Research @GoogleAI #GoogleResearch. Interpretability, analysis, and visualizations for LLMs. Opinions my own.

[Hadas Orgad @ ICML @OrgadHadas]

Excited that our paper on Actionable Interpretability got accepted to ICML!
And just in time -- we also heard that our Actionable Interpretability workshop will be happening again, in COLM!
See you in Korea 🇰🇷 and SF🌉
[Arxiv paper link in the comment]

[Ian Tenney (@[email protected]) @iftenney]

More here: https://t.co/h8XinkE9eS

[Ian Tenney (@[email protected]) @iftenney]

At ICML? Interested in how we can do more with interpretability to have practical impact on the rest of AI? Come to our workshop this Saturday!

[Hadas Orgad @ ICML @OrgadHadas]

Hope everyone’s getting the most out of #icml25. We’re excited and ready for the Actionable Interpretability (@ActInterp) workshop this Saturday!
Check out the schedule and join us to discuss how we can move interpretability toward more practical impact.

[Tal Haklay @tal_haklay]

🚨Meet our panelists at the Actionable Interpretability Workshop @ActInterp at @icmlconf!

Join us July 19 at 4pm for a panel on making interpretability research actionable, its challenges, and how the community can drive greater impact.
@nsaphra @saprmarks @kylelostat @FazlBarez

[Mor Geva @megamor2]

Going to #icml2025? Don't miss the Actionable Interpretability Workshop (@ActInterp)! We've got an amazing lineup of speakers, panelists, and papers, all focused on leveraging insights from interpretability research to tackle practical, real-world problems ✨

[Ian Tenney (@[email protected]) @iftenney]

Submission deadline extended to May 19!

Working on or have thoughts on real-world applications of interpretability, and how we can use it in practice? Consider submitting to our workshop at ICML 2025. More at https://t.co/KnpWajCUuT and below👇

[Hadas Orgad @ ICML @OrgadHadas]

🎉 Our Actionable Interpretability workshop has been accepted to #ICML2025! 🎉
>> Follow @ActInterp

@tal_haklay @anja_reu @mariusmosbach @sarahwiegreffe @iftenney @megamor2

Paper submission deadline: May 9th!

[Tyler Chang @tylerachang]

Presenting our work on training data attribution for pretraining this morning: https://t.co/aypbwhN27I -- come stop by in Hall 2/3 #526 if you're here at ICLR!

[Ian Tenney (@[email protected]) @iftenney]

Check out our new work on scaling training data attribution (TDA) toward LLM pretraining - and some interesting things we found along the way.
https://t.co/rbxcL4qoYb and more below from most excellent student researcher @tylerachang ⬇️

[Tyler Chang @tylerachang]

We scaled training data attribution (TDA) methods ~1000x to find influential pretraining examples for thousands of queries in an 8B-parameter LLM over the entire 160B-token C4 corpus!
https://t.co/4mglIOAjyB

[Ian Tenney (@[email protected]) @iftenney]

Try out LLM Comparator to help make sense of LLM evaluations!

Alongside the Gemma 2 launch, we've released a Python library to help run rater models, bulletizing, and clustering - so all you need are prompts and model responses. Scripts and notebooks at https://t.co/mQ93xx4z3F

[Minsuk Kahng @minsukkahng]

With the Gemma 2 launch, we’ve updated our LLM Comparator tool with a Python library! This lets you more easily run the tool with your own models. https://t.co/YzJlo3TDBf

[Ian Tenney (@[email protected]) @iftenney]

New open-source tool: LLM Comparator can help make sense of side-by-side model evals. In-browser demo at https://t.co/Dhqw09pz8Y, and read more in the blog post below.

[Minsuk Kahng @minsukkahng]

Very excited to open-source LLM Comparator!
This new #visualization tool lets you analyze LLM responses side-by-side. It’s been used for evaluating LLMs @Google, and we're proud to release it as part of Google's Responsible GenAI Toolkit.
https://t.co/xXwvBzlz2g

[Ian Tenney (@[email protected]) @iftenney]

Very, very cool work from @ghandeharioun and colleagues - Patchscopes is a super flexible way to decode knowledge from internal states of an LLM, bridging between interpretability, causality, and control.

[Google AI @GoogleAI]

Being able to interpret an #ML model’s hidden representations is key to understanding its behavior. Today we introduce Patchscopes, an approach that trains #LLMs to provide natural language explanations of their own hidden representations. Learn more → https://t.co/WfY1FYa1Wt

[Jeff Dean @JeffDean]

We're also releasing a Responsible Generative AI Toolkit that provides resources to apply best practices for responsible use of open models such as the Gemma models, including:

Guidance on setting safety policies, safety tuning, safety classifiers and model evaluation.

The Learning Interpretability Tool (LIT) for investigating Gemma's behavior and addressing potential issues.

A methodology for building robust safety classifiers with minimal examples.

https://t.co/cnA6dJtChn

[Ian Tenney (@[email protected]) @iftenney]

Super excited for the Gemma model release, and with it a new debugging tool we built on 🔥LIT - use gradient-based salience to debug and refine complex LLM prompts! https://t.co/m0fsFwhPip

[Google for Developers @googledevs]

Explore Gemma model’s behavior with The Learning Interpretability Tool (LIT), an open-source platform for debugging AI/ML models.

➡️ Improve prompts using saliency methods
➡️ Test hypotheses to improve model behavior
➡️ Democratize access to ML debugging

https://t.co/pEyQAi75nk

[Ian Tenney (@[email protected]) @iftenney]

@osanseviero @iislucas @GoogleAI @osanseviero great idea! Will see if I can get a space set up. Can I DM if I hit any snags?

[Ian Tenney (@[email protected]) @iftenney]

🧵(1/6): Excited to announce the v1.0 release of the @GoogleAI Learning Interpretability Tool (🔥LIT), an interactive platform to debug, validate, and understand ML model behavior. This release brings exciting new features and a simplified Python API. https://t.co/vCfjONCfXh

[Ian Tenney (@[email protected]) @iftenney]

🧵(6/6): Check out the full release notes at https://t.co/YDbxoyYjdQ

If you find LIT useful for your project, please cite our EMNLP paper (https://t.co/nPRbQTKtK9), and drop us a line on GitHub or Twitter!

[Ian Tenney (@[email protected]) @iftenney]

🧵(5/6): Many thanks to all of the contributors who supported this release: @RyanMullins , @nadamused_ , @minsukkahng , @mahimapushkarna , @cjqian , James Wexler, Bin Du, Cibi Arjun, and Oscar Wahltinez - as well as all those who have helped build LIT since the project began.