Inseq

Open-Source Interpretability for Generative Language Models 🔎 🐛

[Inseq @InseqLib]

Hello world! 🐛

[Gabriele Sarti @gsarti_]

After a year of restless development, I'm finally happy to announce Inseq, a new tool to democratize post-hoc interpretability of sequence generation models 🐛 https://t.co/YlxGuW4bVO #nlproc #xai

Some highlights 👇 1/

[Gabriele Sarti @gsarti_]

Model Internals-based RAG Evaluation (MIRAGE) 🌴 is accepted to #EMNLP2024 Main!

➡️ To celebrate, here's our new MIRAGE demo combining @InseqLib and Transformers-specific LRP: https://t.co/wPgnFNAJYk.

Reach out if you want to catch up in Miami! 🤗🏖️

[Gabriele Sarti @gsarti_]

Very hyped for the new beautiful viz that just landed in the @InseqLib main branch! 🔥 This will empower users to explore attribution tensors more flexibly and intuitively.

h/t to @_ddjohnson for his awesome work on the treescope toolkit powering this release!

[Inseq @InseqLib]

Thanks to the new treescope integration, @InseqLib now supports interactive visualizations for multidimensional attributions (show_granular), token highlights (show_tokens) and improved viz for attribute_context CLI! 🚀 Install main, will appear in v0.7

https://t.co/O2PxXBIr4o

[Daniel Johnson @_ddjohnson]

By popular demand, the Treescope pretty-printer from the Penzai neural net library can now be installed separately, and supports both JAX and PyTorch!

And that's not all: Penzai itself now has less boilerplate and includes more pretrained Transformer models!

[CIS, LMU Munich @CisLmu]

🎓 We're thrilled to host Gabriele Sarti (@gsarti_) in our PhD seminar series tomorrow, July 16th, from 12:00-13:00 in Oe67 BU 101! Join us for his talk on interpreting context usage in generative language models, featuring the Inseq toolkit and PECoRe framework. 🕛Don't miss it!

[Inseq @InseqLib]

The 🐑 PECoRe / 🌴 MIRAGE demo on @huggingface Spaces is powered by our new attribute-context CLI command released in v0.6, and allows to export the code to reproduce your results locally with 🐛 Inseq.

Check it out ➡️ https://t.co/txaCqxgyuP

[Gabriele Sarti @gsarti_]

⚠️ Citations from prompting or NLI seem plausible, but may not faithfully reflect LLM reasoning.

🏝️ MIRAGE detects context dependence in generations via model internals, producing granular and faithful RAG citations.

🚀 Demo: https://t.co/OMeM32TNoY

Fun collab w/ @Jirui_Qi, @AriannaBisazza & @raquel_dmg! Check it out ⬇️

[Eliana Pastor @eliana__pastor]

Today, we had the first seminar of our #XAI course!

@gsarti_ presented the @InseqLib to interpret LMs and the PECORE framework to identify & attribute context dependence in LMs! 🚀🌟

Thank you, it was so interesting! 🤗
Great start to our series!

https://t.co/YY8zpCcUMO

[Javier Ferrando @javifer_96]

[1/4] Introducing “A Primer on the Inner Workings of Transformer-based Language Models”, a comprehensive survey on interpretability methods and the findings into the functioning of language models they have led to.

ArXiv: https://t.co/UBh2ZLmYkr

[Inseq @InseqLib]

@MorenoLaQuatra And you haven't found the easter egg yet! 😇

[Inseq @InseqLib]

Today @InseqLib hit 300 ⭐️ on Github!
A huge thank you to all our awesome users ❤️ Onwards to the next 300! 🤺

[Inseq @InseqLib]

@InseqLib v0.6 is out now on PyPI! 🔥

New CLI command for context attribution (@gsarti_), new perturbation-based methods by @hmohebbi75
& @casszzx and optimizations incl. multi-gpu support! ⚡️

Huge shoutout to our contributors! ❤️
Release notes ⬇️
https://t.co/onjSKzeQqY

[Inseq @InseqLib]

Want to learn how to detect and attribute context usage in LMs using @InseqLib? This @Gradio demo using our CLI can teach you how! 🐛

[Gabriele Sarti @gsarti_]

The official 🐑 PECoRe 🐑 demo to detect and attribute context dependence in LM generations is now available on @huggingface Spaces! 🚀

Includes code examples, a usage guide, useful presets for various dec-only & enc-dec models, and more! Check it out ⬇️ https://t.co/ZgQm3LdqDq

[Gabriele Sarti @gsarti_]

🐑 PECoRe repository is now public (https://t.co/cxoyWeMLql) and all model/datasets are available on @huggingface (https://t.co/h6QllPB1Xg)!

🐑Interested in using PECoRe on your models? Have a look at the @InseqLib implementation (`inseq attribute-context`)!

[Inseq @InseqLib]

Tracker for Mamba gradient-based methods support: https://t.co/yYwsfqSzMg

[Inseq @InseqLib]

Did someone say Mamba support? 👀

(Actually gradient-based methods are not working out-of-the-box because of some in-place variable overwriting, but should be fixable!)

[Arthur Zucker @art_zucker]

`mamba` is now available in transformers. PEFT finetuning example: https://t.co/2vPQBgA9iu
Thanks @_albertgu and @tri_dao for this brilliant model! 🚀 and the amazing `mamba-ssm` kernels powering this!

[Gabriele Sarti @gsarti_]

Excited to present my recent work on @InseqLib and the #ICLR2024 PECoRe interpretability framework for the @SheffieldNLP group this afternoon! Many thanks @casszzx for inviting me! 🤗

🐛 Inseq: https://t.co/YlxGuW4bVO
🐑🐑 PECoRe: https://t.co/e5takAk53n

[Hosein Mohebbi @hmohebbi75]

Wow!! This is great news🥳
Thanks @InseqLib! I’m excited to try it out!

[Inseq @InseqLib]

@huggingface Authors: @hmohebbi75 @wzuidema @gchrupala @afraalishahi

[Inseq @InseqLib]

Value Zeroing, a faithful approach for analyzing context mixing in Transformers, is now available on @InseqLib main branch for all @huggingface text generation models! 🔀

🔍Paper introducing VZ: https://t.co/lbSwujzWli
🐛VZ in Inseq: https://t.co/sL76YxwdbO

[Inseq @InseqLib]

After two years with Poetry, Inseq just moved to @astral_sh's new blazing fast package manager uv! Our CI installation step is now ~80% faster! 🔥
Congrats to @charliermarsh and the team on the release, and godspeed for the cargo-like experience you are planning to build!

[Inseq @InseqLib]

Interested in using @InseqLib for your work? Refer to our new tutorial to get started ➡️ https://t.co/BOHz8vVjAC 8/8

[Inseq @InseqLib]

Mousavi et al. attribute the generation of dialogue models, finding increased influence for more refined versions of dialogue history https://t.co/tJNn5WahN6 (@mahedmousavi S. Caldarella G. Riccardi @sislab7) 3/

[Inseq @InseqLib]

Wang et al. propose a metric for LMs factual reliability and show its relation to models’ sensitivity to in-context distractors https://t.co/1FhGXXxqDw (W. Wang @bazril @alexandrabirch1 W. Peng @EdinburghNLP) 7/