Evan Hernandez

ph.d. @mit | building @openevidence (we're hiring!) | formerly @google @uwmadison | #nlproc and loud music

[Evan Hernandez @evanqed]

I will be at ICLR this week to share this work! Stop by and say hi, or let me know if you want to chat about LM interpretability :)

[Jacob Andreas @jacobandreas]

Linearity of relation decoding in transformer LMs (led by *newly-minted Dr.* @evanqed and @arnab_api)

> Understanding how LMs transform internal knowledge representations to output predictions

https://t.co/gttqzgBCL9

[Evan Hernandez @evanqed]

It's been awesome to see this work come together - an agentic interpretability system that iteratively experiments on a model to understand how it works!

I believe "LMs with tools" is a powerful paradigm for auto interp. Lots of room for new tools + LMs to increase the scope!

[Tamar Rott Shaham @TamarRottShaham]

Accepted to #ICML2024!! 🚀

Meet MAIA- a Multimodal Automated Interpretabilty Agent that helps users understand AI systems.

Given a user query (eg "label a model’s feature"), MAIA designs experiments iteratively by forming and updating hypotheses based on experimental results.

[Sarah Schwettmann @cogconfluence]

Thanks @_akhaliq!

MAIA is next in our line of work on Interpretability Agents that interrogate other models, and the functions of their components, using iterative experimentation.

Project page: https://t.co/opOqLum3B8

w/ @TamarRottShaham @f_x_wang @AchyutaBot @evanqed @jacobandreas @ antoniotorralba

[Evan Hernandez @evanqed]

Bigger picture, REMEDI is a first step towards methods that can detect anomalies in LM generations (hallucinations, false assertions, etc) before the model even finishes generating. I think these tools have the potential to be valuable guardrails for safely deploying LMs!

[Evan Hernandez @evanqed]

You’ve heard of model editing, now get ready for 📝 representation editing 📝

In our new paper, we find directions in LM rep space that make the LM assert a fact is true.

Paper: https://t.co/9MohRDwgCJ
Code: https://t.co/ErKJFZhn8O
w/ @belindazli @jacobandreas

[Evan Hernandez @evanqed]

In fact, those directions also do an okay job of detecting errors in the LM’s background knowledge-- which suggests LMs could naturally represent state info and factual info in similar ways.

[Jacob Andreas @jacobandreas]

Exciting to see @OpenAI scaling up @evanqed & co's work on learned models for neuron labeling (https://t.co/8ghZ6YJkwu)! One technical observation, and one practical one:

[David Bau @davidbau]

The NSF has invited us to propose an $18m (RI-1) project for infrastructure to help **you** (academic researchers) to study large language models.

What should we build to help your LLM research?

Respond with your thoughts.

(Or answer 3Q's on a form https://t.co/PXd1lgwwWz)

🧵

[Evan Hernandez @evanqed]

Proud of Eric + team for pushing this work! You don't need a giant LM for everything. Relatively small (< 1B param), specially pretrained language models are effective for processing specialized text like EHR notes. Can beat ICL+big models with only O(100) FT data points

[Eric Lehman @lehmer16]

Recent results have suggested that LLMs encode a surprising amount of clinical knowledge. These results raise an important question about the role of smaller specialized clinical models: do we still need clinical LMs? We explore this question in our paper: https://t.co/CK7ZAoBy7W

[Evan Hernandez @evanqed]

If you’re attending ICLR this week, come say hi! This work has been a long time coming and we’re excited to share it:

Oral: 4/28 4:15AM EST (yikes)
Poster: 4/28 9:30PM EST

[Evan Hernandez @evanqed]

Check out our #ICLR22 oral: MILAN, a procedure for generating language descriptions of neurons!

With @cogconfluence @davidbau @ teona @ torralba @jacobandreas

📄Paper: https://t.co/kcFGlLGS0N
💻Website: https://t.co/t2Xtjv3lfs
🔍Demo: https://t.co/NIUBne0EiZ

[Evan Hernandez @evanqed]

Ultimately, MILAN is another tool for our interpretability toolkit. The irony is not lost on us that it relies on a black-box model to understand black-box models. But we think it serves an important purpose in helping practitioners make decisions about their own models.