Ian Covert

Postdoc @Stanford, previously @uwcse @GoogleAI and @Columbia. Interested in deep learning and explainable AI

[Ian Covert @ianccovert]

Making this class with Su-In, Hugh and Chris was one of the most fun things I did in grad school. We covered a ton of material, definitely check out all the slides we made https://t.co/PFfm6omdOy I'm excited to see how the course evolves in the next couple years!

[Sahil Verma @Sahil1V]

📣 📣 📣 Our new paper investigates the question of how many images 🖼️ of a concept are required by a diffusion model 🤖 to imitate it. This question is critical for understanding and mitigating the copyright and privacy infringements of these models! https://t.co/bvdVU1M0Hh

[James Zou @james_y_zou]

Very excited to introduce locality alignment, an efficient post-training algorithm to improve your ViTs + VLMs, essentially for free🚀

Local align = new self-supervised objective ensuring that encoder captures fine-grained spatial info. No new data needed. Here's the idea 1/3

[Soham Gadgil @soham_gadgil]

How to perform dynamic feature selection without assumptions about the data distribution or fitting generative models? We develop a learning approach to estimate the conditional mutual information in a discriminative fashion for selecting features. https://t.co/6dCHlJJA9m

[Ian Covert @ianccovert]

This was work done with @HughChen18 @scottlundberg and of course our advisor @suinleelab

NMI version: https://t.co/5OWuagexz6
arXiv version: https://t.co/MqwdY9pp28

[Ian Covert @ianccovert]

In our recent Nature MI paper, we looked at the surprising number of algorithms that estimate Shapley values (whose computation scales exponentially with the number of players). There are a lot, we counted at least 24 papers on this topic!

Paper: https://t.co/5OWuagexz6

[Ian Covert @ianccovert]

Large models are tough because you may not be able to query the model thousands of times to get attributions (e.g., KernelSHAP). This is something we've tackled in a couple other papers

FastSHAP (ICLR'22): https://t.co/dGGUEaHs5j
ViT Shapley (ICLR'23): https://t.co/yhraLMTDLQ

[Su-In Lee @suinleelab]

My amazing PhD student @ianccovert will present our work on ViT Shapley at #ICLR2023 soon -- Mon 1 May 11 CAT!

[Chris Lin @chrislin97]

We have an upcoming paper at ICLR 2023 on a new feature attribution method for explaining representations learned by unsupervised models!
https://t.co/kiLQbF8S4i
This was joint work with the fantastic @HughChen18 @ChanwooKim_ and my advisor @suinleelab. (1/n)

[Parul Pandey @pandeyparul]

Check out this course on #XAI by @suinleelab
& @ianccovert. Very practical and nicely curated. Also points to some great papers on the topic. The course covers a broad set of principles and techniques. Slides are available here: https://t.co/g92FV8Bnko
#ResponsibleAI

[Ian Covert @ianccovert]

This was joint work with the fantastic @ChanwooKim_ and @suinleelab from @uwcse/@uw_wail. And big shoutout to collaborators @SudarshanMukund @neiljethani + Rajesh from the paper this work builds on https://t.co/dGGUEaHs5j (9/9)

[Ian Covert @ianccovert]

The question we’re trying to answer is *which patches influence the prediction.* And Shapley values are a surprisingly simple approach: they're like leave-one-out, but the effect of removing a patch is averaged across all sets of preceding patches (3/n)

[Ian Covert @ianccovert]

Our experiments used three image datasets and models as big as ViT-Large (arXiv needs to be updated), but there's still plenty of room to scale this up. My guess is that 1) it gets better with more data, 2) it can help learn better representations than the original task (8/n)

[Ian Covert @ianccovert]

TL;DR: we fine-tune a ViT to directly predict its Shapley values, without using a dataset of ground truth examples. And it works quite well (2/n)

[Ian Covert @ianccovert]

If you want to know what your ViT pays attention to...you might not want to use attention values! Shapley values can do this better, and now they can even do it efficiently. Check out our new paper (ICLR spotlight) https://t.co/yhraLMTDLQ 🧵⬇️