Martin Wattenberg

Human/AI interaction. Visualization as design, science, art. Professor at Harvard, and part-time at Google's People+AI Research initiative.

[Laura Wattenberg, Baby Name Wizard @BNW]

https://t.co/x0MHQHF7WA

[Martin Wattenberg @wattenberg]

@filip_rejmus @viegasf Moreover, different heads have a different level of focus on position, which leads to "sharper" or "blurrier" spiral shapes. Finally, the input sequences we used are different lengths, so there are more points at earlier positions than later ones—that affects the shape as well.

[Martin Wattenberg @wattenberg]

Visualize transformer attention!

AttentionViz, created by Catherine Yeh and expanded by Yida Chen, helps you explore transformer self-attention by visualizing query and key vectors in a joint embedding.

Paper: https://t.co/GKom1BN5Zl
Website: https://t.co/rCeA2e5xKv

[Martin Wattenberg @wattenberg]

@filip_rejmus @viegasf You'll notice that some spirals are more "connected" than others. These correspond to heads that pay attention to multiple nearby positions, so the queries / keys trace out a path between positional encodings. There's a kind of linear interpolation between position vectors. (2/n)

[Martin Wattenberg @wattenberg]

Here's a close-up of hue and brightness heads in a vision transformer. Work done with tremendous help from Aoyu Wu and @chenxcynthia!

[Martin Wattenberg @wattenberg]

Language models have some beautiful spiral plots reflecting positional patterns. And a vision model has heads that arrange images according to brightness and hues. But there’s a lot more to find! What else can you see?

[Martin Wattenberg @wattenberg]

Toasters have blinking lights, cars have speedometers. Should chatbots have dashboards too?

A speculative essay: The System Model and the User Model: Exploring AI Dashboard Design
https://t.co/nurFiNd9sF

[David Bau @davidbau]

I want to show the NSF there would be broad support+utility for a "National Deep Inference" service for >100b LLMs.

If your research would be enabled by an inference service on open LLMs w API access+overrides to internal activations, params, gradients:

Please Like this thread!

[Harvard SEAS @hseas]

Students explore the aesthetics of computing in new computer science course at SEAS. "CS73: Code, Data, and Art" is co-taught by @wattenberg and @viegasf, and teaches students how to create abstract art and communicate data sets through visualizations. https://t.co/se8D0Tkbqd

[Martin Wattenberg @wattenberg]

Starting to get used to Mastodon. Find me at @[email protected]

[Martin Wattenberg @wattenberg]

For information on the application process, see https://t.co/11XIcnahCO

[Martin Wattenberg @wattenberg]

Thinking about grad school next year? Interested in visualization, machine learning interpretability, or human/AI interaction? Consider Harvard. @viegasf and I are continuing to build our lab!

[The Guardian @guardian]

The labyrinthine patterns traced by birds on the wing – in pictures https://t.co/dfhQk78xWC

[Martin Wattenberg @wattenberg]

I'm teaching with @OpenProcessing for the first time, and am completely impressed with how polished and friendly the system is. Every detail is on point. Last class a student spontaneously said, "OpenProcessing is just so great." I agree!

[Martin Wattenberg @wattenberg]

The project is a systematic study of how a neural network represents more features than it has neurons, under a variety of conditions. (Oh, and I can't tweet without a typo, apparently. That's @AnthropicAI of course!)

[Martin Wattenberg @wattenberg]

Tiny neural networks have a surprisingly rich inner life, and may hold clues to how their larger cousins work. This image: evolution of feature vectors during learning. Full story: https://t.co/5lULpDZQBE (in collaboration with the great interpretability team at @AnthopicAI)

[Golan Levin @golan]

SMASHOMANCY is a "divination system for cracked phone screens" and @djbaskin retains pole position as one of my favorite living creatives.

[Martin Wattenberg @wattenberg]

@gro_tsen I like this paper: https://t.co/mNpwfLI0EA Sample finding: If you ask people to review the last restaurant they went to, they're less polarized than if they freely choose a restaurant to review, evidence for your second hypothesis

[Martin Wattenberg @wattenberg]

@sharoz @RuthRosenholtz Wonderful, thanks! Part of the explanation should say why the "3" doesn't stand out when everything's the same color. The shape strongly contrasts with any other individual digit in a side-by-side comparison. And in fact it would stand out if all the other digits were 1's and 7's