tom white

creations with code and networks

[tom white @dribnet]

@_lyraaaa_ @tahsin_mayeesha final_vectors.npz 🙏

[tom white @dribnet]

@wendlerch @jeremyphoward @aryaman2020 @jatin_n0 @voooooogel Thanks Chris! Another use of this contrastive synthetic data technique from years past was steering text-to-image generators away from putting text in the image. https://t.co/plOxPsOjyO

[tom white @dribnet]

@NeelNanda5 @ry_serene @jd_pressman @AlecRad @CasperKaae @hugo_larochelle @OleWinther1 Also: AFAIK no text to image engines currently support steering vectors, but my @pixray tool (which proceeded @midjourney, etc) *did* support using these and by default would apply a vector to suppress text appearing in the image. https://t.co/IpHT0TnaGe

[tom white @dribnet]

if it makes you feel better: i also introduced the idea of generating useful steering vectors from contrastive synthetic data in my 2016 paper - a whole section on augmenting inputs with low pass gaussian filter to derive a steering vector that produces less blurry samples. https://t.co/5eHRe3HZJn

[tom white @dribnet]

@bcherny @Rahatcodes Goodhart's Law here we come 🤬

[tom white @dribnet]

the models, they just wanna converge

[tom white @dribnet]

saxophone print + top-150 SigLIP image probe

though mknn model agreement (a la platonic representation hypothesis) is not part of the test time compute process, it climbs naturally as the as the optimization evolves

[tom white @dribnet]

@sebkrier one-shotted just enough to win

[tom white @dribnet]

45% mutual kNN between CLIP and SigLIP — not bad for two model families trained on different data with different objectives when probed with this print. revisiting ImageNet so I can build a toolbox for navigating more uncharted waters without class labels (stay tuned)...

[tom white @dribnet]

unsure how AI interprets this print? treating the image as a linear probe on your favorite vision model and scraping a diverse dataset for maximum activations provides a coherent suggestion.

[tom white @dribnet]

"pirate ship" (ImageNet class 724)

[tom white @dribnet]

or query your favorite vision model for semantic nearest neighbors - here's OpenAI-CLIP's top hits across CC3M using the baseball_player print as a probe

[tom white @dribnet]

traffic sign, baseball player, pomegranate

[tom white @dribnet]

not seeing it? don't worry - your favorite imagenet model is.

[tom white @dribnet]

@johnhewtt https://t.co/MWjB6gkZDR

[tom white @dribnet]

Weapons-grade piggy bankness: One drawing. No training. Subtract the style, get a direction in SigLIP space. Sort 50K ImageNet images by cosine similarity: 41 of the top 50 are piggy banks (P@50 = 82%). The drawing is the classifier.

[tom white @dribnet]

Weapons-grade piggy bankness: One drawing. No training. Subtract the style, get a direction in SigLIP space. Sort 50K ImageNet images by cosine similarity: 41 of the top 50 are piggy banks (P@50 = 82%). The drawing is the classifier.

[tom white @dribnet]

piggy bank (ImageNet class 719)

[tom white @dribnet]

@DlSPUTED thanks! its a chonky 18in screenprint

[tom white @dribnet]

@hyhieu226 enjoying this slow takeoff and will genuinely miss it

[tom white @dribnet]

@farmgeek fwiw: the email was garbage but the disclosure within the app is actually pretty good - they show at the document/file level what was accessed

[tom white @dribnet]

@RT_Artwork Got this too. 5 minutes before call they ask you to use the riverside client and forward you to a website clone (riverside dot name - BEWARE) with their malware installer (I stopped there).

(would be up for a themed exhibit on scams showcasing all artists with this invite! 😂)