UW RAIVN Lab

The computer vision and reasoning lab in the Allen School at the University of Washington, led by Ali Farhadi and Ranjay Krishna.

[Ai2 @allen_ai]

Last year Molmo set SOTA on image benchmarks + pioneered image pointing. Millions of downloads later, Molmo 2 brings Molmo’s grounded multimodal capabilities to video 🎥—and leads many open models on challenging industry video benchmarks. 🧵

[Prior @ AI2 @Ai2Prior]

📢Applications are open for summer'25 internships at the PRIOR (computer vision) team
@allen_ai:

Come join us in building large-scale models for:
📸 Open-source Vision-Language Models
💻 Multimodal Web Agents
🤖 Embodied AI + Robotics
🌎 Planet Monitoring

Apply by December 11, 2024!

[UW RAIVN Lab @RAIVNLab]

Congrats [Dr. [Dr. Aditya] Kusupati]!!🪆🪆

[Aditya Kusupati @adityakusupati]

This (& graduation) happened last week & I am a (fake) Dr. now!

I owe it all to my advisors, mentors, collaborators, friends, and family! -- I wrote a 6-page acknowledgment in my thesis without realizing😅

Thanks for all the fish @uwcse, @RAIVNLab, @uw_wail & @GoogleDeepMind🪆

[UW RAIVN Lab @RAIVNLab]

Very cool to see our RAIVN lab alum Rowan in the latest GPT-4o demo!!🚀🚀

[Rowan Zellers @rown]

Excited to introduce GPT-4o.

Language, vision, and sound -- all together and all in real time.

This thing has been so much fun to work on. It's been even more fun to play with -- with moments of magic where things feel totally fluid and I forget I'm video chatting with an AI.

[Ainaz Eftekhar @ainaz_eftekhar]

🎉 Very Excited to present our recent work on “Selective🔍 Visual Representations for Embodied-AI🤖” next week at ICLR in Vienna🇦🇹!!

📣📣Important update! Our code and pretrained models are now available through our project website 🌐: https://t.co/LpojdZrFZn🚀

👋Come to my poster, say hi, and learn more about our findings! (Poster #111, Session 8, on Friday, May 10th at 4:30 PM)

[Gabriel Ilharco @gabriel_ilharco]

CLIP models have become a lot better since 2021

[UW RAIVN Lab @RAIVNLab]

Check out🪆MatFormer🪆co-led by @adityakusupati: it’s a simple yet powerful general-purpose architecture with flexibility and elasticity built within. It works across modalities and enables super cool things at web-scale tasks🔥🔥

[Aditya Kusupati @adityakusupati]

Announcing MatFormer - a nested🪆(Matryoshka) Transformer that offers elasticity across deployment constraints.

MatFormer is an architecture that lets us use 100s of accurate smaller models that we never actually trained for!

https://t.co/wzR7To7HZu 1/9

[Mitchell Wortsman @Mitchnw]

E) The attention logit growth instability is still present when replacing softmax with pointwise alternatives.

Side note: If you're interested in learning more about replacing softmax with a pointwise alternative like relu^2/√seqlen, checkout https://t.co/dDZZjxblSr!

(12/15)

[Mitchell Wortsman @Mitchnw]

Sharing some highlights from our work on small-scale proxies for large-scale Transformer training instabilities: https://t.co/mCNFJYO2z8

With fantastic collaborators @peterjliu, @Locchiu, @_katieeverett, many others (see final tweet!), @hoonkp, @jmgilmer, @skornblith!

(1/15)

[UW RAIVN Lab @RAIVNLab]

Exciting work with open source code to facilitate research on medium sized language models! 🎉

[UW RAIVN Lab @RAIVNLab]

Check out this cool work led by @DJiafei on collecting robot data without a robot! 🤖🦾

[Jiafei Duan@CVPR2026 @DJiafei]

🚨Is it possible to devise an intuitive approach for crowdsourcing trainable data for robots without requiring a physical robot🤖?
Can we democratize robot learning for all?🧑‍🤝‍🧑

Check out our latest #CoRL2023 paper->
AR2-D2: Training a Robot Without a Robot

[UW RAIVN Lab @RAIVNLab]

If you are at #CVPR2023, come check out https://t.co/ynbl1oniyD on Monday, June 19 from 9am - 12pm in West room 223-224. Speakers include @sarahmhpratt from RAIVN lab as well as @liuziwei7 @phillip_isola @hyojinbahng @lschmidt3 and @denny_zhou!

[Kaiyang Zhou @kaiyangzhou]

We're organizing a tutorial on Prompting in Vision at #CVPR2023 w/ @liuziwei7 @phillip_isola @hyojinbahng @lschmidt3 @sarahmhpratt @denny_zhou

Please visit our website at https://t.co/orrmWhl22Q to know more about this event

[UW RAIVN Lab @RAIVNLab]

Checkout CREPE led by @zixianma02, a new large-scale benchmark for vision-language model!!

Drop by the poster at CVPR 2023.

[Zixian Ma@CVPR @zixianma02]

Have vision-language models achieved human-level compositional reasoning? Our research suggests: not quite yet.

We’re excited to present CREPE – a large-scale Compositional REPresentation Evaluation benchmark for vision-language models – as a 🌟highlight🌟at #CVPR2023.
🧵1/7

[UW RAIVN Lab @RAIVNLab]

Check out our new work on improving large-scale nearest-neighbor search!

[Aditya Kusupati @adityakusupati]

Introducing💃AdANNS: A Framework for Adaptive Semantic Search🕺

TL;DR: Up to 90× faster nearest neighbor retrieval and 2× lower memory cost for web-scale search.

Applies to vector search at scale & improves all "retrieval" augmented models!

https://t.co/yEnrr4oCwa

[1/8]

[Aditya Kusupati @adityakusupati]

Introducing💃AdANNS: A Framework for Adaptive Semantic Search🕺

TL;DR: Up to 90× faster nearest neighbor retrieval and 2× lower memory cost for web-scale search.

Applies to vector search at scale & improves all "retrieval" augmented models!

https://t.co/yEnrr4oCwa

[1/8]

[Vaishaal Shankar @Vaishaal]

1/9 I am excited to announce that our workshop "Towards the Next Generation of Computer Vision Datasets" will be happening at ICCV 2023 in Paris. We will feature DataComp submissions, other data-centric papers, and invited talks by experts. https://t.co/mMY0OOHLro

[UW RAIVN Lab @RAIVNLab]

Excited to announce DataComp! Let us create stronger models using better-filtered data.

[Gabriel Ilharco @gabriel_ilharco]

Introducing DataComp, a new benchmark for multimodal datasets!

We release 12.8B image-text pairs, 300+ experiments and a 1.4B subset that outcompetes compute-matched CLIP runs from OpenAI & LAION

📜 https://t.co/nz4I9Xwt8e
🖥️ https://t.co/XcAlKmMW1G
🌐 https://t.co/Um7OjgvGlA

[Gabriel Ilharco @gabriel_ilharco]

Introducing DataComp, a new benchmark for multimodal datasets!

We release 12.8B image-text pairs, 300+ experiments and a 1.4B subset that outcompetes compute-matched CLIP runs from OpenAI & LAION

📜 https://t.co/nz4I9Xwt8e
🖥️ https://t.co/XcAlKmMW1G
🌐 https://t.co/Um7OjgvGlA