Assaf Arbelle

Father of 3 amazing daughters, Husband, Manager - AI-Vision Grp & Research Scientist @IBMResearch.
Mastodon: @[email protected]

[Assaf Arbelle @ArbelleAssaf]

Just last week @leokarlin and @Eli_Schwartz presented our BrAD paper at @CVPR, and now we are releasing our first demo for Unsupervised Cross-Domain Image Retrieval on @Huggingface 🤗 with @Gradio!
Come check it out! ❤️
https://t.co/P0OZgKJ5d9
@IBMResearch

[Roei Herzig @roeiherzig]

🚨 Rough luck with your #ICCV2025 submission?

We’re organizing the 4th Workshop on What’s Next in Multimodal Foundation Models at @ICCVConference in Honolulu 🌺🌴

Send us your work on vision, language, audio & more!

🗓️ Deadline: July 1, 2025 🔗 https://t.co/t2DmcZAlWM

[Sivan Doveh @SivanDoveh]

IPLOC accepted to ICCV25 ☺️
Thanks to all the people that were part of it 🩷

The idea for this paper came by a lake during a visit to Graz for a talk. It has traveled with me through too many countries and too many wars, and it’s now a complete piece of work.

[merve @mervenoyann]

IBM released Granite-Vision-3.1-2B, a small vision LM with impressive performance on different tasks 😮🔥

it comes with transformers and vLLM support from the get-go 💗
you can run it in Colab T4, so I built a notebook to put it to test, find it on the next one ⤵️

[Assaf Arbelle @ArbelleAssaf]

Very cool work by @chancharikm, Brandon Huang and the team!
We just scratched the surface of what is possible with Sparse Attention Vectors (SAVs).

[Chancharik Mitra @chancharikm]

🎯 Introducing Sparse Attention Vectors (SAVs): A breakthrough method for extracting powerful multimodal features from Large Multimodal Models (LMMs). SAVs enable SOTA performance on discriminative vision-language tasks (classification, safety alignment, etc.)!

Links in replies!

🔎Using just ~20 attention heads & only few-shot examples, SAVs:
- Outperform both LoRA and few-shot baselines
- Work with image, text, & interleaved inputs
- Extract features without finetuning - ready to go at test time!

This project was a cross-collaborative effort between researchers from UC Berkeley, Carnegie Mellon University, and MIT-IBM Research (@berkeley_ai, @CMU_Robotics, @MITIBMLab). Many thanks to all of the collaborators and co-authors on this work:

Brandon Huang, Tianning (Ray) Chai, @ZhiqiuLin, @ArbelleAssaf, @RogerioFeris, @leokarlin, @trevordarrell, @RamananDeva, @roeiherzig

[Jochem Gietema @gietema]

There are some approaches to improve few-shot performance on VLMs: from simple fine-tuning on more images to using newer techniques such as Task Vectors - compact implicit representations of in-context examples compressed in the model’s attention heads (https://t.co/aVfwb091qy).

[Assaf Arbelle @ArbelleAssaf]

Check out our latest Live Multimodal Dataset!

[Nimrod Shabtay @NimrodShabtay]

Introducing LiveXiv, a new, challenging and maintainable scientific multi-modal live dataset
Paper: https://t.co/QiRrgubUom
Github: https://t.co/ttd4MYad8E
Dataset: https://t.co/4blQxQDiLz

[Assaf Arbelle @ArbelleAssaf]

@harel_nimrod Congratulations!! 🎉

[Roei Herzig @roeiherzig]

🚨𝐌𝐮𝐥𝐭𝐢𝐦𝐨𝐝𝐚𝐥 𝐓𝐚𝐬𝐤 𝐕𝐞𝐜𝐭𝐨𝐫𝐬 (MTV) is accepted to @NeurIPSConf!🥳

We show how to perform many-shot ICL in LMMs using MTV representation---a compact implicit representation of many-shot examples stored in the model’s attention head.

Kudos to all collab.&authors

[Eli Schwartz @Eli_Schwartz]

With the recent widespread discussions on X about OpenAI's newest o1 model🍓arithmetic skills, this feels particularly timely. Excited to share that NumeroLogic, our work on improving LLMs' numerical reasoning capabilities, has been accepted to @emnlpmeeting! See you in Miami! 🥳

[Avihu Dekel @AvihuDekel]

Tokenization algorithms like BPE are the standard practice in NLP 📚
However, they have been mostly overlooked for discrete acoustic units (DAUs) 🎙️
We show that tokenizing DAUs yields significant task performance and speed improvements. 📈
Accepted to #Interspeech2024 🇬🇷

[Roei Herzig @roeiherzig]

This fantastic work was done by the outstanding two undergrads, Brandon Huang and Chancharik Mitra, as well as @ArbelleAssaf, @leokarlin, and @trevordarrell.

and..Happy 4th of July!🇺🇸

[Eli Schwartz @Eli_Schwartz]

Happening now, join us for a great workshop! #CVPR

[#3 MMFM Workshop @MMFMWorkshop]

Good morning 🤩 @leokarlin

[#3 MMFM Workshop @MMFMWorkshop]

Our first speaker @songhan_mit

[Sivan Doveh @SivanDoveh]

The MMFM workshop takes place now at @CVPR
I'll be updating from the mmfm account if you want to follow 😁

[AK @_akhaliq]

Trans-LoRA

towards data-free Transferable Parameter Efficient Finetuning

Low-rank adapters (LoRA) and their variants are popular parameter-efficient fine-tuning (PEFT) techniques that closely match full model fine-tune performance while requiring only

[Leonid Karlinsky @leokarlin]

Thanks for the highlight @_akhaliq!
We offer a simple and nearly-data-free way to move (large quantities) of custom PEFT models within or across LLM families or even across PEFT configurations. Useful for LLM cloud hosting when old base models need to be deprecated & upgraded

[Hilde Kuehne @HildeKuehne]

🚨 @CVPR Challenge alert!🚨

Think you have a good MML for documents, infographics and text-rich images?

Challenge “What is Next in Multimodal Foundation Models?” extended the deadline and opened for all:
https://t.co/e5wt5EKVPC

$10K in prizes
Deadline: 5th June 2024 (11 days)!

[#3 MMFM Workshop @MMFMWorkshop]

In case you haven't noticed, we have $10K winner prizes for the top teams!
🤑🤑🤑
Also, we extended the deadlines for Phase 1 and Phase 2 to June 5th.

Challenge: https://t.co/1PCRHsP6Cc
Git: https://t.co/j5rONVxu4r

@CVPR