Ivan Vulić

Research Prof@Cambridge; Interested in (way) too many things, but mostly (and rarely) (re)tweets about NLP, ML, IR, language(s); (likes parentheses)

[Lucas Caccia @LucasPCaccia]

RAG and in-context learning are the go-to approaches for integrating new knowledge into LLMs, making inference very inefficient

We propose instead 𝗞𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 𝗠𝗼𝗱𝘂𝗹𝗲𝘀 : lightweight LoRA modules trained offline that can match RAG performance without the drawbacks

[Han Zhou @ICLR 2026 @hanzhou032]

Automating Multi-Agent Design:

🧩Multi-agent systems aren’t just about throwing more LLM agents together.

🛠️They require mastering the subtle art of prompting and agent orchestration.

Introducing MASS🚀- Our new agent optimization framework for better prompts and topologies!

[Benjamin Minixhofer @bminixhofer]

We achieved the first instance of successful subword-to-byte distillation in our (just updated) paper.

This enables creating byte-level models at a fraction of the cost of what was needed previously.

As a proof-of-concept, we created byte-level Gemma2 and Llama3 models.

🧵

[Yi Xu @_yixu]

🚀Let’s Think Only with Images.

No language and No verbal thought.🤔

Let’s think through a sequence of images💭, like how humans picture steps in their minds🎨.

We propose Visual Planning, a novel reasoning paradigm that enables models to reason purely through images.

[Benjamin Minixhofer @bminixhofer]

We created Approximate Likelihood Matching, a principled (and very effective) method for *cross-tokenizer distillation*!

With ALM, you can create ensembles of models from different families, convert existing subword-level models to byte-level and a bunch more🧵

[Ivan Vulić @licwu]

We've got plenty of exciting ideas flying around, so consider applying to carve them further with us!

[Jonas Pfeiffer @PfeiffJo]

I am hiring a Student Researcher for our Modularity team at the Google DeepMind office in Zurich🇨🇭

Please fill out the interest form if you would like to work with us! The role would start mid/end 2025 and would be in-person in Zurich with 80-100% at GDM
https://t.co/Vfypj91KHy

[Fabian David Schmidt @fdschmidt]

📣Happy to (pre-)release my Fleurs-SLU benchmark to evaluate massively multilingual spoken language understanding on SIB & Belebele. Work done at @Mila_Quebec with @davlanade @gg42554 @licwu

Datasets:
https://t.co/wqSfkT3VA3
https://t.co/882nh8znY1

Details to follow👇

[River Yijiang Dong @river_dong121]

Thrilled to share our updated paper: "UNDIAL: Self-Distillation with Adjusted Logits for Robust Unlearning in Large Language Models"
We propose a new robust LLM unlearning method via Self-Distillation on Adjusted Logits (UNDIAL).
📄 Paper: https://t.co/vqX1YuFF5e

[Hannah @h_sterz]

Do you DARE?

Introducing a multiple-choice VQA benchmark ✨DARE✨ with:
- 4 main robustness evaluation ⛓️
- 5 diverse categories 🧩
- Extensive analysis of 4 widely used VLMS 🤖

[Markus Frohmann @FrohmannM]

Introducing 🪓Segment any Text! 🪓

A new state-of-the-art sentence segmentation tool!
Compared to existing tools (and strong LLMs!), our models are far more:
1. efficient ⚡
2. performant 🔝
3. robust 🚀
4. adaptable 🎯
5. multilingual 🗺

[Ivan Vulić @licwu]

As someone who spent years working in multilingual NLP, I am so happy that we're finally seeing (L)LMs and (N)MT systems working in tandem towards the shared cause. The idea in this work is so simple & sweet, and yet it moves! 🌍🌏🌎

[Fabian David Schmidt @fdschmidt]

Introducing NLLB-LLM2Vec! 🚀

We fuse the NLLB encoder & Llama 3 8B trained w/ LLM2Vec to create NLLB-LLM2Vec which supports cross-lingual NLU in 200+ languages🔥

Joint work w/ Philipp Borchert, @licwu, and @gg42554 during my great research stay at @cambridgeltl

[Han Zhou @ICLR 2026 @hanzhou032]

Which output is better?
[A] or [B]? LLM🤖: B❌
[B] or [A]? LLM🤖: A✅

Thrilled to share our preprint in addressing preference biases in LLM judgments!🧑‍⚖️We introduce ZEPO, a 0-shot prompt optimizer that enhances your LLM evaluators via fairness⚖️

📰Paper: https://t.co/ZkMvJnFFMC

[Chengzu Li @li_chengzu]

Excited to introduce TopViewRS: VLMs as Top-View Spatial Reasoners🤖

TopViewRS assess VLMs’ spatial reasoning in top-view scenarios🏠just like how you read maps🗺️

Spoiler🫢GPT4V and Gemini are neck-and-neck, each excelling in different setups but neither even close to us humans

[Benjamin Minixhofer @bminixhofer]

Introducing Zero-Shot Tokenizer Transfer (ZeTT) ⚡

ZeTT frees language models from their tokenizer, allowing you to use any model with any tokenizer, with little or no extra training.

Super excited to (finally!) share the first project of my PhD🧵

[Neil Houlsby @neilhoulsby]

Adapters are just a great way to share/benefit from new capabilities without handing around the kitchen sink.

Congrats to the AdapterHub folks for adding support for quantized training (Q-LoRA and friends).

[Ivan Vulić @licwu]

If we align LLMs through preferences, perhaps we should also evaluate them the same way (and respect transitivity)? The answer is: yes, we should. The trick, however, is how to make evaluation tractable. If you are into the whole "LLM-as-Judges" line of work, check this paper!

[Yinhong Liu @YinhongLiu2]

🔥New paper!📜
Struggle to align LLM evaluators with human judgements?🤔
Introducing PairS🌟: By exploiting transitivity, we push the potential of pairwise preference in efficient ranking evaluations that has better alignment!🧑‍⚖️
📖https://t.co/W4wSHQqdYc
💻https://t.co/q5ZMGkvaaj

[Sebastian Ruder @seb_ruder]

🚨 A belated update:
Our survey on "Modular Deep Learning" has been published in TMLR.

Check out the updated version:
https://t.co/q5j55xdXjb

[Edoardo Ponti @PontiEdoardo]

I am still looking for PhD students starting in September 2024! The deadline to apply for the CDT in NLP is the 11th of March.

If you wish to do research in modular and efficient LLMs, here are some highlights of my lab's research from the past year ⬇️🧵

[Ivan Vulić @licwu]

Think globally, act locally? Well, we were thought-experimenting whether LLMs would understand people from different places around our hometowns better than we ever might... And then we have eventually decided to make an actual (non-thought) experiment out of these thoughts! 👇👇

[Nikola Ljubešić @nljubesic]

Interested in commonsense reasoning in dialectal texts? The DIALECT-COPA shared task is the perfect fit for you, providing train and dev data for four official South-Slavic languages and two out of three related test dialects https://t.co/im2CzBFZjY @vardialworkshop @naaclmeeting

[Edoardo Ponti @PontiEdoardo]

We scaled sparse fine-tuning (SFT) to LLMs (such as Llama 2) by making it both parameter- and memory-efficient!

(q)SFT instruction tuning performance is often better than (q)LoRA with comparable speed and memory load.

Paper: https://t.co/wGew8XQvdW
Code:
https://t.co/zElZ7BCbJ6 (SFT PEFT) https://t.co/sOB4WVOHm5 (experiments)

@AlanAnsell5 @licwu @h_sterz @annalkorhonen