Andrew Rouditchenko 🇺🇦

PhD student at MIT working on multi-modal and multilingual speech. I was an intern at @AIatMeta and @Apple MLR.

[Andrew Rouditchenko 🇺🇦 @arouditchenko]

Do you really need audio to fine-tune your Audio LLM? 🤔 Answer below:

Introducing Omni-R1, a simple GRPO fine‑tuning method for Qwen2.5‑Omni on audio question answering. It sets new state‑of‑the‑art accuracies on the MMAU benchmark for Audio LLMs.

https://t.co/8xdWzDmSpS

[Umberto Cappellazzo @Umberto_Senpai]

How do AVSR models balance what they hear and what they see?
Introducing Dr. SHAP-AV, the first large-scale Shapley-based analysis of modality contributions in audio-visual speech recognition.
6 sota models 2 benchmarks 3 analyses
🌐Project page: https://t.co/Ajjk2FpS0w
🧵👇

[Andrew Rouditchenko 🇺🇦 @arouditchenko]

@rdesh26 Thanks for sharing!

[Andrew Rouditchenko 🇺🇦 @arouditchenko]

Do you really need audio to fine-tune your Audio LLM? 🤔 Answer below:

Introducing Omni-R1, a simple GRPO fine‑tuning method for Qwen2.5‑Omni on audio question answering. It sets new state‑of‑the‑art accuracies on the MMAU benchmark for Audio LLMs.

https://t.co/8xdWzDmSpS

[Puyuan Peng @PuyuanPeng]

𝐅𝐫𝐨𝐦 𝐮𝐧𝐞𝐦𝐩𝐥𝐨𝐲𝐚𝐛𝐥𝐞 𝐦𝐚𝐭𝐡 𝐮𝐧𝐝𝐞𝐫𝐠𝐫𝐚𝐝 → 𝐭𝐨 𝟗,𝟎𝟎𝟎 𝐆𝐢𝐭𝐇𝐮𝐛 𝐬𝐭𝐚𝐫𝐬 & 𝟒 𝐫𝐞𝐬𝐞𝐚𝐫𝐜𝐡 𝐬𝐜𝐢𝐞𝐧𝐭𝐢𝐬𝐭 𝐨𝐟𝐟𝐞𝐫𝐬 (𝐌𝐒𝐋, 𝐞𝐭𝐜.)

👉My journey of doing 𝐏𝐡𝐃 𝐢𝐧 𝐀𝐈: https://t.co/3VFIVORVIo

[Andrew Rouditchenko 🇺🇦 @arouditchenko]

@anmol01gulati Congrats!

[Jiawei (Joe) Zhou @jzhou_jz]

🎙️ Another #MultimodalAI workshop we are organizing—this one zeroes in on speech & language foundation models!
📚 Dive into #SpeechAI, audio, and language tech. Learn how to build foundation models and hear from both academia and industry experts.
🗓Sep 4–5, 2025 | @TTIC_Connect

[Andrew Rouditchenko 🇺🇦 @arouditchenko]

@chenwanch1 Your rebuttal leaves them speechless

[Peyman Milanfar @docmilanfar]

If your PhD advisor dressed like this, you probably didn't use neural nets in your thesis

[yobibyte @y0b1byte]

Finally, after all these years of being mocked, ffmpeg enthusiasts win!

[Heng-Jui Chang @hjchang87]

💡Bridging speech, sound, & music representations with one universal model?

We introduce USAD ✅
📚 Distills knowledge from domain-specific SSL models
🎯 Matches expert models across speech/audio/music tasks

📄 https://t.co/IHkZIGbTZg
🧑‍💻 https://t.co/TQ0V4wv7bA

[Herman Kamper @HermanKamper]

Learn to figure out what is worth figuring out: https://t.co/Tqy5oRibn8

[Andrew Rouditchenko 🇺🇦 @arouditchenko]

@nizumical +another benchmark that merges them into one benchmark 😅

[Andrew Rouditchenko 🇺🇦 @arouditchenko]

Congrats to Edson for leading our Contrastive Audio-Visual Masked Autoencoders 2.0 Project (CAV-MAE Sync), accepted at #CVPR2025!

Check out Edson's thread for more details ⬇️

[Edson Araujo @edsonroteia]

🚀 Excited to announce our #CVPR2025 paper: CAV-MAE Sync: Improving Contrastive Audio-Visual Mask Autoencoders via Fine-Grained Alignment!
We introduce a simple yet effective method for improved audio-visual learning.
🔗 Project: https://t.co/IW4yHNO3JP
🧵 (1/7)👇

[Andrew Rouditchenko 🇺🇦 @arouditchenko]

Link the MMAU leaderboard (Massive Multi-Task Audio Understanding and Reasoning Benchmark) - it should hopefully be updated soon with Omni-R1 https://t.co/7cXejB3teZ

[Andrew Rouditchenko 🇺🇦 @arouditchenko]

Code, models, and datasets on the way!
Thank you to my co-authors: Saurabh Bhati, @edsonroteia, Samuel Thomas, @HildeKuehne, @RogerioFeris, James Glass.
Collaboration through @MITIBMLab

[Andrew Rouditchenko 🇺🇦 @arouditchenko]

Granite-speech audio LLM from IBM. The level of data detail here is great especially comparing to ie. Whisper paper

[arXiv Sound @ArxivSound]

``Granite-speech: open-source speech-aware LLMs with strong English ASR capabilities,'' George Saon, Avihu Dekel, Alexander Brooks, Tohru Nagano, Abraham Daniels, Aharon Satt, Ashish Mittal, Brian Kingsbury, David Haws, Edmilson Morais, Gakuto Kurata, Ha… https://t.co/Hjg6Hhh6L6

[arXiv Sound @ArxivSound]

``Granite-speech: open-source speech-aware LLMs with strong English ASR capabilities,'' George Saon, Avihu Dekel, Alexander Brooks, Tohru Nagano, Abraham Daniels, Aharon Satt, Ashish Mittal, Brian Kingsbury, David Haws, Edmilson Morais, Gakuto Kurata, Ha… https://t.co/Hjg6Hhh6L6