Gabriele Serussi

[Gabriele Serussi @GSerussi]

@NimrodShabtay Congrats!

[Gabriele Serussi @GSerussi]

@ChaimBaskin

[Gabriele Serussi @GSerussi]

🧵 1/ New paper: HiMu — Hierarchical Multimodal Frame Selection for Long Video QA

Your brain doesn't rewatch a 1-hour video to answer a question. It splits the problem: speech → audio, visuals → vision, "right after" → temporal ordering.

We built a system that does the same

[Gabriele Serussi @GSerussi]

7/ 💻 Code: https://t.co/UqAAduNxvr
🌐 Project: https://t.co/Ud3YIO0KMX
📄 Paper: https://t.co/ZQKyTWAFsv

w/ Dan Ben-Ami Kobi Cohen @ChaimBaskin @ INSIGHT Lab, Ben-Gurion University

[Gabriele Serussi @GSerussi]

We evaluated 13 top Video-LLMs—and they struggle. 📉
Most score 31–42% (chance is 20% on 5-way MCQ).
Key bottlenecks:
🔍 retrieval (finding the right moments)
🧠 fusion (combining evidence over time)
Let’s build smarter Video-LLMs 🚀
🤗 https://t.co/SDRlLhwuJN
#VideoLLM

[Gabriele Serussi @GSerussi]

Can Video-LLMs really connect the dots across time? 🕵️‍♂️🎬
Many benchmarks are solvable from a single lucky frame - so models can pass without true temporal reasoning.
We introduce HERBench, a harder VideoQA benchmark for multi-evidence integration
🔗 https://t.co/DhiqevEdxC

[Gabriele Serussi @GSerussi]

We also introduce MRFS (Minimum Required Frame Snapshots) 📏
It measures how many snapshots are truly needed to answer.
HERBench: 5.5 snapshots on avg ✅
Older benchmarks: ~2–3 ❌
HERBench genuinely requires multi-step visual aggregation.