Aleksandar Gavric

University Assistant at Technische Universität Wien (Vienna University of Technology)

[DailyPapers @HuggingPapers]

NVIDIA just released a finetuned GR00T N1.7 robot policy on Hugging Face

Trained on the LIBERO benchmark for Panda arm manipulation

Ready to deploy via LeRobot

[Sadao Tokuyama @tokufxug]

独ライプツィヒ大がEU等の資金援助を受け、脳神経外科向け空間コンピューティングプラットフォームを開発。

まずは教育・研修に導入し、将来的には脳室ドレーン留置術などの臨床応用を目指す。

手術の正確性と安全性を高め、教育から臨床まで医療の未来を形作る革新的な取り組み。（詳細はリプ欄）

[Vai Viswanathan @vai_viswanathan]

🥇 CVPR BEST PAPER: D4RT 🏆

• New state of the art across 4D reconstruction tasks
• One transformer encodes a video into a global scene representation that a lightweight decoder reads to jointly infer depth, motion, and camera.
• You can query the 3D position of any pixel at any time independently — skipping dense decoding and making it fast and scalable.

[AK @_akhaliq]

ByteDance presents APT2

Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generation

[Tenny Yin @tennyyin]

🔎Can robots search for objects like humans?
Humans explore unseen environments intelligently—using prior knowledge to actively seek information and guide search. But can robots do the same? 👀

🚀Introducing WoMAP (World Models for Active Perception): a novel framework for embodied open-vocabulary object localization that combines the reasoning power of VLMs 🧠with the physical grounding capabilities of world models 🌎.
🌐 https://t.co/e6ZUra86S1 🧵(1/N)

[SkalskiP @skalskip92]

CVPR 2025 papers pt. 2 - SAMWISE

SAMWISE adds language understanding and temporal reasoning to SAM2; you can segment and track objects in videos just by describing them

more papers: https://t.co/1VlLn2BWxl

↓ more

[SkalskiP @skalskip92]

CVPR 2025 papers pt. 1 - Gaze-LLE

Gaze-LLE simplifies gaze target estimation by building on top of a frozen DINOv2 visual foundation model; SOTA performance; open source code and model

more papers: https://t.co/1VlLn2BWxl

↓ more

[Rohan Paul @rohanpaul_ai]

Brilliant paper from @Meta having the potential to significantly boost LLM's reasoning power.

Why force AI to explain in English when it can think directly in neural patterns?

Imagine if your brain could skip words and share thoughts directly - that's what this paper achieves for AI.

By skipping the word-generation step, LLMs can explore multiple reasoning paths simultaneously.

Introduces Coconut (Chain of Continuous Thought), enabling LLMs to reason in a continuous latent space rather than through word tokens, leading to more efficient and powerful reasoning capabilities.

🧠 The key Solution in this paper

Current LLMs are constrained by having to express their reasoning through language tokens, where most tokens serve textual coherence rather than actual reasoning.

So this paper proposes a novel solution where instead of decoding the hidden state into word tokens, it's directly fed back as the next input embedding in a continuous space.

Let me explain the mechanism simply:

In normal LLMs, when the model thinks, it has to:
1. Convert its internal neural state into actual words
2. Then convert those words back into neural patterns to continue thinking

What Coconut does instead:
It directly takes the neural patterns (hidden state) from one thinking step and feeds them into the next step - no conversion to words needed. It's like letting the model's thoughts flow directly from one step to the next in their raw neural form.

Think of it like this: Instead of having to write down your thoughts on paper and then read them back to continue thinking (like regular LLMs do), Coconut lets the model's thoughts continue flowing naturally in their original neural format. This is more efficient and lets the model explore multiple possible thought paths at once.

-----

The method uses special tokens <bot> and <eot> to mark latent reasoning segments, and employs a multi-stage training curriculum that gradually replaces language reasoning steps with continuous thoughts.

Key insights of the paper:

→ Coconut achieves 34.1% accuracy on GSM8k math problems, outperforming baseline Chain-of-Thought (30.0%)

→ The continuous space enables parallel exploration of multiple reasoning paths, similar to breadth-first search

→ Performance improves with more continuous thoughts per reasoning step, showing effective chaining capability

→ Latent reasoning excels in tasks requiring extensive planning, with 97% accuracy on logical reasoning (ProsQA)

[LangChain @LangChain]

🤖Agentic RAG with VoyageAI, Gemini and LangGraph

Agentic RAG adds intelligent agents that can retrieve, verify, and act on data autonomously.

This a great fit for complex situations like medical diagnoses or customer service

https://t.co/d0AJmfYPnm

[Jaemin Cho @jmin__cho]

Check out M3DocRAG -- multimodal RAG for question answering on Multi-Modal & Multi-Page & Multi-Documents (+ a new open-domain benchmark + strong results on 3 benchmarks)!

⚡️Key Highlights:

➡️ M3DocRAG flexibly accommodates various settings:
- closed & open-domain document contexts (from a single-page doc to a corpus of many long docs)
- single & multi-hop questions
- diverse elements (text, table, image, etc.)

➡️ M3DocVQA is a new open-domain DocVQA benchmark where models should answer multi-hop questions (across multiple pages and documents) 3K+ PDFs (w/ 40K+ pages)

➡️ Strong results on 3 benchmarks (M3DocVQA/MMLongBench-Doc/MP-DocVQA), including SoTA results on MP-DocVQA

🧵👇

[Victoria X Lin @VictoriaLinML]

💫 Introducing Mixture-of-Transformers (MoT) , our latest work advancing modality-aware sparse architectures for multimodal foundation models, led by @liang_weixin, in collaboration w/ amazing colleagues at @AIatMeta.
https://t.co/1LMdVZkZdN (1/n)

[AK @_akhaliq]

DimensionX

Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion

[Xiang Yue @xiangyue96]

🌍 I’ve always had a dream of making AI accessible to everyone, regardless of location or language. However, current open MLLMs often respond in English, even to non-English queries!

🚀 Introducing Pangea: A Fully Open Multilingual Multimodal LLM supporting 39 languages! 🌐✨

https://t.co/lHP1CSNNVe
https://t.co/RkMdE4JSQg

The Pangea family includes three major components:
🔥 Pangea-7B: A state-of-the-art multilingual multimodal LLM capable of 39 languages! Not only does it excel in multilingual scenarios, but it also matches or surpasses English-centric models like Llama 3.2, Molmo, and LlavaOneVision in English performance.

📝 PangeaIns: A 6M multilingual multimodal instruction tuning dataset across 39 languages. 🗂️ With 40% English instructions and 60% multilingual instructions, it spans various domains, including 1M culturally-relevant images sourced from LAION-Multi. 🎨

🏆 PangeaBench: A comprehensive evaluation benchmark featuring 14 datasets in 47 languages. Evaluation can be tricky, so we carefully curated existing benchmarks and introduced two new datasets: xChatBench (human-annotated wild queries with fine-grained evaluation criteria) and xMMMU (a meticulously machine-translated version of MMMU).

🙌 This is a joint leading effort with @yueqi_song. Also kudos to the amazing team @AkariAsai, @seungonekim, @Jeande_d, @simi_97k, @anjali_ruban, @lintangsutawika, @Sathya8NR, @gneubig for their hard work!

Check out more results and insights we conclude from our training in the thread below. 👇

[AI at Meta @AIatMeta]

Today we released Meta Spirit LM — our first open source multimodal language model that freely mixes text and speech.

Many existing AI voice experiences today use ASR to techniques to process speech before synthesizing with an LLM to generate text — but these approaches compromise the expressive aspects of speech. Using phonetic, pitch and tone tokens, Spirit LM models can overcome these limitations for both inputs and outputs to generate more natural sounding speech while also learning new tasks across ASR, TTS and speech classification.

We hope that sharing this work will enable the research community to further new approaches for text and speech integration.

[Tony Wu @tonywu_71]

🚀 Exciting news: ColQwen2, our latest SOTA visual retriever, now supports similarity maps with colpali-engine 0.3.2! Read this thread for more insights. (1/N 🧵)

Can’t wait to try this out? Check out the companion cookbook here: https://t.co/7rjqDlMH4K 👀

[AK @_akhaliq]

Google presents VidPanos

Generative Panoramic Videos from Casual Panning Videos

[Zhenjun Zhao @zhenjun_zhao]

nvTorchCam: An Open-source Library for Camera-Agnostic Differentiable Geometric Vision

@daniel_lichy, Hang Su, Abhishek Badki, @jankautz, @0razio

tl;dr: in title

https://t.co/5jd6utKXcZ
https://t.co/MM1t2x7ArB

[Xihui Liu @XihuiLiu]

#neurips2024 To promote research in long video generation, we propose a new dataset, LVD-2M, with long-take videos (at least 10 seconds without cuts), large motion, diverse contents, and annotated with temporally dense captions.
https://t.co/fUkw6bL6bH
https://t.co/UYIWCgD1wY

[Huaxiu Yao ✈️ ICML 2026 @HuaxiuYaoML]

🚀 We introduce MMed-RAG, a powerful multimodal RAG system that boosts the factuality of Medical Vision-Language Models (Med-LVLMs) by up to 43.8%! 🩺💡

🔍 MMed-RAG enhances alignment across medical domains like radiology, pathology, and ophthalmology with a domain-aware retrieval mechanism. And it tackles three key challenges in alignment of multimodal RAG ✏:

1️⃣ Direct Copy Homework from Others❌ Think it by Self ✅
MMed-RAG helps Med-LVLMs avoid blindly copying external information by encouraging the model to rely on its own visual reasoning when solving complex problems. 🧠

2️⃣ Cannot Solve Problems by Self❌ Learn How to Copy ✅
When Med-LVLMs are unsure, MMed-RAG teaches the model to intelligently use retrieved knowledge, pulling in the right information at the right time, boosting accuracy, and reducing errors. 📚🔍

3️⃣ Copied Homework is Wrong❌ Avoid Interference from Incorrect Homework ✅
MMed-RAG prevents models from being misled by incorrect retrievals, reducing the risk of generating inaccurate medical diagnoses. 🚫🛑

📊 The results speak for themselves: a 43.8% improvement in factual accuracy across tasks like Medical VQA and report generation. This makes Med-LVLMs more reliable and trustworthy in critical healthcare applications! 🌟

MMed-RAG has the potential to be extended beyond healthcare, offering solutions for more general domains where factual accuracy and reliable retrieval are critical!🌍💡

Led by @richardxp888, and nice collab. w/ @WeijiaShi2, @wangshengpkucn, @linjunz_stat, @james_y_zou.

Check out our full project for more details:
👉 Paper: https://t.co/Q2gm3oPJwF
👉 Code (release soon): https://t.co/JSAr83zk38

[AI at Meta @AIatMeta]

New AI research from Meta – CoTracker3 Simpler and Better Point Tracking by Pseudo-Labelling Real Videos.

More details ➡️ https://t.co/b1uoFo7S3g
Demo on @huggingface ➡️ https://t.co/5o5IzC35Nl 

Building on our previous work on CoTracker, this new model demonstrates impressive tracking results where points can be tracked for a long time even when they're occluded or leave the field of view. CoTracker3 achieves state-of-the-art, outperforming all recent point tracking approaches on standard benchmarks — often by a substantial margin.

We've released the research paper, code and a demo on Hugging Face — along with models available under an A-NC license to support further research in this space.