Ziqi Gao

PhD @uwcse @RAIVNLab

[Ziqi Gao @ziqigaothu]

Vision-language grounding is typically formulated around language-conditioned visual localization: language specifies or guides what to look for, and the model localizes the corresponding visual entities.

Can grounding instead explicitly recover the correspondence on both sides?

Excited to share ConCor-1: Vision-Language Grounding as Bidirectional Concept Correspondence
📄 https://t.co/CMQKvG4uOm

🧵

[Ziqi Gao @ziqigaothu]

Excited to share our #ECCV2026 work: Synthetic Visual Genome 2 (SVG2)! 🎬

📄 Paper: https://t.co/hYIzOzYI5X
🌐 Project: https://t.co/fn9JGjti04
🤗 Data: https://t.co/v6AAHxKBoj
🤗 Model: https://t.co/xPjqzN2iq7
💻 Code: https://t.co/F9mGuFCczn

Video scene graphs provide a structured representation for video understanding and reasoning, explicitly capturing objects, their attributes, and how their relationships evolve over time. However, dense, high-quality video scene graph annotations remain scarce and costly to collect at scale.

We introduce SVG2: ~636K videos with 6.6M object trajectories, 52M attributes, and 6.7M relations, built through a fully automated annotation pipeline. It captures objects and their relationships across space and time, from spatial arrangements to actions and interactions.

Building on SVG2, we train TraSeR, a trajectory-grounded vision-language model for video scene graph generation. It combines context from each object’s full trajectory with local temporal details to capture both object semantics and changing relationships.

On multiple benchmarks, TraSeR outperforms all evaluated open-source baselines and surpasses GPT-5 in object and attribute prediction. Its generated scene graphs also improve downstream video question answering.

Huge thanks to my co-authors and collaborators for making this work possible! @JieyuZhang20, @wizdom_dominic, @jjaesungpark, Tario You, @DaniOgbu, @Michael3014018, @weikaih04, @yinuoy01, @winsonhan, Quan Kong, Rajat Saini, @RanjayKrishna.

[Ziqi Gao @ziqigaothu]

@nmwsharp @uwcse Congrats!!

[Ziqi Gao @ziqigaothu]

6/
Huge thanks to my co-authors and mentors, @JieyuZhang20, @LukeZettlemoyer, and @RanjayKrishna!

🌐 Project: https://t.co/qyHJOFOLkB
📄 Paper: https://t.co/CMQKvG4uOm
🤗 Model: https://t.co/8mufIqYYax
🤗 Data: https://t.co/Ty4q3Qqt1S
🎨 Demo: https://t.co/fi5thbfFzC

Would appreciate any feedback!

[Ziqi Gao @ziqigaothu]

Vision-language grounding is typically formulated around language-conditioned visual localization: language specifies or guides what to look for, and the model localizes the corresponding visual entities.

Can grounding instead explicitly recover the correspondence on both sides?

Excited to share ConCor-1: Vision-Language Grounding as Bidirectional Concept Correspondence
📄 https://t.co/CMQKvG4uOm

🧵

[Ziqi Gao @ziqigaothu]

5/
The gains are largest where correspondence is hardest:

📈 +48% correspondence F1 on the long-caption benchmark
📈 +29% on zero-shot LVIS-Minival
For LVIS-Minival, ConCor-1 can take the entire category vocabulary in a single forward pass, without splitting it into chunks.

[Ranjay Krishna @RanjayKrishna]

📣 Call for Papers — 1st Workshop on Multimodal Digital Agents (MDA) @ ECCV 2026

How should AI agents see and act across web, desktop, and mobile interfaces? We're bringing together work on vision-centric digital agents — perception, reasoning, benchmarks, safety, and more.

🗓️ Sept 8, 2026 (ECCV) · ✍️ Submit by July 29 (AoE)
📄 Full paper (≤14 pages, ECCV-accepted welcome) or extended abstract (≤4 pages) — non-archival, 🏆 best paper award
🎤 Speakers include Graham Neubig, Ranjay Krishna, Qianhui Wu & Alexandre Drouin

Submit / details → https://t.co/DQsfFPtyzp

#ECCV2026 #ComputerVision #AIagents #Multimodal #GUIagents

[Ziqi Gao @ziqigaothu]

@YueYangAI @allen_ai @thinkymachines Congrads!

[Grigory Sapunov @che_shr_cat]

1/
The "20 tokens per parameter" Chinchilla scaling law is flawed. It is an artifact of your tokenizer. Scaling shouldn't be measured in tokens at all. It should be measured in bytes. 🧵

[Weikai Huang @weikaih04]

Thrilled to announce our latest project at @allen_ai @RAIVNLab: WildDet3D

Humans understand objects in 3D effortlessly -- we see a mug on a desk, judge the distance to a parked car, or estimate the height of a building across the street. For CV / Robotics models, this remains surprisingly hard.

We've built great models that each handle a piece of the puzzle: FoundationPose for 6-DoF pose over tabletops, MoGe 2 for accurate metric depth estimation, SAM for 2D segmentation and tracking. But they're fragmented -- each solves one sub-task, none gives you the full picture: where is this object in 3D, how big is it, and how is it oriented?

Monocular 3D object detection is exactly this task -- recovering the full 3D bounding box of any object from a single RGB image. It's the missing link that connects 2D perception to real-world 3D understanding for robotics, AR/VR, and embodied AI.
vehicles
So why hasn't anyone cracked open-world 3D detection? Data.

Existing 3D datasets (Omni3D, COCO3D) cover fewer than 100 categories, locked to driving corridors and indoor rooms. And the annotation methods -- BEV labelling, point cloud labelling -- fundamentally don't scale to in-the-wild scenes where you don't have LiDAR or a well-reconstructed point cloud. And objects are much more diverse in size/pose compared with vehicle and furniture.

To tackle this: We designed a human-in-the-loop pipeline to change this. We build complex pseudo-3D box generators using different algorithms/models. Then, 1700+ human annotators from Prolific select the best candidate and verify quality.

Along with thousands of annotators for several months, we got the result: WildDet3D-Data -- 1M total images, 13.5K categories of objects, with 100k of all human-verified 3d detection images. That's 138x more category coverage than Omni3D. Street food carts, violins, traffic cones, sculptures -- objects no 3D dataset has ever covered.

With this data, we trained WildDet3D -- a single geometry-aware architecture built on SAM 3 and LingBot-Depth that unifies every way you'd want to interact with a 3D detector:

- Text: "find all chairs"
- Box prompt: click a 2D box, get its 3D box (geometric, one-to-one)
- Exemplar prompt: draw one box, find all similar objects (one-to-many)
- Point prompt: click on an object

And when you have extra depth -- LiDAR, stereo, anything -- just pass it in. The model fuses it and gets substantially better: +20.7 AP on average. No depth? It works fine without it.

Results on our new in-the-wild benchmark (WildDet3D-Bench, 700+ open-world categories): 22.6 AP text / 24.8 AP box -- up from 2.3 AP for the previous best. With depth: 41.6 AP text / 47.2 AP box. Also SOTA on Omni3D (34.2 AP text / 36.4 AP box) with 10x fewer training epochs, and strong zero-shot transfer to Argoverse 2 and ScanNet (40.3 / 48.9 ODS).

[Yinuo Yang @yinuoy01]

Check out our new work, You Only Judge Once: a 4B multimodal reward model that scores all N responses in a single forward pass. It's both faster (up to N×) and more accurate than scoring each in isolation. As a GRPO reward, it also delivers more gains on open-ended generation than single-response RMs.

📄 https://t.co/E43wElF15H
🤖 https://t.co/vsKOHyVNS3
📊 https://t.co/xm1Vsx1hYl
💻 https://t.co/tTuhWywIHC

[Ai2 @allen_ai]

Today we're releasing WildDet3D—an open model for monocular 3D object detection in the wild.

It works with text, clicks, or 2D boxes, and on zero-shot evals it nearly doubles the best prior scores. 🧵

[Ai2 @allen_ai]

Grounding lets vision-language models do more than describe—they can point to where a robot should grasp, which button to click, or which object to track across video frames.

Today we're releasing MolmoPoint, a better way for models to point. 🧵

[Ziqi Gao @ziqigaothu]

Molmo2 goes beyond high-quality captioning and QA by enabling accurate pointing and tracking in images and videos — a leap for VLMs, fully open-source. Happy to contribute to the AI-generated video detection dataset and pointing evaluation benchmark. Huge thanks to the team!

[Ai2 @allen_ai]

Last year Molmo set SOTA on image benchmarks + pioneered image pointing. Millions of downloads later, Molmo 2 brings Molmo’s grounded multimodal capabilities to video 🎥—and leads many open models on challenging industry video benchmarks. 🧵

[Ai2 @allen_ai]

🗓️ Tue Dec 16, 1–2pm PT: AMA with researchers + engineers from our Olmo & Molmo teams, hosted by r/LocalLLaMA (@LocalLLaMAsub).
💬 Ask your questions now—we’ll start answering when the AMA begins!