Atin Pothiraj

Research Assistant @ MURGe-Lab w/ @mohitban47, Founding Eng @getsapien

[Atin Pothiraj @AtinPothiraj]

Turns out AI video can render a perfect pillow, then forget it should dent when a ball lands on it. These physics slip-ups are everywhere once you start looking, and they're exactly what current metrics tend to miss.

We built PQSG to catch exactly these physics failures, and I'm thrilled to see it accepted to #ECCV2026! Thread👇

[Yue Zhang @zhan1624]

AI video looks incredible until something moves.

Paper towels dissolve instead of soaking up water. Balls bounce off pillows like they're on a trampoline. Gravity stops being a law and starts being a suggestion.

Physics is the thing AI video still struggles with. We introduce our #ECCV2026 paper --- Physics Question Scene Graph (PQSG), a fine-grained evaluation framework for measuring physical plausibility in text-to-video generation.

– Diagnose, not just score
Identify exactly which objects, actions, and physical interactions fail in generated videos.

– Dependency-aware evaluation
Evaluate physics only when prerequisite objects and actions are correctly generated.

– Human-aligned physical realism
Achieve a stronger correlation with human judgments than existing video evaluation metrics.

[Jaemin Cho @jmin__cho]

Look closely at modern AI video generators, and you'll notice subtle (and not-so-subtle) physics slip-ups that standard metrics completely miss.

To pinpoint these errors, we introduce our #ECCV2026 work, Physics Question Scene Graph (PQSG), a framework that evaluates AI-generated videos for physical realism, as well as objects and actions, using fine-grained evaluation questions and dependencies.

We also release FinePhyEval, a fine-grained evaluation benchmark. It has evaluated leading models like Sora, Veo, Wan, and Cosmos-Predict, and now it's time to evaluate yours!

Great work led by @atinpothiraj. Check more 🧵👇

[Elias Stengel-Eskin @EliasEskin]

If we want video generation to act as a world model, models must follow physics laws – something current models struggle with. Many metrics miss these failures, so in our new #ECCV2026 paper we introduced PQSG:

PQSG measures physical realism by decomposing a prompt into a graph of object, action, and physics questions and their dependencies, leading to fine-grained eval of exactly what went wrong in a video! Scores correlate better with human judgments and will continue to improve w/ better VLMs!

🧵👇

[Atin Pothiraj @AtinPothiraj]

Huge shoutout to @jmin__cho @zhan1624 @EliasEskin @mohitban47 for their phenomenal support and mentorship throughout.

[Atin Pothiraj @AtinPothiraj]

Turns out AI video can render a perfect pillow, then forget it should dent when a ball lands on it. These physics slip-ups are everywhere once you start looking, and they're exactly what current metrics tend to miss.

We built PQSG to catch exactly these physics failures, and I'm thrilled to see it accepted to #ECCV2026! Thread👇

[Yue Zhang @zhan1624]

AI video looks incredible until something moves.

Paper towels dissolve instead of soaking up water. Balls bounce off pillows like they're on a trampoline. Gravity stops being a law and starts being a suggestion.

Physics is the thing AI video still struggles with. We introduce our #ECCV2026 paper --- Physics Question Scene Graph (PQSG), a fine-grained evaluation framework for measuring physical plausibility in text-to-video generation.

– Diagnose, not just score
Identify exactly which objects, actions, and physical interactions fail in generated videos.

– Dependency-aware evaluation
Evaluate physics only when prerequisite objects and actions are correctly generated.

– Human-aligned physical realism
Achieve a stronger correlation with human judgments than existing video evaluation metrics.

[Pranav Ravella @pranavravella]

At Sapien, we love surrounding ourselves with people who are curious, sharp, and always up for a challenge.

Yesterday’s NYC trivia night delivered all of that and more. Incredible turnout, great conversations, and a reminder that there’s something really special about getting a bunch of sharp people together to solve hard problems.

Huge thank you to everyone who came out. We’re excited to keep building this community and the team behind it.

We’re growing quickly at Sapien. If you’re interested in joining, roles are in the comments below.

[Elias Stengel-Eskin @EliasEskin]

🚨 Excited to share DART, a multi-agent multimodal debate framework that uses disagreement between VLM agents to address visual uncertainty. VLM debate stagnates and VLMs can struggle with which tools to call – we use disagreement to recruit visual tools (e.g. OCR, spatial reasoning, captioning).

➡️ VLMs are capable debaters but have similar perceptual shortcomings, so their debates often stagnate on perceptual disagreements.

➡️ DART uses disagreement to detect uncertain perceptual claims and selects from a pool of expert vision tools to resolve perceptual differences.

➡️ Introduces tool-based agreement/confidence scores that re-score each agent based on agreement with the expert tools.

➡️ DART outperforms existing single-agent and multi-agent methods across diverse VQA tasks (A-OKVQA, MMMU, NaturalBench)

🧵👇

[Atin Pothiraj @AtinPothiraj]

@vaidehi_patil_ @mohitban47 Huge congratulations! Well-deserved!

[Zaid Khan @codezakh]

🥳 Honored and grateful to be awarded an NDSEG Fellowship in Computer Science! 💫🇺🇸

Big thanks to my advisor @mohitban47 for his guidance, and shoutout to my lab mates at @unc_ai_group, collaborators, internship advisors, and mentors for their support 🤗

Excited to continue working on environment / data generation and adaptable agents!

[Atin Pothiraj @AtinPothiraj]

Going to be presenting CAPTURe at #ICCV2025 today! Come check it out at Poster 280 during Poster Session 2 (Exhibit Hall 1 from 3-5pm).

[Elias Stengel-Eskin @EliasEskin]

Check out 🚨CAPTURe🚨 -- a new benchmark and task testing spatial reasoning by making VLMs count objects under occlusion.

Key Takeaways:
➡️ SOTA VLMs (GPT-4o, Qwen2-VL, Intern-VL2) have high error rates on CAPTURe (but humans get very low error ✅) and models struggle to reason about occluded objects.

➡️ CAPTURe allows us to controllably test model sensitivity to different features (color, object arrangement, num. of objects).

CAPTURe stands for Counting Amodally Through Unseen Regions, which requires a model to count objects arranged in a pattern by inferring how the pattern continues behind an occluder (an object that blocks parts of the scene).

CAPTURe requires recognizing object patterns, reasoning, and counting, making it a useful testbed for evaluating VLMs on whether they understand occluded patterns and possess spatial understanding skills.

🧵👇

[Zaid Khan @codezakh]

How can an agent reverse engineer the underlying laws of an unknown, hostile & stochastic environment in “one life”, without millions of steps + human-provided goals / rewards?

In our work, we:
1️⃣ infer an executable symbolic world model (a probabilistic program capturing environment dynamics) offline from one life (1 episode), with dynamic graph routing for credit assignment.
2️⃣ develop Crafter-OO, our reimplementation of the Crafter environment that exposes a structured, object-oriented symbolic state and a pure transition function that operates on that state.
3️⃣ implement 20+ executable scenarios to test knowledge of core mechanics + mutators that generate illegal distractor states to probe world model understanding.
4️⃣ introduce an evaluation protocol that measures the ability to distinguish plausible future states from implausible ones + the ability to generate future states that closely resemble reality.
5️⃣ show that the inferred world model can be used as a simulator for planning.

Thread 🧵👇

[Hanqi Xiao ✈️ ICML2026 @hanqi_xiao]

🚨Excited to announce General Correctness Models (GCM): 🔎We find no special advantage using an LLM to predict its own correctness, instead finding that LLMs benefit from learning to predict the correctness of many other models – becoming a GCM. Huge thanks to @vaidehi_patil_, @hyunji_amy_lee, @EliasEskin, @mohitban47 for a great collaboration!

Highlights:
➡️Training 1 GCM is strictly more accurate than training model-specific CMs for all models it trains on (including CMs trained to predict their own correctness).
➡️GCM transfers without training to outperform direct training on OOD models and datasets.
➡️GCM (based on Qwen3-8B) achieves +30% coverage on selective prediction vs much larger Llama-3-70B’s logits.

[Elias Stengel-Eskin @EliasEskin]

🚨 Announcing Generalized Correctness Models (GCMs) 🚨Finding that LLMs have little self knowledge about their own correctness, we train an 8B GCM to predict correctness of many models, which is more accurate than training model-specific CMs, and outperforms a larger Llama-3-70B’s self-emitted confidences in downstream selective prediction tasks.

We motivate GCMs and analyze them by answering 2 questions:

❓ RQ1: Are LLMs better than other LLMs at predicting their own correctness? We find that they are not, instead historical information (past LLM outputs and their correctness) drives performance, motivating cross-model transfer and training of GCMs!

❓ RQ2: How can we use historical information from multiple models for correctness prediction? Within RQ2, we explore 3 further subquestions, informing the design of GCMs:

1⃣ How does confidence prediction generalize across models? GCMs transfers strategies across models and datasets, even beating models trained directly on OOD datasets.

2⃣ What information should GCMs condition on? The exact way an LLM phrases an answer is a strong predictor for correctness + strategies leveraging world-knowledge seem to drive generalization.

3⃣ How do alternative methods for encoding history (e.g. post hoc calibration, ICL) compare? Including historical information ICL can aid larger models to predict correctness but underperforms GCMs, and post hoc calibration can complement GCMs to reduce calibration error.

🧵👇

[Atin Pothiraj @AtinPothiraj]

Highly recommend attending!

[Mohit Bansal @mohitban47]

Looking forward to giving this Distinguished Lecture at Virginia Tech on Friday & meeting the awesome faculty+students there! 🙂

Will also discuss some of our new work in multi-agent compositional privacy risks+mitigation, ambiguity/loophole exploitation by LLMs, and bridging MLLMs+Diffusion models (as well as real-world applications of classroom education engagement and mixture-of-agents medical multimodal reasoning).

Feel free to join/stop by! 👇👇

[Duy Nguyen @duynguyen772]

🎉 Thrilled to share that our paper LASeR on using multi-armed bandits to select reward models for LLM training has been accepted to #NeurIPS2025!

Check the post for more paper details and camera-ready updates. See you in San Diego!
👇

[Atin Pothiraj @AtinPothiraj]

@RonNachum Winners hydrate correctly

[Jaemin Cho @jmin__cho]

📢 Introducing RotBench, which tests whether SoTA MLLMs (e.g., GPT-5, GPT-4o, o3, Gemini-2.5-pro) can identify the rotation of input images (0°, 90°, 180°, and 270°). Even frontier MLLMs struggle at this spatial reasoning task that humans solve with >98% Acc.

➡️ Models struggle even with various forms of auxiliary information (such as captions, depth maps, and segmentation maps), CoT reasoning, ICL, or other guided reasoning approaches.

➡️ Fine-tuning on OOD data helps, but does not completely eliminate the problem.

🧵👇

[Hanqi Xiao ✈️ ICML2026 @hanqi_xiao]

🎉 Excited to share that TaCQ (Task-Circuit Quantization), our work on knowledge-informed mixed-precision quantization, has been accepted to #COLM2025 @COLM_conf!

Happy to see that TaCQ was recognized with high scores and a nice shoutout from the AC – big thanks to @EliasEskin @yilin_sung and @mohitban47 for the mentorship!

[Atin Pothiraj @AtinPothiraj]

@hyunji_amy_lee @unccs @mohitban47 @unc_ai_group Welcome to Chapel Hill! 🥳

[Atin Pothiraj @AtinPothiraj]

Thrilled to see CAPTURe accepted to #ICCV2025 - another huge thanks to @EliasEskin @jmin_cho and @mohitban47 for their mentorship and guidance!
We created CAPTURe to test spatial reasoning in VLMs by having them count objects arranged in patterns that are occluded. 🧵👇 for more!

[Elias Stengel-Eskin @EliasEskin]

🎉 Excited to share that CAPTURe has been accepted to #ICCV2025! CAPTURe is a new benchmark for VLM reasoning that requires completing patterns to count objects that are occluded from view.

We find that SOTA VLMs struggle with both counting and reasoning about partial patterns! Great work led by Atin, see 🧵👇 for more details!

[Jaemin Cho @jmin__cho]

Sharing some personal updates 🥳:
- I've completed my PhD at @unccs! 🎓
- Starting Fall 2026, I'll be joining the Computer Science dept. at Johns Hopkins University (@JHUCompSci) as an Assistant Professor 💙
- Currently exploring options + finalizing the plan for my gap year (Aug 2025 - Jul 2026), so feel free to reach out! 🔎

Endless thanks to my amazing advisor @mohitban47, the @uncnlp group, my partner @HeesooJang2, and my family. I couldn’t have done this without your constant support 🙏

Also, a heartfelt shoutout to all the collaborators I’ve worked with over the years—your ideas, encouragement, and hustle have meant the world. Excited for what’s ahead. Let’s keep building together! ❤️