We often hear that "computer vision has been solved.โ But is it really so?
๐ Excited to share our new work: ๐๐ฉ-๐๐ฟ๐ฒ๐ป๏ฟฝ๏ฟฝ๏ฟฝ: ๐๐ป ๐ข๐ฝ๐ฒ๐ป ๐๐ฒ๐ป๐ฐ๐ต๐บ๐ฎ๐ฟ๐ธ ๐ณ๐ผ๐ฟ ๐๐ป๐๐๐ฟ๐๐ฐ๐๐ถ๐ผ๐ป๐ฎ๐น ๐๐ผ๐บ๐ฝ๐๐๐ฒ๐ฟ ๐ฉ๐ถ๐๐ถ๐ผ๐ป ๐ฃ๐ฟ๐ผ๐ฏ๐น๐ฒ๐บ ๐ฆ๐ผ๐น๐๐ถ๐ป๐ด ๐๐ถ๐๐ต ๐๐๐บ๐ฎ๐ป-๐๐ ๐๐ผ๐น๐น๐ฎ๐ฏ๐ผ๐ฟ๐ฎ๐๐ถ๐๐ฒ ๐ฃ๐ฟ๐ฒ๐ณ๐ฒ๐ฟ๐ฒ๐ป๐ฐ๐ฒ๐.
In this paper, we define ๐ถ๐ป๐๐๐ฟ๐๐ฐ๐๐ถ๐ผ๐ป๐ฎ๐น ๐ฐ๐ผ๐บ๐ฝ๐๐๐ฒ๐ฟ ๐๐ถ๐๐ถ๐ผ๐ป ๐ฝ๐ฟ๐ผ๐ฏ๐น๐ฒ๐บ ๐๐ผ๐น๐๐ถ๐ป๐ด ๐ถ๐๐ฉ๐ฃ๐ฆ as a broader formulation of image editing: given a real input image and a natural-language instruction, a system must produce an edited output that realizes the requested transformation while satisfying explicit preservation, geometric, physical, and usability constraints.
๐งฉ To support this direction, we introduce ๐๐ฉ-๐๐ฟ๐ฒ๐ป๐ฎ, an open benchmark designed for professional-grade visual editing and problem solving.
๐๐ฉ-๐๐ฟ๐ฒ๐ป๐ฎ contains:
โ 12K high-resolution real-image instruction pairs
โ 16 instruction-based visual task types
โ Tasks spanning restoration, enhancement, computational photography, physically grounded object insertion, semantic manipulation, geometry-driven structural editing, and typography recovery
โ Real-world images with native aspect ratios and high-resolution details
๐ We also introduce ๐๐ผ๐ด๐ฅ๐ฒ๐๐ฟ๐ถ๏ฟฝ๏ฟฝ๐๐ฒ๐ฟ, a dual-track retrieval and curation pipeline that combines targeted web search, agentic query refinement, verification, and traceability to construct diverse and legally traceable benchmark data.
โ๏ธ For evaluation, we propose ๐๐ฐ๐๐ถ๐๐ฒ ๐๐น๐ผ, a human-AI collaborative preference protocol. Instead of relying purely on automatic metrics or fully human annotation, Active Elo combines:
1. ๐๐ฉ-๐๐๐ฑ๐ด๐ฒ, a logic-gated, multi-dimensional VLM evaluator
2. selective routing of ambiguous high-quality comparisons to expert human raters
3. reliability-weighted Elo updates to aggregate mixed human and AI supervision
This allows us to evaluate models at scale while preserving alignment with expert human preferences.
๐ We benchmark 21 systems, including proprietary, open-source, and agentic models. Our results reveal persistent gaps in instruction adherence, physical reasoning, structural control, and fine-grained detail preservation.
๐ค Finally, we develop ๐๐ฉ-๐๐ด๐ฒ๐ป๐, a lightweight agentic baseline that combines planning, editing, and verification. The results suggest that closed-loop reasoning is a promising direction for professional-grade instruction-following visual editing.
๐ก The main takeaway: as visual AI moves toward real workflows, the challenge is no longer only to generate visually plausible images. Models must also understand intent, preserve constraints, reason about structure and physics, and verify whether the edit actually solves the requested visual problem.
๐ฃ๐ฟ๐ผ๐ท๐ฒ๐ฐ๐: https://t.co/Jzl3UhtD7m
๐๐ผ๐ฑ๐ฒ: https://t.co/ShuNqGnCc4
#ComputerVision #GenerativeAI #MultimodalAI #ImageEditing #AIAgents #Benchmarking #CVArena #TAMU
BREAKING: Claude can now research like a Stanford PhD student.
Here are 9 insane Claude prompts that turn 40+ research papers into structured literature reviews, knowledge maps, and research gaps in minutes (Save this)
๐ Excited to share that our paper "๐ฅ๐ฒ-๐๐น๐ถ๐ด๐ป: ๐๐น๐ถ๐ด๐ป๐ถ๐ป๐ด ๐ฉ๐ถ๐๐ถ๐ผ๐ป ๐๐ฎ๐ป๐ด๐๐ฎ๐ด๐ฒ ๐ ๐ผ๐ฑ๐ฒ๐น๐ ๐๐ถ๐ฎ ๐ฅ๐ฒ๐๐ฟ๐ถ๐ฒ๐๐ฎ๐น-๐๐๐ด๐บ๐ฒ๐ป๐๐ฒ๐ฑ ๐๐ถ๐ฟ๐ฒ๐ฐ๐ ๐ฃ๐ฟ๐ฒ๐ณ๐ฒ๐ฟ๐ฒ๐ป๐ฐ๐ฒ ๐ข๐ฝ๐๐ถ๐บ๐ถ๐๐ฎ๐๐ถ๐ผ๐ป" has been accepted to EMNLP 2025 (Main Track)! ๐
Large Vision-Language Models are game-changers โก๏ธโ they enable powerful multimodal applications across domains. But one major issue remains: hallucinations ๐. These errors limit trust and make deployment in real-world scenarios risky, especially in embodied applications such as autonomous vehicles and robotics.
We introduce a new fully-open framework called ๐ฅ๐ฒ-๐๐น๐ถ๐ด๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ that tackles this challenge head-on using the interplay of two awesome techniques, RAG + DPO with some tailored designs:
โข ๐ ๐ฅ๐ฒ๐๐ฟ๐ถ๐ฒ๐๐ฎ๐น-๐๐๐ด๐บ๐ฒ๐ป๐๐ฒ๐ฑ ๐ฃ๐ฟ๐ฒ๐ณ๐ฒ๐ฟ๐ฒ๐ป๐ฐ๐ฒ๐: Instead of brute-force preference data, we use image retrieval to generate more realistic visual-textual contrasts, creating a dual-preference dataset.
โข ๐ง ๐ฟ๐๐ฃ๐ข (๐ฟ๐ฒ๐๐ฟ๐ถ๐ฒ๐๐ฎ๐น-๐ฎ๐๐ด๐บ๐ฒ๐ป๐๐ฒ๐ฑ ๐๐ถ๐ฟ๐ฒ๐ฐ๐ ๐ฃ๐ฟ๐ฒ๐ณ๐ฒ๐ฟ๐ฒ๐ป๐ฐ๐ฒ ๐ข๐ฝ๐๐ถ๐บ๐ถ๐๐ฎ๐๐ถ๐ผ๐ป): An extension of DPO that incorporates visual preference objectives during fine-tuning, grounding outputs in actual visual cues.
โข ๐ ๐ฆ๐ฐ๐ฎ๐น๐ฎ๐ฏ๐น๐ฒ & ๐ฅ๐ผ๐ฏ๐๐๐: Our method consistently reduces hallucinations across VLM sizes and architectures, while also improving general VQA benchmarks.
โข ๐ ๐๐บ๐ฝ๐ฎ๐ฐ๐: By improving trustworthiness, RE-ALIGN opens the door to safer applications in healthcare, autonomous driving, and beyond.
This milestone is a step toward building more reliable and aligned multimodal AI systems. ๐ Immensely grateful to my brilliant co-authors for making this journey possible!
๐ Preprint: https://t.co/hWCcwgSVuL
๐ Website: https://t.co/mdkE7yTOR1
๐ป Code: https://t.co/B5W0GrfxXw
The Llama 3.2 1B and 3B models are my favorite LLMs -- small but very capable.
If you want to understand how the architectures look like under the hood, I implemented them from scratch (one of the best ways to learn): https://t.co/ODlwRfONOz
Interested in what density models (e.g., EBMs) and Lyapunov functions have in common, and how they can help provide for safe(r) reinforcement learning? @katie_kang_'s new BAIR blog post provides an approachable intro to Lyapunov density models (LDMs): https://t.co/MiHkzw2us0
Letโs rush to 200k followers and talk about when you can start earning GMT during this weekendโs Discord AMA! To get there, we are doing another Uncommon Genesis Sneaker giveaway by 10th April! To enter the crowd prize:
1โฃ Follow us
2โฃ Retweet
3โฃ Tag 3 friends & comment below
We reached 70k followers milestone! To thank to our users, we will give away ONE uncommon and TWO common Shoeboxes to three lucky winners if we reach 100k followers by 30th Mar! To enter the crowd prize:
1โฃ Follow us on twitter
2โฃ Retweet
3โฃ Tag three friends and comment below
How to make steady progress in my research?
I worked so damn hard but "IT JUST DOESN'T WORK!"๐ค
How can I unblock myself quickly and make good progress toward the goals?
Below I compiled a list of tips that I found useful. ๐
"Fair Mixup: Fairness via Interpolation"
Happy to share that the work done during my internship at IBM Research AI was accepted at ICLR 2021!
Paper: https://t.co/uN28ntnwfV
Code: https://t.co/vaQigPhooR
with Youssef Mroueh
https://t.co/XvpCWnnSjg How to do Research
At the MIT AI Lab (1988).
Almost all advices are still valid more than three decades later. Highly recommended.
I had a TON of fun talking to Lex about the game-theoretic perspective on coordinating with people and value alignment, capitalizing on leaked information from humans, modeling humans as rational under different beliefs, and also personal stories! https://t.co/JcJGLkdNZz
CNNs are biased towards high-frequency textural information.
New work on fixing CNN's over-reliance on texture through a curriculum that exposes texture slowly.
Results in better features that generalize both to new datasets and to new tasks.
Paper: https://t.co/clqThZHxbh
If youโre looking for a paper for your Friday readings, maybe check out our new work!
Joint work with @hugo_larochelle & @animesh_garg!
If you want to improve your CNN but donโt want to add trainable parameters or add any regularization loss, give our paper a try! ๐