Rongchang Zhao

Researcher on medical image analysis.

[李富贵 @shenxiankk]

上海交大整了套《动手学大模型》（Dive into LLMs），GitHub 上 Star 数已经飙过 4.2 万，想系统啃大模型又愁没好教程的，这个算对口。

从微调部署一路铺到安全对齐，11 个前沿课题全在里面，随便拎几个感受下：

提示学习与思维链、知识编辑、数学推理、模型水印、越狱攻击、大模型隐写、多模态、GUI Agent、RLHF 安全对齐……

每章都挺实在——PPT 讲义、详细文档、能直接跑的 Jupyter Notebook 代码，三样凑齐，到手就能练。

最近还跟华为昇腾联手出了国产化课程线，把大模型开发全流程拆开讲，PPT、实验手册、配套视频都有，等于多开了一条上手路子。

项目地址：https://t.co/QsmSyd09Bq

[Rongchang Zhao @ByronsZhao]

@Michael_J_Black Exactly

[Michael Black @Michael_J_Black]

PhD Advising vs Vibe Coding #1:
One of the key skills a PhD advisor learns is context switching. Students come into your office every 30-60 minutes and you need to remember their project, where you left off, and what they were supposed to do next. It's a learned skill and exactly what you need to manage AI agents.

[Dominik Schnaus @dominik_schnaus]

DINOv2 has never seen a caption, and Qwen3 has never seen an image. We still aligned their embedding spaces without a single image-caption pair.

It even works when the images and the captions come from different datasets.

Project page: https://t.co/2YHSkXVz1D

⬇️

[elvis @omarsar0]

Banger paper from NVIDIA.

One exciting trend I am seeing is building verifiable environments for your agents and training the harness alongside the model.

Co-evolving the harness and the model is a big part of owning the intelligence stack. And many frontier AI companies have started doing that.

This paper shows how this works:

VERA turns benchmark trajectories into more than 9,000 restartable sandboxes with rubric scoring, and keeps only environments that run and can be scored from observable evidence.

It then updates both the model weights and the harness.

A harness edit is kept only if it passes self-tests and adds at least 5 points on the development set.

The system rejects a model checkpoint if its score drops by more than 20%.

At 9B, the co-evolved agent beats the strongest single-axis baseline by 10.3 points on AutoCoWorkBench and 13.0 points on AutoMedBench. At 27B, it scores 71.6 on AutoCoWorkBench, above Claude Opus 4.8.

The environment corpus is open-sourced.

Paper: https://t.co/Lt9DXDRQp2

Chat with Paper: https://t.co/fXlmBpm4m6

[Jinyan Su @SuJinyan6]

More detailed summarized transcription of the RSI workshop mentioned in my previous article. https://t.co/SJAzCPx0yJ

[elvis @omarsar0]

Banger paper from CMU on harness learning.

(bookmark it)

Also, pay attention to this important new AI engineering skill of improving agents by editing their harness code instead of their weights.

Seeing a huge shift towards this.

The authors train a proposer model with RL to read a task, the current harness and an execution report, then write a code edit to the harness.

The reward is the score of the revised harness. The solver model never changes.

A trained 4B proposer beats its 35B teacher at single-step revision on Reasoning Gym, including task families it never saw in training. A proposer trained on HotpotQA keeps improving harnesses on MuSiQue and 2WikiMultihopQA.

Paper: https://t.co/wpOoUqPF6r

Chat with Paper: https://t.co/1JHXwPwbhM

[Alvin Zhang @ZAlvin39105]

1/8

How do we train a model to adapt to tasks it has never seen, without updating its weights at test time?

With Harness Learning, we train a model to revise harnesses using execution feedback. At test time, the model applies this learned skill to improve harnesses on unseen tasks, while model weights stay fixed.

Improving agent harnesses takes experimentation. We want a model to learn from those revisions and make better edits on unseen tasks.

Models trained with harness learning nearly double their performance on challenging unseen reasoning and multihop question-answering tasks.

[Salman Khan @KhanSalmanH]

The interesting question now is not just what these models can do, but how this changing landscape should shape 𝘄𝗵𝗮𝘁 𝘄𝗲 𝘄𝗼𝗿𝗸 𝗼𝗻 𝗻𝗲𝘅𝘁.

📄 Paper: https://t.co/eNyy9h6G4H
🌐 Webpage: https://t.co/sAQCQubEyd
✍️ Read our 4-min blog: https://t.co/eacBaOEzAF

[Scholarship for PhD @ScholarshipfPhd]

200 papers in one year.

That number is startling. But the bigger question is what happens when AI makes academic publishing scalable.

If publication volume can be generated faster than ever, maybe universities should stop focusing mainly on counting papers and pay more attention to quality, provenance and genuine intellectual contribution.

[Shufan Sun @sofine_sun]

Excited to share our new 𝗼𝗽𝗲𝗻-𝘀𝗼𝘂𝗿𝗰𝗲 work, 𝗛𝗔𝗥𝗠𝗢𝗡𝗬, a framework for reconstructing compositional 3D scenes from single indoor images.

Much like a 3D artist, our VLM agent builds the scene in a 𝗵𝗶𝗲𝗿𝗮𝗿𝗰𝗵𝗶𝗰𝗮𝗹 𝗼𝗿𝗱𝗲𝗿: first the room structure and camera, then wall-mounted objects, free-standing furniture, and finally dependent decorations. After each stage, it compares its render against the input image, reflects on what went wrong, and iteratively corrects the scene to prevent errors from accumulating.

Even with older VLM backbones, our framework remains 𝗰𝗼𝗺𝗽𝗲𝘁𝗶𝘁𝗶𝘃𝗲 𝘄𝗶𝘁𝗵 𝗚𝗣𝗧-𝟲 𝗔𝘀𝘁𝗿𝗮 𝗶𝗻 𝘃𝗶𝘀𝘂𝗮𝗹 𝗾𝘂𝗮𝗹𝗶𝘁𝘆, 𝘄𝗵𝗶𝗹𝗲 𝗮𝗰𝗵𝗶𝗲𝘃𝗶𝗻𝗴 𝗯𝗲𝘁𝘁𝗲𝗿 𝗴𝗲𝗼𝗺𝗲𝘁𝗿𝗶𝗰 𝗮𝗹𝗶𝗴𝗻𝗺𝗲𝗻𝘁 for single-image 3D scene reconstruction.

We also release 𝗛𝗔𝗥𝗠𝗢𝗡𝗬𝟯𝟬𝟬, a benchmark for single-image 3D scene reconstruction with 300 scenes across three difficulty tiers: easy, medium, and complicated.

🔗 Project: https://t.co/rxCKUzkHlD
📊 Leaderboard: https://t.co/viJWQyVHVu
🤗 Data: https://t.co/idDjT0Qz1r
📄 Paper: https://t.co/BqnnrUeFVy

[Michael Black @Michael_J_Black]

This is a thoughtful essay by @ezraklein on the risks of recursive self improvement when we know that the AIs training themselves are demonstrably not aligned with us. https://t.co/8RbRqZASQy

[Jitendra MALIK @JitendraMalikCV]

Toru raises fundamental concerns with which I agree. It is amazing to see the progress in VLMs such as Astra, building on the work of generations of scientists. But Astra doesn't cite what it is building on (I have co-authored a couple of in-hand rotation papers which might have been used in the Astra pen-spinning demo). I fully acknowledge that VLMs make the result of previous research much more accessible to the general public, just like encyclopedias and Google Search did earlier, and that is a good thing. However, in the past, society had mechanisms like patent disclosures and paper citations as ways of doing credit assignment. Should we not accuse these AI models of plagiarism if they don't cite their sources? Somewhat relatedly, I note the open letter by 25 Fields Medalists in response to the Navier Stokes result complaining about the disruption caused to the mathematics research process. Some people interpreted it as turf protection, but the point was more subtle. If certain kinds of creative work stops because incentives are disrupted, it is like farmers eating their seed corn.

[Toru @ToruO_O]

Inspired by recent thoughts from senior researchers whom I deeply respect (e.g. @JitendraMalikCV @Michael_J_Black @Ken_Goldberg @phillip_isola), I wrote down some thoughts as a junior researcher too.

The recent Astra demos in dexterous manipulation are truly impressive. But they also raise two questions that I don’t think academia has good answers to yet:
- How should credit be assigned when an agent synthesizes many prior works into a new research result?
- How can academia attract and retain talents when so much of its incentive structure relies on credit?

I reflect on these questions in more detail in a blog post. Curious to hear what others think, and debates are very welcome!

https://t.co/52qCV6CPMY

[Letian Wang @Letian_Wang_6]

Earlier, quite a few people who attended CVPR told me they came away pretty disappointed — including some extremely senior people in the field. At the time, I didn’t fully understand why. But after I attended ECCV, I finally did.

There was a surprising amount of frustration at the conference, especially since it happened right after the GPT-6 release. People were openly wondering what traditional CV research is going to look like from here, and how much of it will eventually just be absorbed by large foundation models.

Actually, when I gave talks about GenCeption to a few academic groups, I had already started to see that kind of frustration.
GenCeption essentially leverages a large-scale pretrained diffusion model to build a single unified model that can match — and often outperform — specialized SOTA models across a surprisingly broad range of vision tasks, including DepthAnything3, D4RT, VGGT Omega, SAM3, Genmo, Lotus-2, and others. (The code is now open-sourced! https://t.co/TElEid3ZI3)
Quite a few PhD students told me the work made them question the direction of their own research.

Part of me was happy, because it meant GenCeption had genuinely changed how some people thought about the problem. But that feeling was quickly outweighed by how much I felt bad and empathized with the junior PhD students. Some of them seemed genuinely unsure where to go next, especially when years of work on specialized architectures or training recipes could potentially be absorbed by a sufficiently capable general pretrained model. GPT-6 seems to be just creating that same feeling on a much broader scale.
It reminds me a lot of what happened with GPT-1 and BERT — tons of NLP PhDs' years of work and entire research directions perish overnight.

There were even some pretty spicy takes floating around: “CV conferences are dead,” “academia is becoming disconnected from where some of the most important progress is actually happening,” “with increasingly noisy review systems and declining review quality, rejection sometimes is almost a signal of truly fundamental work,” and “conferences increasingly reward work delicately engineered around established topics or methodologies, rather than work truly at the frontier.” Maybe it's too harsh and too absolute, but the frustration behind them was definitely real.

Conferences used to feel so fruitful, but this time it was hard to get genuinely excited. A lot of papers seemed to solve problems that no longer felt that important, or make progress that felt increasingly incremental. In the era of paper inflation, most people are busy trying to survive the research game, rather than thinking about how to go big after one or two years of diving deep into something. Most valuable skills such as critical thinking and taste never get trained well enough, from the beginning, all the way to the end.
On the other hand, most people simply don't have enough context or a big enough picture about what problem actually matters. They get stuck optimizing for local problems that their immediate environment tells them are important. I suffered from this for a long time too, but luckily I gradually started to see a much bigger picture.

My advice to junior PhD students is: when you start a project, try to collect as much context as possible before you even begin — by all means.

One way is to explore the boundaries of existing models as aggressively as you can. Play with the SOTA models extensively. Keep throwing harder and harder problems at them until they fundamentally break. Understand not just what they can do today, but where their capabilities end and why.
Another way is to put yourself wherever the strongest signals are. That could be a visionary academic lab, a team at an industry frontier lab, a genuinely insightful seminar, or simply a senior student or researcher who has seen a much broader landscape than you have. Expand your local context as much as possible.

In an era where things are moving this fast, the information gap alone can be large enough to make years of carefully built work suddenly meaningless. Looking up and making sure you're walking in the right direction matters much more than keeping your head down and perfecting every step along the way — though, of course, being able to do both is the best :)
We stand on the shoulders of giants not just to go further, but to see further — to understand which problems will actually matter in the future.
If a problem is likely to be easily swallowed by large models once they have sufficient data, attention, or business interest, either avoid spending years engineering a specialized solution for it, or contribute the data and benchmarks that will help move the frontier forward faster. The problems that deserve the deepest thinking and genuinely new methods are the ones where existing models fundamentally cannot do the thing you need them to do.

What I actually worry about is not that conferences will suddenly “die,” but that if they start becoming genuinely irrelevant to where the frontier is moving, they will simply fade away.
And maybe one early sign of that is that a lot of people I used to want to meet aren’t really coming to conferences anymore — they’re busy with the big things happening at frontier labs, or, well, busy making money.
It reminds me a little of what happened to control theory as the field matured. As many of the core problems were gradually solved, parts of the community started inventing increasingly artificial use cases, making increasingly unrealistic assumptions, and playing increasingly elaborate mathematical games within them. That possibility genuinely scares me — not because the work becomes unnecessarily sophisticated, but because it can become disconnected from problems that actually matter.

I still believe that conferences will remain places people genuinely want to come to: to exchange ideas, reconnect with old friends, meet new people, and simply have fun together. We just need to keep doing the right things to make that happen :)

[Michael Black @Michael_J_Black]

What is the role of academic computer vision research in the age of increasingly powerful large models? Is GPT-6 Astra a step change? How can a researcher have an impact today in academia?

These are the questions I ask myself as I head off to ECCV 2026, a conference I’ve attended since 1992. One of my papers this year is VIGA, a method that takes an image as input and outputs a 3D Blender scene that represents that image. This is a classical inverse-graphics task and VIGA was the first method to solve it using an agentic approach.

The idea is now several years old and the first version of the paper was rejected. This delayed publication significantly. After it was accepted at ECCV, it was quickly surpassed by people using Claude Code for the same purpose. Today GPT-6 Astra blows away all previous results. But we still head off to ECCV to tell the community about our invention that is now fully out of date.

The way academic work often progresses is that one reads recent papers, notices that they have limitations, comes up with a new idea, explores this, publishes it, etc. Any published paper I read today is based on ideas that are at least a year old. And those ideas were based on the literature of the time, which was also a year old. That means that any paper I see at ECCV is likely two years out of date. In AI today, two years means your work is likely irrelevant.

At CVPR this summer I noticed that many authors have not gotten the message. They continue to work on “old” problems that have a long history. This history is based on assumptions about how the “vision problem” will be “solved”. The truth is that it is being solved in a very different way and many of these problems are no longer relevant. Another group of papers focuses on very niche problems where large models likely fail because of insufficient data or lack of business interest. The impactful papers were largely from industry and had long author lists and massive data+compute behind them. These papers were also out of data, describing systems that had been released months before, but at least they served to provide the community with more complete documentation and analysis of commercial systems.

So what should academics do? First, we need to put aside the tools we’ve used for years and start from scratch. Every project should start by trying really hard to solve the problem with existing tools. I would like to see every paper begin with a detailed experimental analysis of how existing models perform and why they fail (if they do). This gives the kind of insight we need today. Then, assuming current models fail, the solution should provide some fundamental insight that will outlive the next release of such models.

Reviewers today still focus on technical novelty. This pushes people to focus on tweaking architectures rather than clearly moving the field forward. Papers need to be judged based on their novel insight and not their novel technical contribution. This is a real shift in thinking but it focuses us on what matters - progress of the field.

If we want there to be a “field” of computer vision, then it can’t become a marginal backwater, focusing on esoteric problems. If you haven’t tried using Astra (or whatever comes next) to solve your problem, then you have not done your homework. This omission should be seen as negatively as not having a previous work section.

Concretely, I think papers should include a new section analogous to “Related Work” where that related work is current models and how they perform on the task. Reviewers should start asking for this and expecting authors to be able to articulate their insights about the limitations of existing large models.

I'm interested in your thoughts.

[Zhenjun Zhao @zhenjun_zhao]

Sparse auto-regressive modeling for scene generation from multi-view images

Thomas Lucas, Maxime Pietrantoni, @WeinzaepfelP, Wonjune Cho, @bardienus, @Vinc3nt_Leroy, @JeromeRevaud

tl;dr: voxel-aligned 3D latent space + occupancy-aware masked autoregressive transformer

https://t.co/lr1M64GZr3

[Mingyu_Jin19 @fnruji316625]

A very inspiring paper on self-improvement: “From RLVR to RLSVR”

A neat idea: open-ended tasks may not be inherently verifiable, but verifiability itself can be created through task design.

RLSVR (RL with Self-verifiable Reward) transforms the original task into a proxy environment where rewards become automatically verifiable, they instantiate this idea with a self-play game called SpyRL:

🎭 One agent is secretly assigned as the “spy” and receives degraded information, while the other agents receive the full input.
🗳️ A group of agents solve the task and then vote on who the spy is
✅ Because the spy identity is known, the reward is verifiable by construction
📈 Training this way improves performance on summarization, creative writing, and math.

It’s an interesting connection between self-supervised learning and RL post-training: instead of manually providing supervision, redesign the learning problem so that supervision emerges from the task itself.

https://t.co/kZFLQv1kjm

[Xudong Han @Xudong07452910]

最近发现 Stanford Online 把研究生课程 CS329A：Self-Improving AI Agents 的完整课程录像放到 YouTube 了，一共 9 讲。

这门课基本围绕一个问题展开：AI Agent 怎么通过和环境不断交互，持续改进自己的能力。

内容从 Test-Time Compute、Verifier 和 RL 讲起，后面会进入工具/代码反馈、多步推理与规划、Deep Research、自我改进，以及 Agent 的长时任务评测。

我觉得比较好的地方是，它把现在很多散落在论文里的方向串到了一起：从「怎么让 Agent 多思考」，一路讲到「怎么验证结果、从反馈学习，再把这些能力放进更长的任务里」。

最近在关注 Agent 自进化的话，这套课很适合系统补一遍。

YouTube：
https://t.co/2X9JEOlW0j

[Furong Huang @furongh]

Self-improving AI is often framed as a model capability: better reasoning, learning from its own outputs, or acquiring new skills.

But the model is only one layer.

The infrastructure around the model—and the way the system is deployed—can also be designed for self-improvement: how an agent allocates computation, evaluates and revises its actions, and composes or redesigns workflows for each task.

At Berkeley RDI’s Agentic AI Summit, I’ll share this vision in my talk, “Reasoning as Control: Toward Self-Improving Agentic Systems,” drawing on our recent work across three levels:

Thinking → Action → Workflow

More capable models will matter. But the next frontier is also building systems that can improve how those capabilities are used in deployment.

Aug. 1, 3:00 PM at UC Berkeley. Come say hello!