Pat Rufino

PhD in CS; Professor and researcher at University of São Paulo @usponline; AI+Drug Discovery; XAI

[Pat Rufino @RuthRufino]

#ExercitoDoLulaDominouTudo

[Pat Rufino @RuthRufino]

#LulaTemOMaiorExercitoDoMundo

[Pat Rufino @RuthRufino]

#LulaSeuExercitoTeAma

[Mo Lotfollahi @mo_lotfollahi]

Excited to share ORCA 🐋, accepted at NeurIPS 2026!

Ask a text-to-image model for "a red cube next to a blue ball" and you can easily end up with two blue objects. Each object looks perfect; what breaks is composition. Colours bind to the wrong object, spatial relations flip, and counts drift.

Part of the cause is the text encoder. CLIP reads a prompt almost like a bag of words: it knows which concepts appear, but not how they relate. SD3 and FLUX added a T5 encoder to keep that structure, yet the failures persist. Our view: the information is there, but nothing in training teaches the model to tie it to what the image looks like.

ORCA adds that missing signal with a single training-only loss, at zero inference cost. On DiT-L/2 it beats vanilla and REPA models trained twice as long, with ~3× gains on colour binding and spatial relations.

This is the work of @arshia_hemmat, done before he even started his PhD with us. Joint with @AmirhVahidi, in collaboration with @AmitisShidani1, @ValiSanian, @HesamAsdz, and @ArianYazda31266.

📄 Paper: https://t.co/pTiI1TfoIx

#NeurIPS2026 #GenerativeAI #DiffusionModels #TextToImage #ComputerVision #MachineLearning

[Pat Rufino @RuthRufino]

@renato_rovai SP, Tatuapé. Número de vermelhos na rua me surpreendeu positivamente em relação às outras eleições. Os amarelos, em comparação, diminuíram (alguns usaram tons bem pastéis). Poucos com a camisa da seleção. A maioria de preto ou branco. Me animei novamente 😊🙌🏼

[Yinghui He @yinghui_he_]

Skill Entropy is accepted to #NeurIPS2026! 🎉🥳

[Mutty @smiurtitkii]

I’ll be presenting our paper “Locality-Aware Continual Unlearning for Diffusion Models” at #ECCV2026!

Sat, Sep 12, 10:30 AM–12:30 PM CEST
https://t.co/pCCubOLnr9
https://t.co/NELvStnHxs

[Pat Rufino @RuthRufino]

@systemdesignone Handbook

[Wei Huang @WeiHuang_USTC]

"How Transformers Learn to Plan via Multi-Token Prediction" has been accepted to #COLM2026!

Why does MTP improve planning? We identify gradient decoupling that enables reverse reasoning: look to the goal, then trace the path back.

📄 https://t.co/Y3ODyYcLJm

[Sasha Malysheva @aimalysheva]

New ICML 2026 paper challenges the Platonic Representation Hypothesis: model width and depth mechanically inflate similarity scores, creating a misleading global convergence trend which disappears after that bias correction.

What survives is local neighborhood alignment across image, text, and video models (similar things stay near each other even across very different architectures). They call it the Aristotelian Representation Hypothesis :)

So while two models may not share the same representation space in any strong global sense, they can still agree on local neighborhoods (what is similar to what). That is probably the part we actually use in retrieval, transfer, and multimodal systems, and which could be transferred between learned approximations of different models.

https://t.co/sNU3eRtmHd

[Simon Weber @SimWeberTUM]

What if attention wasn't about matching tokens, but operating in function space?

Glad to share our #ICML2026 paper:
📄 Functional Attention: From Pairwise Affinities to Functional Correspondences

w/ @Jiefang_Xiao @GaoMaolin @stevenygd Daniel Cremers
📄 https://t.co/rhn9NtwrBm

[Han Zhao @hanzhao_ml]

Excited to share our latest #ICML2026 paper on dataset selection/valuation during the post-training stage, led by @cindy2000_sh, in collaboration with our fantastic colleagues from Meta!

The core idea is simple but powerful: treat dataset selection as portfolio allocation. With a principled way to estimate the “expected return” of an auxiliary dataset for a target task, we can leverage classic Markowitz mean-variance optimization to select data more effectively for post-training.

Check out the thread below from @cindy2000_sh for more details!

[Jiarui Liu @Jiarui_Liu_]

Excited to share that our work 📝 "PaperMentor: A Human-Centered Multi-Agent Writing Tutor for AI Research Papers on Overleaf" has been accepted to #ACL2026 Demo!

Most AI writing tools either fix grammar or simulate peer review with a score. Neither gives drafting-stage, text-anchored feedback on narrative, structure and presentation.

PaperMentor comments rather than rewrites: It is a human-centered, multi-agent writing tutor that delivers expert-level, actionable feedback as native inline comments right inside Overleaf, while leaving every revision to you.

It pairs a curated library of 40+ expert skill files (distilled from senior researchers' writing advice) with 12 specialized agents covering methods, results, formatting, terminology, venue norms and more.

In a user study, 90.6% of comments were rated actionable and PaperMentor significantly outperformed a GPT-5.2 baseline without the skill library on both validity and actionability.

Anyone can extend or contribute to the skill library with simple text edits!

📝 Arxiv link: https://t.co/0STd6kasBw
🔗 Live demo: https://t.co/4MNrmxyE0U
💻 Code with skill library: https://t.co/iKKQ1orecq

🧵 How it works below 👇

[Sharon Li @SharonYixuanLi]

Best-of-N sampling is often used to boost LLM performance, but the selection relies on external evaluators, adding cost and bias. What if you could select the best output without any external scoring at all?

Introducing our #ACL2026 paper ModeX: Evaluator-Free Best-of-N Selection for Open-Ended Generation! (led by Hyeong Kyu Choi @HyeonggyuC)

💡 Our key insight: among multiple LLM generations, high-quality outputs tend to cluster together semantically. The best answer is the modal one: the generation that captures the dominant consensus.

How ModeX works:
1⃣ Build a similarity graph over N candidate generations
2⃣ Recursively apply spectral clustering via the Fiedler vector to isolate the dominant semantic cluster
3⃣ Select the centroid of that cluster as the final output

No reward models. No external evaluators. No auxiliary inference. Just the texts themselves.

📊 Results across text summarization (CNN/DailyMail), code generation (HumanEval), and math reasoning (Math-500) show ModeX consistently outperforms single-path and multi-path baselines, achieving state-of-the-art among evaluator-free methods.

We also provide theoretical justifications connecting our graph-based mode selection to kernel density estimation, grounding the approach with principled foundations.

📄 Paper: https://t.co/QfPXgqmUHF
💻 Code: https://t.co/YdCpupdnMp

Sometimes the best signal is already hiding in the samples; you just need to find the mode. 🎯

[Huan Sun @ COLM2026 @hhsun1]

Our work, titled "When Benign Inputs Lead to Severe Harms: Eliciting Unsafe Unintended Behaviors of Computer-Use Agents", has been accepted to #ICML2026! It's critical to proactively and systematically study when an agent goes misaligned and performs harmful actions, even without any adversarial manipulation. "Proactive safety", in contrast to passive patching after a safety incident, is what we need to measure and forecast safety risks before wide deployment.

[Stephan Rabanser @steverab]

Very excited to share that our paper "Towards a Science of AI Agent Reliability" was accepted at ICML 2026! See you in Seoul! 🎉

We just released our camera ready version with three important updates (details below). We also recorded a short video on the paper's contributions.

Main changes (full discussion at https://t.co/1a5r1jNFF4):
1️⃣We have added the latest set of frontier models to our evaluation (GPT 5.5, Gemini 3.1 Pro and 3.5 Flash, and Claude Opus 4.7) and find that they are not meaningfully more reliable than previously released models. Agent reliability is still far from being solved.
2️⃣We have updated the definition and measurement of our outcome consistency metric, which contained a typo in the pre-print we initially released. This caused us to under-estimate outcome consistency in our initial set of results. We have updated the paper and our codebase to the corrected metric. Despite this change, our new results show that outcome consistency is still surprisingly low across many reported models.
3️⃣We discovered multiple issues in our HAL Generalist Agent scaffold that we used for our experiments on GAIA. Notably, we discovered multiple instances of answer leakage and agents cheating on our evaluation. This caused us to slightly over-estimate both accuracy and reliability. At the same time, we noticed that the scaffold was overly constrained in terms of permissible software library imports. This caused us to slightly under-estimate both accuracy and reliability. We have done a rigorous audit of the scaffold and have fixed those issues. Overall, we saw that our resulting accuracy and reliability numbers are not meaningfully impacted by this change when compared to our original numbers.

📄Our paper: https://t.co/HAKHzASrOZ
📊Our dashboard: https://t.co/apbtxtsdvz
🎥Short video: https://t.co/uqIourw6C6

Joint work w/ @sayashk, @PKirgis, @khl53182440, @SaitejaUtpala, and @random_walker.

[Xin Eric Wang @xwang_lk]

𝐓𝐡𝐞 𝐁𝐢𝐭𝐭𝐞𝐫 𝐋𝐞𝐬𝐬𝐨𝐧 𝐨𝐟 𝐀𝐠𝐞𝐧𝐭𝐢𝐜 𝐌𝐞𝐦𝐨𝐫𝐲: memory should be a derived capability that exists because it makes an agent better at acting over time.

𝐖𝐨𝐫𝐥𝐝𝐌𝐞𝐦𝐀𝐫𝐞𝐧𝐚 is designed around this principle. Rather than evaluating memory as a storage problem, WorldMemArena evaluates memory through 𝐚𝐜𝐭𝐢𝐨𝐧–𝐰𝐨𝐫𝐥𝐝 𝐢𝐧𝐭𝐞𝐫𝐚𝐜𝐭𝐢𝐨𝐧, instrumenting the full write → maintain → retrieve → use lifecycle across 400 multimodal, multi-session tasks.

And it exposes the findings that should mark the end of the storage-centric era:
→ Storage ≠ use. Better memory storage and retrieval do not necessarily produce better task performance. Optimizing the component we designed does not optimize the capability we actually care about.
→ Harness-based memory performs best where memory is hardest. Agents that can write files, reorganize context, create artifacts, and interact with persistent environments adapt most effectively in long-horizon settings. They are costly and unstable today, which is exactly what many Bitter Lesson transitions look like before scaling and learning take over.

The deeper move is in what gets measured. Memory shouldn't get a score; it should be inferred from capability: how much does remembering improve performance over time.

WorldMemArena drags evaluation off the static object and into the action–world loop, the only place you can tell whether an agent has developed memory or is just simulating it convincingly.

[Lin Ai @_Lin_Ai_]

🎉 #ICML2026 paper from my @AdobeResearch summer '25 internship!

TL;DR: SteER proactively pauses mid-process to invite user steering + adapts a live persona — +7.83–22.80% improved alignment.

📄 https://t.co/sGIjUh11mb

#ICML2026 #LLM #DeepResearch #LLMAgents

[Gouki Minegishi @GoukiMinegishi]

Our paper was accepted as a #ICML2026 Spotlight!

Reasoning in LLMs has improved largely by chaining local steps. But is that the whole story?
Humans occasionally make inferential "leaps" across domains, a faculty known as analogy.
We design a synthetic task to show how small Transformers acquire analogical reasoning, and find that the same signatures appear in pretrained LLMs.

arxiv: https://t.co/1WCizIKWly
code: https://t.co/82kOKCtJo7

[Peter Chen @PeterLauLukCh]

Check RACO, accepted as an 𝗢𝗿𝗮𝗹 paper to #ICML2026 (𝗧𝗼𝗽 𝟬.𝟳%)✨

we propose a new conflict-averse optimization scheme for LLM multi-objective finetuning, with counterintuitive theoretical acceleration and better empirical pareto frontier.

paper: https://t.co/pDvjn5fYR4