Jeonghwan Kim

PhD student @IllinoisCDS @UIUC_NLP | Research @Meta | Prev. @Amazon

[Jeonghwan Kim @MasterJeongK]

🚨 New Paper Alert! 🚨
Excited to share our most recent work was accepted at NeurIPS 2025 as a Spotlight!
👉 https://t.co/UAJaFIDcj3

We introduce PLUM, a new Large Multimodal Models (LMMs) and PARTONOMY, a benchmark for part-level visual understanding and grounding👇

[Manling Li @ManlingLi_]

Welcome!

[Jayden Teoh @jayden_teoh_]

I don’t quite understand the motivation for forcing the text embeddings toward an isotropic Gaussian. Language naturally occupies an anisotropic semantic space.

Vision pretraining has many possible downstream readouts, e.g., segmentation, robotics, captioning, localization, etc. Therefore, enforcing an isotropic gaussian distribution makes sense here to best prime the pretrained representations for all possible downstream tasks.

Text captions, in contrast, are already compressed semantic descriptions. Their role here is mostly to guide vision pretraining, i.e., provide a stable training signal for the vision encoder, not to become a general-purpose text representation.

I wish there was an ablation experiment where SIGReg is removed from the text side while kept on the image side. Or perhaps use VisReg (@HaiyuWu1 , @randall_balestr ) on the text side since it gives more flexibility than directly forcing full isotropic-Gaussian structure.

[Lukas Kuhn @lukaskuhn77]

🔥 We introduce LeVLJEPA: the first fully non-contrastive end-to-end vision-language pretraining method competitive with CLIP & SigLIP 💪🏼

👀 No negatives. No temperature. No momentum encoder. No teacher-student.

TL;DR: LeVLJEPA learns image to text structure by prediction: each modality predicts the other's embedding, while SIGReg keeps each embedding isotropic Gaussian. 🧵

📄 https://t.co/1qBXor8qTf

[Jiwan Chung @JiwanChung]

I’ll be at @icmlconf to present this work!

If you’re working on MLLMs, representation learning, agents, or related topics, feel free to DM me - I’d be happy to connect.

I’m also graduating this August and open to job opportunities starting around March next year, so please reach out if there may be a good fit.

[Jiseon Kim @ ACL2026 @jiseon_kim1]

I'm at #ACL2026 through July 7! Would love to connect if you work on LLM evaluation, computational moral cognition, pluralistic alignment, or how models handle conflicting human values.

Catch me at Poster Session G (Grand Hall), July 7, 11:00–12:30 👋

[Gill @gurtej__gill_]

Honestly, looking at the Mamba-3 paper gives me a massive sense of relief.

For a while, it felt like we were trapped in a Transformer bottleneck, knowing the O(N^2) KV cache was unsustainable, but watching every linear alternative compromise on state tracking or retrieval.

Tri Dao and Albert Gu's team cut the state size in half compared to Mamba-2 while keeping the same perplexity & its kinda wild.

Boosting accuracy by 1.8 points via complex valued updates and MIMO without a decode latency penalty is pure engineering elegance.

We talk constantly about scaling inference for agentic workflows, but rarely about the crushing hardware costs.

Mamba-3 makes me genuinely optimistic that we can actually afford the agentic future we're building. It's a brilliant piece of work.

Read the full paper here:
https://t.co/yqnRmCUXrJ

[Haonan Chen @HaonanChen_]

Can robots perform complex assembly tasks, like motherboard assembly, by composing simple behaviors?

Yes!

Introducing CoStream: Composing Simple Behaviors for Generalizable Complex Manipulation
https://t.co/4HxZKvCXWc

[Sagnik @ ICML 🇰🇷 @saagnikkk]

Bit late to the party. But will be there @icmlconf in Seoul 🇰🇷 !

Our paper in the intersection of RL and optimization will be presented as an oral presentation on 7th July in Hall C at 10:00 AM.

Please drop by !

[Jim Fan @DrJimFan]

Today, we give robots a /skills library that self-evolves and compounds indefinitely! Introducing ASPIRE: a robot solving its 100th task is no longer as clueless as solving its first. Coding agents observe multimodal sensory traces from simulation and real robots, launch an evolutionary search over control programs, and distill the best know-how into an ever-expanding library.

ASPIRE is a new type of continual learning: "training" is skill refinement instead of gradient descent.
"Trained model" is a repo of sensorimotor skills instead of floating weights.
“Distributed training” is a panel of agents each practicing a different skill instead of sharded minibatches.

Here's the beauty: ASPIRE gives the tired terms "sim2real transfer" and "cross-embodiment transfer" a whole new meaning. Bridging the sim-to-real gap is notoriously brutal. An end-to-end policy has to swallow both the visual shift (sim looks toyish next to a real camera) and the subtle contact physics it never quite gets right. ASPIRE sidesteps the mess, because it doesn't ship pixels or weights across the gap, but ships the know-how. The robot still has to practice in the real world, not zero-shot, but it gets there way faster because it isn't rediscovering the strategy from scratch. Same for going single-arm to bimanual hardware, which usually requires new data and retraining from zero. ASPIRE achieves up to ~10x cut in "transfer learning” tokens (yes, tokens are the new unit of *training* compute ;)

Check out our gallery of 150+ tasks and 90+ skills the robots taught themselves, all on the website! Kind of wild that we can ship the "learned weights" as an HTML page rather than a GGUF. We'll open-source the full stack so your own robot library starts compounding from ours!

Deep dive in thread:

[Hyeonjeong Ha @hyeonjeong_ai]

I’ll be attending #ACL2026 to present MM-PoisonRAG!

Let’s grab coffee if you’re around :)

📄Paper: https://t.co/6xrmzWu128

[Friedemann Zenke @hisspikeness]

1/7 Stop by our #ICML2026 spotlight poster next week!
https://t.co/VhEdW65CPz

We unify self-supervised learning (SSL) algorithms: contrastive, VICReg, stopgrad and more, under a single principle: latent distribution matching (LDM).
Paper: https://t.co/9RrpyCJhrx

[Jinu Lee @ ACL2026 @jinulee_v]

I won't be visiting ICML, but I am here to share my favorite restaurants in Seoul! Happy to give menu-wise recommendations as well. Happy ICML 🇰🇷 (note: I don't know much vegan/GF/... options)

Korean cuisine: https://t.co/XEcFTMnEl7
Others: https://t.co/RJU7iKVvmT

[Jing Yu Koh @kohjingyu]

>Why has progress on computer use been so slow?
I actually think progress has picked up a ton this year. The frontier models (anything past Opus 4.6 / GPT-5.4) are actually usable now. In 2025 none of the models were really reasonable beyond very simple tasks.

>Computer use is so clearly verifiable.
Most computer use tasks have parts that are verifiable, and parts that are not (which is why rubrics can be helpful in grading CUA tasks). This also makes it a bit more complex to RL compared to math/coding. Computer use is also harder to write unit tests for (since you don't always have privileged/API access), so many people use LLM-as-a-judge grading, which is finicky in its own ways.

[Paras Chopra @paraschopra]

Predicting your own latents is a breathtakingly beautiful concept.

Read these papers (or ask claude to explain) and you'll get why:

- Next-Latent Prediction Transformers Learn Compact World Models https://t.co/9EH9HsSEpa

- Learn from your own latents https://t.co/XKQnV52Opz

[Alex Chaloner @alex_chaloner]

"Efficient Pre-Training with Token Superposition"

This paper finds that we can group up tokens to create a much more efficient training regime, achieving 2-4x speedups for the same loss.

[Liu Jiayu ✈ ACL 2026 @JiayuJeff]

📈 Nowadays, agents are crushing leaderboards. But when you ask one painfully normal question:

👤 You: “Hi, I'm Jeff. My phone number is 1234567890. I returned a desk lamp and filed a refund request on June 22 at 10:13 PM. Can you check the current status of my refund?”

Sounds easy.
😵‍💫Then the agent enters API hell:

(1) It dives into a huge API pile and is not sure what to search for.
So it keeps trying: "refund", "return", "status", "order status"…

(2) It actually retrieves a promising refund-related API.
But instead of using it, it keeps searching and drifts to less relevant tools.

(3) It finally calls a refund API, but the endpoint fails.
Instead of changing the plan, it loops around similar APIs again.

(4) Then a shipping-status API returns "delivered."
Clearly not the refund status, but the agent treats it as evidence and keeps going.

After many turns, it has searched, called APIs, collected values, and looked very busy 😂

🚀 To evaluate failures like this, we introduce PlanBench-XL: an interactive benchmark for evaluating long-horizon planning of LLM tool-use agents in large-scale noisy tool ecosystems.

PlanBench-XL includes:

🛠️ 1,665 executable tools
🛒 327 retail-domain tasks
🔍 retrieval-limited tool visibility
🎯 implicit sub-goals and intermediate evidence
🚧 retrieval-time blockers that simulate missing, failing, or misleading tools

Across 10 leading LLMs, we find that planning with massive tools is still far from solved. In our hardest setting, GPT-5.4 solves only 11.36% of tasks.

📄 Paper: https://t.co/SbUSn1QcOD
💻 Code: https://t.co/5FLYg6UiMI
🤗 Dataset: https://t.co/9RHNTPyMmZ
🌐 Project Page: https://t.co/c4EG9t3cJn

Feedback, discussions, and benchmark attempts are very welcome! 🙌

[Alisa Liu @alisawuffles]

I'm joining OpenAI next week!🥹 The job search turned out to be really challenging but also super rewarding, so I wrote a small blog to share what I learned along the way and hopefully make the process a little less mysterious for the next person. https://t.co/6FigSBdenD

[Hyeonjeong Ha @hyeonjeong_ai]

Thrilled to share that NarrativeTrack is accepted to #ECCV2026 🇸🇪!

NarrativeTrack is a benchmark for evaluating fine-grained entity-centric reasoning of MLLMs in long-videos.

This work is done during an internship at Apple! Huge thanks to my amazing collaborators🙌

Details👇

[Liu Jiayu ✈ ACL 2026 @JiayuJeff]

Thrilled to share that CostBench has been accepted to ACL 2026 Main! 🎉

I will be presenting the poster at Poster Session F:
📍 Grand Hall
🗓️ Tuesday, July 7
🕘 09:00–10:30
If you are interested in anything about agents, I would love to chat and hear your thoughts! 🚀

[Manling Li @ManlingLi_]

Planning with the views:

Can VLMs predict how each camera move changes the view, and plan many such moves ahead?

We introduce ViewSuite with 6 DoF camera control and ~165K task instances, testing:
Path-to-View
View-to-Path
Interactive View Planning

A sharp Planning Gap emerges:
+ can roughly "track" how camera action changes views
- cannot "compose" a plan towards a target view at all

We then try to teach VLMs with Reinforcement Learning. - RL cannot teach VLMs such planning ability, only 2.5% success rate with Qwen2.5-VL-7B.
+ With View Graph Distillation (our RL-Graph-SFT framework), 2.5% → 47.8%

Below, we answer these questions:
Q1. What are the failure modes?
Q2. How can we make RL work?
Q3. What has the model learned? Can we open up the model to see before/after? Can such spatial priors transfer to other view related tasks?

Led by @James_KKW, great to work with @LINJIEFUN @zhengyuan_yang @shiqi_chen17 @wzenus @drfeifei @jiajunwu_cs Leonidas Guibas, Lijuan Wang.

A joint efforts with @StanfordAILab @StanfordSVL @MSFTResearch.