Jia-Huei (Dylan) Ju

PhD student at IRLab Amsterdam

[Xueguang Ma @xueguang_ma]

What excites me here isn't that Claude Code can find the right information within a project folder using bash tools. It's the question of what the role of the retriever should be in the agentic era.

In AgentIR, we showed that a retriever should be aware of the agent's reasoning context in order to improve agentic search.

In RISE, we take this further: the role of retrieval in agentic search is not merely to select documents that fit within the LLM's context window, but to construct an interaction space: a bounded subset of the corpus that the agent can explore through associated tools. That space needs a boundary, supplied by retrieval, and the objects within it should be processed so the agent can interact with them easily.

technical report: https://t.co/BP4h9wcfJm
code: https://t.co/oJlOpdOnqO

[Sumit @_reachsumit]

Understanding the Behaviors of Environment-aware Information Retrieval

Analyzes how LLMs learn to adapt query formulation to different retrievers via RL, showing that optimal query styles are retriever-specific.

📝 https://t.co/D3EznzAhBN
👨🏽‍💻 https://t.co/e9C7GPrsHM

[Ben Clavié @bclavie]

I have a deeper note to make about this: we need to rethink how we approach retrieval research if we want to have an LLM moment.

I think a problem we have as a sub-field is a lack of openness to early research that might be paving the way to what comes next, even if it's not all that good yet.

Let me explain: You might've noticed that we wrote both paper&blog in a way that almost doesn't care that the results are impressively good (essentially sparse SOTA for ~bert-sized model models).

The reason for this is that I truly don't think that the retrieval performance matters here, beyond proving that the method contains signal.

There has been a lot of progress in (applied) retrieval world and embeddings in the last few years, but one thing is still pretty apparent: we understand very little about how things work, and why they work. We've developed better methods, but they are largely a result of more compute + more refined pipelines + more training, etc. It works, but it's brute-forced, and the results are improving but not revolutionising the world.

Between 2020 and now, our understanding of what makes a retrieval model "good" has progressed, but not to the extent that we know what *makes* it work. ColBERT's maxsim operator, perhaps unbeknownst to @lateinteraction at the time, is still one of the most informative tool, because it shows us what's possible when we go beyond expressivity-limited scoring operators, even if it's still incredibly naive.

One thing that I'm very proud of is that at @mixedbreadai, we made a bet that the way representations are expressed almost matters less than how it is used, which has justified a lot of our (very time-consuming) engineering and research decisions, but I think it's the right decision to actually understand how neural representations can lead to better retrieval. We've pushed late interaction pretty far, and we are very much working on the next steps of late interaction, one discovery at a time.

Information retrieval is a family of tools. Single-vector models, multi-vector models, SPLADE, etc... are just some of the tools in our toolkit. Making them iteratively better is not, IMO, how we get to the end goal. Understanding *what* makes a given method better and going all-in on figuring out what its representations can tell us about training and representation dynamics is, I think, the right way forward.

Back to my original point: we need to encourage more out-there, kooky ideas that are currently borderline useless but show great promise towards the future!
One of my problem with some of the formal review cycles is that far too much importance is placed on what, I feel, should be an entirely separate, more engineering-focused paper: does this run in XXms? How does it perform on TREC-DL? What's the index size?

These are valid questions. But they shouldn't be asked of work that is exploring concepts. To me, it feels like rejecting Attention is All You Need because it's pointless to rely on a quadratic method to convey information across tokens, it'll never scale.

I want to read more exploratory work whose limitation is basically "okay yeah, we can't really deploy this, but there is undeniably something going on here. It might take 3 more papers, but we need to understand it. We can make it scale later.". I want more of these to spark discussions at conferences about the why, with the how staying at the conceptual level -- "how" at production-scale can come later.
Honestly, I even want to read papers that aren't quite sure why something works, but that have some informed opinions about it and want to show that it does.

When writing the SAE+BM25 paper, I actually had started a whole section on efficiency, making the model more or less sparse, how it impacts performance and vocab distribution, etc... Then I decided to take it all out: pages are limited, and that's not the message I want the reader to get out of the paper. The message is that these indexable sparse structures emerge from dense models. That's incredibly cool. It opens up the door to dozens of follow-up studies. Maybe it's one of the early signals that will lead to a major breakthrough in a paper or ten.
An efficiency study would be spending time and space on things that are worth studying, but are an entirely orthogonal point that should be made separately.

I have the freedom to do this because I have the incredible luxury of working at a very strongly-minded industry lab. We get daily feedback from users and know what matters and what doesn't, and if the paper gets rejected, it will have absolutely no impact on my work or career. Many, many extremely talented researchers don't have that luxury, and spend precious human and GPU-weeks on optimising for the wrong problems, too early.

And optimising for efficiency before discovering the true performance potential is one of the best ways to miss the big discovery in exchange for a smaller but more guaranteed payoff.

In LLM world, the massive breakthroughs came from similar freedom from worries. Scaling to GPT-2 made absolutely no sense from a publishable unit or GPU-rentability point of view, but it paved the way to understanding the generative potentials of transformers.

I think retrieval is key to the knowledge economy we're going to live in. The smartest agents will need knowledge from the world, no matter how genius they are.

I'm very happy that we're doing this work, but I'd love to see many more people have the ability to join this gigantic effort.

We'd benefit immensely as a field from supporting and celebrating exploratory research that let us develop the new generation of tools that will power this agentic knowledge era.

[Eugene Yang @EYangTW]

RAGTIME is coming back for the 2nd edition!
Come join us to evaluate your search agents!
We once again feature multilingual and fully human evaluation
This is the ultimate way to know whether your systems are doing better

[Sumit @_reachsumit]

Latent Terms: Dense Retrievers Contain Trivially Extractable BM25-ready Zipfian Vocabularies

@bclavie et al. extract indexable, BM25-ready sparse features from frozen dense retrievers using reconstruction-trained Sparse Autoencoders.

📝 https://t.co/WRIaCu2xIm

[Sumit @_reachsumit]

OmniRetrieval: Unified Retrieval across Heterogeneous Knowledge Sources

@jinheonbaek et al. introduce a framework unifying retrieval across diverse knowledge sources via source-native queries.

📝 https://t.co/ehomQyO3UK
👨🏽‍💻 https://t.co/U06yvJVzOO

[ACLRollingReview @ReviewAcl]

📢 ARR-May reviewers can now try REVAS, an experimental review support tool.
REVAS gives feedback on review quality criteria and ARR reviewer heuristics, but does not suggest review content or scores.
🔗 https://t.co/MOysgnz4UQ
#ARR #EMNLP #ACL #NLProc

[Sumit @_reachsumit]

DeepWeb-Bench: A Deep Research Benchmark Demanding Massive Cross-Source Evidence and Long-Horizon Derivation

Presents a deep research benchmark of 100 tasks requiring massive evidence collection, reconciliation, and derivation

📝https://t.co/2D8VYZX7Zi
👨🏽‍💻https://t.co/ZsJm3LtzTU

[Sumit @_reachsumit]

Does RAG Know When Retrieval Is Wrong? Diagnosing Context Compliance under Knowledge Conflict

Presents an inference-time probe that elicits separate contextual & parametric answers to find when retrieved context overrides a model's parametric knowledge

📝https://t.co/bXZ0HuW5mk

[Computational Linguistics Journal @CompLingJournal]

How Can We Effectively Expand the Vocabulary of LLMs with 0.01GB of Target Language Text? This article explores a very important question for low-resource languages by experimenting with various techniques across 10 languages.A must-read at:https://t.co/T0B9db6zlZ #CLJournal #NLP

[Sumit @_reachsumit]

Rethinking Agentic Search with Pi-SERINI: Is Lexical Retrieval Sufficient?

Shows that a BM25 with sufficient retrieval depth can match or outperform dense retrievers for deep research when paired with LLMs in an agentic loop.

📝https://t.co/r6cnZyRZ9i
👨🏽‍💻https://t.co/LxvRJNbFvU

[Sumit @_reachsumit]

LLM-Oriented Information Retrieval: A Denoising-First Perspective

Argues that denoising is the primary bottleneck in LLM-oriented IR, and presents a taxonomy of signal-to-noise optimization techniques across the full RAG pipeline.

📝 https://t.co/dXK5mPmUKM

[Sumit @_reachsumit]

Hierarchical Abstract Tree for Cross-Document Retrieval-Augmented Generation

Introduces a Tree-RAG framework with a hierarchical abstract tree, scaling Tree-RAG to cross-document multi-hop QA.

📝 https://t.co/cE1JlnEFwn
👨🏽‍💻 https://t.co/sO7vbUEW1m

[Sumit @_reachsumit]

Efficient Listwise Reranking with Compressed Document Representations

Naver introduces a listwise reranker that compresses documents into 8-token fixed-size embeddings, enabling an 8B model to run faster than smaller rerankers

📝https://t.co/cEzMqYhkEv
👨🏽‍💻https://t.co/XagRVf1MZx

[Seohong Park @seohong_park]

What if we represent a state as a "list" of similarities to all other states? In our recent ICLR paper, we studied this "dual" representation.

Come visit our poster at #4608 10:30a-1p on Fri (morning, 2nd day)!

Paper: https://t.co/zYKFjyO0i4
Blog post: https://t.co/lw1Port5k6

[Shanshan Zhong @zhongshsh33]

We built SkillLearnBench, the first benchmark for continual skill learning in LLM agents.

- Stronger LLMs ≠ better skills
- Self-feedback → recursive drift
- Best method still covers only ~45% of the gap to human-authored skills

Find more interesting findings from the paper!

[Sumit @_reachsumit]

Unified and Efficient Approach for Multi-Vector Similarity Search

Introduces the first native hierarchical graph index for multi-vector data, using an edge-weight function that reduces search latency by up to 14x.

📝 https://t.co/1i5qJtA6ye
👨🏽‍💻 https://t.co/I3JRFm3LWG

[Sumit @_reachsumit]

LaSER: Internalizing Explicit Reasoning into Latent Space for Dense Retrieval

Alibaba proposes a self-distillation framework that compresses explicit Chain-of-Thought reasoning into latent tokens for dense retrieval.

📝 https://t.co/wdyyuh11sG
👨🏽‍💻 https://t.co/KWXomiaUks

[Sumit @_reachsumit]

SE-Search: Self-Evolving Search Agent via Memory and Dense Reward

Tencent introduces a self-evolving search agent that improves RAG-based question answering through memory purification, atomic query generation, and dense reinforcement learning rewards.

📝https://t.co/TUg77NDc5j

[Sumit @_reachsumit]

DeepResearch-9K: A Challenging Benchmark Dataset of Deep-Research Agent

Baidu presents a 9K question benchmark across three difficulty levels for evaluating deep-research agents, along with an open-source RL training framework

📝https://t.co/2YZucyyKBg
👨🏽‍💻https://t.co/fy5uKNQPb1