Agis P.O.F.

Research in NLP, Dialogue and indoors localization. Opinions are my own. Likes and retweets are not endorsements

[Agis P.O.F. @agispof]

@thesupermanmx I highly doubt anyone who has actually built an information retrieval agent for a production system is surprised by this. Semantic Search IMO is just another component not the solution. I remember an older paper that said that beyond 50k documents ss is just random retrieval.

[elvis @omarsar0]

New systematic review on RAG for enterprise applications.

Enterprise RAG adoption still has a long way to go.

The assumption is that RAG systems are production-ready.

But the gap between lab prototypes and enterprise deployment is wider than most realize.

This systematic literature review analyzes 77 high-quality studies on RAG + LLM systems for enterprise knowledge management and document automation, covering publications from 2015 through mid-2025.

The findings reveal a concentrated technology stack:

- 63.6% of implementations use GPT-based models.
- 80.5% rely on standard retrieval frameworks like FAISS or Elasticsearch.
- 66.2% favor cloud-based infrastructure for scaling.

But the "lab to market" gap is significant. While retrieval and classification tasks frequently use rigorous validation methods like k-fold cross-validation (93.5%), generative evaluation predominantly relies on static hold-out sets due to computational constraints. Only 13% of studies deploy RAG prototypes in live corporate environments.

Five recurring challenges dominate:

- hallucination control (48.1% of studies)
- data privacy and security (37.7%)
- latency and scalability (31.2%)
- domain adaptation (23.4%)
- difficulty measuring business impact (15.6%)

The technical metrics are well-covered. Precision, recall, and accuracy appear in 80.5% of studies. ROUGE and BLEU in 44.2%. But human-in-the-loop evaluation appears in only 19.5%, and real-world case studies measuring end-user outcomes remain rare.

Fine-tuning on in-domain data typically adds 10-20% factuality gains over zero-shot approaches. Hybrid retrieval combining dense vectors with knowledge graphs appears in 23.1% of studies and often boosts both explainability and precision.

The research provides a data-driven roadmap for bridging academic prototypes and production systems. The technology works in controlled settings, but privacy-preserving retrieval, sub-100ms latency, and business-centric evaluation frameworks remain open challenges for enterprise deployment.

🔖 (bookmark it)

Report: https://t.co/QKedUXUzsJ

[DAIR.AI @dair_ai]

Banger paper from NVIDIA.

Bigger models aren't always the answer. However, the default approach to improving AI systems today remains scaling up. More parameters, more compute, more cost.

But many tasks don't require the full power of a massive model.

This new research introduces ToolOrchestra, a framework that strategically coordinates multiple AI models with external tools based on task complexity.

Instead of routing everything through one large model, an orchestrator decides dynamically. When is a tool necessary? Which model size fits the task? How should components coordinate?

The researchers trained Orchestrator-8B, a specialized 8-billion parameter model that makes intelligent routing decisions. It determines when external tools are needed versus when model inference alone suffices.

On HLE, Orchestrator achieves a score of 37.1%, outperforming GPT-5 (35.1%) while being 2.5x more efficient.

They also release ToolScale, a synthetic dataset of tool usage examples across diverse scenarios for training orchestration capabilities.

What it matters: strategic orchestration of smaller models with targeted tool usage can match or exceed monolithic large model performance while cutting computational overhead.

Paper: https://t.co/iNvqIHGTES
Learn how to build AI Agents in our academy: https://t.co/zQXQt0PMbG

[Akshay 🚀 @akshay_pachaar]

NVIDIA just dropped a paper that might solve the biggest trade-off in LLMs.

Speed vs. Quality.

Autoregressive models (like GPT) are smart but slow - they generate one token at a time, leaving most of your GPU sitting idle.

Diffusion models are fast but often produce incoherent outputs.

TiDAR gets you both in a single forward pass.

Here's the genius part:

Modern GPUs can process way more tokens than we actually use. TiDAR exploits these "free slots" by:

1. Drafting multiple tokens at once using diffusion (the "thinking" phase)
2. Verifying them using autoregression (the "talking" phase)

Both happen simultaneously using smart attention masks - bidirectional for drafting, causal for verification.

The results:

↳ 4.71x faster at 1.5B parameters with zero quality loss
↳ Nearly 6x faster at 8B parameters
↳ First architecture to outperform speculative decoding (EAGLE-3)
↳ Works with standard KV caching, unlike pure diffusion models

The training trick is clever too - instead of randomly masking tokens, they mask everything. This gives stronger learning signals and enables efficient single-step drafting.

If you're building real-time AI agents where latency kills the experience, this architecture is worth paying attention to.

Link to the paper in the next tweet.

[Ethan Mollick @emollick]

It begins.

This is another sign that LLMs are going to be able to work with structured & unstructured spreadsheet data soon. This will unlock a lot of use cases (projections, financials, valuations, etc.) and having a spreadsheet source of truth will tend to lower hallucinations

[Aakash Kumar Nain @A_K_Nain]

Seven years ago, the paper Attention is all you need introduced the Transformer architecture. The world of deep learning has never been the same since then. Transformers are used for every modality nowadays.
Despite their nearly universal adoption, especially for large language models, the internal workings of transformers are not well understood.

Through our paper titled, Transformer Layers as Painters, we aim to understand the flow of information in a pretrained transformer. We present a series of experiments for both decoder-only and encoder-only frozen transformer models. Note that we do not perform any kind of fine-tuning on these pretrained models. Here are our findings:

1. Do layers speak the same language?
To check this, we ran a few experiments. For example, What if we skip some layers? For example, if we changed the output flow (layer 4 -> layer 5 -> 6) to (layer 4 -> layer 6) by skipping layer 5 entirely, how much does the performance deteriorate?  What if we switch the order of neighbuoring layers? For example, feeding the output of layer 4 to layer 6, then sending the output of layer 6 to layer 5, then to layer 7.

Based on a series of experiments, we noticed that the representation space of a transformer-based model can be partitioned into three types: beginning layers, middle layers, and ending layers. The middle layers seem to share a common representation space.

2. Are all layers necessary?
If the middle layers share the representation space, can we drop some layers without getting a performance hit? To check this, we send the output of the Nth layer directly into the input of layer N + M (where M > 1), thereby “skipping” M − 1 layers. We found that at least a few middle layers can be dropped without catastrophic failure.

3. Are the middle layers all doing the same thing?
Does shared representation represent redundancy? If yes, how much redundancy can we get rid of?

We ran an experiment where, instead of skipping some of the layers, we replaced their weights with those of the center layer, effectively looping on this layer for (T − 2N + 1) times, where is the total number of layers (32 for Llama2-7B, 24 for BERT-Large).

We found that even though the middle layers share the representation space, they perform different functions, and sharing weights among the middle layers is catastrophic.

4. Does the layer order matter?
If the layers perform different functions in a shared representation space, how much does the order of these functions matter?

We performed two sets of experiments on this. First, we ran all these layers in the reverse order of how they were trained. Specifically, we take the output of (T − N)th layer and send it into the input of (T − N − 1)th layer, then the output of this layer into (T − N − 2)th layer, and so on down to Nth layer.

In the second variation, we ran the middle layers in a random order. The order matters to an extent as randomizing and reversing the middle layer order have graceful degradation.

5. Can we run the layers in parallel?
If the presence of the layers is more important than the order in which they’re executed, can we run the layers independently from an early input and merge their results? We found out that we could do this without a major hit except for math-heavy benchmarks.

6. Does the order matter for some tasks more than others?
We noticed that for all variants (reversed, skip, parallel), ARC and GSM8K (i.e. reasoning and mathematical tasks) had the steepest decline. Performing well on these benchmarks requires a model to be sound both structurally, and semantically. This would be consistent with the hypothesis that some degree of order-dependent reasoning is happening within a single pass of the model. Therefore, we conclude that some tasks are more order-dependent than others and require more than just semantics.

7. Does looping help parallelized layers?
What if we loop the mean output of the parallelized layers M times back into the same layers for a fixed number of iterations? The hypothesis is that if a layer performs only a certain subset of functions, then iterating the parallelized layer from the previous experiment should improve performance compared to a single execution of the parallelized layer. We found that this is true with the optimal number of iterations proportional to the number of parallelized layers.

8. Which design variants are least harmful?
Repeating a single layer is the worst. Randomizing the layer order and looped-parallel do the least damage.

[LangChain @LangChain]

⚡ Using knowledge graphs to boost RAG accuracy ⚡

Check out this guide to constructing and retrieving information from knowledge graphs in RAG applications with @neo4j and LangChain.

➡ https://t.co/0e4NYimlOd

Graph RAG is gaining momentum as a powerful addition to traditional vector search retrieval methods.

This approach leverages the structured nature of graph databases, which organize data as nodes and relationships, to enhance the depth and contextuality of retrieved information.

Graphs are great at representing and storing heterogeneous and interconnected information in a structured manner, capturing complex relationships and attributes across diverse data types.

In contrast, vector databases can struggle with such structured information, as their strength lies in handling unstructured data through high-dimensional vectors.

In your RAG application, you can combine structured graph data with vector search through unstructured text to achieve the best of both worlds.

[anton @abacaj]

Haven't seen this paper discussed yet... it has been said before that fine tuning disrupts world knowledge in LLMs (catastrophic forgetting)

This paper shows a way to fine tune llama-2 with millions of instruction data w/o disrupting that, effectively injecting new knowledge

[Thodoris Georgakopoulos @tgeorgakopoulos]

"Το 19% των πρόσφα��ων αποφοίτων πανεπιστημίων ή μάστερ στην Ελλάδα είναι λειτουργικά αναλφάβητοι. 1 στους 5 νέους αποφοίτους τριτοβάθμιας εκπαίδευσης δεν έχουν ούτε καν τις βασικές δεξιότητες" https://t.co/g1h5eAxHix

[Graham Neubig @gneubig]

We're excited about all the interest in our Gemini report and working to make it even better!

This week we made major improvements, switching to the @MistralAI instruct model, and working with the Gemini team to reproduce their results. Updates below.

[Sasha Rush @srush_nlp]

Goddamn, wild times.

Here's the paper you need to read understand today:
https://t.co/M9AfFtTYcv

have fun.

[Graham Neubig @gneubig]

An interesting tidbit from the Mamba paper (https://t.co/0yD3bjG5i0): the Transformer vs. Transformer++ comparison. Transformer is the original version, and Transformer++ is the LLaMa-2 version (SwiGLU/RoPE/training tweaks).

Architectures/algorithms make a huge difference!

[elvis @omarsar0]

Magicoder: Source Code Is All You Need

Better instruction data leads to better code models!

This work proposes a series of fully open-source LLMs for code that close the gap with top code models while having no more than 7B parameters.

It's trained on 75K synthetic instruction data. Uses open-source references for the production of more diverse, realistic, high-quality, and controllable data.

The model outperforms state-of-the-art code models with similar or even larger sizes on several coding benchmarks, including Python text-to-code generation, multilingual coding, and data-science program completion.

MagicoderS-CL-7B based on CodeLlama surpasses ChatGPT on HumanEval+ (66.5 vs. 65.9 in pass@1).

It's exciting to see these results which are a strong indicator of the importance of diverse and high-quality data for LLMs. The potential to improve synthetic data for LLMs is also very interesting area.

[Valeriy M., PhD, MBA, CQF @predict_addict]

RIP Monte Carlo @GoogleDeepMind releases the code for Conformal Monte Carlo.

https://t.co/1c1jj1AXRo

#conformalprediction

[Matt Green @mattgreencomedy]

Rishi Sunak snubs Greek PM over Elgin Marbles!

[Sebastien Bubeck @SebastienBubeck]

We trained a small transformer (100M params) for basic arithmetic. W. the right training data it nails 12x12 digits multiplication w/o CoT (that's 10^24 possibilities, so no it's not memorization🤣).

Maybe arithmetic is not the LLM kryptonite after all?🤔
https://t.co/dldvBhb2Qj

[David Whitney @david_whitney]

If you ever felt like you got something from Rob C Martins mostly bad books, and are contending with how awful he is online, here are some vastly superior reads that you will

1. Likely enjoy more
2. Be able to recommend instead

[Julian Togelius @togelius]

It is happening! @yannakakis and I have started working on the second edition of our textbook on AI and Games. We will try to incorporate feedback from the community as much as possible, so please send your suggestions for changes/additions to [email protected].

[elvis @omarsar0]

I've been enjoying the recent lectures in what is one of my favorite courses of all time (Stanford CS224N).

If you are interested in NLP with Deep Learning and want to catch up on topics like RLHF, Prompting Techniques, Multimodal DL, and NLG, check out the updated playlist for the course.

Great way to catch up on the latest LLM papers and emerging AI topics.

Bookmark this one!

https://t.co/fAWQTP8oAg

[Lilian Weng @lilianweng]

Feeling a bit intimidating to write about it but work on attacks can lead to good insights for mitigation. Plan to write about mitigation work separately later.

Also want to thank all the researchers who shared disclosure reports w/ us so far. 🙏🙏🙏

https://t.co/TkQnKARPgT