Matt Barta

Founder at @DeployQL, specializing in RAG. Get in touch!

[Matt Barta @MtBarta]

We've open sourced our code for evaluation and dataset bootstrapping for RAG applications.

If you're working on RAG, we want to make it easier for you to move to production. Part of that is trusting that your retrieval pipeline is the right balance of quality and cost.

[DeployQL @DeployQL]

Working on RAG applications? We're building a way to bootstrap better retrieval and ranking algorithms for Gen AI.

We've open sourced our repo to evaluate against MTEB and soon bootstrap your own dataset.

Here's why this matters. 🧵

[Yam Peleg @Yampeleg]

Huggingface just made an 11T magical training set that made 3 different SOTA models..

The best 135M, 360M, 1.7B models to date.

Can't wait to read about the dataset, the previous SmolLM dataset was a work of art.

[Ksenia Se @Kseniase_]

I've found this method really interesting👇 It's about embeddings – which have not enough attention imo.

So, a group of researchers suggested an in-context learning (ICL) strategy to make models smarter.

Their novel bge-en-icl model improves text embeddings by providing few-shot learning in the model's input.

Here's a closer look:

[Matt Barta @MtBarta]

Huge improvements on image retrieval embeddings.

[Manuel Faysse @ManuelFaysse]

🚨 New model alert: ColQwen2 !
It's ColPali, but with a Qwen2-VL backbone, making it the best visual retriever to date, topping the Vidore Leaderboard with a significant +5.1 nDCG@5 w.r.t. colpali-v1.1 trained on the same data ! 🚀 (1/N)
https://t.co/nFFknVWbjG

[Hiten Shah @hnshah]

An Early Access process for your startup can be a game changer for future growth.

The real challenge? Landing good design partners.

Here's how to do it right:

[Tony Wu @tonywu_71]

Want to dig deeper into ColPali? 👀 I’ve just dropped a series of cookbooks that covers similarity map generation and fine-tuning ColPali (with optional quantization). Find these crispy new recipes at https://t.co/7rjqDlMH4K! 👨🏻‍🍳 (1/N)

[Leonie @helloiamleonie]

Ok, I’ll bite: What’s ColPali?

(And why should anyone working with RAG over PDFs care?)

ColPali makes information retrieval from complex document types - like PDFs - easier.

Information retrieval from PDFs is hard because they contain various components:
Text, images, tables, different headings, captions, complex layouts, etc.

For this, parsing PDFs currently requires multiple complex steps:
1. OCR
2. Layout recognition
3. Figure captioning
4. Chunking
5. Embedding

Not only are these steps complex and time-consuming, but they are also prone to error.

This is where ColPali comes into play.

But what is ColPali?

ColPali combines:
• Col -> the contextualized late interaction mechanism introduced in ColBERT
• Pali -> with a Vision Language Model (VLM), in this case, PaliGemma

And how does it work?
During indexing, the complex PDF parsing steps are replaced by using "screenshots" of the PDF pages directly.
These screenshots are then embedded with the VLM. At inference time, the query is embedded and matched with a late interaction mechanism to retrieve the most similar document pages.

Here are some more resources to learn more about ColPali:
🎓 ColPali paper: https://t.co/Xw8djk7bSg
🤗 Blog post by the author of ColPali: https://t.co/yYxXCy3uEF
💻 ColPali Weaviate notebook: https://t.co/4z1pqyzwaB

[Sumit @_reachsumit]

Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wisely

Microsoft categorizes data-augmented LLM queries and proposes strategies to tackle challenges in specialized domains.

📝https://t.co/nFyLHp6u1T

[Matt Barta @MtBarta]

I wrote a bit more here: https://t.co/wCc8p3skv6

[Matt Barta @MtBarta]

It's hard to write C++ in open source.

I don't consider myself a C++ developer by any means. I've made a ton of mistakes.

Here are some of the pros and cons I've found:

[Matt Barta @MtBarta]

The reasons why C++ let you write very performant code are the same reasons why it's hard to write.

Our C++ code needs to be fast, reliable, and likely support multiple languages, including CUDA. Very few languages offer the same level of integrations as C++.

[Matt Barta @MtBarta]

Such a good idea.

Token embeddings -> pooling -> late chunks.

Add some quantization, and you've got a compact database!

[Victoria Slocum @victorialslocum]

Optimizing your chunking techniques is one of the top places to improve performance in your RAG pipelines, but what’s the best one?

@JinaAI_ just released a new method called late chunking that takes the same amount of storage space as naive chunking, but solves the problem of lost context similarly to ColBERT.

You can implement it super easily with just a few extra lines in your embedding step!
Blog: https://t.co/ydo4nKXOXr
Notebook: https://t.co/ocXOcKD7k9

Thanks so much to @DanielW966 again for the awesome collaboration 💚

📄 Papers
Late Chunking: https://t.co/Oa2CrDSqtD
ColBERT: https://t.co/FhDf6bTycF

[Theophile Gervet @theo_gervet]

Excited to finally share what I have been working on at @MistralAI. Meet Pixtral 12B, our first-ever multimodal model:
- Drop-in replacement for Mistral Nemo 12B
- SOTA multimodal capabilities without compromising on SOTA text-only capabilities
- New 400M parameter vision encoder trained from scratch, paired with a 12B multimodal decoder based on Mistral Nemo
- Handles variable image sizes and aspect ratios
- Supports multiple images in 128k context window
- Apache 2.0 license

Blog: https://t.co/2pbjn6lPDd
Weights: https://t.co/CidiN2Emtu
Try it: https://t.co/p4IuFcEsSb

1/N

[Ben Clavié @bclavie]

"Just use a re-ranker" is a loaded sentence

Everything here does the same thing, but w/ very different mechanisms, AND they are all commonly used.

But what even is Point/List-wise? And why are there so many methods? How do I try them? Why should I try them? So many questions...

[Leonie @helloiamleonie]

Whether you're working on RAG, search, or RecSys,
the "retrieve & re-rank pipeline" is something you will come across often.

Using only one algorithm to find the best search results is often not enough.

Either it's too compute-intensive,
or not accurate enough.

That's where the "retrieve & re-rank pipeline" comes into play:

1. Retrieve:
• Use a light-weight model
• to reduce the candidates from millions to hundreds
• e.g., vector search, bi-encoder, etc.

2. Re-rank:
• Use a more sophisticated (and compute-intensive) model
• to further reduce the candidates from hundreds to the top few
• e.g., cross encoder models

You will find this technique not only in search but also in advanced RAG and RecSys.

[Matt Barta @MtBarta]

@softwaredoug @jxnlco I was lucky to hear this one. You made a great point about incrementally improving search vs attempting large projects.

Does reddit let you speak officially on the search infra? Would love to hear more about it.

[Matt Barta @MtBarta]

Here's the link: https://t.co/WkOkt1kLAH

Would love to hear any feedback!

[Matt Barta @MtBarta]

Have you thought about how filters are implemented for vector search?

I wrote about some of the considerations and how we thought about it when implementing filters for LintDB.

[Matt Barta @MtBarta]

Filters are conceptually simple. You only return documents that match the filter.

It's not as simple in practice. It depends on how many documents will match the filter and what data structures you're using.