antoine

founder swe + ml, phd in ai/nlp

[antoine @antoinelouis_]

Need a dense multi-vector retrieval model in your language? We've got you covered with ColBERT-XM, a new modular retriever that supports 81+ languages and can easily be extended to many more. [1/4]

📑 Paper: https://t.co/9OWpjjgpFE
🤗 Model: https://t.co/LvQSsozQnf

[antoine @antoinelouis_]

excited for this collab between https://t.co/okchcKUOCt and https://t.co/YEfj0QIpzp! the largest online repositories of legal agent skills now officially integrated into the largest open-source legal AI platform.

[Samuel @5am2x]

700 stars on GitHub !

@lawveAI legal skill repo just crossed a milestone I’m very happy to see.

We’re still early.

https://t.co/BEYeDUaiFo

[antoine @antoinelouis_]

@tomaarsen very exciting!

[antoine @antoinelouis_]

@antoine_chaffin @LightOnIO @perplexity_ai Congrats Antoine! Looking forward to all the cool things you'll build there

[antoine @antoinelouis_]

@LyalinDotCom @geminicli @LyalinDotCom is a GUI for the VS Code extension (similar to CC or Codex) in the roadmap?

[antoine @antoinelouis_]

@gortron @lancedb Very cool work!

[antoine @antoinelouis_]

@tomaarsen Thanks for the nice words Tom!

[Manuel Faysse @ManuelFaysse]

For companies with modern embedding-based retrieval system, this paper has an insane density of actionable key takeaways - which btw align well with my practical experience! Awesome work @antoinelouis_

[Jo Kristian Bergum @jobergum]

Aligns with my thinking and experience

[tomaarsen @tomaarsen]

Wow, this is definitely one of my favorite papers of the last while. It details various Sparse/Dense/Multi-vector search combinations, and whether it makes sense to use them together.

Definitely give it a read, or at least read @antoinelouis_' thread below:

[antoine @antoinelouis_]

Overall, these insights suggest that for specialized domains, fine-tuning a dense bi-encoder model generally yields optimal results when (even limited) high-quality domain-specific data is available, whereas fusion should be preferred when such data is not accessible and domain-general retrievers are used.

But that’s not all! The paper also dives deeper into topics like:
- Language-specific retrievers trained on a budget using mMARCO, vs. larger-scale multilingual models like mE5 and BGE-M3;
- Effect of task-adaptive pre-finetuning on downstream performance;
- Efficiency of different retrieval approaches (BM25, DPR, SPLADE, ColBERT, monoBERT) in terms of index size, retrieval latency, and inference FLOPs;
- Alignement of score distributions from learned retrievers under different normalization techniques;
- Complementarity of different matching paradigms to understand the impact of fusion;
...and more.

Interested? Check it out here:
📄 Paper: https://t.co/xvdvJuaTZw
💻 Code: https://t.co/NStJPsG2kX
🤗 Domain-specific models: https://t.co/GGF6oQ5JBB
🤗 Domain-general models: https://t.co/PQIKTOB5Fo

[antoine @antoinelouis_]

Can hybrid search boost retrieval performance in a non-English, highly specialized domain like law?

In my final PhD paper (COLING 2025), I investigate this question by combining various retrieval techniques across two realistic scenarios:
1️⃣ zero-shot, where we assume **no** domain-specific data is available and assess the performance of fusing domain-general models;
2️⃣ in-domain, where we assume **limited** domain-specific data is available and assess the performance gains of fusing fine-tuned retrievers. (1/N 🧵)

[antoine @antoinelouis_]

For example, in the DPR+BM25 combination, we find that integrating BM25 offers minimal benefits once DPR is domain-tuned, with equal weighting between both systems consistently leading to worse performance. This contrasts starkly with the out-of-domain setting, where fusing the two consistently improves performance over using either one alone, regardless of the weighting.

[antoine @antoinelouis_]

Such an important release! BERT, modernized with years of research in language modeling.

Now I need to see this for French. A long-awaited update to the legendary CamemBERT. Maybe it’s time for a new French king: DagoBERT! 👑

[Ben Clavié @bclavie]

It's finally out!

We at @answerdotai, @LightOnIO and friends are releasing ModernBERT 🎉

It does exactly what it says on the tin: It's BERT, but not 2018 BERT, no, it's 2024 BERT, with all the 2024 bells and whistles.

They're slot-in replacements for BERT, at both model sizes.

[antoine @antoinelouis_]

Couldn’t agree more! Super exciting to see a modern BERT that’s way more efficient than similar models and can handle up to 8,192 tokens at once.

[Jeremy Howard @jeremyphoward]

The popular buzz around GenAI has obscured the role of encoder-only models. These are the workhorses of practical language processing, the models that are actually being used for such workloads right now in many scientific and commercial applications.

[antoine @antoinelouis_]

@antoine_chaffin Very exciting work @antoine_chaffin @bclavie (and team), huge congrats for the release!

[antoine @antoinelouis_]

And would you generally agree with the following suggested settings: https://t.co/0sNf0PUbrh? What is your own recommendation for a robust configuration that best prevents 'database locked' errors when doing concurrent writing (5-10 workers)? Asking as I'm still having them even in WAL mode...

[antoine @antoinelouis_]

@simonw @moojii @ImSh4yy Simon, would you mind sharing what it means in practice for a Django write? Does it mean placing every save, create and delete operation inside a 'with django.db.transaction.atomic():'?