We surfed the dense and late-interaction waves… now it's time to SlideOn sparse retrieval 🐕
🚀 Meet SlideOn and mSlideOn, fully open SOTA sparse retrievers at 174M and 339M parameters, trained by @LightOnIO R&D
Models:
https://t.co/fedmPHd15A
Blog: https://t.co/b8oiNDSGRz
This is a great day for sparse retrieval enjoyers. A very strong release with an Apache license, efficiency very similar to SPLADE-v3, and what looks like a way to avoid custom LexMAE/RetroMAE/CoCondenser pretraining - just multi-vec pretrain!
Introducing SPARSEUP: the first model release from @Linkup_platform , and the missing sparse companion of DenseOn and LateOn.
Same backbone, same data, <150M, 56+ on BEIR-13. The model is open-source, use it!
Blog: https://t.co/Eo83WVH1WB
Model: https://t.co/bP4yGemPtM (Apache 2.0)
The most recent OpenSearch model based on Alibaba's NewModel already had a RoPE-enabled context length, but for workflows still on a BERT-based sparse encoder, ModernBERT's extended max sequence length might come in handy too
PLAID-style late-interaction search has always had a slow half: exact rescoring. I made it nearly free in next-plaid with one int8 operation. Works on existing indexes, CPU only. Top bar is 1.6.5, bottom is 1.7.0. The "decompress" segment is gone. Thread.
🚨I've just released Sentence Transformers v6.0!
MultiVectorEncoder joins the family: ColBERT-style late interaction models are now a first-class model type, for training, inference & interpretation, alongside dense, sparse & reranker models.
Big thread 🧵
Hierarchical pooling is very strong to reduce the footprint of late interaction without degrading results
We recently showed that training for MUVERA/SMVE improved performance retention
What if we trained for hierarchical pooling?
We get even stronger (5×) lossless compression!