jonah

sparse retrieval

[jonah @drexalt]

@usr_bin_roygbiv It is not healthy for the country

[jonah @drexalt]

@usr_bin_roygbiv bro get in ventrilo for the daily

[Sumit @_reachsumit]

Rescaling MLM-Head for Neural Sparse Retrieval

@yjoonjang et al. find that pretrained encoders with large MLM-head scales face degradation in sparse retrieval, and introduce a zero-cost rescaling correction to stabilize training.

📝 https://t.co/fLYrT4oRVz

[jonah @drexalt]

Bringing some of the mystery out of stable LSR training

If you’ve struggled training SPLADE models across backbones (noob lol who would think it’s difficult DEFINITELY not me) give this one a read :)

[Youngjoon Jang @yjoonjang]

The scale of MLM-Head may be the problem of your SPLADE models. Rescaling the head could resolve this problem. Check our new paper about this!
https://t.co/Q577MwQLzF

[jonah @drexalt]

get this alpha straight into your veins

[Antoine Chaffin @antoine_chaffin]

Party is over, time to regularize ColBERT models to fix efficient ANN
MUVERA and SMVE promised to simplify multi-vector retrieval infrastructure but broke on modern ColBERT models
We found a fix, and it does the exact opposite of what we expected

[Antoine Chaffin @antoine_chaffin]

Whether you are GPU poor or GPU rich, today's release of PyLate has something for you!
GPU maxxers: MaxSim kernels greatly speed up training while lowering the memory requirements
CPU enjoyers: TACHIOM enables lightning fast multi-vector indexing and search directly on CPU

[jonah @drexalt]

@antoine_chaffin pooling? i cant even swim!

[jonah @drexalt]

- sparse maxsim
- contrastively-trained sae
- strong results on BERT and 8b nemo

this is a great one

[Sumit @_reachsumit]

No More K-means:Single-Stage Sparse Coding for Efficient Multi-Vector Retrieval

@Veritas2026 et al. replace vector clustering with efficient sparse autoencoders & natural inverted indexing to accelerate multi-vector retrieval.

📝https://t.co/jux1WjXkML
👨🏽‍💻https://t.co/CdRW68YnF4

[jonah @drexalt]

@capemox @mattjustram @yjoonjang I have an old neobert lexmae/dupmae checkpoint that might help testing
https://t.co/FFFT8YPAHj

[jonah @drexalt]

@mattjustram @capemox @yjoonjang agree CLS is not irrelevant, maybe some traumatic DupMAE/LexMAE experiences leaking through haha.

but generally I am wondering the far opposite, whether maxsim pretrain is comparable/better for SPLADE than DupMAE, has not been tested I think

[jonah @drexalt]

@capemox @mattjustram @yjoonjang I am pretty sure it would not be, since ColBERT isn’t using the MLM head anymore and the CLS token-maxxing doesn’t seem super aligned. I think the LateOn contrastive pre train probably stronger

[jonah @drexalt]

@capemox @mattjustram this might serve as a jumping off point from @yjoonjang, I tried DupMAE on NeoBERT and had good results
https://t.co/4twgAGr8ES

[jonah @drexalt]

@cloneofsimo kyochon is by far the best

[jonah @drexalt]

@antoine_chaffin @ManuelFaysse the pi-serini results show off the gpt5.5 dominance really well
https://t.co/lwAnkyGGOJ

[Jheng-Hong Yang @mattjustram]

someone already wrote a love letter to pi, by @badlogicgames.

so we wrote a love paper to pi :)

with my teammates @xuzihuan4 and @lintool.

a few days ago, i promised i’d share some fun plots once Pi-Serini joined the BrowseComp-Plus deep research agent party.

now, it’s about time.

here weeeee goooooo.

bear with the sloppy images first.
the serious one is at the end.
the question was simple:

how far can we push deep research with BM25 + pi?

turns out: weirdly far.

[Mixedbread @mixedbreadai]

Introducing mxbai-rerank-v3-listwise: reranking that goes beyond binary relevance.

It reads the whole candidate set, resolves conflicts, and ranks by directives like recency, source priority, and multi-step rules.

+11% NDCG@10 on average across multiple domains, modalities, and languages in runs with Wholembed v3.

Available today in preview in Mixedbread.

[jonah @drexalt]

Yes, the same team behind the SoTA Sparse index SEISMIC is also behind what appears to be SoTA multi-vector index. what is in the water in Pisa

[Sumit @_reachsumit]

Efficient Multivector Retrieval with Token-Aware Clustering and Hierarchical Indexing

Presents a multivector retrieval system that uses token-aware clustering to allocate centroids based on token frequency & semantic variance.

📝https://t.co/f0MB0mFL2b
👨🏽‍💻https://t.co/YLF9PHbkAH

[jonah @drexalt]

@lateinteraction I think they mention clustering 64 core, retrieval single core. pretty nuts

[jonah @drexalt]

@menhguin Korean boomers are degen pumping quantum computing stocks and crypto lmao

[jonah @drexalt]

@realsigridjin