On a career break
Prev. Director of AI and Staff Research Engineer @Hyperbots_Inc
IIT Bombay 5th Year CS PhD @cfiltnlp @iitbombay
Building in stealth 🚀
🇮🇳Releasing resources for Multilingual Search in 11 Indian languages!
1⃣INDIC-MARCO (Translated version of MSMARCO in 11 Indian Languages): https://t.co/R7K69S0cSR
2⃣Indic-ColBERT (11 Multilingual ColBERT Models): https://t.co/XE4YoMZYOt
Paper: https://t.co/a9Rl3Jo87w
i have something to release/share tomorrow that i think is very badly needed and has been so for a few years - it’s a tiny start and won’t be complete but i hope people will find it as useful and fun as i have enjoyed producing it
as someone who proudly reposted the release before it had any traction and who loves the team, y’all here are massively and absurdly overhyping an otherwise perfectly good (even great) release
calm down and, unironically, be calibrated for once..!
overwhelming evidence for late interaction / multi-vector models yet again :-)
> even after finetuning, single-vector models lag far behind multi-vector embeddings, which achieve significant performance gains and exhibit greater robustness to catastrophic forgetting.
The “grep-is-all-you-need” nonsense arguments arise from the fact that too many people think neural search means single-vector IR, which do in fact suck. But we’ve known that since 2019.
Quoting @aaxsh18, CEO of Mixedbread:
> late interaction cant stop winning
Of course not. In fact, my lab is simultaneously building RLMs as the next paradigm for LLMs *and* developing the next paradigm for retrieval (stay tuned!).
Retrieval will not go anywhere: if you have a large corpus with, say, billions of tokens over which you issue many queries, you necessarily need to build some index data structures that enable fast sub-linear access.
RLMs may internally choose to build such an index when it proves to be an effective tool, but fundamentally RLMs are about long one-off context. You wouldn’t typically put an RLM over a million documents and expect that to be the optimal system design.
(Thank you for the question @jayitabhattac11 !)
For those interested in making OSS contributions to the RLM repo, I've added a bunch of random thoughts and TODOs of what to add in a *messy* Markdown file on the GH repo.
Feel free to tackle any of them, or any other things you think are meaningful. I'll be pretty active here or on the repo. Once I finish some other related work, I might open up a Discord channel or something for people who want to make longer standing contributions to the repo / discuss the direction of where to take it. Cheers!
https://t.co/EVK7g0vzf0
@a1zhang IMO, RLMs are as “language model”-y as modern “LLMs” or Reasoning Models are truly “statistical models of language”.
All three are a bit of a stretch BUT in the same way.
Pedantically, all three are language processing systems, eg recursive/reasoning language processing system.
> You’ll implement ColBERT to understand multi-vector search [and] apply ColPali for patch-level image retrieval.
So happy to see the great folks at @DeepLearningAI@AndrewYNg host a course on late interaction (ColBERT, ColPali et al) after their short course on DSPy :D
Please consider applying to the program. Over two years, my research skills, perspective on research have all been broadened and sharpened. This is an exceptional group, in the way they groom you, and allow you a room for exploring wild ideas. Pls reach out if you have questions!