wamanarix

[wamanarix @wamanarix]

@TheAhmadOsman maybe if you convince NVIDIA and friends to rent/sell local machines via 20$ or 100$ monthly subscriptions

[wamanarix @wamanarix]

@hu_yifei that looks like context compression when you reached the maximum context length, not text generation

[wamanarix @wamanarix]

@fchollet A high percentage of trading volume is driven by machines

[wamanarix @wamanarix]

@hxiao Interesting experiment. People talking about search as code but not showing numbers. Can you add also a cost associated to the amount of text retrieved, or number of grep calls? It is free in your local setup but imagine you pay fee every time.

[wamanarix @wamanarix]

@paradite_ You may want to read about emergence.

[wamanarix @wamanarix]

@jobergum Less LLM tokens for the agent, but what if the user pays for each retrieved piece of text that ends discarded? You can't expected a generalist agent to optimise search every time you want to search for something.

[wamanarix @wamanarix]

@jobergum @aiDotEngineer but is it really more efficient? are you not moving costs of search and ranking to the agent? it has to do more searches with more keywords, it has to navigate over more results, etc.

[wamanarix @wamanarix]

@TheAhmadOsman how much GPU do you need to be able to run this? what is the electricity bill?

[wamanarix @wamanarix]

@_iamperegrine_ @turbopuffer Ok, so measuring the ANN deterioration versus exact NN, not really a measure of recall in finding the right document. Even for this, 96% is still hard to believe for these latencies, is this using hnsw?

[wamanarix @wamanarix]

@patpcj At the end people will use SOTA models at the higher level, with full context of the tasks, and delegate to smaller models narrower tasks. Nevertheless, these do not have full context, so your agentic search tool won't compete at the same level.

[Tony Wu @tonywu_71]

Very excited to release late-interaction-kernels (LIK): fused Triton kernels for MaxSim, the scoring step behind ColBERT, ColPali & LateOn. 🚀

Numerically equivalent to PyTorch at a fraction of the memory, with day-0 support in PyLate & colpali-engine. (1/N 🧵)

[wamanarix @wamanarix]

@capythanh @burkov You can probably create better apps vibe coding yourself

[wamanarix @wamanarix]

@jonbratseth Less costly in LLM tokens for the agent, perhaps, but not in search "tokens". Tools with high level of customization send the burden of search to the agent, that was not trained for this, why not a fine-tuned LLM in the search tool?

[wamanarix @wamanarix]

@turbopuffer How do I rank using a lightgbm or even a simple logistic regression model combining multiple features?

[wamanarix @wamanarix]

@CyrusRashtchian @GoogleResearch well, I see they mention MCPs in the retrieval step. Can I plug any MCPs for search there?

[wamanarix @wamanarix]

@p0 How does grounding work here? Do you provide attribution to the URLs or sources where you get this information from? (E.g. crunchbase) Is this like Exa AI entity knowledge graph search? How many companies do you currently cover?

[Silvio Martinico @SilvioMartinico]

The late-interaction multivector retrieval ecosystem is exploding right now.
To help separate the signal from the noise, we put together an "Awesome Multivector Retrieval" list organizing the top models, engines, libraries, and datasets all in one place 📚 🧵👇

[Mixedbread @mixedbreadai]

By now, everyone knows that single-vector embedding models are hugely limiting for modern workflows.

But they contain than you think: you can extract sparse Latent Terms from them.

And it turns out that BM25 is all you need to turn this vocabulary into a strong retriever.