A search stack that has been operating for years typically accumulates layers of fixes. Over time, debugging search issues requires reconstructing behavior across the whole stack. New changes require broader validation. Experiments become harder to interpret and iterate.
The question shifts from:
"What would create the most value for users and the business?"
to:
"What can we safely change within the constraints of the current stack?"
That shift has a cost.
A search stack can keep working even as it starts getting harder to maintain.
In this new blog post, we look at how to recognize when a search system is moving from a healthy regime into a straining one and what the recurring cracks usually mean.
There are plenty of benchmarks comparing search engines that measure parts of the search under controlled conditions.
But how a search system behaves in production is shaped by interaction of multiple factors.
In this new blog post, we break down those factors.
If the retrieval engine is programmable and self-debuggable, then the agent becomes the primary operator and orchestrator of search. Much like coding agents, we'll likely see specialized search agents emerge.
About time people find out more about @vespaengine and what it has to offer!
At @searchplex, we've been migrating to and building vespa-powered search solutions for our clients. And it's miles ahead of other options in the market today!
Tomorrow I’ll be joining the @vespaengine team for a live webinar on how to migrate from Elasticsearch to Vespa Search. Do join us if you're exploring Vespa Search.
📅 𝐃𝐚𝐭𝐞: October 15
🕙 𝐓𝐢𝐦𝐞: 10 AM ET | 4 PM CET
🔗 𝐑𝐞𝐠𝐢𝐬𝐭𝐞𝐫 𝐡𝐞𝐫𝐞: https://t.co/SalTqXnn2v
Thanks to Vespa for featuring us!
If your team would like to accelerate migrating to Vespa, building on it, or optimizing scalable Al search solutions, check out our full range of services in this blog post.
https://t.co/iNOGDiZ987
Our first time attending the Berlin Buzzwords conference: spent the days listening to engineers dealing w/ search infra challenges. Some clear patterns emerged around where teams are hitting walls with current approaches. Our notes from the conversations: https://t.co/JqtmIG9blU
Just picked up the @searchplex merch & flyers.
It's the first time I've created something physical to represent the company I've been quietly building — and I have to say, it hits different.
We're all set and heading to #BerlinBuzzwords today!
If you're around, say hello 👋🏼
Spreading Vespa cheer in Prague today — wearing what we run.
https://t.co/lpjlfs7JGz powers the @searchplex stack and my holiday look today ;-)
Thanks for the cool swag, @vespaengine 😎
PostgreSQL and pgvector: Now Faster than Pinecone, 75% cheaper, 100% open-source.
Introducing pgvectorscale, an open-source PostgreSQL extension that builds on pgvector, enabling greater performance and scalability.
Here’s how pgvectorscale helps pgvector outperform specialized vector database like Pinecone:
1️⃣ StreamingDiskANN: A new vector search index that overcomes limitations of in-memory indexes like HNSW the index on disk, making it more cost-efficient to run and scale as vector workloads grow. Inspired by the DiskANN paper from Microsoft.
2️⃣ Statistical Binary Quantization (SBQ): Developed by researchers at Timescale, this technique improves on standard binary quantization techniques by improving accuracy when using quantization to reduce the space needed for vector storage
3️⃣ Written in Rust, giving the PostgreSQL community to contribute to vector support.
📈The result?
On our benchmark of 50 million Cohere embeddings (768 dimensions each), PostgreSQL with pgvector and pgvectorscale achieves 28x lower p95 latency and 16x higher query throughput compared to Pinecone for approximate nearest neighbor queries at 99 % recall, all at 75 % less cost when self-hosted on AWS EC2.
We also tested it against Pinecone’s p2 high performance index, see the blog post at the end of this post for full results (spoiler: It’s just as impressive).
Pgvectorscale is open-source under the PostgreSQL license and free for you to use on any PostgreSQL database for your AI projects.
To get started, see the pgvectorscale github repo: https://t.co/UvqcO3DHLk
Or try it on Timescale Cloud on any new database service.
Eager to learn more about pgvectorscale and how it works? Head over to our blog post with all the details: https://t.co/pUern554Ct
"How do we improve search relevance?" - I've gotten this question many times with clients and it led to implementing the logging layer first, before getting to measuring & eventually improving the relevance. Great to see an open-source initiative to tackle the foundational part.
Foundations of Vector Retrieval
This 185-page monograph provides a summary of major algorithmic milestones in the vector retrieval literature, with the goal of serving as a self-contained reference for new and established researchers.
📝https://t.co/rGLja22Nh7