VECTOR DATABASES ARE THE WRONG ABSTRACTION. Here’s a better way: introducing pgai Vectorizer, a new open-source PostgreSQL tool that automatically creates and syncs embeddings with source data, just like a database index.
❌ Why vector databases fail
Vector databases treat embeddings as independent data, divorced from the source data from which embeddings are created, rather than what they truly are: derived data.
This pitfall means that many AI projects that start out as simple vector search implementations inevitably evolve into a complex orchestra of monitoring, synchronization, and firefighting.
😓 Keeping embeddings in-sync is hard
In an attempt to avoid stale embeddings, engineering teams have to build and maintain a maze of ETL pipelines, juggle multiple databases (vector DB, metadata store, lexical search), and manage complex queuing systems for updates.
Add monitoring for data drift, alert systems for stale results, and validation checks across systems - and you have a brittle infrastructure that inevitably breaks down, leading to stale embeddings and wasted engineering hours.
What if you could just use Postgres instead?
✅ Pgai Vectorizer: Vector embeddings as database indexes
Pgai Vectorizer treats embeddings like database indexes. It automatically creates, updates, and maintains embeddings as your data changes. Just like an index, the database handles all the complexity: syncing, versioning, and cleanup happen automatically.
This means no manual tracking, zero maintenance burden, and the freedom to rapidly experiment with different embedding models and chunking strategies without building new pipelines.
��Why did we build pgai Vectorizer?
Our team at @timescaledb built pgai Vectorizer because many developers regard PostgreSQL as the “Swiss army knife” of databases, as it can handle everything from vectors and text data to JSON documents.
We think an “everything database” like PostgreSQL is the solution to eliminate the nightmare of managing multiple databases, making it the ideal home for vectorizers and the foundation for AI applications.
⚙️How does pgai Vectorizer work?
Check out the code snippet below – it takes just 6 lines of SQL to put your embedding creation pipeline on autopilot with pgai Vectorizer!
Under the hood, pgai Vectorizer checks for modifications to the source table (inserts, updates, and deletes) and asynchronously creates and updates vector embeddings in an external worker.
🧑💻 Sounds exciting! How can I get started?
Pgai Vectorizer is open-source under the PostgreSQL license and available for free to use on any PostgreSQL database. You can find installation instructions on the pgai GitHub repository (see end of post). It’s also available as a managed service in Timescale’s PostgreSQL cloud platform.
📚Learn more
[1] Pgai github repo: https://t.co/hut1MxuwPZ
[1] Technical explainer post: https://t.co/A9hOz482Rg
Share this post with your followers to let them know about pgai Vectorizer and comment your reactions and questions.
🤖 🎉 🎉 Announcing pgvectorscale (100% open source)!
Makes PostgreSQL faster than Pinecone: 28x lower p95 latency and 16x higher query throughput than Pinecone—at 25% of the cost. PostgreSQL licensed.
We explain why and how this extension makes PostgreSQL the best database for AI applications in this blog post: https://t.co/gDsWFqUzwR
PostgreSQL for Everything 🤝 AI for Everything
Postgres 👏👏 for 👏👏 Everything! 👏👏
PostgreSQL has become the de facto database platform.
Here’s why this matters:
1. Data is flooding the world:
Everything — our cars, our homes, our cities, our factories, our farms — is becoming a computer spewing tons of data.
2. Databases are flooding the world:
Two decades ago developers had maybe 5 database to choose from. Today that number is closer to 500, just to keep up with the data flood.
3. But more databases = more complexity = more problems:
Faced with the flood, we have had no choice but cobble together Rube Goldberg architectures with several different “purpose-built” database types: a relational database, a vector database, a time-series database, etc.
4. More complexity means less time to build:
Complex architectures are more brittle, need more complex application logic, and slow down development.
5. Instead of building the future, we are now maintaining the plumbing:
Developers are forced to spend their time and energy on the wrong things.
The answer is simplicity.
The answer is Postgres, for everything.
Thanks to extensions, Postgres is now a versatile platform. Thanks to pgvector, it’s a vector database. Thanks to TimescaleDB, it’s a time-series database. Thanks to PostGIS, it’s a geospatial database.
Postgres is also still the same rock-solid relational database that developers and businesses have trusted with their core workloads for decades.
By choosing Postgres, developers can get back to the real work we are here to do: building the future.
Want to learn more? Read this longer post that I recently wrote:
https://t.co/rEwmCZ2wEs
Want to join the movement? Retweet this post and join the discussion.
#PostgresForEverything
Timescale + @popsql = a developer’s dream come true! 🥳
🚀 Building the Best PostgreSQL GUI for everyone, everywhere 🚀
We’re excited to announce that the PopSQL team is joining Timescale to help build the best PostgreSQL developer experience for the cloud era. ☁️
Congrats to the team at @GetSwitchboard on the Series A! And welcome to @jebgmiller and the team at Icon.
Switchboard is my go-to tool for 1:1s and interactive meetings. Try it here: https://t.co/s8jRu5xCmd
https://t.co/Pj56uXkaKl
In partnership with @NYT's 1619 Project, @NPR, @Newsela and @Tolerance_org, Quizlet is launching a free, digital library of educational materials on the history of systemic racism and other forms of oppression in the U.S.
Quizlet is sharing our State of Remote Learning Report 2020. With over 50 million monthly active users, our online learning platform caters to a diverse user base across geographies & stages of education.
Read on to see our findings for back to school:
https://t.co/aXoMJOSUZO
Today Quizlet announced a $30M Series C round of funding led by @generalatlantic. We have a great team that’s eager and committed to continue helping students learn effectively around the world.
https://t.co/ga1noCCTAg @tonywan
Thrilled to announce Maven's $45m Series C to double down on building a better healthcare system for women and families. Huge thanks to our incredible team, board, investors, and advisors for getting us here, and very excited for where we're going!
https://t.co/2cwEGkZoP2
"Consumer intent on mobile is up for grabs," which is why @Button, an app-to-app commerce connector, was able to raise a new $30 million round from Icon Ventures & Capital One Growth Ventures among others. Scoop @WSJVC https://t.co/4wp2cPPXJD