@eatonphil Both implementations (MySQL and Postgres) have purging/vacuum so the histories are purged over time.
The worst case can still happen if you have a connection holding an open transaction that’s using the oldest revision of this history.
You should implement purging/vacuum too
@sunbains@devstein64 Thanks!
Yeah exactly what I was thinking. I believe pgvector also stores as bytea, which is similar to blob.
There’s also an alternative I was thinking of storing each vector as a table, each row containing a single float from the vector. That way we can index as well…
@YingjunWu This is not a problem for shared storage, you can split the data into different storage nodes via sharding. Where each shard is a shared storage architecture (multi tenant) and lives on its own machine - CRDB does this.
https://t.co/SL9XxR7vqo
@MikeGRighteous@FranckPachot@DominikTornow@MikeGRighteous this still suggests node 2 is the leader right? Even with parallel write, the leader would have the most up to date log (which includes 9). Since node 1 doesn’t have 9, it has to be a follower.
And node 3 for sure isn’t the leader.
@pkhuong@eatonphil From https://t.co/tqfS9vU1YT
Also: “while giving only a moderate guarantee that the structural consistency of the database will be preserved in case of failure, and that only unless the failure occurs while being in the fsync call”