We built external indexing for pgvector in Postgres.
External indexing offloads the resource-intensive index creation process to external machines to reduce the impact on database performance.
Read the blog post from @D4RK7ET below 👇
https://t.co/1r8J2Nyudg
These days, column stores are used everywhere for analytics workloads. But how good are they, really?
I like this paper from the early days of column stores because it explains how column stores are so effective for analytics and why their benefits are hard to replicate with the traditional row store architecture.
The obvious reason column stores are effective compared to row stores is that they allow queries to only access the columns they need and ignore the others. However, even when that behavior is simulated in row stores (for example, using materialized views containing only the columns needed by a query), column stores perform far better on analytical workloads. Why?
The authors find a couple of reasons! First, better block iteration. On analytical queries that evaluate a large number of tuples, a row store needs to iterate through tuples one at a time, extracting data from each tuple to perform predicate evaluation or aggregation. A column store can operate on blocks of values from the same column in a single function call, which takes advantage of CPU caching and parallelism (which is even more true now than it was back then).
Second, compression. Column stores put similar data from the same column together, which can enable enormous space savings, and potentially enormous speedups, through compression, for example on a very sparse column. By contrast, a row store has to store whole rows, so even if one column is very sparse or otherwise low-entropy, it is still stored in its entirety.
Third, late materialization. Even if a column store needs to return rows to answer a query, it can perform most operations (predicates, joins) only on a handful of columns and then materialize rows later for the data the query returns. This can greatly reduce time spent processing values in columns that are returned but not evaluated.
if you ever begin to think you have a novel idea, just remember that some coked out aerospace engineer from the 50s had had that same idea 70 years ago, and the only difference is that you have cheap computing power at your fingertips
Footnotes are not for dumping bits of your argument that wouldn’t fit in the body. They are for displaying bits of fun but totally irrelevant antiquarian knowledge
@angeris ... where the artist is in control well past where we'd expect them to lose it. And the neighborhood doesn't do that either! It's a sadness where you're like, "okay, it's clearly not overwhelming. You're not overwhelmed at all. ur just brooding"
@angeris No, no, I don't think it's whisper-yness! Nothing to do with quiet or softness. Like, here's Brian Eno's take on limits of form. Howling, voice cracking, etc are emblematic of blues; you get to 90's R&B and Lana's contemporary Adele, and audiences appreciate the talent...
@badcryptobitch@matthew_d_green Knowing when to break rules is an important life skill. Adolescence is a great time to learn it if you're in a safe environment.
@matthew_d_green Is Johns Hopkins CTY still doing residential programs in the summer? I TA'd for a similar program from Duke and I feel like the kids got into an okay amount of trouble outside the classroom.
@angeris It adds a lethargic quality and tons of artists use it sparingly (so you can look at two different songs by the same artist and be like, oh yeah, that's the difference) but her thing is just to do it every single time
@angeris Everyone is overthinking it. It's not the lyrical subjects or how they approach themes or whether they're from California. Lana just doesn't use the top of her range so it doesn't sound like she's straining.
1/ Starting tomorrow, a fresh run of the Proofs, Arguments, and Zero-knowledge reading group will meet Wednesdays in the @__zkhack__ Discord at noon EST.
The first session will give a broad overview of the SNARK design space + the costs of key operations.
Simone Weil and her brother André grew up to become respected intellectuals, in philosophy and math respectively. A letter that André wrote to Simone in 1940 told of a bridge between branches of math. “Just as God defeats the devil: this bridge exists.” https://t.co/PrP65mMvHQ