oldmoe مُحَمَّد

Xoogler, Rubyist, SQLitist

[oldmoe مُحَمَّد @oldmoe]

@IWAN_RG Amazing work msA, thanks for open sourcing this

[oldmoe مُحَمَّد @oldmoe]

@penberg It's a delicate balance

From one side you get isolation, limited blast radius and better concurrency

But you have to live with schema syncing, distributed queries and file descriptor exhaustion challenges

M:N works too and you get the best, and also the worst of both!

[oldmoe مُحَمَّد @oldmoe]

@glcst @HSVSphere Wouldn't just setting the page cache size properly (or the mmap size if multiple processes use the db) deliver the needed memory-then-desk function but transparently?

[oldmoe مُحَمَّد @oldmoe]

@drmingdrmer Great comparison!

[oldmoe مُحَمَّد @oldmoe]

@kellabyte Using how many client processes/threads?

Typically a cluster should perform better with more concurrent clients until it reaches its saturation point

But regardless, that throughput number sounds very ... low?

[oldmoe مُحَمَّد @oldmoe]

@lemire @Adev I would argue it is technological & governing advancement, power, expansion & wealth, in that order

And indeed there is a clear "current" advantage in the Western approach to tech and government. But history didn't end yet and new models will emerge, exactly like this one did

[oldmoe مُحَمَّد @oldmoe]

@Adev @lemire So I bring up the colonial past and you mention the state post the 70s. Who's straw manning?

I am not really sure what is being argued here? That European colonialism didn't happen? or that it wasn't an economic boost for Europe?

Both are simply false anyway

[oldmoe مُحَمَّد @oldmoe]

@Adev @lemire This is totally wrong. Small colonies existed for Scandanavia while Germany & Italy had vast colonies in Africa.

Pretending the systematic stealing of resources accompanied with oppressing the natives didn't happen or was inconsequential economically is bizarre to say the least.

[oldmoe مُحَمَّد @oldmoe]

@lemire Of course there are multiple other factors, but we can't escape the fact that England, France, Germany, Spain, Italy, Portugal and The Netherlands benefited a lot from their colonies and their resources, at the right moment too. Citing other examples doesn't negate this fact.

[oldmoe مُحَمَّد @oldmoe]

@lemire I said most, and that truly applies to most of the lines to the left of that imaginary line, no?

[oldmoe مُحَمَّد @oldmoe]

If you are building a really high performance data ingestion pipeline, what matters most is how many pages you are touching per tx and in which cache level (L1, L2, L3) would they fit.

WAL helps, but it will largely depend on how many records are written per page in a single tx

[oldmoe مُحَمَّد @oldmoe]

Don't do expensive stuff in a loop

Prepare all you need first then iterate and do whatever

[oldmoe مُحَمَّد @oldmoe]

user1: why is this slow?

rails dev investigates: oh, it’s N+1 AR queries.

user2: why is this slow?

C dev investigates: oh, it’s N+1 malloc().

the similarity is uncanny.

[oldmoe مُحَمَّد @oldmoe]

@nateberkopec It is actually a very promising result, I think that piece of code should be in the zjit test suite

[oldmoe مُحَمَّد @oldmoe]

@Iztok @brennandunn @patio11 This, the moment you need to set aside time to talk to a potential customer means a completely different engagement model than self-serve, and hence a completely different pricing structure is needed.

[oldmoe مُحَمَّد @oldmoe]

@webcodr @carlcamilleri @dhh @Beelinkofficial That too

[oldmoe مُحَمَّد @oldmoe]

@carlcamilleri @dhh @Beelinkofficial Memory bandwidth

[oldmoe مُحَمَّد @oldmoe]

Spent a little time optimizing a hot route on a Ruby Roda + SQLite JSON API app

15 db queries: 1 refactored, 3 new indexes & all now use prepared statements

DB time was ~5ms per req, now ~500us per req

Most of the req time was in DB, reqs now are under 1 ms

Roda + SQLite FTW!