I built a sharded, durable key-value store from scratch in Go.
No tutorials. No frameworks. Just the primitives that real
databases are made of.
Here's what I learned across 6 stages π§΅
A benchmark taught me more about distributed systems than any
tutorial:
Unsharded, 4 cores: 152 ns/op
Sharded, 4 cores: 65 ns/op
But the real lesson was when I broke it on purpose (all keys β 1 shard):
458 ns/op. Worse than unsharded.
People ask how I learn this stuff.
The honest answer: I don't read more. I build smaller.
One stage at a time. One problem at a time. One bug at a time.
Depth compounds. Breadth doesn't.
The hardest part of building a database isn't the code.
It's the ordering constraints.
Snapshot before truncate.
Log before memory.
Rename before truncate directory.
Every one of them exists because someone, somewhere, lost data
by getting the order wrong.
Write-ahead log in one sentence:
Every write goes to disk BEFORE it goes to memory.
If you crash between the two, the log has it. If you crash the other
way, you lose it silently.
Order is not a suggestion. It's the whole thing.