working at HFT and complex Fintech systems for 8 years, I’ve noticed something:
Most engineers think databases become slow because of complex queries while problems lies somewhere else.
Interviewers now ask things like:
“How would your database survive 10M writes/minute without melting the disk?”
And candidates immediately jump to:
- sharding
- caching
- replicas
But the real bottleneck often starts much deeper: Database internals.
These 10 concepts are what actually allow modern databases to handle insane write throughput in production.
Bookmark this thread. Read till the end.
I was looking for cost effective retention options for our AWS Managed Kafka brokers and set out to do a boring cost exercise, make a table, move on. Didn't quite work out that way.
Wrote up what I learned, including the tradeoffs:
https://t.co/lWUok9oOLt
@divyaporwal_ I use Macbook Pro at work(48 gb M4 Max) and got a personal M4 Air 16 Gb recently for personal use, Honestly the Air doesn’t disappoint me and super value for money, but if you plan to run local models then pro seems better choice
I recently wrote about how we tackled API latency for our apps. We ended up building a custom decorator based cache tooling on top of Amazon ElastiCache. It gave us massive speed gains while keeping the code clean.
https://t.co/epIq6p6FDY
@arpit_bhayani Esop in Indian startups is very grey area as the odds of making multiples over it are lower..I joined Rivigo as my first company few months before its slump sale and a lot of my seniors saw their esops go to 0 because it was a distress sale.