@system_monarch I don't think 50M API calls would be made, Zomato calls the firebase once, and then the firebase announces it to the topic (the topic contains the devices registered for that notification )
Learning database internals ๐
Imagine tracking a user's profile.
User changes:
๐ Location
๐ท Profile picture
๐ Bio
100 times.
A log-structured database doesn't update the old recordโit keeps appending new versions.
Later:
๐น Compaction removes outdated versions
๐น Merging combines cleaned segments
๐น Hash indexes point directly to the latest profile data
Simple idea. Powerful architecture.
Now I understand why systems like Bitcask are so fast ๐
Learning database internals ๐
Imagine tracking a user's profile.
User changes:
๐ Location
๐ท Profile picture
๐ Bio
100 times.
A log-structured database doesn't update the old recordโit keeps appending new versions.
Later:
๐น Compaction removes outdated versions
๐น Merging combines cleaned segments
๐น Hash indexes point directly to the latest profile data
Simple idea. Powerful architecture.
Now I understand why systems like Bitcask are so fast ๐
Ever wondered WHY SQL databases became dominant? ๐ค
The answer lies in how database models evolved over time ๐
1. Hierarchical Model
Best for one-to-many relationships
Data stored like a tree structure
Similar to how modern JSON documents look today
But handling many-to-many relationships became difficult
2. Network Model
A record could have multiple parents
Supported:
many-to-one
many-to-many relationships
Every record acted like a linked node
But there was a catch โ ๏ธ
Developers had to manually manage the traversal paths (โaccess pathsโ) to fetch data.
Complex systems = painful queries.
3. Relational Model
This changed everything.
Data organized into tables/tuples
Relationships handled using foreign keys & joins
Introduced the game changer โ Query Optimizer
Instead of developers manually deciding how to fetch data,
the database automatically chooses:
โ best indexes
โ optimal execution order
โ fastest access path
That abstraction is one of the biggest reasons SQL databases scaled in adoption. ๐
Understanding database evolution helps you understand why modern systems are designed the way they are.
#SQL #NoSQL #Databases #SystemDesign #Backend #SoftwareEngineering #DataEngineering #Programming #TechTwitter
@system_monarch Thanks for such a wonderful share.
I am even trying to understand the distributed systems
Written a small article related to caching -
https://t.co/gI0EVDFHBx
Would really appreciate some insights.
Spent this weekend diving into research papers.
Just published my first article on caching, summarizing โScaling Memcache at Facebook.โ
https://t.co/gI0EVDGfr5
Curious to knowโhave you explored better approaches? Letโs discuss
@abhijitwt Theyโre volatile โ everything lives in memory, so a restart wipes it out
No real persistence guarantees, unlike databases that use things like WAL(write - ahead logs)
RAM is expensive compared to disk, so scaling can get costly
Getting image generations to match a reference layout is still trickyโmodels tend to hallucinate placements. Any tips on reducing that and getting outputs that stick closely to the reference + prompt?
Getting image generations to match a reference layout is still trickyโmodels tend to hallucinate placements. Any tips on reducing that and getting outputs that stick closely to the reference + prompt?
System Design fundamentals every engineer should know:
When building scalable systems, the first question is: single server or distributed?
Start with a single server โ it's simpler, easier to iterate on, and perfectly fine for early-stage products and predictable loads. But once you hit 10x growth, things change fast.
At scale, you need:
โ Horizontal scaling + sharding for data
โ Load balancers to distribute traffic
โ Database replication (master for writes, replicas for reads)
โ Cache layers (read-through strategy, LRU eviction)
โ CDNs for static content delivery
โ Stateless architecture so any server can handle any request
โ Message queues to decouple services asynchronously
โ GeoDNS + multiple data centers for availability & latency
Choosing the right database matters too:
โข Relational DB โ complex joins, many-to-many relationships
โข Document DB โ self-contained "tree" data (one-to-many)
โข Graph DB โ highly interconnected, "anything relates to everything"
And don't forget observability:
Logging + metrics (host, aggregated, business KPIs) + CI automation = a system you can actually trust.
The hardest part isn't building it. It's knowing when and what to scale. ๐
#SystemDesign #SoftwareEngineering #ScalableSystems #BackendEngineering #TechLearning
Weโve launched our new website with a fresh new theme.
Please check it out: https://t.co/vZP0ZsTebg
Would love to hear your feedback. Thanks Everyone for all the support! ๐
This is when dot product stopped being โmathโ.
It became:
normalization
intent extraction
constraint logic
A way to ask:
โHow much of this belongs in that direction?โ
A math formula I memorized years ago finally turned into code intuition.
For the longest time, dot product was just a formula I used.
Aโ B = โฃAโฃโฃBโฃcosโกฮธ
I could calculate it.
I couldnโt feel it.