🚨Rice “Mikel is gutted but he spoke about how much he loves us. I think we have come so far as a club. When I came to this club it was a QF exit then a Semi Final exit and now a final exit. One more step and we go again.
One thing about us, this wont define us.
We will be back”
Fun fact: Redis does not just rely on the standard operating system memory allocator. Even though it's written in C, it overrides the native `malloc` implementation with highly optimized third-party memory allocators like `jemalloc` (by Facebook) or `tcmalloc` (by Google) to achieve efficiency and low-fragmentation performance.
I firmly believe that understanding theory is fine, but things become interesting when you sit down to implement it. This is where you find gaps in your understanding and understand the nuances. and hence...
Today, we dive into the source code of Redis to look at how and why Redis overrides the default `malloc` implementation.
This is the 18th video in the Redis Internals series. Like always, we keep our focus on execution and not just theory, looking closely at how an in-memory database handles massive amounts of small object allocations and deallocations without crumbling under memory fragmentation.
In the video, I talk about memory pages, how native allocators interact with the OS, and the classic problem of fragmented memory. We also look directly at the Redis source code and see how they wrapped malloc with zmalloc to abstract out the implementation.
By the way, 18 videos are now live:
1. Why Single-Threaded Redis Is Fast
2. Writing a TCP Echo Server
3. Wire Protocols
4. Implementing RESP
5. Implementing PING
6. Understanding Event Loops
7. Implementing Event Loops
8. Implementing GET, SET, and TTL
9. Implementing DEL, EXPIRE, and Cleanup
10. Evictions and Implementing first-eviction
11. Implementing Command Pipelining
12. Implementing AOF Persistence
13. Objects, Encodings, and Implementing INCR
14. Implementing INFO and allkeys-random Eviction
15. The Approximated LRU Algorithm
16. Implementing the Approx LRU Algorithm
17. How Redis Caps Its Memory Usage
18. How and Why Redis Overrides Malloc
If you have ever wondered how scale-critical systems avoid performance degradation over time and handle advanced manual memory management with custom concurrency and defragmentation controls, this deep dive is for you.
Hope this helps you better understand database internals and spark that engineering curiosity.
Give it a watch.
Arteta: "We start to make some very important decisions if we want to reach another level. And we're going to have to show that ambition. We are more than capable of doing it, but it's going to demand us to be very, very ambitious, very fast and very smart."
API Design Playbook
Giveaway Alert!!!
• Core API fundamentals.
• Clean & scalable design principles.
• Popular patterns used in real-world systems.
• Practical concepts for interviews & building projects.
(24 hours only.)
To get it for free:
1 Follow @systemdesignone [MUST]
2 Like & Retweet to get DM
3 Reply "Playbook"
Then I'll DM you the details.