Aayush Gupta

MSCS CU Boulder | ex-Software Engineer @Flipkart | Learning about and working on Databases and Distributed Systems | views are my own...

[wangbin579 @wangbin579]

This paper on database optimizers is definitely worth a read! If you’re into database technologies — especially a PostgreSQL fan — you’ll find it well-explained and easy to follow. Thanks to @ashishjayamohan for the recommendation!

[wangbin579 @wangbin579]

If you're interested in databases, this paper is a must-read! It provides an in-depth analysis of the challenges faced when implementing MVCC in most databases and their solutions.

[Kaivalya Apte - The Geek Narrator @thegeeknarrator]

Going deep into real world systems’ internals is the 2nd best way to learn good #SystemDesign.

Just released the next episode with @MarcJBrooker where we talk about internals of @awscloud Aurora DSQL. While doing this we also talk about general database design and important implementation details that will help you understand what goes behind implementing a system like Aurora DSQL.

This is a gem, trust me on this.

Go watch now:
https://t.co/8gWxYbYpWg

If you are interested in learning about databases, I have 43 episodes just on Databases here https://t.co/iv0bBNs0D6

If you are interested in learning about the JVM internals, I have an ongoing series (member-only). Do check it out. https://t.co/uwfVQC2aa9

If you are interested in learning about Streaming systems, here is the playlist to look at: https://t.co/DIqfFHbtQt

If you are interested in learning about #Kafka here is the playlist: https://t.co/LCNsv5KasL

In general if you are interested into Databases, Distributed Systems, Stream processing, Backend Engineering, The GeekNarrator is something you will love.

Thanks

[Andy Pavlo (@andypavlo.bsky.social) @andy_pavlo]

New @CMUDB Course: Database Query Optimization!
A survey of how to build a SQL optimizer from scratch covering foundational and state-of-the-art implementations. All lectures available on Youtube: https://t.co/I8dMflK9dP

[Roy Carrilho @RuiCarrilho5]

Found Low-Level Programming University, a roadmap for becoming a low-level dev. Seems pretty cool and regularly updated!

[Kaivalya Apte - The Geek Narrator @thegeeknarrator]

I started my 2025 reading white-paper(s) on Bε-trees. A thread that captures everything you need to know about them and more..

Like, repost and share..👇

[v @iavins]

Here's a fascinating story of how researchers used Bloom filters cleverly to make SQLite 10x faster.

I'll also explain some database internals and how databases implement joins.

Let's dive in!

[Phil Eaton @eatonphil]

7 databases in 7 weeks

cool idea :)

https://t.co/3CTcxqproh

[Phil Eaton @eatonphil]

Three basic suggestions for getting into distributed systems

- Read DDIA
- Follow MIT 6.5840
- Do fly dot io distributed systems challenge

(no particular order)

[Phil Eaton @eatonphil]

Josh has such great btree libraries in different languages. Love this new one for type safe btrees in C.

https://t.co/NHPCRsvrTS

[Abhinav Upadhyay @abhi9u]

I see a rising interest in memory allocators. Before looking at how modern allocators work, which are too complicated for various reasons, it helps to study how the older and more simpler implementations were designed.

On that note, I found a great paper from Poul-Henning Kamp who designed the new malloc in FreeBSD 2.2. This implementation was optimized for virtual memory based systems.

Many ideas which are now commonly used are touched upon. PHK later on designed the popular Jemalloc implementation as well, which is used widely.

[Abhinav Upadhyay @abhi9u]

I've had a bit more chance to read OSTEP. It is a very good intro to OS, but if your aim is to do systems programming you need to go deeper.

The kernel is deeply intertwined with the peculiarities of the hardware architecture and without looking at the implementation details, you will have holes in your understanding.

I'd suggest supplementing OSTEP with a few more resources in the replies. Each has its own unique point of views. (The OSTEP authors recommend many of these themselves as additional resources on the website).

[Phil Eaton @eatonphil]

This is a very good paper on benchmarking courtesy of Justin Jaffray.

We definitely need more practical books and writing on benchmarking practices geared at senior+ developers.

https://t.co/Icp4ZsS21K

[Colin Breck @breckcs]

Blogged: Predicting the Future of Distributed Systems
https://t.co/qwEYZ2SMkd

[Dominik Tornow @DominikTornow]

Research papers aren't just theory, they're the backbone of innovation

Take @TigerBeetleDB, which implements Viewstamped Replication & Protocol-Aware Recovery

Two cutting-edge research papers equip the beetle with top tier safety & liveness guarantees

https://t.co/DrSqqeNFPR

[CedarDB @cedar_db]

Have you ever wondered why existing database systems focus on either analytical or transactional performance?

Learn why this is the case and how a hybrid storage engine can deliver high performance for combined workloads:

https://t.co/HUxIgiuGBd

[Vipul Vaibhaw @vaibhaw_vipul]

Release 0.1.0-alpha is out! 🎉
This is our very first raft-rs release, and we're driven to make it one of the best Raft implementation out there!

It has some optimizations like -
- runtime cluster membership (makes it more scalable)
- default leader
- leader preference
- It does not assumes storage to be non-malicious. It can repair its storage. (inspired from tigerbeetledb)
- It doesn't use any serialisation library, it works on bytes directly.
- zero-copy support.
- log compaction, etc.

🚀 Check it out: Release v0.1.0-alpha.

It is built on top of tokio runtime.

We're in this for the long haul and have some exciting plans ahead.
If you're interested in contributing, we've got several open issues where we'd love your help! 🌟

[Abhinav Upadhyay @abhi9u]

AI engineer using OpenAI wrappers

Vs

@karpathy showing how to build your own LLM from scratch in C

[Alex Miller @AlexMillerDB]

Funemployment means fun blog posts! Please enjoy a categorization of replication algorithms by failure handling strategies; their optimizations, tradeoffs, and complexities; and how this all affects their resource efficiency, latency, and availability.
https://t.co/yGh75oMsgX

[Peter Kraft @petereliaskraft]

Want to learn how database locks actually work?

Check out this incredibly thorough review by database legend Goetz Graefe, which dives deep into how databases use locks to protect your data and the integrity of your transactions.

One of the most interesting distinctions in this paper is between locks and latches. Locks provide concurrency control between transactions--they're heavyweight, are meant to be held for a long time, and support complex scheduling and deadlock detection policies. However, as a result, they're expensive to acquire and release, requiring thousands of CPU cycles.

By contrast, latches protect individual data structures from concurrent accesses by different threads/processes. They're lightweight (tens of CPU cycles per acquire/release), are held only while the data structure is being read or updated, and have minimal scheduling or deadlock detection capabilities and thus must be used very carefully. You might grab a latch before physically modifying a B-tree page in memory to ensure no one else concurrently writes to that page.