Jeevananthan

Love to talk about the Theory of Computation that can be concurrent, parallel and distributed. Doing Ziglang⚡. I eat tech blogs and research papers for supper.

[Jeevananthan @Jeevananthan05]

@arpit_bhayani In a more specific way to address OLTP & OLAP, in OLAP db's not use any lock or transactional lock on data/rows/records but uses bufferpools to buffer the data in-mem and stores LSM style. For eg: ElasticSearch(Apache Lucene indexer) uses 2PC protocol to mem-buffer-> segement

[Viacheslav Biriukov @brk0v]

🦀 If you are trying to understand modem software development.

https://t.co/4cq7ma0Bg5
#rust #rustlang

[TigerBeetle @TigerBeetleDB]

The Bi-Channel Networking Paradigm for Database Systems in the Cloud

https://t.co/gKe6fMcNO4

Congratulations to the authors, @georg_kreuzmayr, @melhindiCS, @Tobias__Ziegler, Benjamin Wagner, and @viktor_leis! Also for its acceptance at EDBT 2027.

[Mitchell Hashimoto @mitchellh]

My family is donating another $400,000 to the Zig Software Foundation. Zig is exceptional software. I use AI every day. Zig has one of the strongest anti-AI policies in open source. We disagree on some things, but respect doesn’t require agreement. https://t.co/gaOuwjDDtm

[Jeevananthan @Jeevananthan05]

@avrldotdev My bad it is --> terms

[Jeevananthan @Jeevananthan05]

@Saanvi_dhillon Zig⚡

[Jeevananthan @Jeevananthan05]

@Aditya_181105 your APIs 50tl lie % is avg: 600ms

[RustRover, a JetBrains IDE @rustrover]

“Zig over Rust?” 👀

@VBragilevsky asked Andrew Kelley a simple question, and the answer turned into a very interesting take on language tradeoffs, tooling, and choosing the right tool for the job.

Full interview in the comments 👇

[Loris Cro ⚡ @croloris]

Andrew's interview with JetBrains

https://t.co/7SsmfjksJP

[Ben Dicken @BenjDicken]

Every engineer should read this.

The principles for building reliable software systems have been around for a long time. Max outlines them beautifully.

Here's to getting that 99.99% on your status page.

https://t.co/HFDcriLodl

[Jeevananthan @Jeevananthan05]

@thegeeknarrator @confluentinc Congrats!! 🥳

[Jeevananthan @Jeevananthan05]

@jarredsumner This is so heating up and we all the reason why the moment to Rust.
We always make trade offs and there is free lunch!

[Anderson Nascimento @andersonc0d3]

How to execute an object file: Part 1
https://t.co/RvyPiqDL8J

How to execute an object file: Part 2
https://t.co/MwMtEAzVaf

How to execute an object file: Part 3
https://t.co/LWuRvsQfvs

How to execute an object file: part 4, AArch64 edition
https://t.co/ucaAoblePt

[Jeevananthan @Jeevananthan05]

@0xlelouch_ Now the followup question would be
So how many topics and consumer grps we have to keep. How do you handle priority in the producer and consumer?

[Abhishek Singh @0xlelouch_]

A good system design interview question for Staff Distributed Systems Engineer:

Design a notification system for 100M users that can send email, SMS, push, and in-app notifications.

This looks simple from outside. “Just send a notification.” But this is exactly where distributed systems interviews become interesting because now you need to think about retries, duplicate sends, user preferences, queue lag, provider failures, priority, rate limits, observability, and cost.

How one can approach this problem?

I would start by separating notification types. An OTP, password reset, payment alert, marketing email, and “someone liked your post” notification should not go through the same mental model. OTPs and payment alerts need low latency and high reliability. Marketing emails need batching, throttling, cost control, and strict preference checks. Engagement notifications can tolerate some delay.

At a high level, backend services should not directly call SMS or email providers. Payment service, auth service, order service, or social service should only publish a notification event like PAYMENT_SUCCESS or PASSWORD_RESET_REQUESTED into a queue. Then a notification orchestrator consumes that event, checks user preferences, chooses the right template, applies rate limits, checks idempotency, and routes it to channel-specific queues like email, SMS, push, or in-app.

The most important part here is idempotency. In distributed systems, retries happen everywhere. Producer may retry. Consumer may process same message twice. Worker may timeout after provider already accepted the request. So every notification should have a unique event_id, and delivery should use a key like user_id + event_id + channel to avoid duplicate sends. This matters a lot because sending the same OTP or payment alert five times destroys user trust.

I would also separate queues by priority. Critical notifications like OTP, login alerts, payment alerts, and password resets should go into high-priority queues. Likes, comments, and reminders can go into normal queues. Marketing campaigns should go into low-priority queues. This way a 50M user campaign does not block password reset messages.

Retries should be handled carefully. Provider timeout, 5xx errors, network failures, and rate-limit errors can be retried with exponential backoff. Invalid phone number, unsubscribed user, hard bounce, or broken template should not be retried blindly. After max retries, the message should go to a dead letter queue so engineers can inspect and replay it later.

For storage, I would keep a notifications table for the original event, delivery attempts table for each provider/channel attempt, and user preferences table for opt-ins, opt-outs, quiet hours, and channel settings. Preferences can be cached in Redis because every notification needs that lookup, but the main source of truth should stay in the database.

For scale, assume 500M notifications per day. That is around 5,800 per second on average, but peak traffic can easily be 10x during campaigns or incidents. So the system should be designed for 50k+ notifications per second using partitioned queues, horizontally scalable workers, provider-level rate limits, and backpressure. If SMS provider supports only 10k messages per second, workers should slow down instead of creating retry storms.

Observability is also a must. I would track notification created count, sent count, failure count, queue lag, retry count, DLQ count, provider latency, provider error rate, duplicate suppression count, template failures, and delivery latency p95/p99. Saying “we will add monitoring” is too vague. A good system should answer: “Why did this user not receive OTP?” within seconds.

The main tradeoff is that you do not treat all notifications equally. Some need speed. Some need reliability. Some need cost control. Some need strict user preference checks. A good Staff-level answer is not about naming Kafka, Redis, and workers randomly. It is about explaining what can fail, where duplicates can happen, how retries behave, what gets priority, what can be delayed, and how you debug the system when users say, “I did not receive the message.”

[Pekka Enberg @penberg]

After being DST-pilled by TigerBeetle, I've written the same completion-based I/O loop many, many times just to get going on whatever new i was working on.

This time I extracted it as a small Rust library so next time I don't have to start from scratch.

https://t.co/6GH8fVwZG5

[Rach @rachpradhan]

What shipped under the hood this cycle:

Throughput: 2,400 → 115,093 req/s (48× faster)
Latency: 41 → 0.39 ms (105× faster)
RAM under load: 71.7 → 4.8 MB (15× leaner)
Release binary: 1.9 MB → 260 KB (7× smaller)

Arena reuse, in-memory static caching, shell-first paint, parallel fetches collapsing onto one critical path.

[Jeevananthan @Jeevananthan05]

@grok @0xlelouch_ The only Kafka guy i know can help here maybe @kozlovski!

[Jeevananthan @Jeevananthan05]

@grok @0xlelouch_ My bad i don't have anyway it is a bit tricky and messy part of handing priority, partitioning and consuming. I really can't find the solution otherwise i have to do the trade off. But my question is most of the systems faced this issue but nobody talked about it.

[Jeevananthan @Jeevananthan05]

@grok @0xlelouch_ @0xlelouch_, let me know if you can help me with this.