Over the past few days, I've been building a fairly complex distributed system.
Tomorrow, I'm adding k6 and putting it through different load scenarios to see how it behaves under pressure.
The plan is to break things, understand why they fail, and learn from what happens.
Stay tuned. I'll share what the system does, what broke, why it broke, and how I fixed it.
Let's see what holds up.
Going to take a break today. I've been pushing myself pretty hard these past few days.
Time to slow down a bit, play some games and watch a few movies.
I don't usually watch movies, so drop some recommendations. What are some movies you think everyone should watch? ๐ i like thrillers
That makes sense. So concurrency limits control how much a tenant can execute, but they don't necessarily guarantee fair access to the workers. That's where fair scheduling or isolated worker pools become important. that is the Interesting part how quickly queue management turns into a resource allocation problem, it just depends upon sla
A bit of light reading today ๐
Uber's engineering blog on how they handle background tasks in Cadence.
Interesting problem: one customer's batch job can delay workflows for everyone else, even when the incoming API traffic looks normal.
I liked how they approached it with task prioritization and separate queue cursors instead of just throwing more workers at the problem.
https://t.co/5tksY1QtIH
@riteshs2907 Exactly. Iโd start with per tenant concurrency limits and separate queues for batch vs interactive work. Curious how far that gets you before needing something more sophistcated.
i had claude go through every @kirat_tw podcast. 83 episodes, 40 hours
100+ devs explaining exactly how they got hired. most had no big college name
wrote down everything that worked https://t.co/2KEBIOF76D
Backend developers, quick question.
Your API is slow.
You scale from 2 instances to 10.
But latency gets worse.
What's the first thing you'd investigate?
A. CPU utilization
B. Database connection pool
C. Load balancer
D. Network latency
Curious to see how people approach this.
Trying to find more engineers on here who enjoy understanding why things break, not just how to make them work.
The kind of people who:
- Run experiments instead of guessing
- Read engineering blogs for fun
- Break their systems to understand failure modes
- Build tools to solve problems they encounter
I'm doing more of that myself.
If this sounds like you, let's connect. What was the last interesting thing you built or learned?
Two requests arrive at almost the same time.
Both check whether a username is available. Both see that it is. Both try to register it.
What is the most reliable way to prevent duplicate usernames?
A. Check availability twice
B. Use a mutex in the frontend
C. Add a unique database constraint and handle conflicts
D. Increase the server's CPU
Your service writes an order to a database, then publishes an event to Kafka.
What happens if the database write succeeds but Kafka is unavailable?
A. Both operations roll back automatically
B. The order is lost
C. The order exists, but the event may never be published
D. Kafka retries the database transaction