Backend engineer (Golang) open to opportunities 🚀
I’ve been building high-scale systems — from handling massive concurrent WebSocket connections to designing distributed backends with Kafka & Kubernetes. I enjoy digging into performance, latency, and system design challenges.
Reading a tutorial and terms like exponential backoff, jitter, and idempotency keys makes sense. Then you build the thing, the API returns a 500, and you watch your system crash.
The happy path is the part everyone writes about. The request succeeds, the response looks like the example, and the system behaves. That is maybe 20% of the actual code you end up shipping.
The other 80% is what you only discover by running the thing under real conditions. A queue that is supposed to be FIFO but is not under load, a dependency that is "always available" until it is not, or a database connection pool that gets exhausted when 3 slow queries stack up during a deploy.
Hence, reading 10 blog posts about a system is no substitute for spending a weekend building it. The bugs you hit while building it (or, better yet, while running it in production) are the actual curriculum. The blog post is just the syllabus.
Reading teaches the happy path. Building teaches the failure modes. So keep coding and keep shipping.
Hope this helps.
Spent some time today diving into Go's pprof and profiling internals. Instead of guessing where performance issues exist, I'm learning how to measure CPU usage, memory allocations, goroutine behavior, and GC activity to make data-driven optimizations. Building small experiments, generating flame graphs, and analyzing hotspots has been a great reminder that writing performant code isn't just about algorithms , it's about understanding what your program is actually doing at runtime. Every backend engineer should spend time with profiling tools before reaching for optimizations. #golang #pprof #performance #backend #softwareengineering
Go is doing a lot before your main function runs. That 1.5MB of extra binary contains the entire runtime: a memory allocator, a garbage collector, a scheduler, a system monitor, and all the machinery needed to support goroutines, channels, and maps. Before you get control, Go has to set all of that up.
A subtle lesson from designing ticket booking systems: a Seat Lock table and an Idempotency table may look similar, but they solve entirely different problems. A Seat Lock table answers, "Who currently owns this seat?" and prevents multiple users from reserving the same seat simultaneously. An Idempotency table answers, "Have I already processed this request?" and prevents the same user from accidentally creating duplicate bookings due to retries, refreshes, or network failures. One protects a resource, the other protects an operation. If you only have seat locks, users can still be charged twice. If you only have idempotency, two users can still compete for the same seat. Production systems need both.
Unikernels are one of those ideas that feel futuristic even though they've been around for years. Instead of running an application on top of a full operating system, the application and only the OS components it needs are compiled into a single specialized machine image. The result can be dramatically smaller attack surfaces, faster boot times, lower memory overhead, and fewer moving parts. While containers became the mainstream answer to lightweight deployment, unikernels remain a fascinating alternative for highly specialized workloads where security, isolation, and efficiency matter more than general-purpose flexibility.
Just read a great piece on CAP theorem and realized how much the usual “pick any 2 of Consistency, Availability, and Partition Tolerance” explanation oversimplifies things. In a real distributed system, partitions aren't optional—they will happen. The actual question is what your system does when they do: sacrifice availability to stay consistent (CP) or sacrifice consistency to stay available (AP). CAP isn't really about choosing between CA, CP, and AP; it's about the consistency–availability tradeoff during a partition. One of those concepts that's widely quoted but often misunderstood.
This is an unbelievable piece of work by Sarthak and something that requires amplification.
Let me explain what he found, in simple terms.
Sarthak is a Class 12 student from the 2025-26 batch, one of the 17 lakh students whose answer sheets went through CBSE's new On-Screen Marking system.
He spent days reading through CBSE's evaluation tenders, scraped all 576 tenders CBSE has issued, and tracked how the rules changed across three versions of the same tender.
The core finding is that the company that won the contract to scan and grade 17 lakh students' answer sheets is Coempt Eduteck.
Coempt used to be called Globarena Technologies. Globarena was the company behind the 2019 Telangana intermediate exam disaster, where software failures led to 3.8 lakh students getting wrong or missing marks, and 23 students died by suicide.
A government committee found systemic failure and negligence. Six months later, Globarena rebranded to Coempt Eduteck.
So a company with that track record won a contract to handle 17 lakh CBSE students. Sarthak's investigation is about how the rules were rewritten to let that happen.
The tender was issued three times.
> First tender, February 2025. It existed, then disappeared from the public GeM portal. Sarthak scraped all 576 CBSE tenders and this one was missing from the archive entirely.
> Second tender, May 2025. Four companies applied including TCS and Coempt. All four failed the technical evaluation. Cancelled.
> Third tender, August 2025. Coempt won. Between the second and third tender, a series of rule changes happened, and every single one made it easier for Coempt to qualify.
Here is what changed, one by one.
01. The old rules disqualified any company with a history of abandoning work, failing to complete contracts, or financial weakness. The new rules deleted this clause entirely. Coempt's Telangana history stopped being a barrier.
02. The old rules disqualified any company that was "blacklisted earlier." The new rules changed this to "currently blacklisted." Because Globarena rebranded after Telangana, removing the word "earlier" effectively erased their past.
03. The rules required Rs 50 crore average turnover over three years. Coempt's exact average came to Rs 50.86 crore. They cleared the bar by less than 1%. Earlier, a smaller company had asked CBSE to lower the bar to Rs 30 crore for fairer competition. CBSE refused. So the bar was kept high enough to block small players, but sat exactly low enough for Coempt to scrape through.
04. Software maturity is measured on the CMMI scale, 1 to 5. The old rules required Level 5. The new rules dropped it to Level 3. Coempt is a Level 3 company.
05. The cooling-off period for engaging retired CBSE officials was cut from two years to one. This makes it easier to use recently retired insiders to influence the process.
06. The old rules required experience with large projects of at least 5 lakh students each. The new rules removed the student count and counted cumulative answer-book volume across small projects instead. Coempt has many small fragmented university contracts. This helped Coempt and hurt TCS.
07. The old rules required bidders to own their own data centre and disaster recovery centre on Indian soil. The new rules allowed third-party MeitY-empanelled cloud hosting. Coempt runs on AWS and Azure. This helped Coempt and hurt TCS, which owns its own data centres. It also means student data is no longer on sovereign, Indian infrastructure.
08. The old rules required the bidder to own or control the complete source code of its software. The new rules deleted this. Coempt's platform runs on Microsoft's proprietary IIS, which they don't own.
09. A last-minute corrigendum, issued right before bid submission, removed CBSE's own power to blacklist the firm if its software failed catastrophically. So even a Telangana-scale failure couldn't get Coempt banned from future government tenders.
10. The penalty structure shifted from punishing mistakes to punishing delays. The old rules fined the vendor for wrong scanning, merged pages, and unscanned books. The new rules dropped those and instead levied Rs 50,000 per day for delays. This incentivises rushed scanning over accurate scanning.
11. The old rules had a hard accuracy threshold, error rate not to exceed 0.5%. The new rules removed this number entirely.
12. The old rules specified proper book and robotics scanners. The new rules just say "sufficient scanners." The definition was vague enough that, as Sarthak notes, the scanning could be done with a phone on a stand.
13. On the security side, the contract required a VAPT (vulnerability and penetration test) certified by CERT-In before go-live, and a restricted beta phase before launch. The system clearly wasn't restricted, because the other researcher, Nisarga, was able to access it and find vulnerabilities four days before go-live. So the mandatory security audit appears to have been bypassed.
These are more than a dozen rule changes, all between the failed tender and the winning tender, all pushing in the same direction, all benefiting the one company with the worst track record in the field.
The security holes Nisarga found last week now have an explanation. The system was built by a vendor that was specifically allowed to skip the security certification, the source code ownership, the data sovereignty, and the quality thresholds the original rules demanded.
Following things need to happen immediately;
1. An immediate CAG audit of the tender process.
2. A parliamentary debate on the topic.
3. An independent investigation into
> Why the first tender vanished?
> Why the disqualification clauses were deleted?
> Why the turnover bar was held exactly where it was?
> Why the security level was dropped?
> Why the blacklisting power was removed at the last moment?
Sarthak, this is genuinely exceptional investigative work. Far better than most journalists with full resources ever manage. Take a bow. :)
Thinking of posting the blogs/papers I’m reading and the systems concepts I’m learning.
Don’t just agree with them — interrogate me on them.
Ask the kind of questions that expose shallow understanding:
edge cases, internals, tradeoffs, failure modes, scaling implications.
If your question makes me uncomfortable, it’s probably a good one.
First one :
Your App Is Slow Because You Don’t Know What a Primary Key Is [https://t.co/UoAq2mYWH0]
One of the biggest advantages of stateless systems is how effortlessly they scale horizontally. Any request can be handled by any server because no application instance owns user state locally. Your login request might hit Server A, the next profile API call might hit Server B, and everything still works seamlessly because the shared state lives in external systems like Redis, databases, or token-based auth.
This probably happened because the system assumed that when a Go context gets cancelled, all related data would automatically get cleaned up everywhere. But Redis Streams and background workers don’t understand Go contexts. So even though the request was cancelled at the gRPC layer, the metadata already pushed into Redis stayed there and kept accumulating over time.
One subtle MySQL/InnoDB mistake is tables being created without an explicit PRIMARY KEY. InnoDB silently generates a hidden clustered index internally when no primary key exists, so everything appears to work fine at first. But as the table grows, the problems start showing up: the hidden key is inaccessible to the application, physical row ordering becomes something you no longer control, and secondary indexes become more expensive internally because they reference the clustered index underneath. It’s one of those design decisions that feels harmless early on but becomes painful at scale. Every serious production table should have an explicitly designed primary key instead of relying on InnoDB’s hidden fallback behavior.
UUIDs are larger (16 bytes vs 4–8 bytes for integers), increase index size, reduce cache locality, and can cause index fragmentation because randomly generated UUIDs are not sequential. This can slow inserts and increase storage overhead at scale. That’s why many modern systems use time-ordered variants like UUIDv7, ULID, or Snowflake IDs
I was just reading about this yesterday.
The first step is capacity math. If one worker processes 1 message every 5 seconds, that’s 0.2 messages/sec. With 10 workers, throughput is only 2 messages/sec. Processing 1 million messages at that rate takes ~138 hours. To finish in under 1 hour, the system needs to process at least ~278 messages/sec. That means the real solution is horizontal scaling plus reducing per-message latency. You’d increase worker concurrency aggressively (for example 1500+ lightweight async workers or hundreds of multi-threaded consumers), batch operations wherever possible, parallelize I/O, avoid synchronous blocking calls, and optimize the slow 5-second processing path itself. In production systems, the bottleneck is usually external APIs, DB writes, or network waits , so techniques like bulk inserts, connection pooling, async event loops, partitioned queues, autoscaling consumers, and idempotent retry handling become critical. The key insight is that queues don’t magically make systems fast , throughput is purely a function of processing latency × concurrency.
This feels like a major shift in how agents will interact with the internet. Humans navigate visual interfaces; agents prefer structured protocols. In the future, websites will likely expose “agent-native” layers the same way we expose APIs today. Something like tools.txt or machine-readable capability manifests makes a lot of sense . A standard way for agents to discover available actions, auth methods, pricing, rate limits, and structured schemas without reverse engineering everything manually. The web is slowly evolving from human-first interfaces to programmable surfaces for autonomous systems.