I help 400,000+ engineers level up, one idea at a time.
🚀 Weekly insights on:
🔹 Software engineering
🔹 Tech leadership
🔹 Career growth
🧠 Read by staff/principal engineers, CTOs & builders at top companies.
👉 Join 46,000+ smart readers: https://t.co/vwmiYoFsBf
If you have the wrong data model, no algorithm can save you
Rob Pike wrote five rules on complexity in his Notes on Programming in C, back in 1989, and they are still valid today.
Most people know the first two (about measuring before improving), but the last three are also important.
Rules 3 and 4 are about keeping algorithms simple. He even lists the data structures that cover almost every practical program. There are four of them: array, linked list, hash table and binary tree.
If I had to select just one, it would be Rule 5. In 1989, a C programmer wrote most of these structures manually, and today you use a library for that.
So, the main question is the shape of the data, meaning the tables, the messages between services and fields in your API.
You can replace an algorithm inside one function. Many services and reports read the same data model, so changing it means a migration.
So, start with the data, and write down what the system stores, who owns each piece and how it changes. The code is then an easier part.
How Google managed to crash many iOS apps around the world
On September 28 we noticed in our React Native mobile app that crashes are occurring from the Firebase Analytics. We couldn't do anything about it.
When I took a look at the issue opened in the Firebase project, I saw that many others have the same problem.
Now, Google provided a postmortem. What they did was that they removed an old feature flag from the backend system for Firebase Analytics. The existing tests passed, but the SDK inside the apps still read that flag. When they rolled out that change globally, each device that fetched it crashed at launch time.
The bad config was served for 𝟮 𝗵𝗼𝘂𝗿𝘀 𝗮𝗻𝗱 𝟭𝟭 𝗺𝗶𝗻𝘂𝘁𝗲𝘀. A developer opened a GitHub issue 18 minutes after the rollout began, and by 18:05 PDT, it already pointed at the config response and asked for a rollback. Google traced the cause at 19:16.
Facebook had a similar issue in May 2020, when a server-side config change made its iOS SDK crash Spotify, TikTok, Pinterest and Venmo. The lesson has been public for more than six years.
Google listed four fixes, and each one you can check in your systems:
🔹 𝗩𝗮𝗹𝗶𝗱𝗮𝘁𝗲 𝗼𝗻 𝘁𝗵𝗲 𝗰𝗹𝗶𝗲𝗻𝘁. The SDK never checked that a flag had a name. Bad server data should not crash the app, so skip the broken flag and keep the cached default.
🔹 𝗧𝗲𝘀𝘁 𝘄𝗵𝗮𝘁 𝗮 𝗱𝗲𝗹𝗲𝘁𝗲 𝗹𝗲𝗮𝘃𝗲𝘀 𝗯𝗲𝗵𝗶𝗻𝗱. The existing tests passed on this change. Block the release when code still reads a flag you removed.
🔹 𝗥𝗼𝗹𝗹 𝗼𝘂𝘁 𝗰𝗼𝗻𝗳𝗶𝗴 𝗶𝗻 𝘀𝘁𝗮𝗴𝗲𝘀. These flags also act as kill switches that turn off a buggy feature remotely. That mechanism is released globally, so the bad config went out to apps everywhere. Treat a config change like a deploy and start with a small group of users (phased rollout).
🔹 𝗣𝘂𝘁 𝗰𝗹𝗶𝗲𝗻𝘁 𝗵𝗲𝗮𝗹𝘁𝗵 𝗼𝗻 𝘁𝗵𝗲 𝘀𝘁𝗮𝘁𝘂𝘀 𝗽𝗮𝗴𝗲. Google's status pages track mostly server health, and it stayed green while apps crashed.
Today is my birthday, so I want to introduce myself to you shortly:
🤓 I am a Serbian guy called 𝗠𝗶𝗹𝗮𝗻 (yes, like that Italian club)
🚀 I'm a 𝗖𝗧𝗢 at a US startup
👨 I'm 𝗺𝗮𝗿𝗿𝗶𝗲𝗱 with children
🏠 I live in 𝗟𝗮𝘇𝗮𝗿𝗲𝘃𝗮𝗰, an hour from Belgrade
🪖 I was two times 𝗵𝗶𝘁 𝗯𝘆 𝘄𝗮𝗿 as a kid and lost everything
🕹️ I wrote my 𝗳𝗶𝗿𝘀𝘁 𝗽𝗿𝗼𝗴𝗿𝗮𝗺𝘀 in the 1980s on a ZX Spectrum
⚙️ I started 𝗰𝗼𝗱𝗶𝗻𝗴 𝗽𝗿𝗼𝗳𝗲𝘀𝘀𝗶𝗼𝗻𝗮𝗹𝗹𝘆 in 2002. and did this in 9+ programming languages (now coding with AI :)
🧓 𝗠𝘆 𝗱𝗮𝗱 was a software developer for more than 40 years.
📄 I've been a 𝗦𝗼𝗳𝘁𝘄𝗮𝗿𝗲 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁 in Enterprise companies and Startups
📝 I worked for 𝘀𝗶𝘅 𝘆𝗲𝗮𝗿𝘀 𝗶𝗻 𝗿𝗲𝘀𝗲𝗮𝗿𝗰𝗵, publishing 20+ papers and obtaining a Ph.D. in Computer Sciences
🏅 I've been a 𝗠𝗶𝗰𝗿𝗼𝘀𝗼𝗳𝘁 𝗠𝗩𝗣 since 2023
🍀 I'm also 𝗮 𝗰𝗲𝗿𝘁𝗶𝗳𝗶𝗲𝗱 𝗰𝗮𝗿𝗲𝗲𝗿 𝗰𝗼𝗮𝗰𝗵 (ICF) and love working with people, seeing them happy and successful
🏢 I like 𝘀𝗺𝗮𝗹𝗹𝗲𝗿 𝗼𝗿𝗴𝗮𝗻𝗶𝘇𝗮𝘁𝗶𝗼𝗻𝘀, because the best ideas win, not politics
♟️ I got my first 𝗛𝗲𝗮𝗱 𝗢𝗳 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴 role in 2014, leading 40+ people.
🌻 I've worked as a 𝗰𝗼𝗻𝘀𝘂𝗹𝘁𝗮𝗻𝘁 in different companies since 2013.
📈 I try to 𝗺𝗮𝗸𝗲 𝘁𝗵𝗶𝗻𝗴𝘀 𝗲𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝘁, 𝗮𝗻𝗱 𝗼𝗿𝗴𝗮𝗻𝗶𝘇𝗲𝗱 and add value quickly
💖 I love 𝗯𝗼𝗼𝗸𝘀, 𝘁𝗲𝗰𝗵𝗻𝗼𝗹𝗼𝗴𝘆, 𝘀𝗰𝗶𝗲𝗻𝗰𝗲, 𝗽𝗵𝗼𝘁𝗼𝗴𝗿𝗮𝗽𝗵𝘆, 𝗦𝘁𝗮𝗿 𝗪𝗮𝗿𝘀, 𝗰𝗼𝗳𝗳𝗲𝗲, 𝘄𝗶𝗻𝗲𝘀, 𝗺𝗶𝗻𝗱𝗳𝘂𝗹𝗻𝗲𝘀𝘀, and 𝗽𝗲𝗿𝘀𝗼𝗻𝗮𝗹 𝗶𝗺𝗽𝗿𝗼𝘃𝗲𝗺𝗲𝗻𝘁 topics
📚 I'm 𝗮𝗻 𝗮𝘃𝗶𝗱 𝗿𝗲𝗮𝗱𝗲𝗿, having a library of 5000 books and reading them every day, and this year I published my first book "Laws of Software Engineering"
✍️ Every week I write the 𝗧𝗲𝗰𝗵 𝗪𝗼𝗿𝗹𝗱 𝗪𝗶𝘁𝗵 𝗠𝗶𝗹𝗮𝗻 newsletter for 50,000+ engineers and tech leaders
💡 I like to write about 𝗦𝗼𝗳𝘁𝘄𝗮𝗿𝗲 𝗗𝗲𝘃𝗲𝗹𝗼𝗽𝗺𝗲𝗻𝘁, 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲, .𝗡𝗘𝗧, 𝗗𝗲𝘃𝗢𝗽𝘀, 𝗢𝗿𝗴𝗮𝗻𝗶𝘇𝗮𝘁𝗶𝗼𝗻𝘀, 𝗣𝗲𝗼𝗽𝗹𝗲, 𝗖𝗮𝗿𝗲𝗲𝗿, 𝗣𝗲𝗿𝘀𝗼𝗻𝗮𝗹 𝗗𝗲𝘃𝗲𝗹𝗼𝗽𝗺𝗲𝗻𝘁, 𝗣𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝘃𝗶𝘁𝘆, and more.
🛠️ I maintain the 𝗗𝗲𝘃𝗢𝗽𝘀 𝗥𝗼𝗮𝗱𝗺𝗮𝗽 (20K+ stars on GitHub), the .NET Developer Roadmap and the Azure cheat sheet
You can find 𝗺𝘆 𝘄𝗿𝗶𝘁𝗶𝗻𝗴:
🔹 on my 𝗡𝗲𝘄𝘀𝗹𝗲𝘁𝘁𝗲𝗿: https://t.co/xVh2nM0nj6 (50k+ subscribers)
🔹 in my 𝗖𝗼𝗺𝗺𝘂𝗻𝗶𝘁𝘆: https://t.co/PDbxHK3dBp (recieve all of my posts in advance to your e-mail + buy my products)
🔹 on 𝗧𝘄𝗶𝘁𝘁𝗲𝗿: https://t.co/9lXEKxcoy4 (65k+ followers)
🔹 on 𝗦𝘂𝗯𝘀𝘁𝗮𝗰𝗸: https://t.co/owHTuadHxb (85k+ followers)
🔹 on 𝗠𝗲𝗱𝗶𝘂𝗺: https://t.co/YxMcJdneuE (20k+ followers)
🔹 on my 𝗕𝗹𝗼𝗴: https://t.co/8pH0qLWapN
This birthday is a special one. Serbia won its freedom, and the students defeated the past 🇷🇸
Also, I would like to THANK YOU all for following and supporting me!
If any of these resonates with you, feel free to DM me.
See you around!
𝗪𝗵𝗮𝘁 𝗶𝘀 𝗮𝗻 𝗔𝗣𝗜 𝗚𝗮𝘁𝗲𝘄𝗮𝘆?
An API management tool known as an 𝗔𝗣𝗜 𝗴𝗮𝘁𝗲𝘄𝗮𝘆 sits between a client and a group of backend services. It performs the function of a reverse proxy by accepting all application programming interface (API) calls, aggregating the different services needed to fulfill them, and returning the right outcome.
Most enterprise APIs are deployed behind API gateways. User 𝗮𝘂𝘁𝗵𝗲𝗻𝘁𝗶𝗰𝗮𝘁𝗶𝗼𝗻, 𝗿𝗮𝘁𝗲 𝗹𝗶𝗺𝗶𝘁𝘀, 𝗮𝗻𝗱 𝘀𝘁𝗮𝘁𝗶𝘀𝘁𝗶𝗰𝘀 are typical duties that API gateways take care of on behalf of a system of API services.
An API service receives a remote request and responds to it. But in reality, nothing is ever that easy. When you host large-scale APIs, take into account 𝘃𝗮𝗿𝗶𝗼𝘂𝘀 𝗰𝗼𝗻𝗰𝗲𝗿𝗻𝘀:
✅ You use a 𝗿𝗮𝘁𝗲 𝗹𝗶𝗺𝗶𝘁𝗮𝘁𝗶𝗼𝗻 𝘀𝘆𝘀𝘁𝗲𝗺 𝗮𝗻𝗱 𝗮𝗻 𝗮𝘂𝘁𝗵𝗲𝗻𝘁𝗶𝗰𝗮𝘁𝗶𝗼𝗻 𝘀𝗲𝗿𝘃𝗶𝗰𝗲 to safeguard your APIs against abuse and excessive use.
✅ You implement 𝗮𝗻𝗮𝗹𝘆𝘁𝗶𝗰𝘀 𝗮𝗻𝗱 𝗺𝗼𝗻𝗶𝘁𝗼𝗿𝗶𝗻𝗴 𝘁𝗼𝗼𝗹𝘀 because you want to know how people use your APIs.
✅ You should link to a 𝗽𝗮𝘆𝗺𝗲𝗻𝘁 𝘀𝘆𝘀𝘁𝗲𝗺 if your APIs are monetized.
✅ If you've chosen a 𝗺𝗶𝗰𝗿𝗼𝘀𝗲𝗿𝘃𝗶𝗰𝗲𝘀 𝗮𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲, a single request can call many different services.
✅ Your clients will still want to be able to access all your services in 𝗼𝗻𝗲 𝗹𝗼𝗰𝗮𝘁𝗶𝗼𝗻 even when you add new API services over time and retire others.
So, 𝗵𝗼𝘄 𝗱𝗼𝗲𝘀 𝗶𝘁 𝘄𝗼𝗿𝗸 in general:
1. API gateway receives an HTTP request from a client.
2. When received, it validates the request first.
3. API gateway checks with an identity provider about authentication/authorization.
4. The rate-limiting rules are then applied to the request.
5. The API gateway finds the backend services and routes the request.
Also, the API gateway can handle 𝗳𝗮𝘂𝗹𝘁𝘀 (𝗰𝗶𝗿𝗰𝘂𝗶𝘁 𝗯𝗿𝗲𝗮𝗸𝗲𝗿), 𝗯𝘂𝘁 𝗮𝗹𝘀𝗼 𝗱𝗼 𝗹𝗼𝗴𝗴𝗶𝗻𝗴, 𝗰𝗮𝗰𝗵𝗶𝗻𝗴 𝗮𝗻𝗱 𝗺𝗼𝗻𝗶𝘁𝗼𝗿𝗶𝗻𝗴.
𝗘𝘅𝗮𝗺𝗽𝗹𝗲𝘀 𝗼𝗳 𝗔𝗣𝗜 𝗚𝗮𝘁𝗲𝘄𝗮𝘆𝘀 are Apigee (now part of Google Cloud) and Tyk API Gateway. The primary public cloud providers offer API gateway tools specific to their platforms: Amazon API Gateway, Azure API Management, and Google Cloud API Gateway.
Communication 101
Whenever you write or talk about something or someone, always ask yourself do your audience have enough context or knowledge about it as you have
If not, you have that to explain first
SQL injection has a known fix and more than 14,000 CVEs
That number is from the OWASP Top 10 for 2025. Cross-site scripting has more than 30,000. The first public write-up of the technique is from 1998.
We know how to stop these attacks. They still work because the fix has to be everywhere, and one missed place is enough.
Here is how six common web attacks work:
𝟭. 𝗦𝗤𝗟 𝗶𝗻𝗷𝗲𝗰𝘁𝗶𝗼𝗻
The app builds a query by putting user input into a string. The attacker sends 42' OR '1'='1 as the id. The condition is now always true, and the database returns every row. The fix is to use parameterized queries.
𝟮. 𝗫𝗦𝗦
The attacker posts a comment to your website with a script in it. The page shows the comment unescaped, so the script runs in the browser of every visitor, with the rights of the site. Escape output by default, and sanitize any HTML that users can submit.
𝟯. 𝗖𝗦𝗥𝗙
You are logged in to your bank and you open another page. That page submits a hidden form to the bank, and your browser adds your cookie to the request. To solve it, just put a CSRF token on every request that changes state. SameSite cookies help, but OWASP says they do not replace a proper CSRF defense in most deployments.
𝟰. 𝗦𝗦𝗥𝗙
Your server fetches a URL that a user gave it. The attacker gives it an internal address, such as the cloud metadata endpoint at 169.254.169.254. The server fetches it and can send cloud credentials back. Keep an allow-list of the destinations your server may call.
𝟱. 𝗖𝗿𝗲𝗱𝗲𝗻𝘁𝗶𝗮𝗹 𝘀𝘁𝘂𝗳𝗳𝗶𝗻𝗴
Bots try usernames and passwords that leaked from another site. The passwords work because people reuse them. The way to solve it is to check for leaked passwords (use Have I Been Pwned) and use multi-factor authentication.
𝟲. 𝗦𝘂𝗽𝗽𝗹𝘆 𝗰𝗵𝗮𝗶𝗻 𝗮𝘁𝘁𝗮𝗰𝗸
The attacker creates a package with malicious code, and you install it. In 2025, the Shai-Hulud worm reached more than 500 npm package versions. Solve it by installing from a lockfile, and wait before you take a brand-new version.
Eventual consistency bugs start before "eventually"
When you read "Eventually consistent", it usually sounds like it will happen with a small delay. And this means that when writes stop, replicas will catch up with the data.
But the question is what happens before this?
Here is what happens:
𝟭. 𝗥𝗲𝗽𝗹𝗶𝗰𝗮𝘁𝗶𝗼𝗻 𝗹𝗮𝗴
The primary confirms a write before every replica has it. A read from the slow replica returns the old value.
𝟮. 𝗥𝗲𝗮𝗱 𝘆𝗼𝘂𝗿 𝘄𝗿𝗶𝘁𝗲𝘀
After a write, the read carries that version. A replica that is behind waits until it catches up.
𝟯. 𝗖𝗼𝗻𝗳𝗹𝗶𝗰𝘁 𝗿𝗲𝘀𝗼𝗹𝘂𝘁𝗶𝗼𝗻
When more than one node takes writes, two of them can accept different writes before they sync. Last write wins keeps one and 𝗱𝗿𝗼𝗽𝘀 𝘁𝗵𝗲 𝗼𝘁𝗵𝗲𝗿 𝘄𝗶𝘁𝗵𝗼𝘂𝘁 𝗮𝗻 𝗲𝗿𝗿𝗼𝗿. A merge keeps both.
𝟰. 𝗤𝘂𝗼𝗿𝘂𝗺
With 3 replicas, we wait for 2 on every write and ask 2 on every read. Because 2 + 2 > 3, every read reaches at least one replica that has the confirmed write.
𝟱. 𝗛𝗶𝗻𝘁𝗲𝗱 𝗵𝗮𝗻𝗱𝗼𝗳𝗳
When a replica is down, another node keeps its write and delivers it when the replica is back.
𝟲. 𝗥𝗲𝗮𝗱 𝗿𝗲𝗽𝗮𝗶𝗿
A read notices that one replica is stale and writes the newest value back to it.
Before we add a read replica, we should go feature by feature and decide if that read is allowed to be old. A product list can be a little behind. The profile a user just saved cannot.
Write that decision down for every screen before the first replica goes live.
Don't use Claude Code the way you did in 2025
Claude Code is released on a daily basis and we need to adjust to what it can do today.
Here are a few things from the last year that need to be adjusted.
𝟭. 𝗧𝗵𝗶𝗻𝗸𝗶𝗻𝗴 𝗸𝗲𝘆𝘄𝗼𝗿𝗱𝘀. Set the reasoning level with /effort. ultrathink is only a one-turn hint now, and "think hard" is plain text.
𝟮. 𝗦𝗸𝗶𝗽𝗽𝗶𝗻𝗴 𝗽𝗲𝗿𝗺𝗶𝘀𝘀𝗶𝗼𝗻𝘀. Since v2.1.283, interactive sessions start in auto mode. Keep --dangerously-skip-permissions for isolated containers and VMs. Also, add deny rules for secrets and production, because they block in every mode.
𝟯. 𝗖𝘂𝘀𝘁𝗼𝗺 𝗰𝗼𝗺𝗺𝗮𝗻𝗱𝘀. They merged into skills, which are popular now. Put new skills in .claude/skills/. Old files in .claude/commands/ still load.
𝟰. 𝗢𝗻𝗲 𝗯𝗶𝗴 𝗿𝘂𝗹𝗲𝘀 𝗳𝗶𝗹𝗲. The docs say to keep each CLAUDE.md under 200 lines, and /doctor proposes the cuts. Anthropic removed over 80% of Claude Code's own system prompt for Opus 5 and Fable 5.
𝟱. 𝗥𝘂𝗹𝗲𝘀 𝗶𝗻 𝗰𝗮𝗽𝗶𝘁𝗮𝗹 𝗹𝗲𝘁𝘁𝗲𝗿𝘀. "ALWAYS run lint" in CLAUDE.md only asks, yet a hook runs it every time its event fires.
𝟲. 𝗣𝗹𝗮𝗻 𝗺𝗼𝗱𝗲 𝗳𝗼𝗿 𝗲𝘃𝗲𝗿𝘆𝘁𝗵𝗶𝗻𝗴. Plan when the approach is more complex and cannot be done in a simple step, or the code is new to you.
If your CLAUDE.md and settings are a year old, read them again with this list open.
A few important commands to keep in mind:
🔹 /goal <condition> keeps Claude working until the condition is met.
🔹 /verify runs your app and checks the real result.
🔹 /code-review finds bugs in the diff, and /simplify cleans it up.
🔹 ultracode: <task> runs the task as a multi-agent workflow.
Back to you, what are your favorite Claude Code commands?
𝗪𝗵𝘆 𝘆𝗼𝘂 𝘀𝗵𝗼𝘂𝗹𝗱 𝗯𝗲 𝗮 𝘀𝗲𝗿𝘃𝗮𝗻𝘁 𝗹𝗲𝗮𝗱𝗲𝗿
Servant leadership is a leadership style that emphasizes the leader's role as a servant to those they lead.
Instead of the traditional top-down hierarchical style, the servant leader works to serve the needs, aspirations, and growth of their team members.
The basic idea of servant leadership is that the higher we move up in the organization, the more people we are there to serve, not the more people who are there to serve us.'
In his book, "𝗧𝘂𝗿𝗻 𝘁𝗵𝗲 𝗦𝗵𝗶𝗽 𝗔𝗿𝗼𝘂𝗻𝗱!" written by L. David Marquet, a former U.S. Navy captain, describes how, when he took the submarine, it had the lowest morale and performance metrics, and he managed to make it one of the most successful submarines in the Navy's fleet.
Here are the main learning points from the book:
𝟭. 𝗟𝗲𝗮𝗱𝗲𝗿-𝗟𝗲𝗮𝗱𝗲𝗿 𝗠𝗼𝗱𝗲𝗹. This approach aligns with servant leadership, shifting from a command-and-control approach to empowering individuals to take on leadership roles.
𝟮. 𝗘𝗺𝗽𝗼𝘄𝗲𝗿𝗺𝗲𝗻𝘁. Marquet's most significant change was empowering crew members to make decisions without always seeking top-level approval. This aligns with the concept of servant leadership, which emphasizes empowering individuals to take initiative and assume responsibility.
𝟯. 𝗖𝗼𝗺𝗽𝗲𝘁𝗲𝗻𝗰𝗲 𝗮𝗻𝗱 𝗖𝗹𝗮𝗿𝗶𝘁𝘆. The author emphasized that competence and clarity are essential for empowerment to be effective. This aligns with the servant leader's commitment to nurturing individual growth, ensuring they possess the necessary skills and understanding to effectively perform their roles.
𝟰. 𝗔𝗰𝘁𝗶𝘃𝗲 𝗟𝗶𝘀𝘁𝗲𝗻𝗶𝗻𝗴. Like servant leaders, Marquet understood the importance of listening. By valuing the perspectives and feedback of the crew, they could make more informed decisions and build a culture of trust.
𝟱. 𝗗𝗲𝗰𝗶𝘀𝗶𝗼𝗻 𝗠𝗮𝗸𝗶𝗻𝗴. Instead of giving orders, Marquet would ask his subordinates how they intended to handle situations, pushing the authority for decisions down the ranks. This decentralization is a core component of servant leadership, where the goal is to create more leaders, not followers.
𝟲. 𝗕𝘂𝗶𝗹𝗱𝗶𝗻𝗴 𝗧𝗿𝘂𝘀𝘁 𝗮𝗻𝗱 𝗖𝗼𝗺𝗺𝘂𝗻𝗶𝘁𝘆. The author established an environment of trust and community by relinquishing control and empowering his crew.
𝟳. 𝗖𝗼𝗺𝗺𝗶𝘁𝗺𝗲𝗻𝘁 𝘁𝗼 𝗚𝗿𝗼𝘄𝘁𝗵. Marquet was committed to the growth and development of his crew, both professionally and personally, much like the emphasis servant leadership places on the growth of individuals.
𝟴. 𝗟𝗼𝗻𝗴-𝗧𝗲𝗿𝗺 𝗩𝗶𝘀𝗶𝗼𝗻. While immediate results were essential, Marquet was more focused on creating a sustainable leadership model that would benefit the ship and its crew in the long run.
We ship 3x more PRs and review them the same way as before
Linear: teams with coding agents went from 21 to 65 PRs a week, and total dev time went up.
Trends #12 is out: TypeScript 7.0, GitHub's 8h outage, METR, DORA, Cursor and more.
https://t.co/3b2rFevBy8
GitHub was degraded for 7h 47m and your repo was still on your laptop
On 17 August, from 13:28 to 21:15 UTC, GitHub was degraded. GitHub features had errors, including PRs, Actions, Copilot and others. At the peak, around 20% of web and API requests failed.
Yet, Git on your machine still worked normally. A regular clone holds the 𝗰𝗼𝗺𝗽𝗹𝗲𝘁𝗲 𝗵𝗶𝘀𝘁𝗼𝗿𝘆 of the repo, so commit, branch, diff and log worked as expected. But GitHub's copy and its features returned errors on some requests.
We usually say "push it to Git" when we mean GitHub because we treat the two as one tool.
𝗚𝗶𝘁 keeps the history, in full, on every machine (having the cloned repo), and this was one of the important features of distributed version control systems. While 𝗚𝗶𝘁𝗛𝘂𝗯 hosts 𝗼𝗻𝗲 𝗺𝗼𝗿𝗲 𝗰𝗼𝗽𝘆 and adds the team features around it: issues, code review, Actions and releases. Basically, it's a social network around Git.
Git had pull requests long before GitHub even existed. git request-pull shipped with Git 1.0 in December 2005, and it prints a summary of your branch for the maintainer, usually sent by email.
GitHub went live in April 2008 and later put review, comments and a merge button on top. The default button makes a merge commit on GitHub's servers, the same as git merge --no-ff.
What is important to note is that every commit is contained in any Git repo clone, but issues, PR discussions and Actions history are stored on GitHub, and we depend on its stability.
𝗪𝗲 𝗮𝗿𝗲 𝘀𝗼𝗳𝘁𝘄𝗮𝗿𝗲 𝗲𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝘀, 𝗻𝗼𝘁 𝗱𝗲𝘃𝗲𝗹𝗼𝗽𝗲𝗿𝘀
What does this mean? This means that 𝘆𝗼𝘂𝗿 𝗷𝗼𝗯 𝗶𝘀 𝗻𝗼𝘁 𝘁𝗼 𝗰𝗼𝗱𝗲 𝗯𝘂𝘁 𝘁𝗼 𝘀𝗼𝗹𝘃𝗲 𝗽𝗿𝗼𝗯𝗹𝗲𝗺𝘀.
This includes:
🔹 Talking to customers
🔹 Do system design
🔹 Writing docs
🔹 Finding ways to resolve different kinds of problems in the most efficient manner
🔹 Analyzing existing and future solutions
🔹 Talking with your peers
🔹 Making plans and executing it
🔹 Learning new stuff
🔹 Do different types of testing
🔹 Troubleshoot existing system
🔹 Using common sense
🔹 And more...
When you get the problem, sit first, try to understand, ask, clarify, then make a plan and execute. Do not jump into coding immediately.
Ultimately, 𝗶𝗳 𝘆𝗼𝘂 𝗻𝗲𝗲𝗱 𝘁𝗼 𝗰𝗼𝗱𝗲 𝘁𝗼 𝗱𝗼 𝘀𝗼𝗺𝗲𝘁𝗵𝗶𝗻𝗴, 𝗱𝗼 𝗶𝘁.
But if you can resolve something without a code, even better.
And remember that 𝗰𝗼𝗱𝗲 𝗶𝘀 𝗮 𝗹𝗶𝗮𝗯𝗶𝗹𝗶𝘁𝘆, 𝗻𝗼𝘁 𝗮𝗻 𝗮𝘀𝘀𝗲𝘁! Every line of code you write needs to be executed somewhere; we must pay for this in runtime, etc.
We don't want to be a 𝗳𝗲𝗮𝘁𝘂𝗿𝗲 𝗳𝗮𝗰𝘁𝗼𝗿𝘆 but a team that produces high-quality software and efficiently solves our clients' problems.
Do you agree?
Azure writes your file 3 times before your app sees 201 Created
How we usually work with Azure Blob is to call blob.UploadAsync(file), get a 201 and move on. But under this command, a few servers in your region run a sequence, and this order enables what the 201 promises.
So, what happens:
𝟭. 𝗗𝗡𝗦 𝗳𝗶𝗻𝗱𝘀 𝘆𝗼𝘂𝗿 𝘀𝘁𝗼𝗿𝗮𝗴𝗲 𝘀𝘁𝗮𝗺𝗽
Your account name resolves to the virtual IP of one storage stamp, a cluster in your region.
𝟮. 𝗔 𝗙𝗿𝗼𝗻𝘁 𝗘𝗻𝗱 𝗰𝗵𝗲𝗰𝗸𝘀 𝘆𝗼𝘂𝗿 𝘁𝗼𝗸𝗲𝗻
The SDK sends PUT Blob over HTTPS. A stateless Front End server authenticates it: Entra ID with RBAC, a SAS, or a shared key. Least privilege for a write is the 𝗦𝘁𝗼𝗿𝗮𝗴𝗲 𝗕𝗹𝗼𝗯 𝗗𝗮𝘁𝗮 𝗖𝗼𝗻𝘁𝗿𝗶𝗯𝘂𝘁𝗼𝗿 role.
𝟯. 𝗜𝘁 𝗳𝗶𝗻𝗱𝘀 𝘆𝗼𝘂𝗿 𝗽𝗮𝗿𝘁𝗶𝘁𝗶𝗼𝗻
The partition key is account + container + blob name. The Front End finds it in a cached Partition Map and forwards the write to the Partition Server that owns it. Sequential names can pile onto one partition, and that 𝗵𝗼𝘁 𝗽𝗮𝗿𝘁𝗶𝘁𝗶𝗼𝗻 is one place 503 Server Busy comes from.
𝟰. 𝗬𝗼𝘂𝗿 𝗯𝘆𝘁𝗲𝘀 𝗴𝗼 𝗶𝗻𝘁𝗼 𝘁𝗵𝗲 𝗰𝗼𝗺𝗺𝗶𝘁 𝗹𝗼𝗴
The Partition Server appends the blob bytes to its commit log stream. The index row stores only a pointer: extent, offset and length.
𝟱. 𝗧𝗵𝗿𝗲𝗲 𝗰𝗼𝗽𝗶𝗲𝘀 𝗯𝗲𝗳𝗼𝗿𝗲 𝗮𝗻𝘆𝗼𝗻𝗲 𝘀𝗮𝘆𝘀 𝘆𝗲𝘀
The primary extent node writes the append and sends it to two secondaries in separate fault domains. It reports success 𝗼𝗻𝗹𝘆 𝗮𝗳𝘁𝗲𝗿 𝗮𝗹𝗹 𝟯 𝗿𝗲𝗽𝗹𝗶𝗰𝗮𝘀 𝗮𝗿𝗲 𝘄𝗿𝗶𝘁𝘁𝗲𝗻. LRS works this way, and ZRS does the same across three availability zones.
𝟲. 𝗜𝗻𝗱𝗲𝘅 𝘂𝗽𝗱𝗮𝘁𝗲, 𝘁𝗵𝗲𝗻 𝟮𝟬𝟭 𝗖𝗿𝗲𝗮𝘁𝗲𝗱
The Partition Server points the index row at the new bytes and reports committed. Your app gets 201 Created with an ETag, a service-computed MD5 and CRC64, and x-ms-request-server-encrypted: true. Encryption with 𝗔𝗘𝗦-𝟮𝟱𝟲 𝗮𝘁 𝗿𝗲𝘀𝘁 is on for every account and cannot be turned off. If your own MD5 or CRC64 did not match what arrived, you get 400 instead.
Big file? Put Blob takes up to 5,000 MiB in one request. Past the SDK's InitialTransferSize, it sends Put Block calls in parallel, up to 50,000 blocks of 4,000 MiB each, and Put Block List commits them in order.
𝟳. 𝗘𝘃𝗲𝗿𝘆𝘁𝗵𝗶𝗻𝗴 𝘀𝗹𝗼𝘄𝗲𝗿 𝗵𝗮𝗽𝗽𝗲𝗻𝘀 𝗮𝗳𝘁𝗲𝗿 𝘁𝗵𝗲 𝟮𝟬𝟭
Geo copy to a second region (GRS or GZRS only) is asynchronous, so a regional failure can lose writes that had not copied yet. Event Grid fires BlobCreated only once the blob is fully committed.
Back to you, which redundancy option do you pick for a new storage account, and why?
Never check a role name in your code
In one of the projects where I worked, we had role checks in many places across the project. Then, we needed to introduce a new role and that was a cumbersome code change in many places.
RBAC stands for Role-Based Access Control. The path we selected was to give users roles and let the code ask "is this user a Manager?"
Yet, the model has one more part. Users get roles, roles get permissions and permissions define what a user can do.
The code should only ask about the permission, for example "can this user approve an invoice?"
Here is the complete flow:
𝟭. 𝗧𝗵𝗲 𝘁𝗼𝗸𝗲𝗻 𝗰𝗮𝗿𝗿𝗶𝗲𝘀 𝗿𝗼𝗹𝗲𝘀. We set up the identity provider to add them at sign-in.
𝟮. 𝗧𝗵𝗲 𝗔𝗣𝗜 𝘁𝘂𝗿𝗻𝘀 𝗿𝗼𝗹𝗲𝘀 𝗶𝗻𝘁𝗼 𝗽𝗲𝗿𝗺𝗶𝘀𝘀𝗶𝗼𝗻𝘀. It reads the mapping from its database and keeps a short cache.
𝟯. 𝗧𝗵𝗲 𝗲𝗻𝗱𝗽𝗼𝗶𝗻𝘁 𝗰𝗵𝗲𝗰𝗸𝘀 𝗼𝗻𝗲 𝗽𝗲𝗿𝗺𝗶𝘀𝘀𝗶𝗼𝗻. It asks for invoices:approve and never names a role.
𝟰. 𝗔𝗻 𝗮𝗱𝗺𝗶𝗻 𝗮𝗱𝗱𝘀 𝗮 𝗻𝗲𝘄 𝗿𝗼𝗹𝗲 𝗮𝘀 𝗱𝗮𝘁𝗮. The admin creates the Auditor role in the identity provider. One table row maps it to invoices:read. The code stays the same, and nothing gets deployed.
In https://t.co/PLYLs2PAEE Core, the endpoint gets [Authorize(Policy = "invoices:approve")]. It replaces [Authorize(Roles = "Manager")]. Behind it, a policy provider turns that name into a requirement. A handler checks it against the user's permissions.
Our Person of the Week is a CTO, author, and coach with over 20 years of experience building software and leading teams. Drumroll, please, for... Milan Milanović!
Thank you for everything you do for the community, dear @milan_milanovic! 🧡
#smashingcommunity
Before you shard your database, check if it needs an index first
When your database becomes slow, we usually discuss different strategies, such as caching, read replicas or sharding.
This order is important, because each step down is costly and is 𝗵𝗮𝗿𝗱𝗲𝗿 𝘁𝗼 𝘂𝗻𝗱𝗼. So, we should start at the top and move down when the previous one is not enough.
Here are the 3 steps, with cheapest first:
𝟭. 𝗦𝘁𝗮𝘆 𝗼𝗻 𝗼𝗻𝗲 𝘀𝗲𝗿𝘃𝗲𝗿
First, read the plan with EXPLAIN ANALYZE, then index what you filter, join and sort on. Buy a bigger machine only after that.
All of it is easy to undo, but note that 𝗲𝘃𝗲𝗿𝘆 𝗶𝗻𝗱𝗲𝘅 𝘀𝗹𝗼𝘄𝘀 𝘄𝗿𝗶𝘁𝗲𝘀, and scaling up is expensive.
𝟮. 𝗧𝗮𝗸𝗲 𝗿𝗲𝗮𝗱𝘀 𝗼𝗳𝗳 𝘁𝗵𝗲 𝗽𝗿𝗶𝗺𝗮𝗿𝘆
Cache what you read most, add read replicas, or precompute heavy joins in a materialized view. You pay with 𝘀𝘁𝗮𝗹𝗲 𝗿𝗲𝗮𝗱𝘀, because replicas lag and a cache can hold old values.
So decide who invalidates each key, and read a user's own writes from the primary.
𝟯. 𝗦𝗽𝗹𝗶𝘁 𝘁𝗵𝗲 𝗱𝗮𝘁𝗮
When one table or one machine can no longer handle the writes or the data size, partition big tables, split data by domain, and shard last. This is the 𝗲𝘅𝗽𝗲𝗻𝘀𝗶𝘃𝗲 𝘀𝘁𝗲𝗽, and the hardest to undo.
Joins across domains move into your code, and resharding later is a lot of work.
Back to you, which step fixed your last slow database?
The software industry is in the fear era:
1. Developers fear that AI will replace them
2. Owners fear that someone will replicate their software quickly
Probably none of this will be true in the short to mid term term
For long term who knows
A clear plan can still solve the wrong problem
I once saw a team that spent a large amount of time optimizing a ten-minute build, but the releases were waiting two days for approvals. They were still late, even with that optimization.
Here is the simple problem-solving loop I use.
𝟭. 𝗗𝗲𝗳𝗶𝗻𝗲 𝘁𝗵𝗲 𝗽𝗿𝗼𝗯𝗹𝗲𝗺
Everything starts from here. Check any evidence you have, then identify what's missing.
In the end, state what you need to solve.
𝟮. 𝗦𝗲𝘁 𝘁𝗵𝗲 𝗴𝗼𝗮𝗹
Define what you want to achieve. If you don't know the goal, don't continue to the next step.
Define what success looks like, then choose a simple approach.
𝟯. 𝗔𝗰𝘁
Take the next useful step, then use the result to test your assumptions.
𝟰. 𝗟𝗲𝗮𝗿𝗻 𝗮𝗻𝗱 𝗮𝗱𝗷𝘂𝘀𝘁
Check progress against the goal. Ask for feedback, and revise your approach based on the evidence.
This order matters because we need to define success before we have the solution. Otherwise, we get a goal that is aligned with our preferred approach.
New evidence can change your understanding of the problem itself. Revisit any step when it changes what you should do next.
For the team in our example, investigating the approval delay could change the entire plan.