Simran

Learner for life. Staff Engg. Curious. Coder. Reader. I write on tech.

[Simran @simranzchawla]

How do you stop your company's AI assistant from answering with documents the person asking isn't allowed to see?

Say you've put every wiki page, HR policy and board deck into one vector index.

An engineer types: "Are there layoffs coming?"

The closest matches in the whole index are the restructuring plan and the severance sheet in the HR folder.

"Don't reveal HR data" in the system prompt won't hold. The chunks are already in context, and someone will find the question that gets them out.

So the usual move is to run the search, then drop every chunk this user can't read.

Nothing restricted reaches the model. It looks safe.

But the search returned its top 5, and all 5 were from HR.

After the filter, the model gets nothing, and it answers anyway: a confident guess, or a "no" that is technically true.

On the questions people actually worry about, the closest matches are usually the restricted ones.

𝗧𝗵𝗲 𝗳𝗶𝗹𝘁𝗲𝗿 𝘄𝗮𝘀 𝗿𝗶𝗴𝗵𝘁. 𝗜𝘁 𝗷𝘂𝘀𝘁 𝗿𝗮𝗻 𝗮𝗳𝘁𝗲𝗿 𝘁𝗵𝗲 𝘀𝗲𝗮𝗿𝗰𝗵.

To fix this, tag every chunk when you ingest it: its classification, the roles that can read it, the country it can be served in.

Then pass the caller's identity into the query, so 𝘁𝗵𝗲 𝘀𝗲𝗮𝗿𝗰𝗵 𝗼𝗻𝗹𝘆 𝗿𝗮𝗻𝗸𝘀 𝗰𝗵𝘂𝗻𝗸𝘀 𝘁𝗵𝗶𝘀 𝘂𝘀𝗲𝗿 𝗰𝗮𝗻 𝗿𝗲𝗮𝗱.

The HR chunks are never candidates. The top 5 are now the best 5 this engineer is allowed to see, like the all-hands notes and the hiring FAQ.

And the clever question has nothing to pull out, because the model never received it.

This is called 𝗽𝗲𝗿𝗺𝗶𝘀𝘀𝗶𝗼𝗻-𝗮𝘄𝗮𝗿𝗲 𝗽𝗿𝗲-𝗳𝗶𝗹𝘁𝗲𝗿𝗶𝗻𝗴.

The tags are a copy of your permissions. Revoke someone's HR access and search keeps serving it until the chunks are re-tagged, so add a live check at query time if roles change often.

Treat untagged chunks as restricted, or the backfill becomes the leak.

And tight filters can hurt recall on approximate indexes, so measure it per role before rollout.

[Simran @simranzchawla]

@MiralineSystems Nobody accepts loudly that they are bringing it back. But out of necessity they are having to

[Simran @simranzchawla]

I don’t understand the eagerness and overconfidence to throw engineering principles out the door the moment a new toy comes into the market.

AI is a tool. It’s malleable enough to be moulded to strengthen a team’s existing engineering principles.

Building a strong engineering culture is the hardest thing an engineering leader does.

Knowing a principle is one thing. Getting a whole team to practise it daily is another.

So it baffles me when teams that already paid that price hand it over to AI the moment it’s convenient.

Code reviews are a good example. I’ve seen teams with a strong review culture drop human reviews, or hand them to AI, at the drop of a hat.

Why? Reviews are messy. Egos get involved. Feedback is delivered badly. Instead of handling that mess, they offloaded the practice entirely.

That’s a mistake. Code review is one of the most efficient tools our industry has ever had.

I’ve learnt a great deal from having my code reviewed. A lot of my technical edge comes from the questions I was asked in those reviews.

It’s a great way to mentor juniors. It’s also a clear indicator of someone’s technical prowess.

And it tells you a lot about a team. Read the comments on their merged PRs and you’ll see what they value: maintainability, readability, even nitpicks. Or you’ll see PRs merged without the diligence they deserve. That’s one of the first things I look at to understand how a team is doing, and to guess which team will cause the most incidents.

AI can make reviews sharper. It shouldn’t become the excuse to stop doing them.

On the contrary, code reviews are required much more today in age of AI slop

[Simran @simranzchawla]

How do you undo a database cutover three days after it happened?

Say you moved orders to a new database on Saturday night.

The switch was clean. The checksums matched, the dashboards stayed green, and everyone went to bed.

On Tuesday, finance finds refunds rounding wrong, and the fix won't be quick.

So you reach for the rollback plan: the old database, still sitting there untouched.

But its data stops at Saturday night.

Three days of customer orders exist only on the new side, so going back means losing them or merging them by hand in the middle of an incident.

𝗬𝗼𝘂 𝗸𝗲𝗽𝘁 𝘁𝗵𝗲 𝗼𝗹𝗱 𝗱𝗮𝘁𝗮𝗯𝗮𝘀𝗲. 𝗬𝗼𝘂 𝗱𝗶𝗱𝗻'𝘁 𝗸𝗲𝗲𝗽 𝗮 𝘄𝗮𝘆 𝗯𝗮𝗰𝗸.

To prevent this, at the moment of cutover, 𝘀𝘁𝗮𝗿𝘁 𝗿𝗲𝗽𝗹𝗶𝗰𝗮𝘁𝗶𝗻𝗴 𝗳𝗿𝗼𝗺 𝗻𝗲𝘄 𝘁𝗼 𝗼𝗹𝗱.

Every order written to the new system also lands on the old one, mapped back to its old shape.

So on Tuesday, the old database is seconds behind instead of three days.

Going back is now the same move you made on Saturday: block writes, let replication catch up, verify, open the other side.

Customers sit through a short pause, and no order disappears.

You fix the bug, and you try again next week.

This is called 𝗿𝗲𝘃𝗲𝗿𝘀𝗲 𝗿𝗲𝗽𝗹𝗶𝗰𝗮𝘁𝗶𝗼𝗻.

While it's running, new features can't use anything the old schema can't represent.

The old database also keeps carrying load, so pick the end date before the cutover, not after.

And run the reverse path in a rehearsal. If you haven't tested the way back, you don't know it works.

Sometimes the new model can't be mapped back at all.

Then you don't have a rollback. Call it a 𝗽𝗼𝗶𝗻𝘁 𝗼𝗳 𝗻𝗼 𝗿𝗲𝘁𝘂𝗿𝗻 and make it a go or no-go decision.

𝗬𝗼𝘂 𝗰𝗮𝗻 𝘀𝘁𝗶𝗹𝗹 𝗿𝗼𝗹𝗹 𝗯𝗮𝗰𝗸 𝗮𝗳𝘁𝗲𝗿 𝘁𝗵𝗲 𝗰𝘂𝘁𝗼𝘃𝗲𝗿, 𝘄𝗵𝗶𝗰𝗵 𝗶𝘀 𝘄𝗵𝗮𝘁 𝗺𝗮𝗸𝗲𝘀 𝗶𝘁 𝗲𝗮𝘀𝗶𝗲𝗿 𝘁𝗼 𝗮𝗽𝗽𝗿𝗼𝘃𝗲.

#data #migration

[Simran @simranzchawla]

How do you find out an attacker is already inside your cluster?

Not trying to get in. Already inside.

Say someone got in three weeks ago with a stolen session.

Since then they've done nothing loud. They list a few buckets, read some configs, run
kubectl get secrets once. Then they wait.

Every one of those actions shows up in your logs.

And every one of them looks exactly like your platform team on a normal Tuesday.

So the usual move is to add more rules and more alerts.

Now you have 400 alerts a day, and the real one is sitting in the middle of them, looking boring.

The logs had the attacker the whole time. The alerting just couldn't tell them apart.

Now change one thing.

Months ago, someone dropped an AWS key into an old config repo. It looks real. It's attached to nothing, and nothing in your systems ever uses it.

Three weeks in, the attacker finds it. Of course they try it.

That one API call pages you.

No tuning, no threshold, no "might be suspicious". Nobody legitimate had a reason to touch that key, so anyone who did is the person you're looking for.

That's a HONEYTOKEN. It can be a key, a fake "CEO salary" row that fires on a bulk export, or a hostname in a doc that nobody should ever resolve.

The catch is that you have to track every one, or your own migration script sets it off at 2am.

And it has to look boring. A careful attacker skips a key that looks planted.

This is not detection by pattern. It is detection by access.

[Kelly Sommers @kellabyte]

I don’t recommend this approach.

Many of the elite engineers I’ve ever met put everything into their daily work cuz they on purpose chose a role/company/challenge they are truly passionate about & put a lot of extra effort into the company

This is a lazy way to judge autonomy.

[John Crickett @johncrickett]

This is lazy hiring.

Not having side projects could signal a lack of autonomy.

Or it could signal they're highly autonomous and highly engaged in their current role leaving no time for side projects.

Or that they have other non-tech related side projects.

Or hobbies, family, etc.

The founder isn't going to hire well.

They're taking a lazy approach to hiring, trying to find good attributes - autonomy - with a lazy approach.

They'd do better learning how to do effective competency based interviews and look for evidence of autonomy. More importantly, evidence of it done in ways that are relevant to the role.

[Simran @simranzchawla]

@kunchenguid When things don't work in production there will be mud-slinging since no-one can expect claude to take ownership

[dax @thdxr]

i see people saying

they're moving to iac because agents
they're not using iac because agents
they're switching to a monorepos because agents
they're moving off monorepos because agents

you're all confused man everyone is doing random shit

[Simran @simranzchawla]

Run reviews against a pre-decided acceptance criteria

At the end of each review cycle, have a different persona(same model works) triage the findings in what’s worth fixing and what should be deferred.

In a few cycles, things would settle.

This is tried and tested approach

[ayoub__dev @ayoub_el77927]

Asking Claude to review your code is a slot machine.

Every review finds brand new bugs. Review again: more. Review a tenth time: it still finds something.

It never runs out of bugs to find... so how do you decide your code is actually done?

[Simran @simranzchawla]

@ayoub_el77927 Run reviews against a pre-decided acceptance criteria

At the end of each review cycle, have a different persona(same model works) triage the findings in what’s worth fixing and what should be deferred.

In a few cycles, things would settle.

This is tried and tested approach

[dax @thdxr]

one thing that hasn't changed is there are still a bunch of companies telling you how to build software

and like always they don't lead with their products. it's all thought leadership on software factories or whatever the term of the week is

make something a lot of people love and then maybe i'll be interested in hearing how you work

[Simran @simranzchawla]

I don't think its right to have the tools influence the architecture of the product or how the codebase is structured.

Having the need to move multiple services into monorepos just because your agent can read the code tells that your AGENTS .md is not setup properly anyways.

I have anyways found claude code, opencode etc to be eager to scan other folders.

[Simran @simranzchawla]

@gagu_sh 💯

[Simran @simranzchawla]

If you've been using AI for development for more than a year, you've stopped treating your language as a badge. The model writes the Go, the Kotlin, the Rails. What you actually bring is judgment: the design, the failure modes, and whether the output is right.

So leading with your language stack now reads a little dated.

Which makes it strange when an "AI-first" company screens candidates by language in the JD. If AI does the typing, the language is a detail. What's worth filtering for is how someone reasons about systems, reviews code they didn't write, and decides what not to build.

Filter for that, and the language question mostly answers itself.

[Simran @simranzchawla]

sure. That much difference is there on language to language basis. This difference can be learnt very easily today, again using AI. Most of the languages share most of these concepts anyways.

The differential can always be covered and this differential should never be the reason for filtering out the resumes.

[Simran @simranzchawla]

This when the recruiters are still rejecting profiles for AI-first companies when a specific coding language doesn’t match the JD

[Somi @somi_ai]

Anthropic rewrote its Messages API in Rust. Zero Rust devs on the team.

Claude wrote the code. Humans wrote the plan and checked old vs new matched on real traffic. CPU dropped 40-50%.

Pretty sure "never do the big rewrite" is dead advice.

@katelyn_lesse on @latentspacepod

[Simran @simranzchawla]

Welcome aboard Vibe Airlines Pvt. Unlimited!

Please fasten your seatbelts: zero human eyes touched today’s flight software. No HITL code review—just 100% blind trust in the behavior test harness.

An AI agent wrote the PR, an AI agent approved it, and because ⁠assert_flight_stable() == True⁠, it auto-deployed straight to production at 35,000 ft.

Why eyeball diffs when the E2E suite is green?

Use promo code ⁠ONEWAYTRIPTOHEAVEN⁠ for 100% off!

[Steve Bauman @stevebauman]

Im sorry im not buying this ‘not reviewing code’ thing. Maybe later, but certainly not now, even with frontier models with max thinking

Every single day I’m catching nonsense. If you’re merging this shit without looking, the systems your working on are either

- not important and/or
- have zero consequences for being wrong