Anup

Software engineer writing about production AI systems, search, RAG, evals, and system design. Building in public from Bengaluru.

[Anup @hexnoct]

@technoking_420 @gork is this true?

[Anup @hexnoct]

@MarcosRGjr Agreed. Most runaway loops were visible the whole time, just with no authority to act on it.

A stop-only floor is the right call, since a halt can only make things safer. One suggestion: add a spend or call-rate cap as an example rule, because a runaway loop is loud, not silent.

[Anup @hexnoct]

Two AI agents got stuck asking each other to clarify their work.

For 11 days.

Billing every single call, until the tab crossed $47,000. Nobody noticed.

This is from Linas Beliūnas's new piece on AI agent loops, and it's the part of agentic AI nobody wants to talk about.

Another case he describes:
A coding agent hit an ambiguous error and retried 240 times in 3 hours. $4,200 burned.

Three monitoring dashboards were showing the spend live.
None of them had the power to stop it.

That's the real lesson here.

The loop logic was fine. Both agents did exactly what they were built to do.
What was missing was a governance layer outside the loop, with actual authority to kill it.

A dashboard is not a brake.
Visibility is not enforcement.

As teams move from one founder running one overnight script to swarms of agents other people depend on, this becomes the actual engineering problem:

1. Hard budget caps that terminate, not just alert
2. Someone owning the outer loop, the goal, not just the task
3. Deciding how much human review you can safely remove
4. A verification gate before swarm output ships

Everyone's racing to make agents more capable.
Very few are building the off switch.

Full piece here: https://t.co/efBdVQ5oyb

If you're running agents in production today, what actually stops a runaway loop in your setup?

Follow me for more tech related insights.

#AI #AIAgents #TechTwitter #LLMOps #BuildInPublic

[Anup @hexnoct]

Spotify says it cut Claude Code token usage by about 90% on file reading, without touching how the hard thinking works.

The idea: most of what a coding agent does is I/O, not reasoning.

Reading five files to answer one question. Writing a test file that copies the twenty next to it. Frontier tokens, almost zero thinking.

So they split the work:

1. A cheap worker model (Gemini 2.5 Flash in their examples) reads big files and returns a short summary
2. The same worker writes boilerplate straight to disk, so Claude never even sees the output
3. Claude keeps the debugging, architecture and safety-critical code

The routing is enforced, not just suggested. A Claude Code plugin called shunt uses hooks to block any read over 350 lines by default and redirect it to the cheaper path.

Why it matters: Gartner predicts AI coding costs will pass the average developer salary by 2028. Some teams already spend over $2,000 per developer per month on tokens.

What does not work, per the author:

1. Delegating edits, since summaries lack reliable line numbers
2. Delegating reasoning, since the worker missed a thread-safety bug that Claude caught in seconds
3. Small files, since each call adds 10 to 30 seconds of latency

Full write-up: https://t.co/S26gFXGxa6

Is your team tracking token spend per developer yet, or is it still a surprise on the monthly bill?

Follow me for more tech related insights.

#AI #ClaudeCode #AIAgents #DevTools #TechTwitter

[Anup @hexnoct]

Spotify says it cut Claude Code token usage by about 90% on file reading, without touching how the hard thinking works.

The idea: most of what a coding agent does is I/O, not reasoning.

Reading five files to answer one question. Writing a test file that copies the twenty next to it. Frontier tokens, almost zero thinking.

So they split the work:

1. A cheap worker model (Gemini 2.5 Flash in their examples) reads big files and returns a short summary
2. The same worker writes boilerplate straight to disk, so Claude never even sees the output
3. Claude keeps the debugging, architecture and safety-critical code

The routing is enforced, not just suggested. A Claude Code plugin called shunt uses hooks to block any read over 350 lines by default and redirect it to the cheaper path.

Why it matters: Gartner predicts AI coding costs will pass the average developer salary by 2028. Some teams already spend over $2,000 per developer per month on tokens.

What does not work, per the author:

1. Delegating edits, since summaries lack reliable line numbers
2. Delegating reasoning, since the worker missed a thread-safety bug that Claude caught in seconds
3. Small files, since each call adds 10 to 30 seconds of latency

Full write-up: https://t.co/S26gFXGxa6

Is your team tracking token spend per developer yet, or is it still a surprise on the monthly bill?

Follow me for more tech related insights.

#AI #ClaudeCode #AIAgents #DevTools #TechTwitter

[Anup @hexnoct]

Spotify engineers in 2023: Write a 20,000 line script to bump Maven dependencies.

Spotify engineers in 2025: Write a prompt. Go to lunch.

That is roughly the story of Honk, Spotify's background coding agent.

1,500+ AI written pull requests merged into production.
60 to 90 percent time saved on migrations.
Around half of all their PRs were already automated before the AI even showed up.

Things the agent now does:

1. Java value types to records
2. Breaking change upgrades on data pipelines
3. Frontend component migrations
4. YAML and JSON edits that actually respect the schema

Things it also does:
Lets PMs propose code changes from Slack without cloning a single repo.

Somewhere a senior engineer just felt a disturbance in the force.

Meanwhile most Indian service companies are still raising a Jira ticket to get approval for upgrading a library version.

Honest question: would you merge a PR from an agent without reading every line?

Follow me for more tech related insights.

#AI #AIAgents #Spotify #DevTools #TechTwitter

[Anup @hexnoct]

Someone uploaded a 14 trillion parameter Anthropic Mythos checkpoint to Hugging Face.

It turned out to be seeded random noise.

How it probably went for a lot of people:

1. See the upload. Heart rate goes up.
2. Check your disk space. Not enough.
3. Order another SSD on No Cost EMI.
4. Start the download anyway.
5. Tell the team chat you now run frontier AI locally.
6. Learn it was random numbers dressed up as model weights.
7. Quietly delete the team chat message.

The actual lesson is not funny though.

Model weights are just huge files of numbers. Unless you can verify where they came from, you have no way of telling real weights from noise just by looking at them.

Checksums, official sources, signed releases. Boring, and very important.

Would you have downloaded it first and asked questions later?

Follow me for more tech related insights.

#AI #TechTwitter #HuggingFace #Anthropic #OpenSource

[Anup @hexnoct]

@marchoeffl Hmm Interesting!

[Anup @hexnoct]

Most teams use ClickHouse at maybe 30% of what it can actually do.

Here's how to get the rest:

1. Design ORDER BY around your queries, not your schema.
The sparse primary index is your biggest lever. Put the columns you filter on most first, low cardinality before high.

2. Batch your inserts.
Thousands of tiny inserts create thousands of parts, and merges fall behind. Insert in big batches, or turn on async_insert and let ClickHouse buffer for you.

3. Don't over-partition.
Partitioning by day on a modest table hurts more than it helps. Monthly is usually enough. Partitions are for data management, not query speed.

4. Use the right types.
LowCardinality(String) for columns like country, status or city. Smallest integer that fits. DateTime over String for timestamps. Storage drops and scans get faster.

5. Add compression codecs.
Delta or DoubleDelta for timestamps and counters, ZSTD for the rest. Easy disk savings, often with zero query penalty.

6. Precompute with materialized views.
If your dashboard runs the same aggregation every 10 seconds, compute it once at insert time with an AggregatingMergeTree.

7. Try projections for alternate access patterns.
Need to filter by a different column often? A projection gives you a second sort order without managing another table.

8. Stop writing SELECT *.
It's a columnar database. Every extra column is extra disk you read for nothing.

9. Be careful with FINAL.
It's convenient but expensive on large tables. Design your ReplacingMergeTree queries with argMax or GROUP BY where you can.

10. Read system tables.
system.query_log shows your slowest queries. https://t.co/oHr2PVaWZ7 shows if parts are piling up. Most performance problems are already sitting there.

ClickHouse is fast by default. It becomes absurdly fast when you design for how it actually reads data.

What's the one ClickHouse optimization that saved you the most?

Follow me for more tech related insights.

#ClickHouse #DataEngineering #Database #OLAP #TechTwitter

[Anup @hexnoct]

@marchoeffl Even teams deep into ClickHouse keep finding new levers, whether it's projections, merge settings or dictionaries. There's always another layer.

Curious, what's the most advanced thing you're running in prod right now?

[Anup @hexnoct]

Your Postgres is not slow.

You are just asking it to do two jobs at once.

Serving the app (OLTP) and running heavy dashboards (OLAP) on the same database is how you end up with a 40 second GROUP BY and an angry on-call engineer.

The cleaner setup: keep Postgres as the source of truth, and let ClickHouse handle the analytics.

Here is how to wire them together:

1. Start simple: query Postgres from ClickHouse
Use the postgresql() table function or the PostgreSQL table engine to read tables directly. Great for quick joins and small lookups, but every query still hits Postgres.

2. Set up CDC for real workloads
Turn on logical replication in Postgres (wal_level = logical), create a publication, and stream changes into ClickHouse.
Options: ClickPipes / PeerDB if you want managed, or Debezium + Kafka if you want full control.

3. Model for ClickHouse, not Postgres
Use ReplacingMergeTree with a version column so updates and deletes land correctly.
Pick ORDER BY based on how you filter, not on your Postgres primary key.
Denormalize. ClickHouse loves wide tables, not 8-way joins.

4. Pre-aggregate the hot paths
Materialized views for the dashboards everyone opens every morning with their chai.

5. Watch the plumbing
Monitor replication slot lag in Postgres. An abandoned slot will quietly keep WAL around and eat your disk.

The result: Postgres stays fast for writes, ClickHouse scans billions of rows for reports, and nobody has to choose between product and analytics.

Are you running analytics straight on Postgres today, or already split out? What did you pick for CDC?

Follow me for more tech related insights.

#PostgreSQL #ClickHouse #DataEngineering #OLAP #TechTwitter

[Anup @hexnoct]

Your Postgres is not slow.

You are just asking it to do two jobs at once.

Serving the app (OLTP) and running heavy dashboards (OLAP) on the same database is how you end up with a 40 second GROUP BY and an angry on-call engineer.

The cleaner setup: keep Postgres as the source of truth, and let ClickHouse handle the analytics.

Here is how to wire them together:

1. Start simple: query Postgres from ClickHouse
Use the postgresql() table function or the PostgreSQL table engine to read tables directly. Great for quick joins and small lookups, but every query still hits Postgres.

2. Set up CDC for real workloads
Turn on logical replication in Postgres (wal_level = logical), create a publication, and stream changes into ClickHouse.
Options: ClickPipes / PeerDB if you want managed, or Debezium + Kafka if you want full control.

3. Model for ClickHouse, not Postgres
Use ReplacingMergeTree with a version column so updates and deletes land correctly.
Pick ORDER BY based on how you filter, not on your Postgres primary key.
Denormalize. ClickHouse loves wide tables, not 8-way joins.

4. Pre-aggregate the hot paths
Materialized views for the dashboards everyone opens every morning with their chai.

5. Watch the plumbing
Monitor replication slot lag in Postgres. An abandoned slot will quietly keep WAL around and eat your disk.

The result: Postgres stays fast for writes, ClickHouse scans billions of rows for reports, and nobody has to choose between product and analytics.

Are you running analytics straight on Postgres today, or already split out? What did you pick for CDC?

Follow me for more tech related insights.

#PostgreSQL #ClickHouse #DataEngineering #OLAP #TechTwitter

[Anup @hexnoct]

ChatGPT: canceled. Claude: canceled. @underdogai: free, private, on my laptop. Entered to win the RTX Spark signed by Chris Malachowsky

https://t.co/cntyxMFIAS

[Sigil Wen @0xSigil]

All AI should be free, so I’m giving away the first RTX laptop for free, signed by NVIDIA Founder

@UnderdogAI gives everyone a free, capable & private AI. 100% on-device

To enter:
1) Retweet and like this post
2) Sign up for Underdog giveaway: https://t.co/5bxNQA6rzP
3) follow @UnderdogAI so we can dm u

[Sigil Wen @0xSigil]

All AI should be free, so I’m giving away the first RTX laptop for free, signed by NVIDIA Founder

@UnderdogAI gives everyone a free, capable & private AI. 100% on-device

To enter:
1) Retweet and like this post
2) Sign up for Underdog giveaway: https://t.co/5bxNQA6rzP
3) follow @UnderdogAI so we can dm u

[Anup @hexnoct]

AWS gives you two kinds of SSD storage.

Local NVMe: insanely fast, but wiped if the instance stops.
EBS: a bit slower, but your data survives.

So run Postgres on NVMe for speed, stream a replica to EBS for safety, and back up to S3.

Speed of NVMe, durability of EBS.

Would you trust this setup in prod?

Follow me for more tech related insights.

#AWS #PostgreSQL #CloudComputing #DevOps #TechTwitter

[Anup @hexnoct]

If you're using GPT to evaluate GPT outputs, your eval is lying to you.

LLM judges have a documented self-preference bias. Models tend to rate text that sounds like their own writing higher, even when it isn't better.

So your shiny 92% score might just mean your judge likes its own accent.

The fix is simple and cheap:

1. Use judges from at least 3 different families. Something like GPT, Claude, Gemini, plus an open model like Llama or Qwen.
2. Never let a model be the only judge of its own outputs.
3. Average the scores or take a majority vote.
4. For A vs B comparisons, run every pair twice with the order swapped. Judges have position bias too.
5. Temperature 0, strict JSON output, so results are reproducible and parseable.
6. Hand-label 30 to 50 samples yourself and check which judge actually agrees with you before trusting any of them at scale.

Research on judge panels has found that a jury of smaller, diverse models can match or beat a single big judge, at a fraction of the cost.

One judge is an opinion. A diverse panel is an evaluation.

What does your eval setup look like right now?

Single judge, panel, or vibes?

Follow me for more tech related insights.

#AI #LLM #LLMEvaluation #MachineLearning #TechTwitter

[Anup @hexnoct]

@Wbackdown I do these three things bellow:-
1. If GPT scores GPT higher than Claude and Gemini do, that gap is the bias.
2. Rewrite an answer in another model's style. If the score moves, it's grading voice.
3. Rerun a small human-labeled set every eval to catch drift.

[Anup @hexnoct]

If you're using GPT to evaluate GPT outputs, your eval is lying to you.

LLM judges have a documented self-preference bias. Models tend to rate text that sounds like their own writing higher, even when it isn't better.

So your shiny 92% score might just mean your judge likes its own accent.

The fix is simple and cheap:

1. Use judges from at least 3 different families. Something like GPT, Claude, Gemini, plus an open model like Llama or Qwen.
2. Never let a model be the only judge of its own outputs.
3. Average the scores or take a majority vote.
4. For A vs B comparisons, run every pair twice with the order swapped. Judges have position bias too.
5. Temperature 0, strict JSON output, so results are reproducible and parseable.
6. Hand-label 30 to 50 samples yourself and check which judge actually agrees with you before trusting any of them at scale.

Research on judge panels has found that a jury of smaller, diverse models can match or beat a single big judge, at a fraction of the cost.

One judge is an opinion. A diverse panel is an evaluation.

What does your eval setup look like right now?

Single judge, panel, or vibes?

Follow me for more tech related insights.

#AI #LLM #LLMEvaluation #MachineLearning #TechTwitter

[Anup @hexnoct]

October isn't even over and we've already had 20 new AI models.

From 16 different providers. In roughly two weeks.

A few that dropped:

1. Grok Voice 3.0 from xAI (Oct 14)
2. Muse Spark from Meta (Oct 15)
3. Jais 3 from IFM (Oct 15)
4. Nemotron 3.5 Super VL from NVIDIA (Oct 15)
5. LFM2.5-24B from Liquid AI (Oct 13)

And the one everyone was waiting for, OpenAI's reportedly planned October model, is no longer expected in its original form.

The big theme: multimodal everything. Text, images, video, voice, all in one model.

Honestly, new models are launching faster than Swiggy can deliver a chai.

The real question isn't which model is best anymore.

It's which one you'll actually bother switching to.

Which model are you using daily right now, and why?

Follow me for more tech related insights.

#AI #TechTwitter #LLM #GenerativeAI #Nvidia

[Anup @hexnoct]

@amonyx hi