Meghana Harishankara

AI Builder @ Salesforce • CMU • AI agents by day, Carnatic vocals + long runs by evening • I write about the eng craft

[Meghana Harishankara @meghanatweets]

@dineshpaii Yup. Who cares if someone important doesn't read or doesn't like?
The point is you learn something new and pen it down somewhere so that you self assess how much you have learnt over time 😊

[Meghana Harishankara @meghanatweets]

One axis your articles misses: you nail when specialization pays off, but a rented (frontier) model can also just be switched off anytime. Maturity tells you when to migrate for efficiency. Revocation risk tells you when you can't afford not to. Wrote up the full argument here: https://t.co/DOoEB2vFZY

[Meghana Harishankara @meghanatweets]

https://t.co/Hck3SakXaP

[Meghana Harishankara @meghanatweets]

https://t.co/Hck3SakXaP

[Meghana Harishankara @meghanatweets]

https://t.co/Hck3SakXaP

[Meghana Harishankara @meghanatweets]

Five patterns, what they share underneath, how Uber/LinkedIn/Airbnb/Meta combine them, and which framework to start with: https://t.co/a1pncCHdf0

[Meghana Harishankara @meghanatweets]

https://t.co/7RjjtJ1ltw

[Meghana Harishankara @meghanatweets]

Your agent system isn't broken because the model is bad.
It's broken because you don't have orchestration. State is undefined. Retries aren't handled. Handoffs between steps have no contract.

The model is fine. The infrastructure around it isn't.

[Meghana Harishankara @meghanatweets]

All five share the same loop underneath.
Create a unit of work → run it → persist the result → continue.

"The model can be chaotic. The orchestration layer shouldn't be."

[Meghana Harishankara @meghanatweets]

@sv_cloud_expert Mostly agree, though I'd push one level deeper: the workflow spec only becomes the product once you've decided what failure means. Most teams skip that. The retry logic and the escalation path are where the real design decisions live, not the happy path.

[Meghana Harishankara @meghanatweets]

https://t.co/7RjjtJ1ltw

[Meghana Harishankara @meghanatweets]

@svembu Not just Databricks but also @coinbase
Here's their implemention metrics: https://t.co/sT0b6ky7HL

[Brian Armstrong @brian_armstrong]

How to keep AI spend flat while token usage grows exponentially: Not with friction and spend alerts. With better defaults, routing, and caching.

Better Defaults (not Usage Caps) – Engineers can choose any model they want, but defaults matter. We’re experimenting with defaulting to open weight models like GLM 5.2 and Kimi 2.7 through our LLM gateway, while still encouraging engineers to choose the right model for the task. 91% of our employees were never hitting their usage caps, so instead of lowering caps and driving up alerts, we're moving to cheaper defaults. Note that code reviews use a diversity of models, so they can check each other's work.

Better Routing – In our custom harnesses, we preprocess prompts and route to the best model for the job, considering cache hits and model pricing. For instance, you may want a frontier model for planning, but not for execution where they can be overkill. Ultimately, humans shouldn't be choosing models - AI can automate this task.

Better Caching – Cache misses are the easiest way to drive your cost up. All of our requests are cache aware, so we’re reusing a warm cache wherever possible. For example, our cache hit rate went from 5% → 60% in LibreChat once properly implemented.

Keep Context Lean – Start fresh sessions when switching tasks. Scope file context narrowly. Disconnect unused tools. Don't just compact. The goal isn't fewer tokens used, it's fewer tokens wasted.

Better Visibility – Our engineers can use as many tokens as they want, from whatever model they want, but we’ve made usage visible – and the more you spend on AI, the more impact we expect.

The goal isn't to suppress usage. It's to build the infrastructure that makes exponential growth sustainable.

Putting this into practice has cut our AI spend nearly in half, while our token usage continues to grow.

[Meghana Harishankara @meghanatweets]

For developers: the 50-year-old while loop isn't the breakthrough. The multi-agent control plane is. The skeptics are right about the shape, but they're missing what we actually put inside it.

Argue about the inside. Everything else is plumbing.

Full post: https://t.co/zgLCdgPvzf

[Meghana Harishankara @meghanatweets]

https://t.co/iNXe07rQ8P

[Meghana Harishankara @meghanatweets]

AI loops is the trending topic these days. But the debate isn't about running an LLM in a while(true) loop—it's about a fundamental shift in how we orchestrate software.

The fight isn't about loops. It's about control.

[Meghana Harishankara @meghanatweets]

The cautionary tales are financial. Uber capped its engineers at $1,500/month after blowing its annual AI budget in four months.

Infinite loops. Missing halt conditions. Massive bills. All starting from a missing break statement.

Don't just schedule. Control.

[Meghana Harishankara @meghanatweets]

@ruparel_amit @jeremybernier Oh yes.
Forgot Big Mug. The only late night coffee shop in Santa Clara.

[Meghana Harishankara @meghanatweets]

@jeremybernier Coupa Cafe in Ramona Palo Alto (8pm).
Verve Coffee in Palo Alto (6pm).
Joe & the Juice in Palo Alto (7pm).
Yummy Future Coffee in Palo Alto (10pm).
Paris Baguette in MTV (10 pm)
Phliz Coffee in Sunnyvale downtown (7:30 pm)

These are some of my late evening favorite coffee shops!

[Meghana Harishankara @meghanatweets]

@mehulmpt You have hit the nail on the head!
All are excellent points that needs to be said out loud, especially if we want @OfficialINDIAai / @SarvamAI to win.

[Meghana Harishankara @meghanatweets]

For India: chips and training aren't doable in 2026 (state-level fights). Weights, inference, and data-flow law are.

@3one4Capital is right, commercial inference is the next mission. Friday made it urgent.

Full post: https://t.co/WiFnQkkoCd

[Meghana Harishankara @meghanatweets]

https://t.co/693N66SSLT

[Meghana Harishankara @meghanatweets]

On June 12, the US ordered Anthropic to suspend access to its frontier models for 'any foreign national, whether inside or outside the United States.'

Sovereign AI just got real.

[Meghana Harishankara @meghanatweets]

The cautionary tales are loud. Project Independence missed 1980. GAIA-X never displaced AWS. Aleph Alpha got acquired by Cohere.

Three countries, three decades, three failure modes. All started with the press release.

Don't crown. Layer.