Good take
My guess is
- demand for intelligence is near infinite
- but 80% of workloads will be running on 99% cheaper models within 12-18 months
- 20% of workloads will still run on latest gen models where IQ maxing is important (scientific breakthroughs, higher level ochestrator agents?)
- rough analogy might be what % of macbooks or gaming PCs sold have the maxed out specs for CPU/GPU, prices are falling much faster than Moore's law here though
- this leads me to think the limiting factor will be energy and compute, not better models
At Coinbase we're working hard on routing prompts to cheaper models where appropriate, and in some cases have been able to keep costs roughly flat, while token usage continues to grow exponentially.
*UBER SETS $1,500 MONTHLY CAP ON SOME AI CODING TOOLS FOR STAFF
$UBER officially reeling in the Claude budget after blowing their AI budget earlier this year.
Undoubtedly more companies to follow
think back to projects you've worked on in the past
it's hard not to imagine they'd have been completed way faster now that we have ai
but everything still feels as slow and as difficult as ever
The more I build agentic systems at Razorpay, the more I understand that - at its core, it is an agentic loop with tool calls, integrations, and retrieval. The hard part is...
actually making it run reliably, at scale, under real production load. And this is what makes system design even more important.
Your AI system is still expected to scale. It will still need microservices, message queues, consistency guarantees, load balancing, work distribution, state management, rate limiting, throttling, fallbacks, service-to-service communication, QoS, etc.
It is great that you are looking into AI and are interested. You should be. Everyone should be. But it is important not to skip system design and cs fundamentals. I know it seems overwhelming, but it is what it is.
First principles are not going anywhere, and that is super essential for actually building applied AI systems and running them reliably at scale. If you are a backend engineer and are kind of skipping these things, pause and reflect once.
It is always good to be great at system design, not because it will help you crack interviews (it will), but because it will make you meaningfully better at your job. Seeing it firsthand.
Remember, you will not be shipping prototypes to production. The difference between prototype and production code is 15 components and 1000 commits.