Below the model, above the GPU. Everyone builds above. Everyone buys below. The infrastructure layer in between is due for a closer look. It's where utilization lives or dies.
Plus, cloud vendors just raised H200 prices 15%. The first jump in twenty years, and the industry's answer is still - go buy more. If this sounds like familiar - you don't have a GPU shortage. You have a 95% idle fleet and a scheduler that was never built to notice.
Nine companies this year will spend $830 billion on data centers. But the crazier statistic is that the average GPU utilization inside them: 5%.
That's not a hardware problem. It's a software problem wearing a hardware bill.
None of this means efficiency is the enemy. It means the way we've pursued efficiency to date hasn't yet impacted consumption; it's raised the demand ceiling every single time. Have you found a plateau yet? Or does every gain get fed right back into the machine?
The most expensive assumption in AI right now is that efficiency reduces demand.
Everyone cheered when DeepSeek trained a frontier model for under $6 million. Finally, proof that AI could get cheaper!
What happened next?
Better algorithms = bigger training runs. Better chips = previously impossible scales. Cheaper inference = the agentic era, where one task burns thousands of times the tokens of a chat reply. 𝘞𝘦 𝘥𝘰𝘯'𝘵 𝘤𝘰𝘯𝘴𝘰𝘭𝘪𝘥𝘢𝘵𝘦, 𝘸𝘦 𝘳𝘦𝘭𝘰𝘢𝘥.
Obviously this is way worse than API overall. However, explicitly nerfing subscriptions leads to huge public backlash, and the rapidly falling cost of intelligence means you'll be able to profitably serve Opus 4.8 level models for $20/month in the near future. We therefore think it's far more likely the labs will withhold new features/models from subscription plans. It will be interesting to see if Mythos ends up being API only. (4/4)
Obviously this is way worse than API overall. However, explicitly nerfing subscriptions leads to huge public backlash, and the rapidly falling cost of intelligence means you'll be able to profitably serve Opus 4.8 level models for $20/month in the near future. We therefore think it's far more likely the labs will withhold new features/models from subscription plans. It will be interesting to see if Mythos ends up being API only. (4/4)
Now you can add loading animations to your Rust terminal app! 🦀
⏳ tui-skeleton — Animated skeleton loading widgets for @ratatui_rs apps
⚡ Show placeholders with smooth animations while your data loads
⭐ GitHub: https://t.co/nSbgrXVXxF
#rustlang#ratatui#tui#widget #library #ux #ui #animation #terminal
@SiliconANGLE We can play nice too! You get the most with Kubernetes by pairing it with TAHO. We fit easily into your stack and works with your current tools. No disruptions. No major rewrites.
A little spice in a headline never hurts though 🌶️ 😁
Central control is the taxi dispatcher model. One office assigns every ride, and when demand spikes the line grows even with cars nearby. A compute fabric is rideshare for infrastructure. Nodes coordinate directly so work finds the fastest path.
Full blog link in first reply.
When “just throw more hardware at the problem” falls apart...
Microsoft’s CEO said the constraint is not chips. It is electricity and power ready data center space.
Our view at TAHO:
• Performance per watt is the north star
• Efficiency beats brute force
AI bubble? Dave Birnbaum argues the next surge is engineering: do more with existing servers, cut power per task, turn spend into productivity. Check out the in depth blog post Dave authored on the subject:
https://t.co/JDvE3039Tg
Containers are convenient but compute hungry.
TAHO gets more from your existing hardware. Start small then expand. Results 10 times faster compute up to 30 times performance about 90 percent lower cost.
Watch the video.
#AI#DevOps#TAHO
Meet TAHO, a computing platform that turns all your servers into one intelligent supercomputer.
Big jobs are broken into pieces and sent anywhere there is capacity.
Once computed, the results are available everywhere, so you never repeat work.
TAHO creates a shared memory compute layer across your fleet. When one server computes, the result is available everywhere.
No repeat work. No wasted cycles. The outcome faster jobs, higher utilization, and big cost savings.
Run serious AI or data pipelines? TAHO is for you.