1/ we’ve built a lot of connectors to make it easier to integrate Muse into your life. Muse runs in a secure VM and we don’t use your data for ads or anything other than to make Muse great for you. you can connect Muse to Gmail, Google calendar, Outlook, Plaid, Opentable, Google Docs, Spotify, Function Health, Withings, Tailscale, Peloton, and more.
token pricing is effectively meaningless now. 3.8 flash looks 13x cheaper when measured per token, but Astra is cheaper per task since it's far more efficient. measure your costs per task, not per token.
HBM splits into two things: capacity (how many layers) and speed (bandwidth/latency). Capacity decides whether KV, batch, and context fit on the card. Speed decides how many bytes you can move each step. Stacking mainly adds GB. It does little for bandwidth and can make the cube hotter, slower, and harder to supply.
The old script was 12/16-Hi and ever-larger memory per GPU. Rubin Ultra rumors now point to 8-Hi / 192GB. At Hot Chips, some labs were asking for 4-Hi. Same direction, different notch: not “no HBM,” but no paying heat and yield for unused capacity.
For inference and agents, the scoreboard is tokens per watt, then tokens per dollar. Capacity is the entry ticket. Bandwidth is the throttle. “4-Hi is enough” means: keep pin speed, make up capacity with NVLink and the rack, and a shorter stack can turn watts into tokens more cleanly. That pushes out the urgency of very tall stacks and hybrid bonding. Flagship training SKUs may still keep high-capacity parts. Do not read this as HBM demand disappearing.
Cloud agents are the future. Say goodbye to the harness running locally on your computer.
All models will move toward agent swarms and cloud computers very soon, much like Grok Bot.
backdrop: kings all asleep/on vacation while minions slave away
august 2024 - first big fail
jan 2025 - only competent vp bails after one terrorist lead demanding an even bigger slice of pie breaks the camels back
summer 2025 - second big fail + the king of terrorists climbs the ranks above the slumbering execs
summer 2026 - slumbering execs + terrorists who only show up for successful announcements move on, leaving just the king of terrorists behind
-----
lesson: if you're high enough up there, you can pretty much be a terrorist and hold orgs hostage while continuing to ask for more and more and more, until there's absolutely no more aura or gold left to plunder, and then just move on to greener pastures to do it all over again!
(honestly, good timing, the king of terrorists is just insufferable, no?)