Microsoft might have just made running large AI models a whole lot cheaper. 🤯
They open-sourced bitnet.cpp, a 1-bit LLM inference framework that can run a 100B-parameter BitNet model on a single CPU without a GPU.
The numbers are pretty wild:
→ Up to 6.17x faster on x86 CPUs
→ Up to 82.2% less energy consumption
→ Completely open source
Imagine running massive AI models without needing expensive GPUs.
This could be a big deal for local AI.
And here we go…GPT-6.2 Astra and GPT-6.2 Sol are yours to make more games with.
Oh, and they’re also super efficient and really good at making slides, building websites, and tackling the work you were doing before you started making games.
Rolling out today to ChatGPT Work and Codex for Plus, Pro, Enterprise, and Edu users.
https://t.co/tjQBDQWB2n
Most "slow AI" complaints aren't about speed. They're about staring at a blank screen.
We ran the same prompt 5 times each way. With streaming, the first words showed up in 1.12s. Without it, nothing appeared until 5.60s.
The full answer took about the same time both ways. One flag changed how fast it felt.
If you're building on Decuff, turn on stream=True.