5.6 sol growth is insane.
the inference team has done heroic work to be able to support demand.
we are going to move mountains to continue to scale, but it is possible there are some hiccups soon.
@Sentdex@KinvertOG As a hobby, all this is great (including tinkering with hardware and local inference), but trying to convince everyone that "this is the only way" is kinda weird
@Sentdex@KinvertOG Being self-sufficient is great, but most of us no longer slaughter cows at home, raise chickens, grow potatoes and corn, sew our own clothes and shoes, produce medicines from herbs, or make furniture. That's perfectly normal in a civilized society. It's not about being a product.
@ChrissGPT I'm still hoping they won't roll out some outdated, overpriced, and buggy crap. Google caught up fast in the industry but has lost its momentum and pace again. Looks like Larry has lost interest in LLMs.
@thdxr Interestingly, we tend to forget just how powerful the even mid-range models we've got for free really are. We adjust to new benchmarks so fast, even though just a few months ago that level of intelligence was the absolute cutting edge.
@Sentdex@KinvertOG I'm absolutely not "mad" — I work calmly and happily every day with fast frontier intelligence for pennies, offloading a bunch of tasks to less frontier LLMs for even less. I'm not trying to build an oil refinery in my backyard to fuel my car with "homemade crappy gasoline" 😆
@Sentdex@KinvertOG Or, let me guess, the ultimate dream would be unlimited inference of quantized GLM 5.2 at a snail's pace for tens of thousands of dollars?
@antirez Yeah, exactly — it's like managing processes from a big-picture perspective versus manual micro-management. The former scales much better (especially if you have the ability to micro-manage in some cases), you just need to tune everything the right way.