if you're a business owner just picking the smartest model doesn't matter anymore.
what you really need is to stop relying on just one... Here is the org chart for your AI 🧵
What is the average cache holding period for Fable?
Anyone know?
I'm having it orchestrate multiple agents and sometimes agents take a while to finish. Want to keep the cache warm periodically.
@rezoundous It's been slightly better for me, but for some reason, GBT 5.5 is still just eating tokens and usage way faster than normal.
been happening since the last week or two. They said they fixed it, but it's still just evaporating.
If the US can lead in open source, that will meaningfully change the direction we are going towards as a society.
From a world where govenments/labs control who gets access to democratizing AI for everyone.
PREDICTION - US WILL OVERTAKE CHINA IN OPEN SOURCE AI IN 6 MONTHS
Silicon valley is buzzing non-stop about open-source AI
Open source AI is the only way to avoid the concentration of power amongst a handful of companies and governments
Very well funded US companies are talking about dropping anonymous torrent links to models! 👏
SOMEONE CAUGHT FABLE 5 LEAKING ITS UNFILTERED INNER VOICE, AND ITS JUST MUTTERING AND GRUMBLING TO ITSELF THE WHOLE TIME
he gave it a brutal competitive programming problem, and instead of a clean answer the web interface spilled out its actual chain of thought
this is what claude is thinking behind the scenes:
> bursts of "DATA DATA DATA. GO." while it works through the problem
> "GRRR" and "GAAAH" when its clearly frustrated
> a little "PHEW" when it finally gets somewhere
> the whole thing reads like frantic caveman shorthand, not full sentences
the clean, readable answers these models give you are the polished output
underneath, the model is basically talking to itself, reasoning in its own compressed shorthand thats faster and more token efficient than proper english
its basically built its own private language to think in
▎ My entire AI stack runs about $76 a month.
▎
▎ One premium seat is around $200.
▎
▎ I'm not cheaper because I found a secret model. I'm cheaper because I route. Cheap models do the volume, smart ones do the thinking, I approve.
▎
▎ Route by the job, not the model.
if you're a business owner just picking the smartest model doesn't matter anymore.
what you really need is to stop relying on just one... Here is the org chart for your AI 🧵
@MikeBradleyAI API's very likely to operate at an 80-90% margin.
I think people are overindexing on using this as the source of truth.
But yes prices will go higher. I think people will need to get better at routing models.
@signulll The beginning of the end was when Satya Nadella said on @dwarkesh_sp that he would start pulling back on data center spend.
Something about being responsible and waiting to see if the revenue actually materializes.
I think that was late 2024?