It’s because large language models are trained on the average of all ideas. When prompted correctly, especially with Claude models, you can get them to come up with new things, but it’s difficult
The AI he’s describing is something that is not an LLM, but we already have all of the technology to run it
OpenAI more stores prompt cache for codex *permanently*.
Think about that for a second.
The prompt cache includes prompts, code, etc.
Anything codex sees.
I hope OpenAI means it’ll be kept for 12 hours and they’ll eat the cost afterwards.
We’ll see if Tibo clarifies.
Day 16/
You can now cache for 12h in the API and for logged in users across Codex & Work we keep cache forever and you don’t have any penalty for coming back to an old conversation or resuming after a few hours and are now guaranteed to hit the cache.
@scaling01 If it’s faster and cheaper per token I don’t really mind the token consumption, it still ends up faster and cheaper overall
Even if the official provider is slow there are others (baseten) with 5x the speed of sol per token, it balances out
And as you mentioned, swarms