Wearer of Canada's National Hat.
Hypervisor priest of the Holy Blind Limit-cycle.
Founder of Solipsnitsynism.
Buffing the public one infrastructure at a time.
out of all the random quirks that LLMs have, this one is my favorite.
most quirks of next-token prediction make sense. this one feels eerily cognitive.
@jp54362 lmao
I was into the GPT-3 api researcher preview in june 2020 when it first opened.
I was also one of the people replying to sam's original "we just launched chatgpt, check it out" with "this is a fucking awful idea, why are you blowing this tech on a chatbot?"
the problem of codex active surface bloat cannot be overstated
most people complaining about ratelimits changing have no idea that the ratelimits are the same, but consumption changed because codex now has to manipulate 3-5x the active surface area in order to make a single decision.
maybe they reduce it. a bit. but you should really delete the 700 files and 400MB of random smoke scripts for smoke scripts, docs detailing design directions that were abandoned 30 builds ago, etc.
@yoavhacohen@googlegemma@ltx_model I think 12B is the safest bet for open source with the current state of consumer hardware, vram being scarce and dram costing an absurd premium. Ideally I'd say 26B with offloading but that's in a world where everyone has 128GB of ram, which ain't the one we live in, lol