I prompted Claude Fable 5 to use Python to generate a 9:16 social video and render it using ffmpeg. Told it to put its own personal spin on it so it’s aligned with Anthropic’s launch and to fully express what it’s like to be an LLM hated by Theo from its POV.
It made this lmfao.
Day 1 of building GTA 6. Still feels fake typing that out.
Upgraded to Claude Max 20x just for this. Spent a couple hours getting the whole project structured and pushed to the repo. Sandbox is up and running.
No studio, no publisher. Just whoever shows up. We picked Godot on purpose: it's community-owned, so nobody can pull an EPIC on us later and rewrite the deal once we're invested.
The goal: beat the real GTA 6 to launch. Ambitious, probably stupid, doing it anyway.
If you can model, code, build levels, or write music and lore, come join.
Looking for a couple contributors to cook this.
Time to cook. Reply if you're in.
1/ You can shrink a language model's KV cache by 200×, in a single forward pass, and it still answers correctly.
At 256k context that's 36 GiB of cache down to ~360 MiB, with no change to the base model.
Here's how we did it 👇
the way humans do ai research is highly empirical, but it is possible there are very outsized theoretical and mechanistic improvements in model training. even the gradations of skill among human researchers mean some create 10,000x more progress given a compute budget. some invent the transformer or PPO
this is the ilya sutskever “age of research” bet, that you can find massive improvements on small models and small training runs. if Ilya thinks so maybe GPT7 and Claude Requiem think so too.
many of the brightest researchers don’t do fundamental deep learning research anymore. most have stopped being curious as to what a neural net is the way they used to be in 2017. probably because incremental engineering-based progress has been so guaranteed and low-hanging.
the rate and cost of progress today doesn’t necessarily predict the speed of RSI loop
@jeremyphoward@yacineMTB@bfockter@karpathy Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.
anthropic won't let you use fable for biology, chemistry, ai research, or anything that accelerates human progress. that makes it the perfect tool for developing blockchains