I spent 48 hours with the Kimi K3 modeling code.
It took:
- 650 mg of caffeine (mandatory)
- 40 cans of LaCroix (optional... world record (?))
- 8 papers
- 6 months off my lifespan
Finally grokked the entire lineage of Kimi K3 and how we got here... every single step, since 2019 GPT-2