What comes after the transformer?
Over the last couple of months, we've been running these crazy interesting experiments, letting LLMs redesign neural nets as they train.
The architectures that emerge are an incredibly strange mixture of learned weights and spliced code.
I really recommend reading this one carefully. Very proud to have been part of this research.