We’re sharing the first phase of our research @constructor_agi. Our mission is to solve continual learning and develop models that learn natively from experience.
Unlike LLMs, humans don’t reread their entire context; with experience, we learn what matters. Our work is a promising first step in that direction: it shows that a model improves when it learns where to look.
Introducing Link: Teaching Attention to have intention.
Link is a learned, sparse read that lets an LLM choose what to read. At multi-million context lengths, it outperforms dense attention on retrieval accuracy by 53% at 4M token context lengths, alongside 9.9x GPU efficiency.