@RhysSullivan Using AsyncLocalStorage, AbortSignals, and promise helpers has been pretty good, and keeps my code compatible with most libraries and LLM code out-of-the-box, but very tempted to switch to effect...
Been burned before by backend API lock-in is why I'm hesitant.
Can a language model learn, end-to-end, what to keep in its own KV cache and what to throw away? Can it learn to forget while it learns to reason?
Deep learning's central lesson: capability emerges from end-to-end optimization, not heuristics/strong inductive biases. But for efficiency, we rely heavily on hand-designed approaches.
🗑️ Introducing Neural Garbage Collection (NGC): we train a language model to jointly reason and manage its own KV cache, using reinforcement learning with outcome-based task reward alone. No SFT, no proxy objectives, no summarization in natural language.
New paper with @jubayer_hamid, Emily Fox, and @noahdgoodman!
Welcome home Reid, Victor, Christina, and Jeremy! 🫶
The Artemis II astronauts have splashed down at 8:07pm ET (0007 UTC April 11), bringing their historic 10-day mission around the Moon to an end.
We're hosting a happy hour for ~50 developers on Accel's rooftop tonight at 6:30 and have a few spots left. DM me to get added or RSVP at the link in the comments
The biggest issue for me with agents is that they are hard to resist. But then you can build yourself some shit into the codebase that you get to regret in record time. And no, I don't think you can vibe yourself back to sanity with better models.