We've published our preprint on Zenodo and are submitting to arXiv (https://t.co/wfY7rDtCgo): https://t.co/ok0nHbEylE
Open-source and reproducible under MIT License. Check it out and let me know your thoughts!
Cc: @EleutherAI @ggerganov@swyx@rohanpaul_ai @Raff_Edward
How do we bypass the quadratic KV-cache memory bottleneck on consumer GPUs?
I built Symphony: a constant-memory sequence modeling engine that processes a 43k+ context payload on an 8GB RTX 4060 with a flat, bounded VRAM footprint of 5.49GB (7B-4bit).
https://t.co/5WEhAdEcdX
2/ Symphony HEP-DNA: A recurrent coordinate pointer network. It tracks rare, high-value tokens (like API keys, variable names, and specific values) in the compressed space and executes exact token copy-paste retrieval.
Result: 100% exact retrieval accuracy with only a ~15% PPL