Rumor has it frontier labs are using looped LMs to scale quality without scaling parameters. But KV-cache memory still grows with every loop. Is that necessary?
No! Our method BeNice! unlocks near-lossless sharing of the final KV-cache across loops: more depth, same KV memory.
This lossless KV-sharing isnβt guaranteed out of the box with looped models. ππfor our method BeNice! to teach your model that sharing is caring!