Top Tweets for #longcontextLLM

5 months ago

#TurboQuant's rotation trick got absorbed by standard quants. Now #KVarN shifts the #LongContextLLM quality-per-memory curve by a full tier, real benchmarks vs hype https://t.co/xyL3fj8HVF

SOUVIK KUNDU @thisissouvikk

over 1 year ago

#ICLR2025 #mamba #longcontextLLM #SSM 🚀🚀Can Mamba be extended to support long context data without training or tuning of the pre-trained weights? To answer this intriguing and pressing question we did a bit digging in to the: 👉General understanding of Mamba towards long context 👉Key bottleneck identification in terms what plays a critical role in Mamba's context forgetful-ness, as we keep on increasing context size! 👉The impact of scaling of the discretization matrix of Mamba towards their performance impact on long context 👉Finally come up with our proposed solution of "MambaExtend" to improve their long context understanding ability! 📌OpenReview Paper link: https://t.co/CZglO7y20w 📌Code: will be available soon! In collaboration with the @USCMingHsiehEE @USCViterbi

thisissouvikk's tweet photo. #ICLR2025 #mamba #longcontextLLM #SSM
🚀🚀Can Mamba be extended to support long context data without training or tuning of the pre-trained weights?

To answer this intriguing and pressing question we did a bit digging in to the:

👉General understanding of Mamba towards long context
👉Key bottleneck identification in terms what plays a critical role in Mamba's context forgetful-ness, as we keep on increasing context size!
👉The impact of scaling of the discretization matrix of Mamba towards their performance impact on long context
👉Finally come up with our proposed solution of "MambaExtend" to improve their long context understanding ability!

📌OpenReview Paper link: https://t.co/CZglO7y20w

📌Code: will be available soon!
In collaboration with the @USCMingHsiehEE @USCViterbi

383

Top Tweets for #longcontextLLM

Last Seen Hashtags on Sotwe

Trends for you

Most Popular Users