Top Tweets for #SVDQuant

SVD is playing an increasingly important role in LLM. It has been widely used for model compression (e.g., our previous Dobi-SVD) and memory-efficient training (e.g., GaLore). Samir, Xiuyu, and Junxian take a creative step further by leveraging SVD to dynamically select a sparse subset of weights for loss and gradient computation. This approach reduces computational cost by up to 2.2× and achieves a measured speedup of up to 1.6× during FINETUNING — all while maintaining accuracy across a range of downstream tasks. Check Xiuyu's post and the website https://t.co/3evkjVfrN7👇