Federico Alvetreti

[Maximo Rulli @MaximoRulli]

Read the full details in our paper!

Preprint: https://t.co/JHNq8C6Xmr

Big thanks to the amazing team: @tvfontanari @simone_petruzzi @FedericoA45200 @krytox205 @DonatoCrisosto1 @GiorgosNik02 @tommaso_mncttn @teelinsan @EmanueleRodola @s_scardapane @devoto_alessio

(6/6)

[Yuxiang Huang @yxyxyyy6]

[1/n] Can a model learn *where* and *how much* information it should attend to, and do so efficiently?

We introduce DashAttention: Differentiable and Adaptive Sparse Hierarchical Attention!
This pushes the accuracy-efficiency frontier in LLMs.