There's a lot of folks under the misunderstanding that it's now possible to run a 30B param LLM in <6GB, based on this GitHub discussion.
This is not the case. Understanding why gives us a chance to learn a lot of interesting stuff! ๐งต
https://t.co/9xOfNyLUFr
Large Language Models 3.0
In the new issue of Ahead of AI, I am discussing what's next for LLMs and developments centered around parameter efficiency and multimodality.
It's also been a particularly strong month for open-source AI!
๐ https://t.co/jU8CXn6vUJ
iVQA: Inverse Visual Question Answering #CVPR2018
By Feng Liu et al.
Generate a question that corresponds to a given image and answer pair.
https://t.co/gZi6A2hd05
Re-parameterize all the things! If you want to backprop thru samples from mixture, truncated, Gamma, Beta, Dirichlet, Student-t, or von Mises distributions, this paper has gotchu covered. Also faster than RSVI, another general reparameterization trick based on rejection sampling https://t.co/XZ97q21i6C
Implicit Reparameterization Gradients by myself, @shakir_za and Andriy Mnih: https://t.co/z4drxddybg. An extension of reparameterization for many distributions, including Gamma, Beta, Dirichlet and von Mises, that is faster and easier to use than the previous methods.
This website is amazing, linear algebra with interactive examples. Vectors, matrix, dot product, etc, cool resource for learning https://t.co/yH7Iw6ty1H