To help explain the weirdness of LLM Tokenization I thought it could be amusing to translate every token to a unique emoji. This is a lot closer to truth - each token is basically its own little hieroglyph and the LLM has to learn (from scratch) what it all means based on training data statistics.
So have some empathy the next time you ask an LLM how many letters 'r' there are in the word 'strawberry', because your question looks like this:
👩🏿❤️💋👨🏻🧔🏼🤾🏻♀️🙍♀️🧑🦼➡️🧑🏾🦼➡️🤙🏻✌🏿🈴🧙🏽♀️📏🙍♀️🧑🦽🧎♀🍏💂
Play with it here :)
https://t.co/pFQGZIAW1k
Em 2013, Ronaldo inaugurou a academia do FC Dinamo Tbilisi. Hoje, 11 jogadores georgianos lá formados podem defrontar o Ronaldo no Euro 🤯
#PartilhaAPaixão
wow. The new model from @LumaLabsAI extending images into videos is really something else. I understood intuitively that this would become possible very soon, but it's still something else to see it and think through future iterations of.
A few more examples around, e.g. the girl in front of the house on fire
https://t.co/wDiCirpmUa
Imagine a massive celestial bubble made of ice and dust surrounding our Solar System. That’s the Oort cloud and this is its scale.
The Oort Cloud lies at the very edges of the Solar system. Even the farthest ones which are still working, like Voyager 1, Voyager 2 and New Horizons didn't go beyond the cloud.
Long-period comets, taking centuries to orbit the Sun, provide evidence for the existence of the Oort Cloud. Scientists believe this vast region is the birthplace of these cosmic travelers.
Have you ever wanted to train LLMs in pure C without 245MB of PyTorch and 107MB of cPython? No? Well now you can! With llm.c:
https://t.co/PoGTZIwASL
To start, implements GPT-2 training on CPU/fp32 in only ~1,000 lines of clean code. It compiles and runs instantly, and exactly matches the PyTorch reference implementation.
I chose GPT-2 to start because it is the grand-daddy of LLMs, the first time the LLM stack was put together in a recognizably modern form, and with model weights available.