...I'm working on
Andrej visualizes tokenization
& as (emojis) to grok what an LLM sees
Eric thinks there is an equivalence step
AFTER tokenization & before interpretation...
"token" sequences (canonical forms)๐ง๐พโโคโ๐โ๐ง๐ฟ
can be "equivalenced" to a single symbol/glyph ๐จโ๐ฉโ๐ฆ
To help explain the weirdness of LLM Tokenization I thought it could be amusing to translate every token to a unique emoji. This is a lot closer to truth - each token is basically its own little hieroglyph and the LLM has to learn (from scratch) what it all means based on training data statistics.
So have some empathy the next time you ask an LLM how many letters 'r' there are in the word 'strawberry', because your question looks like this:
๐ฉ๐ฟโโค๏ธโ๐โ๐จ๐ป๐ง๐ผ๐คพ๐ปโโ๏ธ๐โโ๏ธ๐งโ๐ฆผโโก๏ธ๐ง๐พโ๐ฆผโโก๏ธ๐ค๐ปโ๐ฟ๐ด๐ง๐ฝโโ๏ธ๐๐โโ๏ธ๐งโ๐ฆฝ๐งโโ๐๐
Play with it here :)
https://t.co/pFQGZIAW1k
@paul_tarvydas agree w/:
standing on the shoulders of giants. But the giants were solving single-CPU, synchronous, sequential problems...problems of the 21st century distributed systems, asynchronous networks, multi-core concurrency, real-time interaction are in a different direction entirely.
everything designed this way b/c
"the power of telling sequential unified narrative"
humans like to box things in to explain process causality
i.e. emergent non unified state never stops growing (expanding universe) hard explain causality
reduced unified state fits into memory
@paul_tarvydas another cool thing (still niche, doesn't get enough eyeballs / love) is "Processing In Memory" paradigm
(getting rid of CPU pipeline bottleneck)
https://t.co/zbXTkN97cD
So let me get this straight.
Jake Tapper is focused on attacking my Mom.
Jared and Ivanka are building a private island paradise on Albanian protected land.
Don Jr married the daughter of Epsteinโs banker, and a startup his fund backs just got a record $620M Pentagon loan.
Eric is taking an Israeli drone company public for $1.5B in the middle of a war with Iran that nobody wanted.
And I know: โBut what about your paintings, Hunter?โ
Please.
@HollyBriden I wrote a song for little Jasper:
๐ตHey Jasp, don't make it bad.
Take a sad song and make it better.
Remember to let her into your heart,
Then you can start to make it better.๐ถ
Israel has used American-supplied munitions to kill tens of thousands of innocent civilians.
America is morally obligated to end support of Israelโs devastation of Gaza and its people. Iโm cosponsoring the Block the Bombs Act to limit the transfer of offensive weapons to Israel.
Matt Brooks bragging that the Republican Jewish Coalition spent millions to buy a congressional seat in Kentuckyโฆ but if you observe the same thing, youโre antisemitic.
A normal developer sees a file with 1 billion rows.
@royvanrijn sees a challenge ๐
This talk is a wild ride through #Java optimization, profiling, memory mapping, parallelism, and shaving milliseconds off a seemingly impossible problem.
Watch: https://t.co/jCWJxpHeLX
@LocasaleLab Nature is the ultimate engineer. Researchers and biologists study at the feet of the Master and sometimes uncover a slice of infinite wisdom. This is where to find health and flourishing.
It has become impossible for a smart young person to be that longtermist without having to bullshit either investors or grant committees, and lose their focus and sincerity along the way.
When I quit academia at 36, the conflict between my intellectual ambition and my basic material needs was still unresolved.
in the 90s, the mayor of DC was arrested for having and using crack cocaine, was 100% filmed doing so, and went to trial for it. afterwards, he ran for mayor again and won. can't decide if this would be easier or harder today with social media and the present political climate.