What's really going on in machine learning? Just finished a deep dive using (new) minimal models. Seems like ML is basically about fitting together lumps of computational irreducibility ... with important potential implications for science of ML, and future tech...
https://t.co/OfevpJezi7
To help explain the weirdness of LLM Tokenization I thought it could be amusing to translate every token to a unique emoji. This is a lot closer to truth - each token is basically its own little hieroglyph and the LLM has to learn (from scratch) what it all means based on training data statistics.
So have some empathy the next time you ask an LLM how many letters 'r' there are in the word 'strawberry', because your question looks like this:
👩🏿❤️💋👨🏻🧔🏼🤾🏻♀️🙍♀️🧑🦼➡️🧑🏾🦼➡️🤙🏻✌🏿🈴🧙🏽♀️📏🙍♀️🧑🦽🧎♀🍏💂
Play with it here :)
https://t.co/pFQGZIAW1k
@poxspi Eureka (from Ancient Greek εὕρηκα) is the awesome feeling of understanding something, of feeling it click. The goal here is to spark those moments in people's minds. Labs because Eureka all by itself is taken... and I always wanted a lab 👨🔬
I recreated an entire GPT architecture in a spreadsheet.
It is a nanoGPT designed by @karpathy with about 85000 parameters, small enough to be packed into a spreadsheet file.
It is great for learning about how transformer works as it shows all the data and parameters going through a transformer pipeline, and all the calculations are actually working inside those cells.
Here is a link to the project where you can download the file and play with it yourself. No coding needed, it is just a spreadsheet. 😉
Spreadsheet is all you need:
https://t.co/dreZlBDq4d
Also thanks to @BrendanBycroft with his LLM visualisation project which inspired this.
Just released Speech-to-Roam, an extension to directly transcribe vocal notes in @RoamResearch using Whisper API.
Speak also to an AI assistant, provide some context by selecting blocks and instantly get the response in your graph !
Here is a demo:
✨ UPDATES:
Been a long time since my last Bitcoin illustration
I've been building a Bitcoin platform! Almost ready to show it off:
1) Illustrations: Visuals to simplify complex Bitcoin topics and BIPs.
2) Bob GPT: Need more clarification ? Ask an AI assistant for all your Bitcoin-related questions.
3) Bitcoin developer RoadMap: Step-by-Step guide to master Bitcoin development
4) Get Daily summaries of active discussions in the Bitcoin-Dev and Lightning-Dev mailing list
I'll be building this platform openly and sharing it's development publicly, so stay tuned for regular updates on its progress
MVP is almost done. Follow along! ✨
When will @matter or @readwise support listening to and highlighting PDFs? still my largest pain point and seems like should be doable with the OpenAI APIs now right?
I know how much of a pain PDFs are to work with but surely AI can help us here
Do you know that @RoamResearch already supports side notes as proposed by @EdwardTufte by its block views?
I'm currently experimenting with a better UI for them in combination with my Roam Studio plugin. What do you think?
“Mendel's concept of the laws of genetics was lost to the world for a generation because his publication did not reach the few who were capable of grasping and extending it”
“Science has provided the swiftest communication between individuals; it has provided a record of ideas and has enabled man to manipulate and to make extracts from that record so that knowledge evolves and endures throughout the life of a race rather than that of an individual.”
“If the aggregate time spent in writing scholarly works and in reading them could be evaluated, the ratio between these amounts of time might well be startling.”