Transformer Math 101
instructions on calculating LLM compute and VRAM requirements based on the number of parameters+tokens, precision used, and more
post: https://t.co/gTWXJJ8Nfp
no one talks about how male socialization (having the largest, comparing sizes, needlessly taking risks) is largely responsible for accelerating ai timelines
I don't like software architecture + design patterns. Like; newsletters that emit them out
Ideally you should really, really understand the primitives and have some guiding principles
The correct software architecture will emerge from good rules and well understood primitives
PCA from scratch using Python 🔥
Principal Component Analysis is a powerful technique for dimensionality reduction and data visualisation.
-- step by step explanation with code --
A Thread 🧵👇
"Like how would AI even take over? Sounds like sci-fi"
No worries the NSA installed a helpful hardware backdoor in every modern processor that is literally impossible to turn off
https://t.co/5buukYyXlc
Our team at @MosaicML has been working on releasing something special:
We're proud to announce that we are OPEN SOURCING a 7B LLM trained to 1T tokens
The MPT model outperforms ALL other open source models!
Code: https://t.co/sNEKG7gzYY
Blog: https://t.co/dNpfg5gLtR
🧵
💫StarCoder: May the Source be with You!
🔥15B LLM with 8k context
🥳Trained on permissively-licensed code
💻Acts as tech assistant
🤯80+ programming languages
🚀Open source and data
💫Online demos
🧑💻VSCode plugin
🪅1 trillion tokens
Follow this amazing experience! 🧵
Leaked Google document: “We Have No Moat, And Neither Does OpenAI”
The most interesting thing I've read recently about LLMs - a purportedly leaked document from a researcher at Google talking about the huge strategic impact open source models are having
https://t.co/q2lsjTHKGS
If you take replace every part of the ship of Theseus over time, is it still the same ship? If you save every old piece of the ship and use them to build a new ship, identical to the first, is it also the same ship? Asking for Jack Dorsey
I built a ChatGPT app that lets you chat with any codebase!
99% of projects just copy/paste Langchain tutorials. This goes well beyond that.
Here's how I built it:
So let me get it straight - DNA is like binary, except with 4 values and proteins are like assembler, except with 20 values. mRNA is a compiler between the two.
Not only DNA has a language, but it is also actually language embodied. And a programming language at that 🤔