With many 🧩 dropping recently, a more complete picture is emerging of LLMs not as a chatbot, but the kernel process of a new Operating System. E.g. today it orchestrates:
- Input & Output across modalities (text, audio, vision)
- Code interpreter, ability to write & run programs
- Browser / internet access
- Embeddings database for files and internal memory storage & retrieval
A lot of computing concepts carry over. Currently we have single-threaded execution running at ~10Hz (tok/s) and enjoy looking at the assembly-level execution traces stream by. Concepts from computer security carry over, with attacks, defenses and emerging vulnerabilities.
I also like the nearest neighbor analogy of "Operating System" because the industry is starting to shape up similar:
Windows, OS X, and Linux <-> GPT, PaLM, Claude, and Llama/Mistral(?:)).
An OS comes with default apps but has an app store.
Most apps can be adapted to multiple platforms.
TLDR looking at LLMs as chatbots is the same as looking at early computers as calculators. We're seeing an emergence of a whole new computing paradigm, and it is very early.
Perception is fundamentally compositional. We make sense of novel stimuli (things we're not used to) by parsing its elements, then composing them following a known template.
The left image and the right image are the same, rotated. Note how you interpret the upside-down face.
We’re introducing GSLM, the first language model that breaks free completely of the dependence on text for training. This “textless NLP” approach learns to generate expressive speech using only raw audio recordings as input. Learn more and get the code:
https://t.co/kRkUaFyZWb
I built a UI creator in Figma using @OpenAI's GPT3.
Define your UI component in simple English, and GPT3 + @figmadesign will create full blown mockup for you. With accurate text, images and logos.
Very impressed with GPT3! It is absolutely incredible⚡️
How it works? Read on...
Twitter is funding a small independent team of up to five open source architects, engineers, and designers to develop an open and decentralized standard for social media. The goal is for Twitter to ultimately be a client of this standard. 🧵
Our new work uses episodic memory to learn language tasks in a lifelong setting. The episodic memory is used in two ways: sparse experience replay during training and local adaptation during inference.
Paper: https://t.co/xbwgSH1ntS
Cyprien, @seb_ruder, @ikekong & @DaniYogatama
Here’s how we trained an 8.3B parameter GPT-2. We alternate row- and column- partitioning in the Transformer in order to remove synchronization and use hybrid model/data parallelism. 15 PFlops sustained on 512 GPUs. Details and code: https://t.co/7eXA6r15yX
I’m always amazed by the disconnect between what we see in the news and the reality of the world around us. As my late friend Hans Rosling would say, we must fight the fear instinct that distorts our perspective: https://t.co/uQRofM4q2u
Thought-speaking: decoding speech from neural activity. Very nice work. https://t.co/VgXMIUFXrR Great to see another instance of TensorFlow being applied to neuroscience. Full PDF is on BiorXiv: https://t.co/1Hm7ktuPdb
Listen to heavy ions collide in the #LHC! 🎶🎇
Data from the ATLAS detector has been transformed into a symphony of sounds via Quantizer, a @medialab sonification platform. Listen to even more ATLAS collisions LIVE at https://t.co/4twDWsUlkv.
Check out a @naturemethods paper where we and our collaborators present a new type of recurrent neural network that can improve the accuracy of automated interpretation of connectomics data by an order-of-magnitude over previous deep learning techniques. https://t.co/CnYbJ6VLAj