perhaps you should have devs use regular $100/$200 usage plans part of the time @bcherny@lydiahallie@trq212
having unlimited toks internally at all times disconnects devs from users
Like Facebook in 2015 when Zuck made devs use 2g on “2g tuesdays”, which led to Lite products
@mattpocockuk I’m currently working on a hook to grab used context & tokens to compaction into the context after every PostToolCall. I want to instruct it to write out notes for itself before compaction.
Trying to get a human to respond from @HeyBilt is a great way to waste time. Someone has hard coded “59 minutes” to the support service and it has no relationship to reality.
New 3h31m video on YouTube:
"Deep Dive into LLMs like ChatGPT"
This is a general audience deep dive into the Large Language Model (LLM) AI technology that powers ChatGPT and related products. It is covers the full training stack of how the models are developed, along with mental models of how to think about their "psychology", and how to get the best use them in practical applications.
We cover all the major stages:
1. pretraining: data, tokenization, Transformer neural network I/O and internals, inference, GPT-2 training example, Llama 3.1 base inference examples
2. supervised finetuning: conversations data, "LLM Psychology": hallucinations, tool use, knowledge/working memory, knowledge of self, models need tokens to think, spelling, jagged intelligence
3. reinforcement learning: practice makes perfect, DeepSeek-R1, AlphaGo, RLHF.
I designed this video for the "general audience" track of my videos, which I believe are accessible to most people, even without technical background. It should give you an intuitive understanding of the full training pipeline of LLMs like ChatGPT, with many examples along the way, and maybe some ways of thinking around current capabilities, where we are, and what's coming.
(Also, I have one "Intro to LLMs" video already from ~year ago, but that is just a re-recording of a random talk, so I wanted to loop around and do a lot more comprehensive version of this topic. They can still be combined, as the talk goes a lot deeper into other topics, e.g. LLM OS and LLM Security)
Hope it's fun & useful!
https://t.co/75mXcUBI8L
Can I just say I loooove Suno. Some of my favorites:
Dog dog dog dog dog dog dog dog woof woof
https://t.co/3yWAqFGDe3
Chemical elements
https://t.co/p7EEc4iYgd
train_gpt2.c header (who did this lol)
https://t.co/6gz25sxiKA
Suno tutorial (in Suno!):
https://t.co/vN5lPa55Tg
Many others. So good. Anyone else favorites?
How to be as "smart" as Auto-Regressive LLMs:
- memorize lots of problem statements together with recipes on how to solve them.
- to solve a new problem, retrieve the recipe whose problem statement superficially matches the new problem.
- apply the recipe blindly and declare victory.
- do not use basic logic.
- do not use common sense to check your solution.
- do not use a mental model of the situation as a sanity check.
- do not simulate the scenario in your mind using your world model.
- when someone tells you your solution is wrong, reply "I'm sorry, you are right" and apply another irrelevant recipe.
Knowledge accumulation is not a substitute for actual understanding.
Gemini 1.5 pro is STILL under hyped
I uploaded an entire codebase directly from github, AND all of the issues (@vercel ai sdk,)
Not only was it able to understand the entire codebase, it identified the most urgent issue, and IMPLEMENTED a fix.
This changes everything