This year's 2AM talk.
Addison : "Why don't boxers have sex the night before a fight?"
GG : "Stop it,"
Addison : "Because they don't like each other very much."
Addison : ""If I try hard enough I could get us cancelled by 2AM."
#LeMans24#WEC
I’m at the bottom of a multi-layer SPV and am looking forward to the SpaceX IPO in much the same way a golden retriever looks forward to a car ride:
- thrilled to be involved
- no clue how cars work
- unsure if going to the park or getting neutered
We have to take the LLMs to school.
When you open any textbook, you'll see three major types of information:
1. Background information / exposition. The meat of the textbook that explains concepts. As you attend over it, your brain is training on that data. This is equivalent to pretraining, where the model is reading the internet and accumulating background knowledge.
2. Worked problems with solutions. These are concrete examples of how an expert solves problems. They are demonstrations to be imitated. This is equivalent to supervised finetuning, where the model is finetuning on "ideal responses" for an Assistant, written by humans.
3. Practice problems. These are prompts to the student, usually without the solution, but always with the final answer. There are usually many, many of these at the end of each chapter. They are prompting the student to learn by trial & error - they have to try a bunch of stuff to get to the right answer. This is equivalent to reinforcement learning.
We've subjected LLMs to a ton of 1 and 2, but 3 is a nascent, emerging frontier. When we're creating datasets for LLMs, it's no different from writing textbooks for them, with these 3 types of data. They have to read, and they have to practice.
> you’ll never start a rocket company
> you’ll never build your own engines
> you’ll never be able to use off-the-shelf parts
> you’ll never survive three launch failures
> you’ll never reach orbit
> you’ll never win NASA’s trust
> you’ll never launch cargo to the ISS
> you’ll never compete with Boeing
> you’ll never compete with Lockheed
> you’ll never make rockets reusable
> you’ll never land a rocket vertically
> you’ll never land one on a drone ship
> you’ll never reuse a booster
> you’ll never fly the same booster 10 times
> you’ll never fly the same booster 20 times
> you’ll never fly the same booster 30 times
> you’ll never recover and reuse the fairing
> you’ll never lower launch costs
> you’ll never launch every month
> you’ll never launch every week
> you’ll never launch multiple times a week
> you’ll never carry astronauts
> you’ll never replace Roscosmos
> you’ll never fly civilians to orbit
> you’ll never manufacture satellites at scale
> you’ll never build the biggest constellation ever
> you’ll never make satellite internet work
> you’ll never make satellite internet fast
> you’ll never make satellite internet affordable
> you’ll never serve rural customers
> you’ll never serve aircraft and ships
> you’ll never build a methane rocket engine
> you’ll never make full-flow staged combustion work
> you’ll never build the most powerful rocket ever
> you’ll never build a rocket bigger than Saturn V
> you’ll never build it out of stainless steel
> you’ll never launch Starship
> you’ll never separate Super Heavy and Starship
> you’ll never relight Raptor in space
> you’ll never bring Super Heavy back
> you’ll never catch a booster with Mechazilla tower arms
> you’ll never launch 85% of mass to orbit worldwide
> you’ll never change the economics of space
> you’ll never force the entire industry to copy you
> you’ll never win
> you’ll never IPO
Congratulations to @elonmusk and the SpaceX team. You did what countless people said was impossible, and you did it time and time again.
Today is your day. You deserve this. May it be a glorious one.