my honest reaction to terence tao's 'stop ai progress' talk at caltech on 9/11
full response video below
shoutout to @SAIRfoundation for uploading high res recordings
Come mothers and fathers
Throughout the land
And don't criticize
What you can't understand
Your sons and your daughters
Are beyond your command
Your old road is rapidly agin'
Please get out of the new one
If you can't lend your hand
For the times they are a-changin'
i miss the old ai
the alphago ai
mutual info ai
em algo ai
i hate the new ai
scale GPUs ai
no solid rules ai
gpt-2 ai
i love the stats ai
boost and bootstrap ai
affine y hat ai
no sandbox hack ai
Bread is hiring!
We're looking for two exceptional individuals to join our team in SF / Bay Area:
Developer Relations:
β Build a vibrant developer ecosystem
β Create killer tutorials/docs
β Python, ML, and OSS familiarity
β Attend conferences, speak publicly, and be an evangelist for our mission.
β 3+ years experience in a similar role.
Machine Learning Engineer
β Run experiments + bake models
β Develop good recipes & share best practices
β Python/PyTorch/huggingface
β Applied experience Prompt Engineering
β 2+ years experience training LLMs
If you want to give people greater control over their AI by making them learn more like humans, click the link below to learn more!
"We really care about being seen positively by society. We care about being in good standing. All these social intuitions that we have, I feel strongly that they're baked in."
Had not heard of context distillation when we wrote the paper back in 2024 but this is great stuff & way ahead of its time!
Our initial paper showed us that prompts could in principle be converted into weight updates β and surprisingly fast with new advances like LoRA, and increasingly capable open source LLMs.
We started Bread to take the next step: making LLMs learn at human speed, autonomously, from the real world, in the real world. That involves
- Taking the same data from inference time
- Using it to structure coherent weight updates repeatedly (like humans learning new concepts).
- Doing that usably and reliably at scale (distillation to + from many model versions)
Technically, that involves solving
1. Ingestion problem: how does incremental inference time data update the configuration of the bake?
2. Digestion problem: how do you structure updates to retain model coherence?
3. Learning infra problem: How do you build a fast training platform which depends upon a rapidly expanding repertoire of new models for on-policy updates?
4. Serving problem: how do you take advantage of GPU + bandwidth resources to serve inference & hot-swap new models created throughout inference time?
The first research paper only put us at the starting line, and our work at Bread is running the marathon.
Please check out the papers in this thread to see some early precursors to Baking! (wish we had known to cite them in the original paper). Itβs satisfying validation to see others recognizing the value in converting prompts to weight updates, and I celebrate these prior works from Askell and Snell which bolster our commitment to and confidence in our mission.
Why Prompt Baking is the only known method for sample-efficient, on-the-job learning.
Our new blog post explores why no other method can achieve the same sample efficiency and composability as baking in prompts, and why weβre placing our long-term bets on this approach for solving human-like on-the-job learning.
Announcing Bread Technologies. Weβre building machines that learn like humans.
We raised a $5 million seed round led by Menlo Ventures and have been building in stealth for 10 months.
Today, we rise π
Our round also included participation from Golden Ventures, Northside Ventures, and a number of angels.
None of this would have been possible without our early employeesβ extremely hard work and monumental technical lifts.
Special thanks to Alexander Detkov & Prof. Matt Thomson at Caltech for their support in the early research that led to Bread!