Dusting off this tweet from April because this gap in shared understanding of LLM capability is *widening*. It's now less "two groups of people speaking past each other" and more a sharp funnel.
- Napkin math somewhere around 6B people (~75% of the population) have barely come in contact with LLMs at all.
- Around 1-2B (~20%) are casual and infrequent users of free-tier ChatGPT-like products. This group sees derpy chatbots and treats them a bit like a better Google search, a writing aid, or etc. Maybe an agent tries to book you a flight. I have non-tech friends who (reasonably, imo) say they have not much use for it at all. Even many professionals outside of math&code are in this tier. For example, execs and many other functions spend a lot of their time talking to other people, so while they understand what is happening intellectually, it is still second-hand and a bit abstract.
- Now we get to professional use of frontier-grade LLMs in math&code. Somewhere around 20M people (0.2%) see first-hand that large, complex projects that used to take them weeks/months can now be completed by agents with a prompt. Building apps, copying apps, translating apps, decompiling apps from binaries... This has all happened very quickly and recently - less than 1 year ago, I was writing code manually by hand, typing memorized computer code commands into a code editor character by character, occasionally pressing Tab to autocomplete a little chunk of code.
- And finally we get to the ~5,000 people (~0.00006%) with access to frontier-grade systems internally. The external world has seen the preview. It looks like swarms of thousands of agents collaborating over weeks on software mega projects: minting zero days, running cyber attacks and defenses at machine speeds, discovering new science, advancing the frontier of mathematics. Things that would have taken top professionals in the industry years of work. Meanwhile, human review and comprehension are starting to fall behind. For example, people are still involved in the "archeology" of the OpenAI-HF incident from many months ago. Mathematicians may be poring over the 722 manuscripts on frontier mathematics for a while.
The funnel is driven by a combination of factors:
1. The impact scales with ambition, problem size, and horizon. A question with a paragraph answer barely stresses the system. You need a reservoir of big, difficult problems that you really care about. This aspect drives the consumer / professional dimension of the funnel.
2. The jaggedness of the system (which I have written about a lot separately). Capability peaks in domains that are digital, verifiable and economically valuable. This is because LLM capability emerges from reinforcement learning on verifiable rewards on a curated environment mixture driven by revenue potential. This aspect primarily drives the area (e.g. math&code) dimension of the funnel.
3. Access. Free-tier, paid-tier, internal.
So this is the weirdness of the moment. The general public has mostly not interacted with these systems. When they have, it looks like a derpy chatbot. The majority of professionals still see only a modest uplift. And a small sliver of professionals are experiencing the vertigo of the curve going vertical. And it is all happening at the same time.
As tech moats diminish, human ones increase. CEO integrity is an under-appreciated moat. In the short term, being the smartest can attract the best talent. In the long term, being someone ppl trust will sustain it 👇
* @luke10ferrari (Bending Spoons) gets the first call from companies who want to sell. He has the reputation of telling the truth+never omitting what someone should here (a value learned the hard way by cheating on his partner in his 20s and feeling so ashamed)
* @alighodsi describes this as never being conflict averse
Many more examples here - would love to hear yours...
h/t Long Strange trip pod @bhalligan
Reading isn’t about storing facts; it’s about altering the lens. You might forget 90% of the pages, but the mind that closed the book will never perceive the world the same way as the one that opened it.
Lots of folks are dunking on Tristan here, but towards the end of the clip he points out a serious security issue. When you submit a grant proposal to NSF, you basically have to outline the research that you will do in the next few years with enough details that an expert in the field can assess whether it’s likely that you’ll complete it. All panelists in that group of reviewers have access to that proposal, and confidentiality is assumed and usually isn’t a problem. But if any panelist pops the proposal into an LLM (say, to summarize or assess), confidentiality has been broken and a private research plan has just been given to a for-profit competitor.
This is an accidental breach, but a serious breach still. Can’t OAI/ANT just scrape these inputs for ideas to try?
Reading a book in public is only performative to people that don’t read books. To a reader, every waiting room, park bench, train carriage, airport gate, and café table is another potential chapter.
Seth Rogen points out how wealthy people once built libraries, museums, parks and observatories for everyone to enjoy, with Keanu Reeves adding national parks to the list.
“Now it’s like, people don’t do that stuff anymore.”
Skydance CEO David Ellison says the combined company will spend more on content than any company in the world. Ellison: “More than Disney, more than Netflix.”