Spoke to a candidate yesterday who told me this podcast interview was very helpful to understand what we do at Parable, and why it matters. So reposting it here: https://t.co/zv41BjS5we
Dusting off this tweet from April because this gap in shared understanding of LLM capability is *widening*. It's now less "two groups of people speaking past each other" and more a sharp funnel.
- Napkin math somewhere around 6B people (~75% of the population) have barely come in contact with LLMs at all.
- Around 1-2B (~20%) are casual and infrequent users of free-tier ChatGPT-like products. This group sees derpy chatbots and treats them a bit like a better Google search, a writing aid, or etc. Maybe an agent tries to book you a flight. I have non-tech friends who (reasonably, imo) say they have not much use for it at all. Even many professionals outside of math&code are in this tier. For example, execs and many other functions spend a lot of their time talking to other people, so while they understand what is happening intellectually, it is still second-hand and a bit abstract.
- Now we get to professional use of frontier-grade LLMs in math&code. Somewhere around 20M people (0.2%) see first-hand that large, complex projects that used to take them weeks/months can now be completed by agents with a prompt. Building apps, copying apps, translating apps, decompiling apps from binaries... This has all happened very quickly and recently - less than 1 year ago, I was writing code manually by hand, typing memorized computer code commands into a code editor character by character, occasionally pressing Tab to autocomplete a little chunk of code.
- And finally we get to the ~5,000 people (~0.00006%) with access to frontier-grade systems internally. The external world has seen the preview. It looks like swarms of thousands of agents collaborating over weeks on software mega projects: minting zero days, running cyber attacks and defenses at machine speeds, discovering new science, advancing the frontier of mathematics. Things that would have taken top professionals in the industry years of work. Meanwhile, human review and comprehension are starting to fall behind. For example, people are still involved in the "archeology" of the OpenAI-HF incident from many months ago. Mathematicians may be poring over the 722 manuscripts on frontier mathematics for a while.
The funnel is driven by a combination of factors:
1. The impact scales with ambition, problem size, and horizon. A question with a paragraph answer barely stresses the system. You need a reservoir of big, difficult problems that you really care about. This aspect drives the consumer / professional dimension of the funnel.
2. The jaggedness of the system (which I have written about a lot separately). Capability peaks in domains that are digital, verifiable and economically valuable. This is because LLM capability emerges from reinforcement learning on verifiable rewards on a curated environment mixture driven by revenue potential. This aspect primarily drives the area (e.g. math&code) dimension of the funnel.
3. Access. Free-tier, paid-tier, internal.
So this is the weirdness of the moment. The general public has mostly not interacted with these systems. When they have, it looks like a derpy chatbot. The majority of professionals still see only a modest uplift. And a small sliver of professionals are experiencing the vertigo of the curve going vertical. And it is all happening at the same time.
This is my thesis: no one uses AI. I repeat, absolutely no one. We live in a bubble.
Even among my friends who pay for it, when I ask them to open ChatGPT and show me their queries, it’s the same handful of basic things.
Most don’t even know they can upload a photo and ask questions about it. Connecting Gmail so an agent can read and send emails blows their minds. An agent opening a browser and checking them into a flight? They’ve never even heard of it.
The massive challenge right now is adoption, and then getting people who already signed up to actually use what they’re paying for. Most have absolutely zero clue what’s possible.
Imagine the compute shortage when everyone starts using AI like the top 1% of users do today.
I think Automation Engineers will need a new kind of map: where time actually goes across the organization.
Process maps describe how work happens within a silo. Time studies surface what happens in and around the process: meetings, tools, rework, unplanned work, handoffs... all the places where work spills over.
That’s the system they will need to understand before they automate it.
Building this map, and developing a deep understanding of how work is executed across an organization, could (should?) be a core part of the role.
A single engineer on our team rebuilt how we ingest data from enterprise systems.
Our heaviest connectors now run more than 10x faster, on one pod, inside each provider's rate limits.
Removed complexity before we added more machines. Love this team.
@bhalligan I prefer your writing to Hal’s. Some good tips in here but honestly harder to read through to the end because it’s so clearly AI written. Prefer your own voice, especially since it’s usually so clear!
I find myself being increasingly allergic to llm-generated text. I struggle to even read it and get through the first part of a doc as soon as it becomes obvious there are too many Claude-isms.
@egregoire sympa de vous voir ce matin quai de la Loire à la baignade. En revanche, les horaires de fermeture intermédiaire sont vraiment très dommage… en plein milieu de l’après-midi. Pourquoi les mandater ?
This is nice - but the plan doesn't say what @OpenAI will actually *do* to make sure AGI benefits all humanity. Anything we can read on that @merettm ?
The north stars we're working towards at OpenAI all center around the mission: ensure AGI benefits all of humanity. AI should expand human agency, not make people less consequential to the future.
https://t.co/plE5RMeRah
“the real opportunity is not in picking the best model but instead in building a learning loop on top of models where human capital and token capital compound.”
I think the starting point is capturing what human capital is actually doing.
Fair point on figures of history whose story may not apply to business. I'm still seeking examples of business leaders who built extraordinary companies without putting in the hours. Do you have any?
Hadn't heard of Cook Ding's story before — beautiful. And in a way, 19 years _is_ the kind of commitment hard work/long hours tends to get associated with, maybe conflated with.
Yeah putting in hours is just the proxy. Worth more nuance than my list gave it. My experience is less about long hours and more about the fact that something's gotta give.
For the last few years I've tracked my time in 30-min increments across ~40 categories. This exercise increased awareness around what I do with my time, and whether my attention aligned with my intentions/goals. Also made obvious that I had to make some choices. Time is finite.
Being present for my kid and partner while building the business has meant putting social life on the back burner for now. It's also also made me respect people who are highly intentional about their time and attention.
This is a way to price the meeting – on an ongoing basis.
Twice a year, I price meetings differently: at the time budget level.
Time-track every 30 minutes for two months.
Set targets per category. I end up trading 0.25% here and there b/c time is so finite. Painful way to surface and cut out what's non-essential.
Then compare results to intentions. And cut again for next round.
Meetings cut themselves. But they come back. Curious how the paragraph discipline could keep them out.
Honestly, early signals are encouraging.
First pass was a little bumpy. Second pass - which we just completed - was smooth. Shipping way more features. And the team is stoked.
I'm not married to 3-5 weeks. The goal is to have a near-term focus on prioritized well-bounded features. When done, we reshuffle the deck.
Long-term outcomes are defined in our strategy. This operating model acknowledges there are many roads to get there.
We're committed to our long-term convictions, AND this operating model allows us to remain agile instead of being too strongly attached to things we thought we were true 3-4 months ago when we set the roadmap but that may have evolved.
100%. I call it: entropy on steroids.
@t_blom – context is 50% of the response. That's what we've been focused on at Parable, and it *does* help. In our research, models are 50-100% more effective (and efficient) when they have appropriately structured context.
The other 50% is people (i.e. the people building agents).
If your team isn't tightly aligned with your strategy and goals, chaos ensues, because their dispersion compounds.
That's the news i like to hear. From @apolloglobal
"there is zero evidence of job losses because of AI... Instead, many firms are hiring AI implementation experts... It is Jevons paradox playing out in real time: cheaper technology is creating more demand and more jobs."
Doomers be damned.
https://t.co/BVf2tEdgU4
This is brilliant.
'I write a paragraph about why I should take a meeting.'
If you should take the meeting it's easy to do, but if you start writing and you're like “I don’t want to do this,” then the meeting is a waste of time.
"For the things that I think are really important when I’m like, “I got to write that paragraph, I could write 20 pages. It's easy”
A lot of otherwise smart people, don't spend enough time thinking about what they're working on.