1/ This week's Q3 earnings call w/ $FB will likely go down as one of the biggest bets in history. IMO they'll spend >$170B over the next decade on building the metaverse. Since it's my largest position, I've been spending a lot of time reflecting. Attached are my "scratch notes"
@signulll You'd be surprised by how many things are not automated because of resource constraints.
For example, for a long time, the restaurants you could see when you opened the app and we were short on drivers were manually modulated by a human being in the Philippines.
@lpolovets I think this is like using the taxi TAM to determine Uber's TAM. Coding's TAM would encompass automating *most* business processes for knowledge workers. So likely trillions of dollars.
Of course, most of the future things to automate haven't been imagined yet.
Composer 2.5 being Pareto dominant in coding per CursorBench is important.
This is after only a few weeks of supplemental training and/or RL in the Colossus 2 cluster.
The 1.5 trillion parameter version of Grok will likely be a much better base model than Kimi. We shall see.
@dwarkesh_sp - Robert J Gordan (history of the American industrialization)
- Thomas Ashbridge (history of the crusades)
- Roger Crowley (history of the age of discovery)
- Kenneth W Harl (classical history)
- Dorsey Armstrong (history on black death)
@GavinSBaker noted that SpaceX might be one of the most important defence tech companies in the world right now.
Case in point.. look at Russian territory gains when Starlink access was cut off in Feb.
This is my sixth conversation with @GavinSBaker.
As always with Gavin, the conversation covers a lot of ground, but we spend the most time on watts and wafers.
We discuss:
- Why the wafer shortage may prevent an AI bubble
- Data centers in space (reframed)
- Elon's Terafab and the new chip companies challenging Nvidia
- Usage-based pricing
- The disaggregation of GPUs
- DRAM, frontier tokens, and open source
Enjoy!
Timestamps:
0:00 Intro
7:55 Anthropic and OpenAI Valuations
12:58 Watts, Wafers, and Infrastructure
14:39 Orbital Compute and Data Centers in Space
22:49 Avoiding the AI Bubble
28:26 Terafab and the Future of US Manufacturing
32:16 Returns to the Frontier
37:23 Continual Learning
42:03 New Chip Companies
48:52 Extending GPU Lifespans and Private Credit
51:22 The Application Layer
57:32 The Token Path and Open-Source Dynamics
1:01:37 Cybersecurity
1:05:46 Diversity Breakdown
1:11:59 Assessing the Big Tech Players in AI
1:19:02 Geopolitics, Personal Safety, and the AI Horizon
“Ukraine tallied Russian casualties at 35,351 last month, with drones causing 96 per cent of them while artillery and small arms fire accounted for the rest”
The Anthrophic Krishna Rao with @InvestLikeBest is great. Says they want to be a platform. Take Gates postulate that platforms capture small % of value for customers / ecosystem.
iPhone - $200B rev = $2T in value
Windows - $17B rev = $170B in value
Anthrophic - $45B rev = $450B in value?
Having spent the past few weeks in Beijing giving talks and attending meetings, here are some quick observations as I wait for my flight to NYC to board:
1. The talk of the town has, of course, been the Xi-Trump meeting, but no one (not even usually well informed elite circle insiders) seems to know what it actually accomplished, other than a continuation of the detente that’s been in place for the past several months. That’s about as good an outcome as one could realistically expect, I suppose, but clearly a real “grand bargain” is not in the cards anytime soon.
2. The Chinese economy seems to be in a steady state, neither improving much nor visibly deteriorating like it was in 24-25. In that sense the government’s stimulus policies have had a positive effect, but the vast majority of industry people I talked to remain very pessimistic about domestic profits and consumption. The dominant sentiment is that the only way for major firms to generate profit growth is through direct overseas expansion.
3. That said, technological advancement is of course very real and quite impressive (although it’s not quite as visible in Beijing as it is in, say, Shenzhen). One interesting and very pleasant side effect of the EV revolution (paired with infrastructure investment) has been that Beijing is now a bike-able city again, given the sharp reduction in exhaust fumes on city streets and the expansion of bike lanes. Armed with a new bike, I could almost explore the city like I used to back in 2000. Hugely nostalgic feeling.
4. Academia is, in general, in a pretty dour mood. STEM subjects and the social sciences/humanities alike have seen very significant funding reductions over the past 2 years, but the latter have of course gotten the worst end of the deal. Political censorship also seems to be visibly ramping up again, with the sheer scale of perceived “red lines” snowballing to levels unprecedented since the early 1990s. As the recent Yang Nianqun incident suggests, administrative regulation of faculty members’ personal affairs has also expanded (i.e., consensual extramarital relationships between adults who were not in a direct teacher-student relationship would almost certainly have gone unpunished as recently as 5 years ago).
5. In general, it’s hard not to notice the steady increase in government presence in everyday life—in both positive and negative ways. The city feels safer and cleaner than it ever has been, and yet the layers of administrative review needed for just about any kind of professional activity have clearly proliferated on a vast scale (made less painful by the digitization of most government services and more uniform law abidance, but still more onerous than it used to be despite all that).
6. The most alarming thing, I suppose, is that general optimism (personal or socioeconomic) seems to be in particularly short supply among the younger generations. This is obvious even among the most intellectually gifted kids at Tsinghua and PKU, where the level of career anxiety seems to be at a level that I have never encountered before. Unsurprisingly, willingness to form families or plan ahead in general at the personal level is very low.
All in all, it was, as always, a very informative couple of weeks. The stay was also made much more pleasant by the fact that I managed to do it before Beijing becomes brutally hot. I look forward to being back more often in the near future.
Codex with Computer Use is magical.. feels a lot like ChatGPT in 2023. Constantly have to stop myself midway through a task with "oh, Codex can do this".
@nachkari Could roughly go another 2-3 years before the regime wouldn't be able to pay fighters (which I would consider the only turning point that would garner capitulation from the regime).