Alex Brogan

rendering 🇦🇺🇺🇸🇬🇧

[Alex Brogan @alexbrogan]

the type of innovation that can happen when the founder owns 100% of the business and answers to no one

[Polymarket @Polymarket]

JUST IN: Dyson launches a $499 AI toothbrush with a built-in camera that identifies gaps between teeth & automatically squirts them with mouthrinse.

[Alex Brogan @alexbrogan]

if there's even a 0.01% chance of self-improving AI taking over humanity, all other concerns should be secondary — crazy to see so many conspiracy theories about this

the only question that matters is whether it can get away from us

OpenAI was founded after Elon thought Google wouldn't take safety seriously. Anthropic was founded because Dario et al. didn't trust OpenAI to take it seriously

now they are both racing to create superintelligence with little regard to safety, with the justification that China will create it first and take America's global hegemony

if OpenAI and Anthropic build something they think is dangerous because "China will do it anyway," they need to be extremely sure China will do that — create a recursively self-improving system — not just keep training models

this is even more baffling because the labs (looking at you Anthropic) keep saying that China is distilling or copying their models. In other words, China is keeping up by copying, not overtaking

if that's true, a slowdown at Anthropic/OpenAI would slow China down too. So:
a) why on earth are we using "China will do it anyway" as the justification?
b) even if we don't trust Beijing, why does that distrust outrank the existence of humanity? China's posture to AGI to-date has arguably been more conservative than the US'
if our technological progress continues the way it's going, in the next 30 years, we will soon have industry, energy, and settlement move off-world, and Washington versus Beijing in relative positions will be a relic of the past. Great power competition still matters, but it does not outrank keeping the first self-improving systems from destroying the option value for the rest of humanity
to those saying "show me the mechanism of how AI gets out of hand," the control/intelligence explosion problem has been written about for almost 60 years, starting with Norbert Wiener in 1960 and I.J. Good in 1965

to the frontier labs (Anthropic/OpenAI) fighting for economic supremacy, realize that, in its current state, AI is already going to be economically transformative. You will capture enormous rents without crossing self-improving superintelligence. Take a breath, pause, and realize your decisions are determining the future of humanity

every person alive today and everyone who will ever be should have the same first incentive: that humanity continues with prosperity and agency intact

Blaise Pascal argues that you should live as if God exists. If you're wrong, you lose a few finite pleasures. If you're right, you gain eternity and avoid going to hell

when the downside of an action is catastrophic and irreversible, and the cost of caution is small by comparison, you do not wait for certainty

we do not have to go far back in history to find a time—COVID-19—where the world neglected warning signs, and the people who waited for proof ended up at the mercy of whatever happened around them

the risk of superintelligence that we cannot control is that same risk magnified to infinity. We are not wagering with only millions of lives. We're wagering with everyone's, forever

if we slow down and the danger was overstated, we only lose time and revenue. If we race ahead and it wasn't overstated, we will not get a second attempt

the world will little note, nor long remember what we say here, but it can never forget what @hilbertspaess did here

[Jacob Coxon @hilbertspaess]

I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.

[Alex Brogan @alexbrogan]

this is going to end badly

[Sheel Mohnot @pitdesi]

Instinct looking to raise $1B, potentially at a $10B valuation.

Compute costs are high and Noah doesn’t want to charge users for the product.

Instinct has raised $350M to date, mostly uses open source models and wants to own chips and data centers

https://t.co/Mrz65nRvSm

[Jamie Cox @jimcox25]

Our convictions

[Alex Brogan @alexbrogan]

@clemissima @danedisonsiegel I don't think the space is big enough...

[Michael Millerman @millerman]

It's clear now that the problem of intelligence was always on the supply side

[Alex Brogan @alexbrogan]

@oneill_c congrats mate - sorely needed!

[Alex Brogan @alexbrogan]

@danlovesproofs @abhargava20

[Alex Brogan @alexbrogan]

@pavelprata @murphcapital @tanthaip @briannekimmel @crisraygoza_ @MartinGTobias @julianweisser that first photo is incredible

[Jack Clark @jackclarkSF]

"we believe the world would benefit if the industry adopted a lawful, verifiable, effective mechanism for coordinated pacing as soon as possible."

[Alex Brogan @alexbrogan]

@majamediaco @_AdamMiller_ @maxmarchione @mudithj @Sachin_Shah_ @JoumanaElomar @jaelubberink_ @alexvalente @willsclips_ @clarejtbirch @oneill_c new term coined: "brain circulation"

[Alex Brogan @alexbrogan]

“I held fast to the notion that until someone has repeatedly said no and adamantly refuses another word on the subject, they're in the process of saying yes and don't know it.” — Dee Hock

[Alex Brogan @alexbrogan]

@sharifshameem some of my favourite books on this:
Where good ideas come from
Borrowing brilliance
American Independent Inventors in an Era of Corporate R&D
The Myths of Innovation
A technique for producing ideas
The Idea Factory

[Alex Brogan @alexbrogan]

it is a golden time to be in SF

[Kevin L. @kevin_jy_lu]

North Beach and new benchmarks for hosting set by @JoumanaElomar

[Alex Brogan @alexbrogan]

@pavelprata thank you for coming my friend!

[Alex Brogan @alexbrogan]

@JoumanaElomar @Sachin_Shah_ @mattboustred @lucy_tanAU @kevin_jy_lu @_AdamMiller_ best view in sf

[Alex Brogan @alexbrogan]

@elonmusk @NecessaryForm

[Matt Stockton @mstockton]

Agree and also disagree. It's truly shocking what happened. If you dig into the OpenAI or metr technical report (like, truly read the PDF), what happened is pure science fiction. As someone fully down the rabbit-hole, I was shocked.

But you have to remember what the public discourse on AI looks like. It's still all like "Will AI take jobs away?" or "Will AI use too much water?" or "Data centers are bad" or "Yeah I have used ChatGPT and it seems a little better than Google"

The reach of frontier AI is so limited outside of the people reading this post right now. Even for folks on the 'leading edge' in GlobalCos, it's not that sophisticated. Like probably most people still haven't experienced AI that can search through their files and do a thing.

Imagine the media trying to explain what happened to the general audience. Imagine the media even truly understanding the technical reports so that they could communicate it effectively.

I agree that people need to know, but I don't know the best way to do that. It is weird times because the understanding gap is just getting bigger, faster. Eventually something has to give here, but I honestly do not know what this all ends up looking like. Accelerated and compounding AI is a very weird thing.

[Philippe Lemoine @phl43]

I watched this the other day and what I find the most striking is that, for the most part, both of them have a purely supply-side approach to forecasting the revenue of OpenAI and Anthropic.

Even Dylan, who is less bullish than Dwarkesh, seems to think that only politics and financial bottlenecks can prevent the kind of hypergrowth that Dwarkesh is talking about, but demand is basically taken for granted.

Nor is this specific to them, to be clear, I made the same reflection to myself countless times in conversation with AGI-pilled people or reading some of the stuff they write.

I constantly find myself wishing that people who believe OpenAI/Anthropic will generate trillions in revenue within 2-3 years explained to me where, in the kind of scenario they envision, that demand is supposed to come from concretely.

Dwarkesh's basic intuition seems to be that AI will soon be able to fully automate many white-collar jobs, so to get a sense of how much revenue OpenAI/Anthropic can generate by replacing them, you can just look at the wage bills for those workers, but that's not how things work.

First, if AI can fully automate most tasks performed by those workers but even a handful of them prove resistant to automation, AI will still generate a lot of value but it will presumably be captured mostly by the remaining workers, their firms or their customers and the current wage bill for those workers will massively overestimate the future revenue of frontier labs.

In that case, demand for AI may not explode in the way Dwarkesh and Dylan expect, because it will be bottlenecked by humans in production.

But let's assume, as Dwarkesh and Dylan presumably do, that AI capabilities will improve so much that it can automate every task those workers perform and their jobs can be fully automated. It still doesn't follow that demand will be as high as they think.

Even when it has become technically possible to replace most white-collar jobs with AI in principle, before it can happen in practice, there will be enormous transition costs and frictions as firms will have to undergo massive reorganization. This will be very slow even if politics doesn't interfere, which it will.

Dwarkesh's argument implicitly assumes that frontier AI companies will be able to charge about the same for tokens that provide services equivalent to that of the worker being replaced, but as long as there is enough competition, the prices they can charge will be pushed down and if he's right about how fast capabilities will improve they may fall very quickly.

Will there be enough competition? Well, if things continue as they have looked so far, OpenAI and Anthropic are ahead of everyone in capabilities but other companies are not that far behind.

So for the price they can charge not to fall very rapidly, you have to assume that the gap in capabilities with their competition starts increasing quickly and that customers of AI services continue to be willing to pay a huge premium for access to state-of-the-art models, which is hardly obvious.

The fact that it’s true now doesn’t mean that it will still be true in 2-3 years, when the capabilities of non-frontier models have improved, the user base of AI services has expanded a lot and they collectively consume a much larger volume of them.

Now, even if prices do fall rapidly, this could in theory be compensated by increased demand in volume. But there is only so much legal, managerial, engineering, etc. services that people want, even if prices fall.

The price elasticity of the demand for those services will eventually fall and, even if improved capabilities shift the demand curve outward, this will only generate so much demand because no matter how good your lawyer gets and how low his rates fall, at some point you don't need more of his services.

Now, obviously with improved capabilities AI may also make it possible to develop new services for consumers, which in turn will create new sources of demand for AI companies.

But even if that's true, and on a long enough time horizon I have no doubt that it will be true, it will still take time before these new markets can generate a large amount of demand for AI services.

That's because even if you assume that AI capabilities improve so much that in a couple of years AI itself will be able to come up with ideas of new services that can take advantage of it, you will still need to let people know about those services, convince people to use them, change their behaviors, etc. and this stuff will take time because however good AI gets those people will still be humans and not AI.

Which brings me to another big issue I have with the kind of purely supply-side analysis Dwarkesh and Dylan do in the podcast, but which again is totally standard in that space, namely that if as many people as they assume lose their jobs because they have been fully automated it's not clear where the demand for the consumer goods and services produced by the customers of Anthropic and OpenAI will come from.

I already mentioned that a while ago, but the thing about AI agents is that even if they reach the point where they can fully replace human workers on the production side, they can't replace them on the consumption side because they don't consume anything except compute and a few B2B services.

The income will still go to someone, but they will probably have a lower propensity to consume and different spending patterns, so there is a macroeconomic problem here about where the ultimate demand for the output of consumer goods and services produced by the customers of OpenAI and Anthropic will come from.

I guess in principle it might be possible to address that with the right public policies, such as fiscal redistribution, but even putting aside the fact that in the kind of scenario Dwarkesh and Dylan envision it's not obvious because a lot of the aggregate demand will be lost in other countries whereas the value will be captured by Anthropic and OpenAI in the US, it will take a while before people can agree on what must be done and make the political deals to do it.

This is also politics getting in the way, but in a very different sense from what they seem to have in mind, which is more something like politicians making it more difficult to build data centers because their constituents think they use too much water or whatever.

Even assuming that the demand for AI will be at the level they assume, it’s not obvious that OpenAI and Anthropic will be able to retain a large part of it as profit, as opposed to upstream suppliers like chip manufacturers, power companies, etc. The relevant question is kind of the production side analogue of the “what will be scarce” question that @alexolegimas asked on the demand side.

Even if you assume that the elasticity of substitution between state-of-the-art and less capable models will still be pretty low even after the latter’s capabilities reach a certain point, which as I already noted is hardly obvious, it’s still not clear that upstream suppliers will not be in a position to capture most of the scarcity rent.

This is a somewhat disorganized rant, but the point is that, for the kind of scenario Dwarkesh and Dylan have in mind (i. e. OpenAI and Anthropic generate several trillions in revenue in 2-3 years) to materialize, many highly non-obvious assumptions have to hold.

I have no doubt that possible replies to the arguments I made could be made, I can think of several myself, but this is just a tweet and it’s already way too long. My own view is that it’s extremely unlikely all those assumptions will hold, but I don’t even care if you disagree with me.

What I would really like to convince AGI-pilled people of, because again I don’t think Dwarkesh and Dylan are being atypical here, is that if they want to make the argument that the kind of timeline they’re defending is plausible there are tons of implicit assumptions that need to be made explicit and a case needs to be made for them.

The fact that people rarely do that frankly suggests to me that, for the most part, they don’t even realize that they need to make those assumptions. They seem to think that people who don’t find their timeline plausible just “don’t know what is coming” and are in denial.

But while it’s probably true that a lot of people are in denial about what AI is capable of I also think AGI-pilled people have very naive views about how improved model capabilities will translate into economic change.

Again, I can hardly be called an AI skeptic, I’m convinced that AI will profoundly transform human civilization and that it will do so much faster than the Industrial Revolution did. I just don’t think it will do so as fast as many people in that space believe.

Maybe I’m wrong, but if people think so, they need to make their case more rigorously because the claims they make are very strong and very strong claims require equally strong arguments. This ain’t it.

[Alex Brogan @alexbrogan]

@Sachin_Shah_ DM'd