Scott Alexander

I have a place where I say complicated things about philosophy and science. That place is my blog. This is where I make terrible puns.

[Scott Alexander @slatestarcodex]

I prefer to think of it as the late pre-truth era.

[Scott Alexander @slatestarcodex]

No I'm not! See the section of the post starting "The third-biggest risk comes from", which explains that part of the motivation for the pacing is that if we get it right, we can monitor giant data centers only, while avoiding the level of authoritarianism involved in interfering with normal consumer hardware.

[Scott Alexander @slatestarcodex]

Congratulations to @DKokotajlo and team on releasing Plan A (https://t.co/OIly7QxiUs) , a followup to AI 2027. My attempt at a gentle introduction to the ideas involved is at https://t.co/6X6Kjuv2l7

[Scott Alexander @slatestarcodex]

@jdysn4 @DKokotajlo I'm not really sure what you're getting at, but to answer the question: if 2023 counts as "pre-LLM boom", then see https://t.co/OLRFNBb1qb , where I write about a paper whose 90% confidence interval for takeoff speed is between 0.8 - 15 years.

[Scott Alexander @slatestarcodex]

@Goolic @DKokotajlo Crazy shut-in terrorists mostly don't own large data centers, so this isn't the main threat model. But yes, Plan A makes them even less likely to own large data centers than usual.

[Scott Alexander @slatestarcodex]

@timberwind @DKokotajlo > panopticon government
> it's regulating one product

(also, it's just AI chips, a small fraction of all chips)

[Scott Alexander @slatestarcodex]

@Chaos2Cured @DKokotajlo Actually, this time we're hope-mongering with ZERO proof of claims!🎉

[Scott Alexander @slatestarcodex]

@__tinygrad__ @DKokotajlo We just want to know where they are so we can appreciate them better ❤️. We think you'll find much to like in the Plan.

[Scott Alexander @slatestarcodex]

@Goolic @DKokotajlo https://t.co/E8GZkXsjW8

[Scott Alexander @slatestarcodex]

@DLM93987699 @DKokotajlo If you read the plan or the blog post, you'll see that this is a plan to delay things in 2040, not a "correction" saying it will actually be 2040. For the latest timelines from AIFP, see https://t.co/t96A2mOwZc

[Scott Alexander @slatestarcodex]

@jdysn4 @DKokotajlo The world's leading AI lab did publish an article saying that RSI had already started within their company, that seems like a pretty big one to me. See https://t.co/iZ3lBIeKln . But yes, this is still speculation based on extrapolating existing trends.

[Scott Alexander @slatestarcodex]

@full_kelly_ @DKokotajlo That's not really accurate - I've gotten in a few stupid fights on Twitter, and want to make sure that if I make enemies for myself I'm not making them for AIFP as well.

[Scott Alexander @slatestarcodex]

@QuinsoLo0 @DKokotajlo See the last section of my blog post "A Is For All Of Us", which is something of a response to this accusation.

[Daniel Kokotajlo @DKokotajlo]

In AI 2027, we predicted that AI would take over the world or irreversibly concentrate power.

In AI 2040: Plan A, we've laid out our positive vision for what should happen instead.

[Scott Alexander @slatestarcodex]

As I said in the article - "You can’t say something like “it gets 85% of questions right”, because that depends entirely on question difficulty. If the questions are things like “will the sun rise tomorrow morning”, then even a 100% hit rate is unimpressive. Instead, we can only match different forecasters against each other and determine who is better or worse. Any anchoring in an absolute space will come from the inclusion of groups whose predictive abilities we intuitively understand (eg the average member of the public, CIA analysts, etc)."

The most commonly-cited stat is that on geopolitical questions, superforecasters beat CIA analysts even though the latter had access to extra classified information. In an ACX contest, the median superforecaster who chose to participate placed in the 70th percentile of ACX readers who chose to particpate, and an especially well-known superforecasting team previously identified as the best placed in the 98th percentile. Metaculus (which aggregates superforecasters) did better than Nate Silver/538 at predicting the 2022 election (first one I found good data on), although that's not a fair metric of individual superforecasters.

[Scott Alexander @slatestarcodex]

Yes, that is the study I linked in my post, and the one I described in my original response to Sterling which he insulted.

Our whole discussion is about whether it's just this one study reporting an in silico simulation that has never successfully been pulled off in real life but maybe will someday, or whether there's more to it.

I haven't been able to get him to give a clear answer.

[Scott Alexander @slatestarcodex]

@SterlingCooley I would accuse you of being thick-skulled, but I suppose under your world-model that's impossible.

[Scott Alexander @slatestarcodex]

@microfounded @cremieuxrecueil It's vacuously true that if an exciting new technology works, good things will happen. I don't think this excuses us from discussing the question of whether it will work, how much it will work, and in what applications it will work.

[Scott Alexander @slatestarcodex]

That's not a stable equilibrium. The person who boosts every new technology and the person who pooh-poohs every new technology blend into identical meaningless noise.

Experts and journalists have already faded partway into noise because they were too obsessed with promoting what they considered socially beneficial ideas at the expense of their duty to explain and investigate. But we're almost as far gone as they are! Half the world thinks AI is a scam, and if you ask them why they'll say something like "I don't believe Silicon Valley because it always hypes things" or "They overestimated crypto, they're probably doing it again".

The only way out is ignoring all this discussion of which values to privilege, and ruthlessly telling the truth.

[Scott Alexander @slatestarcodex]

Although one can imagine a world in which the medical establishment works completely differently, and a world in which we have more technology than the current world, these are totally different problems from putting ultrasound in a ring.

I think if you put ultrasounds in a ring and say "this will change everything", and you mean "this will change everything if we additionally completely rework the medical system and invent lots of new unrelated technology, and even then it's the reworking and the new technology that will change everything and the ultrasound is mostly irrelevant", then it is fair for people to highlight the unspoken premises of your argument and make it clear that you're being misleading.

[Scott Alexander @slatestarcodex]

It is of course their right to spend their money however they want, but if they release an article implying false things, I think it's fine to correct them.

If you want to develop something in secret without the public having any opinion, don't make a giant splashy campaign to advertise it to the public asserting specific facts about the product!