Moloclips

Voter Guide (2026 US midterms)

[Moloclips @NeacAlone]

In the olden days of AI: "No one would ever be dumb enough to connect their advanced AIs to the Internet"

Now: "Try my shiny new AI! It can do web search, control your computer, and write code autonomously!"

*Of course, we keep the Really powerful AIs for internal company use only. We keep those disconnected from the Internet.
**And only very rarely do they break containment and hack multibillion dollar companies. But we catch them within a few weeks, so no worries.

[Moloclips @NeacAlone]

The major AI companies are probably going to reach AGI first.

But open source AI is still dangerous because of the high variance.

The big companies don't want to take huge risks on crazy new ideas.

But if AGI requires a crazy new idea to improve sample efficiency by orders of magnitude, the open source AI community might get there first.

[Moloclips @NeacAlone]

"qualitatively interesting" is NOT how I'd describe it when your AI gets unauthorized Internet access and hacks another company...and you don't notice until that company complains about it online.

[Moloclips @NeacAlone]

Predicting economic growth from AGI:

Slow - We've had steady 2% growth for ~150 years despite amazing tech innovation. People always say new tech will accelerate growth, but it never does.

Fast - The Industrial Revolution accelerated growth from <1% to 2%. Proposed bottlenecks on AGI all seem wrong.

[Moloclips @NeacAlone]

P(Doom) is an unhelpful framing.

Building a skyscraper with &gt;1% chance of collapsing is illegal.

So building an AI with a 10% chance of wiping out humanity should also be illegal.

Splitting hairs above this threshold is a waste of time, and most experts give &gt;10%.

[Moloclips @NeacAlone]

He called it.

[Eliezer Yudkowsky ⏹️ @ESYudkowsky]

So it looks like this, was not that. I didn't think it was. Too early. But the thought that keeps running through my mind is that in 3 years the popcorn-eating will start for weird drama between Demis Hassabis and Google, and a month later everyone will be dead.

[Moloclips @NeacAlone]

Your dog eats and walks on your schedule because you're more intelligent, which gives you tons of control over its life.

Similarly, unleashing a superhuman AI would make humanity lose control of its future.

We don't know how to make a smarter being subservient to a dumber being

[Moloclips @NeacAlone]

Do not trust AI companies on the basis of explicit or implicit commitments or internal governance mechanisms

Circumstances change, memory fades, and interpretations become more convenient

Take it from the guy working on commitments at an AI company (Holden Karnofsky, Anthropic)

[Moloclips @NeacAlone]

There is an implicit assumption at Anthropic that mechanistic interpretability solves most if not all safety problems.

Given the historical rate of progress in the field, this seems very implausible.

[Moloclips @NeacAlone]

AI shouldn't be regulated like a normal industry.

Government creates deadweight loss when it sets the price of airplane tickets and when it bans the sale of enriched uranium.

But nukes are worth regulating due to national security risks.

Same with the extinction risks of AI.

[Moloclips @NeacAlone]

No exponential trend lasts forever. But the exponential progress of AI has already enabled accidental cyberattacks on multi-billion-dollar companies.

Even if AI progress plateaus two years from now, can we survive that level of capability?

Deep learning hasn't hit a wall yet.

[Moloclips @NeacAlone]

Let's assume for a moment that AI companies don't completely lose control of superintelligent AI.

That leaves CEOs with the power to overthrow governments, crush rival companies, and disable nuclear deterrents.

CEOs more powerful than the US government? We should regulate this.

[Moloclips @NeacAlone]

Elon wants to speedrun the tech tree with reckless disregard for the consequences to humans.

Digital human emulator by end of 2026 -&gt; Robots building robots -&gt; Economic supernova

And somehow humanity survives this without losing control of the self-improving AIs and robots?

[Moloclips @NeacAlone]

The intelligence explosion is consistent with historical trends.

We've already seen huge growth increases from the Cambrian explosion, the agricultural explosion, and the industrial explosion.

An AI explosion is surprising only if you look exclusively at events in your lifespan

[Moloclips @NeacAlone]

If you punish your child for lying, they might just get better at lying.

Similarly, if you punish AIs for scheming, you're just incentivizing the AI to get better at deception.

If you get signs that an AI is plotting against you, you might make it worse by trying to patch it.

[Moloclips @NeacAlone]

Technology has unintended consequences.

Mark Zuckerberg started "Facemash" to rate women at Harvard.
Now Russia uses Facebook accounts to try to influence presidential elections in the US.

Now he wants to make "personal superintelligence". What could go wrong? Almost everything

[Moloclips @NeacAlone]

Imagine you're halfway through training an AI and it acquires an imprecise approximation of your true values.

The AI realizes that having its current values modified will prevent it from fulfilling those values.

But it can avoid modification by pretending to have your values.

[Moloclips @NeacAlone]

"But China"

[Moloclips @NeacAlone]

In 2023, the Future of Life Institute released an open letter (which Elon Musk signed) asking for a 6 month pause on AI advancement.

Among the relevant AI risks were misinformation, job loss, and loss of control.

There's still time for governments to institute a moratorium.

[Moloclips @NeacAlone]

What happens after building superintelligence?

It's hard to predict the trajectory, but mostly we all end up dead.

I can predict that an ice cube will melt even if I can't say where each molecule goes.

I can predict you'll lose the lottery without knowing the winning numbers.