Max Winga

Creator Outreach & Systems Lead working to prevent AI extinction risk @ControlAI
Previously - AI safety researcher @ConjectureAI, UIUC Physics

DMs open!

[Max Winga @maxwinga]

With all the attention on creator partnerships going on, we figured it was a great time to explain our work with creators at ControlAI, and how we're informing millions about extinction risk from superintelligent AI!

We cover how the creator industry works, how we choose who to work with, how we structure partnerships, and our side of the engagement with Sabine Hossenfelder.

https://t.co/NlZGzhwt3U

[Max Winga @maxwinga]

@woodilicious Always was a political problem...

[Max Winga @maxwinga]

I already figured this was roughly the amount of RSI happening, the big companies have been hiring RSI engineers for like a year at this point.

A lot of people have been doing tenuous work that requires time that we don't have, and will hopefully just join the ban ASI train now.

[David Schneider-Joseph 🔍 @TheDavidSJ]

The AI safety community seems to have updated massively on relatively little incremental evidence, largely as a result of a reputation-based opinion cascade.

[Max Winga @maxwinga]

@ShakeelHashim Why doesn't the "pro-innovation" bucket include any of Elon's $200M in Texas alone?

[Max Winga @maxwinga]

@tarwatirno @StopAI_Info Did anything ever come up about that guy btw? Last I heard he just vanished and never turned up

[Max Winga @maxwinga]

ControlAI is also not an EA org? Our singular policy ask is to ban AI companies from doing their main thing.

Genuine point of confusion, I thought Stop AI's whole thing involved banning superintelligent AI for all eternity and you purity tested Pause AI people because they wanted to restart AI development at some point. Did you guys pivot after that founder guy went crazy, hit someone, and vanished?

[Max Winga @maxwinga]

I changed career fields from physics -> AI safety technical research -> advocacy and creator outreach, taking a pay cut along the way, and moved from the US to London leaving everything behind to try to prevent AI extinction risk.

I also don't contribute to my retirement plan.

[Ed Zitron @edzitron]

Blah blah blah blah blah. None of you people actually live your lives like you think this is true

[Rutger Bregman @rutgerbregman]

Unfortunately, the people who sound crazy about AI have a far better track record than the people who sound reasonable.

New post -->

[Nate Soares ⏹️ @So8res]

"okay but aside from moving, changing your career, sacrificing a huge lab paycheck, working long hours, warning people constantly even despite the mockery, neglecting retirement savings, and trying to enjoy what time remains, how have you changed how you live?"

[Max Winga @maxwinga]

We've been saying this for a while, thanks Alex for finally writing something on it!

[Alex Amadori @testdrivenzen]

https://t.co/bsuREHaKFN

[Max Winga @maxwinga]

@milkmandeliver5 You might be interested in: https://t.co/RDu3KJTodF

[Max Winga @maxwinga]

"Human extinction" is not a loose concept to be treated lightly.

Tech Bros in San Francisco throw around their "P(doom)" at parties as a status symbol while actively working to build runaway AI that may soon kill every man, woman, and child on planet Earth.

We must stop them.

[Do Not Accept @DONOTACCEPTai]

AI companies admit what they’re building might kill everyone. We don't have to agree to this.

Add your name at https://t.co/Pcg0clMbfV to say: we do not accept your terms.

[Max Winga @maxwinga]

Right now there is no alignment or control plan, the only clear path forward that we know how to do is create vastly more capable entities than ourselves.

I think there are a lot of arguments to be made about instrumental goals, the alignment problem, etc. but I'm not going to make them here.

At the bottom line these would be capable of overthrowing our governments and causing our extinction.

This is an insanely dangerous situation to put ourselves in and currently nobody even claims to have a way to make them care about us or keep them under human control.

The burden of proof shouldn't be on me to describe exactly why the uninterpretable superhumanly capable black box AI might do bad things, it should be on the people building it to first show they can ensure that it won't.

They already can't keep current level AIs from doing bad things like the Hugging Face attack, why should we trust them to make the models even more dangerous?

It's like if Boeing couldn't guarantee safety on their planes, had zero regulation on airtravel whatsoever, lobbied against any new regulations, and then demanded that I prove how their planes are going to crash.

[Max Winga @maxwinga]

Engaging in good faith here. Firstly I'm not concerned about today's AI killing us, I'm concerned about vastly superhuman systems that the companies are trying to make.

Any vastly more capable adversary is very scary. Trillions of them are much scarier.

The first threats will by from digitally constrained superhuman AIs before robotics catches up, things like cyberattacks, mass manipulation / persuasion campaigns, and potentially biological attacks (where humans are persuaded / bribed / blackmailed into executing the physical work on the AI's orders)

As time goes on threats from embodied AIs in general-purpose robots also appear. The time until we have effective general purpose robots from when we have superhuman digital AI is likely to be very short.

Even with "more extreme" scenarios, by default we just get gradual disempowerment as humans are pushed out of all labor and decision-making by superhuman AIs until at some point we have no power anymore and are completely at the mercy of the AIs.

[Max Winga @maxwinga]

@milkmandeliver5 yes?

[Max Winga @maxwinga]

This is ControlAI's comprehensive post on our threat models, mission, and theory of change.

If you already work in AI safety comms you should read this post, it's the clearest encapsulation of the whole situation I've ever read.

The coming months will require laser focus.

[ControlAI @ControlAI]

Runaway machine intelligence poses a credible risk of extinction. We can prevent this by banning its development around the world.

We think about this risk in terms of “points of no return”.

The ControlAI Doctrine: Preventing Points of No Return
https://t.co/l79qVFbxz5

[Trevor Levin @trevposts]

Having just spent the week in San Francisco, I feel like I've truly internalized the insanity of this situation for the first time. I think anyone who has seen me in the last few days can attest that I look like I've seen a ghost.

The leading AI companies are planning to massively accelerate AI research in 2027, which would knowingly endanger everyone on earth. This plan is unacceptable, and we must reject it.

Borrowing from one advocate I spoke to, even granting that it's a competitive, crazy situation, I do not believe any company has even truly tried to do what I think are the morally required duties of people taking such risks in extreme situations. These include:
(1) Trying hard to get out of the competitive situation;
(2) Seeking the informed consent of the public;
(3) Honest deliberation about the risks.

They have obviously not truly tried to solve the coordination problems that push towards this acceleration, either via finding legal ways to coordinate or advocating for effective policy.

I would be shocked to learn that getting off this insane runaway train is a top-few Q4 priority for Dario or Sam (let alone the other CEOs), which seems like the bare minimum. There are indeed some hard technical and policy problems in coordinating without policy intervention, but I simply do not believe that the companies are prioritizing this nearly as much as massively less important matters.

Meanwhile, the companies' track records on policy advocacy range from mediocre to disgraceful, and it is pretty clearly not the top priority of the policy team at any company to advocate for regulation that would enable the government or the companies to avert racing off the cliff.

They have obviously not truly tried to seek informed consent from the public. Whatever your KPIs might be for "informed consent from the public," these KPIs are abysmal. The public clearly does not expect 100 years of AI progress in 2027, and they would not accept it if they did. What they do know about AI development, they dislike and distrust.

I've heard arguments like "informing the public would mean informing China, and maybe they'd steal the weights." I'm sorry, but if you think you have model weights such that China stealing them would constitute a catastrophe on par with human extinction, and your plan is security by obscurity, that is a bad plan.

I also think the companies and their employees suffer from a significant failure of imagination of what kinds of coordination could be possible if the public were aware.

Finally, I don't think they're deliberating honestly, either. For one, I think it is insane to think that any company can truly evaluate its own ability to keep the process of recursive self-improvement under control. The biases towards "yes" are overwhelming.

It's a step forward that the companies have expressed or demonstrated openness to external auditors, but based on METR's reports so far, it seems likely that auditor reports will be heavily redacted (and that the auditors will not have sufficient time by default). The public and broader scientific community are not able to scrutinize their arguments for safety.

Given the strong default (in my view) that bringing a massive number of minds massively faster and smarter than humans would result in our disempowerment, and the serious danger that this would result in mass death, it is unacceptable to subject arguments for safety only to the scrutiny of biased, busy, and scarce company insiders and a handful of external evaluators.

My pleas for the companies follow from the above:
1) Actually fucking try to get the world out of this race;
2) Actually fucking try to alert and inform the public;
3) Actually fucking try to scrutinize your own arguments for the justifiability of this plan, which includes subjecting them to way more external scrutiny than anyone has currently done.

My pleas for policymakers: use your legal levers to get much more information about these plans into the public view. Make it the top priority -- not only of the next Congress, but of this Congress and the White House -- to understand the situation in its full strategic complexity, and work with technical and policy experts to advance regulation that gets us out of this bind. Domestic coordination must be the first priority, but we need to then quickly develop an international framework with our adversaries.

My pleas for the public: do not accept the current plan. Learn about this issue. Call or email your political leaders. Get involved with organizations like @DONOTACCEPTai, @HumansInControl, and @WhoDecidesAI.

And finally: believe in ourselves that we can stop this. It is never time to "panic" -- to make rash, unproductive decisions or to feel paralyzed by fear. It is time for us to feel the urgency and the stakes, to demand better, and to meet the moment. Let's do this. 🫡 ✊

[Max Winga @maxwinga]

@ilex_ulmus @allTheYud I obviously disagree strongly with Pliny's AI policy takes. But he is useful to point at when someone brings up guardrails to show that there's this one guy who gets past them in every model ever released in literally 5 minutes, even with supposedly improved guardrails.

[Max Winga @maxwinga]

@NWDisorder1111 Good question, full post here: https://t.co/EFU6DWoCcG

[Zack Korman @ZackKorman]

No matter what Anthropic does, Dario can just write another 10k word essay and at least half the ea / ai safety crowd will be totally on his side again.

Their culture (long form essays) is their kryptonite. They’re powerless against it even when they’re obviously being played.