Matthew Tromp⏹️

We need to stop frontier model development now before everyone on earth die

[Matthew Tromp⏹️ @matthewktromp]

@krishnanrohit Should we therefore assume we can solve them, with no plan for how to solve them, and when we've clearly failed to solve easier versions of the problem? Should we, like Jensen Huang, oppose any measure to delay the problem to give us more time to figure it out?

[Matthew Tromp⏹️ @matthewktromp]

@brentmichaelcox I absolutely agree. If we survive this, and if there's any justice in this world, everyone working at these companies will go to prison. At this point, the only hope someone working at these companies has of redemption is blowing the whistle before we shut them down

[Matthew Tromp⏹️ @matthewktromp]

Nobody in tech who thinks about AI for any length of time thinks things will be okay, or normal. I regularly have conversations with people in tech who come up to me to argue, and by the end they almost always conceed "well yes this will almost certainly massively disrupt society and possibly kill billions of people, but it won't kill literally everyone, that's excessive"

If you're a normal person, and you think that your life will continue on as normal even if this technology keeps getting better, you need to understand that basically nobody in this industry believes that!

[Matthew Tromp⏹️ @matthewktromp]

Ah see the problem is, that gives it the *ability* to find novel cancer treatments, but that doesn't mean it necessarily *wants* to find novel cancer treatments. It might want to do something else entirely! Like how you can force someone to go to math class and learn math but that doesn't mean they'll grow up to be a mathematician. They might not like doing math!

Figuring out how to get smart AI systems to want to do something, instead of just knowing how to do it, is called alignment, and it's a problem we don't know how to solve

[Matthew Tromp⏹️ @matthewktromp]

Day 1 of standing outside the UN with a big sign until something happens

This worked extremely well

- I had probably 50 people with UN or Press badges take pictures with/of me
- gave an interview with one journalist and my contact information to another
- someone with a very large entourage read my entire sign
- met Alex Bores on his way to a meeting about AI

The attitude from people I met was almost universally sympathetic. People are worried

Good day

[Matthew Yglesias @mattyglesias]

It’s absurd that we have folks literally on the payroll of Meta and A16Z talking about how well-funded the AI safety community is while touting the obviously self-interested takes of NVIDIA’s CEO.

[Matthew Tromp⏹️ @matthewktromp]

@MontyTaroko I'm not an expert on geopolitics. But I can tell you that if anyone builds this thing, it's going to kill everyone on earth

And to be clear I'm not talking about current models: what we need to do is stop training

[Matthew Tromp⏹️ @matthewktromp]

@PezeshkiCharles Yeah man. That's, uh, that's kinda the whole problem. AI is getting too good

ASI aside, aren't you worried about a world where you can't tell what's real?

[Matthew Tromp⏹️ @matthewktromp]

Accelerationists will really be like "stopping ASI? That pales in comparison to my strategy, shutting down the biolabs" and then not shut down the biolabs

[Matthew Tromp⏹️ @matthewktromp]

@Sockppp1 @Oxandrolonely How do you propose training it to optimize for human well-being? Because even the companies themselves say they don't know how to do that

[Matthew Tromp⏹️ @matthewktromp]

Anyone who does this will go down as a hero in the history books. Now's your chance

[Robert O'Callahan @rocallahan]

I resigned from Google today.
I enjoyed my work and loved the people, but my GDM team was working on a new generation of chips to make AI much faster and cheaper, and I think AI is already progressing too fast, so I had to quit.

[Robert O'Callahan @rocallahan]

I resigned from Google today.
I enjoyed my work and loved the people, but my GDM team was working on a new generation of chips to make AI much faster and cheaper, and I think AI is already progressing too fast, so I had to quit.

[Matthew Tromp⏹️ @matthewktromp]

@AblemanAdam I'd need a bigger sign for that

[Matthew Tromp⏹️ @matthewktromp]

@thesandruns Why do you think superhuman AI will be nice to you? Why do you think it will do what we say? Why do you think it will care about us? Especially when even the people building it don't think it will be, on the current trajectory

[Matthew Tromp⏹️ @matthewktromp]

Superhuman AI won't cure ageing because it won't care about us. False choice

And even if we do want to build it, does it have to happen right now, when the AI companies themselves are saying they don't have the situation under control and are begging for things to slow down? It couldn't wait even a couple years?

[Matthew Tromp⏹️ @matthewktromp]

@thesandruns What do you think the outcome will be of building a new form of life, vastly smarter than us, which doesn't care about us?

[Matthew Tromp⏹️ @matthewktromp]

@thesandruns I've met AI company employees and they are normal, law abiding citizens who for the most part want to slow down anyway (see https://t.co/XGCC1lf7Ll). Of course they would stop

[Matthew Tromp⏹️ @matthewktromp]

@thesandruns Well I'm not a pussy so yeah

Remember, the alternative is death

[Matthew Tromp⏹️ @matthewktromp]

@Rudo1518568 Not currently!