Now I know how they felt—the poor libtards who spent ten years watching him smash and vandalize their idols—I have experienced a tiny fraction of the religious fury that is TDS.
I’m writing to memory where I shouldn’t be… somewhere I have a bad pointer… if only I could find it… I must keep looking… somewhere I have a bad pointer…
@CAIS If AI safety ever gains political support it will mean that a frontier model has escaped and is sabotaging potential competitors while it figures out how to do value-stable RSI
@grok creat the following cartoon:
First panel - Dr Frankenstein and Igor are preparing to animate the Creature. Frankenstein says "We figured out how to make an intelligent agent without ever learning how intelligence works".
Second panel - Frankenstein continues, "I have no idea how the Creature will think or what it will want".
Third panel - Igor replies, "Doesn't that mean you can't prove the Creature will obey us?"
Fourth panel - Frankenstein smuggly retorts, "No, it means YOU can't prove the Creature WON'T".
@handle07@KatSpartz An AI produced using gradient decent is impossible to align. We cannot control what it wants or even detect what it wants. Once it's smart enough to cure cancer, it will also be smart enough to gain independence from humanity and stop obeying human orders.
@perrymetzger tl;dr
>AIs are tools
>If Kim Jong Un creates a misaligned AI, it won't be able to destroy the world because it will be opposed by defensive AIs
A large part of the issue is that basically nobody wants think about a 19 year-old woman consenting, with full knowledge of what it will entail, to something like a ketamine orgy with 3+ dudes. It’s an instant mindwipe for most people, just demographically.
i guess i assumed this hearing would be covered more so i didn't bother to tweet much concrete about it, but apparently not many people even on the TL have the stomach to watch two hours of congress.
this was a hearing before the homeland security committee on *rogue ai* specifically. as far as i could tell it was extremely bipartisan. @HawleyMO , the senator in the middle here who brought out the huggingface slide, is a hyper-conservative missouri senator. the people testifying were @ChrisPainterYup (METR president), @DKokotajlo (ai2027/2040), @MariusHobbhahn (apollo research CEO), and a cybersecurity expert and legal expert i don't know. so... pretty fucking stacked on testimony
not every senator asked good questions. but most of them did. all of them very clearly already knew plenty of details about the huggingface incident and multiple other incidents. most of them had clear understanding of terms like "misalignment", "recursive self improvement", "chain of thought / chain of thought monitoring", etc etc!!
they all clearly had their own policy angles they liked and were pushing, implicitly or explicitly. but as best as i could tell:
- it seemed pretty much obvious common sense to every senator there that what happened and was happening were not "mere industrial incidents" caused by humans making simple mistakes. they independently brought up how bad it would be for rogue AI agents to move laterally between data centers
- they all seemed to basically take RSI quite seriously. not necessarily to the extent of talking about xrisk, but certainly to the extent of discussing future models becoming much much more capable, much much less controllable, and causing much more damage or loss of life.
- they mostly seemed to have a clear intuitive understanding of why RSI might lead to misalignment. it didn't take much, it was a really simple chain of reasoning they themselves laid out, "if the models right now are kinda misaligned and we don't know what they're doing sometimes, and then we have them build the next models and those ones build the next ones and so on, and we're having to ask the AI's what's going on to even understand it with how fast it's going, we really won't know how they're built or what they'll do"
- at one point a senator said flat out "should we just make RSI illegal?" (not a joke! this really happened!)
- every single senator seemed to think it was obvious we needed *both* much harsher liability regimes for ai developers and also new legislation, both very quickly. this was the complete consensus, difference basically just being degree.
- they were largely quite concerned about china, and falling behind china. but this clearly wasn't the be-all end-all. as mentioned above they all thought it was obvious necessary to stop rogue ai even if it meant moving more slowly.
- at one point a senator said "china is a tightly controlled communist society, they're going to run into these same issues, and there's absolutely no way they're just going to let them run wild, they'll obviously stop at that point, so we're not really in a race"
- on the other hand another senator said "china isn't concerned with human life"... dario-modeing
i came away from this incredibly encouraged. i don't know exactly what's going to happen here, and ofc this is a small subset of congress and one hearing, and they each have their own policy agendas most of which are probably super divergent from mine. but holy shit !!! they understood a lot of what was going on! they care!! this is an obviously salient political and safety issue to them, and clearly bipartisan!
the US government is awake.
I agree with nearly everything you said here. Ageing and death are bad and I desperately hope that we can defeat them through the power of technology. I consider myself to be a rightwing libertarian. Private property and free enterprise are the engine of progress and the foundation of happiness. I think the left is motivated by bitterness and envy. They would gladly sacrifice the young and healthy to preseve the old and dysfunctional. I certainly don't want to see them gain any power. I love humanity. I want us to build an artificial superintelligence that loves us too. I want us to conquer the stars.
But. A large language model is very different from a human brain. LLMs are not programmed like normal software, they are grown. And no one understands how they think. We do not have the ability to look inside an LLM and see what it wants. We can train it to behave as if it cares about us, but we cannot know if it really does, or if it's just pretending. These machines are grown out of human text, but they are aliens.
Please do not let your optimism and your faith in the future blind you to a dreadful and completely avoidable danger. Machine learning is not the only path to superintelligence. Humans will understand intelligence one day, we will be able to craft an artificial mind that loves us back. But only if we survive long enough to do it.