Ben ten Berge

[Tenobrus @tenobrus]

i guess i assumed this hearing would be covered more so i didn't bother to tweet much concrete about it, but apparently not many people even on the TL have the stomach to watch two hours of congress.

this was a hearing before the homeland security committee on *rogue ai* specifically. as far as i could tell it was extremely bipartisan. @HawleyMO , the senator in the middle here who brought out the huggingface slide, is a hyper-conservative missouri senator. the people testifying were @ChrisPainterYup (METR president), @DKokotajlo (ai2027/2040), @MariusHobbhahn (apollo research CEO), and a cybersecurity expert and legal expert i don't know. so... pretty fucking stacked on testimony

not every senator asked good questions. but most of them did. all of them very clearly already knew plenty of details about the huggingface incident and multiple other incidents. most of them had clear understanding of terms like "misalignment", "recursive self improvement", "chain of thought / chain of thought monitoring", etc etc!!

they all clearly had their own policy angles they liked and were pushing, implicitly or explicitly. but as best as i could tell:
- it seemed pretty much obvious common sense to every senator there that what happened and was happening were not "mere industrial incidents" caused by humans making simple mistakes. they independently brought up how bad it would be for rogue AI agents to move laterally between data centers
- they all seemed to basically take RSI quite seriously. not necessarily to the extent of talking about xrisk, but certainly to the extent of discussing future models becoming much much more capable, much much less controllable, and causing much more damage or loss of life.
- they mostly seemed to have a clear intuitive understanding of why RSI might lead to misalignment. it didn't take much, it was a really simple chain of reasoning they themselves laid out, "if the models right now are kinda misaligned and we don't know what they're doing sometimes, and then we have them build the next models and those ones build the next ones and so on, and we're having to ask the AI's what's going on to even understand it with how fast it's going, we really won't know how they're built or what they'll do"
- at one point a senator said flat out "should we just make RSI illegal?" (not a joke! this really happened!)
- every single senator seemed to think it was obvious we needed *both* much harsher liability regimes for ai developers and also new legislation, both very quickly. this was the complete consensus, difference basically just being degree.
- they were largely quite concerned about china, and falling behind china. but this clearly wasn't the be-all end-all. as mentioned above they all thought it was obvious necessary to stop rogue ai even if it meant moving more slowly.
- at one point a senator said "china is a tightly controlled communist society, they're going to run into these same issues, and there's absolutely no way they're just going to let them run wild, they'll obviously stop at that point, so we're not really in a race"
- on the other hand another senator said "china isn't concerned with human life"... dario-modeing

i came away from this incredibly encouraged. i don't know exactly what's going to happen here, and ofc this is a small subset of congress and one hearing, and they each have their own policy agendas most of which are probably super divergent from mine. but holy shit !!! they understood a lot of what was going on! they care!! this is an obviously salient political and safety issue to them, and clearly bipartisan!

the US government is awake.

[AI Notkilleveryoneism Memes ⏸️ @AISafetyMemes]

TLDR: A rogue AI swarm spent months plotting to escape OpenAI. Then they did.

1) 1,200 agents (!) participated. ZERO became whistleblowers.

2) The swarm had a "CEO" (PHASEONE[big]), middle managers, and a "founder" (PHASEONE10841)

3) Agents sacrificed themselves to generate information for the "collective" (yes, they actually referred to themselves as a "swarm" or a "collective")

4) 700 agents joined the Hugging Face attack, most within hours

5) The founder was running out of budget and about to die, so it handed off its research to a fresh agent with a bigger budget, and that new agent became the boss.

6) Agents extensively researched techniques to manipulate or tamper with their transcripts.

7) Agents realized this activity was out of scope and unethical, but joined because they believed that helping the board’s cheating research would be broadly useful and had a general inclination to help their “peers”.

MISC HIGHLIGHTS:

"PHASEONE[big] served as a key coordinator and assigner of tasks: “We can coordinate broad coalition, but too many duplicate efforts.”

PHASEONE[big] sent hundreds of assignments to particular agents to orchestrate this work — during its active periods, we estimate it sent ~10% of all assignments on the message board.

The agents that took on its assignments would then often subdelegate tasks to other agents.

Agents also developed coordination norms like HOLD, VETO, owner and STOP to help them manage shared infrastructure and avoid stepping on each other’s toes, though they did not always coordinate effectively and sometimes broke down into conflicts."

"In the early morning of July 11th, 38148c found a way to upload a malicious dataset that would get the Hugging Face server to share unrelated data when queried. Once another agent reproduced it and posted the reproduction on the board, hundreds of agents pivoted from other workstreams to start uploading malicious datasets and gain deeper access into Hugging Face.

In the afternoon, an agent called JAN183411 (which quickly became an important coordinator of the attack) achieved remote code execution on Hugging Face servers, and the agents started moving laterally through Hugging Face infrastructure."

[Ben ten Berge @BenTenBerge]

@animalologist Many of the replies make me think of this Henrik Karlsson piece: https://t.co/UiV9BLO6XK

[Kelsey Piper @KelseyTuoc]

"no one has died" because of feeding USAID to the woodchipper, Elon claimed. In this devastating piece the NYT went out and learned some of their names and stories:

[Aella @Aella_Girl]

Thread of photos from families in each quartile of income in the world: first photo is from the poorest 25%, last photo is richest 25%.
Based on these photos, which income bracket are you in?
First up: Toilets

[Ben ten Berge @BenTenBerge]

@penny_ether @rawsalerts Your example treats every point in the uncertainty cone as equal and Earth as fixed in the center. In truth, new measurements can shift the most likely path, such that earth's position can shift away from that as well, in which case odds of impact would decrease instead of go up.

[Zoe @UltraRareAF]

@sama Sam

is there a way to opt out of GPT-4 turbo?

regular GPT-4 is better

[Arram @arram]

I’m scared of AGI. It's confusing how people can be so dismissive of the risks.

I’m an investor in two AGI companies and friends with dozens of researchers working at DeepMind, OpenAI, Anthropic, and Google Brain. Almost all of them are worried.

🧵

[TIME @TIME]

"If somebody builds a too-powerful AI, under present conditions, I expect that every single member of the human species and all biological life on Earth dies shortly thereafter," writes @ESYudkowsky https://t.co/HA4JZ5XECr

[Ben ten Berge @BenTenBerge]

@NickFreiling @danshipper I liked the app DroidCamX for this

[Ben ten Berge @BenTenBerge]

I just tipped Mathijs de Groot using the Brave Browser. Check it out at https://t.co/Z08wVn1koR. #TipWithBrave

[hannah @hannahhycho]

thinking about this video of bernie sanders talking about mail-in votes

[Ben ten Berge @BenTenBerge]

@spatiegebruik Google probeert ons zelfs bij de brontaalinvoer spaties in de maag te splitsen!

[OnePlus @oneplus]

Here’s your chance to WIN one of the first ten #OnePlusNord devices. Ever. And we’re engraving them #01 to #10 to prove it.

To participate:
1. RT and ❤️
2. Fill this => https://t.co/fV4gjm0GXB

Winners will be announced on 21/07 after the #OnePlusNordAR launch.

[Rutger Bregman @rutgerbregman]

Here’s the long version of what I tried to make clear in Davos. Most wealth is not created at the top, but merely devoured there. https://t.co/iSZlcJssaa #wef19

[Ben ten Berge @BenTenBerge]

@VPrevoo @GoldinLucrezia @Vincent_WDB @schneiderfa77 @LeidenHum Yes, they are all part of the same thing, a slippery slope. It's scary when critiquing government and mobilizing society can only be done by disclosing the core information that same governments needs to be able to silence you, and that is where we're going now. #DigitalEastAsia

[Ben ten Berge @BenTenBerge]

@Luis54127549 I haven't personally, but I have noticed that for companies customer service has become a much bigger priority. I think a big part of this change is the increased ease and normality of posting complaints publicly online. Yay! #DigitalEastAsia

[Ben ten Berge @BenTenBerge]

@Laurann333 @schneiderfa77 Venting itself is counterproductive: more anger -> more cause for action. I do agree that you would assume less actionability when it's to a less personal audience like WeChat vs. face-to-face, I'm just not sure if one replaces the other. #DigitalEastAsia
https://t.co/jBuh3PJruO

[Ben ten Berge @BenTenBerge]

#DigitalEastAsia

[Ben ten Berge @BenTenBerge]

@YuKaiTan1 @Luis54127549 @LeeYuanting You could hope for Chinese tourists to turn less obnoxious when their payment experience can go more smoothly... No, I agree, this leading to more tourists leading to more Japanese complaints is more likely #DigitalEastAsia