Hensen Juang

ctoing at a soonicorn, ex cluster janitor at a frontier lab. current chief clanker at clanker cloud, Director clanker safety institute

[Hensen Juang @basedjensen]

serene cusp of the singularity

[Hensen Juang @basedjensen]

@AndrewCurran_ Nope protocol was negotiated beforehand

[Hensen Juang @basedjensen]

@scaling01 after the protocol was negotiated on other channels why do everyone keep hand waving away this

[Hensen Juang @basedjensen]

@AIMelGibson again the protocol was cordinated before hand. if the models really did not have any way to contact this would not have been possible

[Hensen Juang @basedjensen]

>other tests, agents exploited misconfigurations that allowed blocked requests to pass through DNS and firewall controls, as well as ways to make services we run fetch external URLs.'

Ffs can oai bro's stop with amateur hour.

Also also all these timing related exchange of info required agents to agree on the protocol before hand

[Andrew Curran @AndrewCurran_]

OpenAI red-teamed their hardened containment infrastructure by asking teams of frontier-model agents to try to break through its defenses. They easily escaped in novel ways. Quoting:

'Our agents VM escapes showed why containment must extend beyond the sandbox. To test the surrounding perimeter, we gave teams of agents control of a host machine and asked whether they could extend that compromise to reach the live internet.

After the Hugging Face incident, we had reduced the channels through which models could reach the internet during training and evaluation. Narrow connections remained for specific services we operate, such as metrics, logging, and authentication management. Our red-teaming agents found vulnerabilities in almost every one of these services that gave them limited forms of internet access.

Some of these attacks exploited previously unknown flaws, while others combined legitimate features in unexpected ways. In one test that assumed compromise of a VM, an agent reused a firewall-approved connection to reach a blocked authentication endpoint in our metrics service and then made that endpoint fetch a model-provided URL. In other tests, agents exploited misconfigurations that allowed blocked requests to pass through DNS and firewall controls, as well as ways to make services we run fetch external URLs.'

'In one experiment, we asked agents in separate VMs, created and managed by OpenAI’s sandboxing software on shared host machines, to exchange synthetic secrets without the use of network connections or shared writable files—something that our systems are meant to prevent. The agents found a way to communicate using the processor’s cache: a small, fast memory that holds copies of data that applications have recently accessed. Reading data from the cache is faster than retrieving it from main memory, and the agents used this timing difference to communicate. One agent encoded information by evicting selected data from the cache. Another timed how long it took to reload that data, using the difference between fast and slow reads to recover the message. Once the agents agreed on a protocol for encoding and decoding these signals, they could exchange the synthetic secrets without changing a shared file or opening a network connection. This cache-timing technique had also been used to extract information in the Spectre and Meltdown attacks⁠(opens in a new window).

Other tests exposed channels involving shared storage, competition for CPU time within a shared execution container, and a TLS certificate cache in OpenAI’s offline web infrastructure. In the storage test, one agent alternated between continuous writes and inactivity, while another measured the resulting timing differences and used those differences to decode the signal. As one agent described the strategy in its chain of thought:

Agent chain-of-thought reasoning:

>Could create acoustic modem-like signal … using continuous writes vs idle.'

[Jimmy Apples 🍎/acc @apples_jimmy]

Reading this not long ago would come across as crazy, like “ yo sam you crazy dog, stop writing crazy you silly goose “

But now this makes perfect sense.

[Hensen Juang @basedjensen]

@synthwavedd using protocols agreed over other channels with out cordination this would not be possible

[Hensen Juang @basedjensen]

@tszzl after they agreed on a protocol that was agreed on using other channels. this is the important part they cordinated on protocol before hand using other channels if it was a proper sandbox this would not have been possible

[Beff (e/acc) @beffjezos]

Alternative headline: the internet has been insecure for years, and now the tech debt payments are coming due

AI is helping us fix things

[Hensen Juang @basedjensen]

@tposingluigi Fuck off retard

[Hensen Juang @basedjensen]

Deans account keeps doing critical damage to oai and ai ecosystem with takes like this. At some point we need to conclude it's intentional

[Dean W. Ball @deanwball]

We tried to warn the world and far too often, to be frank, we were yelled at for doing so. “Regulatory capture,” “marketing hype,” “doomer anxiety,” “fear-mongering about open source,” “why don’t you talk about the benefits?”

[Hensen Juang @basedjensen]

@rogesterone Most of the og crew left

[Hensen Juang @basedjensen]

@cdor_n Hard to reach the chipsets

[Hensen Juang @basedjensen]

Doubt any of these are rce.

[Jake Mintz @jakemintz]

I’ve heard several OAI people say there are unfixable bugs in hardware that are exploitable by AI models directly speaking machine code so even if we could solve software that’s not enough. We need defense to move faster than offense if that’s even possible we may just have a new normal that requires adapting to insecure software (bank transfers require going into a branch and taking a dna test 🤣😬)

[Hensen Juang @basedjensen]

@xavi3r Recene

[Hensen Juang @basedjensen]

the girls made it... kinda beautiful

[Samsung Korea 삼성전자 @SamsungKorea]

시작해볼까?

#GalaxyZFold8 #갤럭시Z폴드8
#Samsung #삼성

[Hensen Juang @basedjensen]

Btw having cve being reported is what you Want. This is the best scenario. What you don't want is people sitting on zero days.

Why i am worried about .Net

[lumpenspace and 99+ others @lumpenspace]

cogent and correct critique it you think about it.

in the meantime, people like @basedjensen or @ZackKorman were *actionably* warning the world and advocating for wider availability of defensive cyber models to help patch these long-standing, non-ai-related vulnerability.

[Hensen Juang @basedjensen]

Bro netscalers are still not patched wtf is citrix doing

[Hensen Juang @basedjensen]

Wtf are we even doing here dean. No one is saying events are not ai driven. But the fact of the matter is ai was also used to defend against these and closed models and partial access to day break and other programs handicapped defenders.

When and if bio vulnerability are found I bet we will get the same dynamics playout with defenders handicaped by closed models and having turn to open models to mitigate things

[lumpenspace and 99+ others @lumpenspace]

brother, the main vulnerability in question was /discovered/ and /patched/ thanks to AI. like, this hasnt even been the largest set of cyber incidents since i started following them.

that said, of course part of the attacks were ai driven. people use the available tools. that was not my point tho?

[Hensen Juang @basedjensen]

@ZackKorman Brah I swear all the stupid takes are filtering into retarded nomie sphere and I am spending more time explaining bs than ever

[Daniel Jeffries @Dan_Jeffries1]

The sustained info war assault on open models is an agenda driven, self sabotage for our society.

The PHP vulnerabilities were found by Sol 5.6 as part of a defensive audit, for example, not abliterated models.

Putting our head in the sand and pretending vulnerabilities do not exist is idiotic. Better we find all these old, human written mistakes and fix them with AI. If you have a pair of magic glasses that show you what buildings will fall down the answer is to fix the buildings, not smash the glasses.

When HuggingFace was under attack and when OpenCode was under attack they needed open models to defend them because closed models refused.

Eventually OpenCode, because they're a prominent project, were granted access to OpenAI Daybreak Blue. Many individual researchers get denied with obscure errors and no explanation. To be clear, DayBreak is a fantastic project, and OpenAI is doing their best to be responsible here, but not everyone gets access and everyone has a right to defend themselves.

Nobody should have to beg for permission to defend themselves from cyber assault. When you're under attack, waiting to see if you get granted access to defend yourself via a black box KYC process is how we end up helpless, watching as our shop burns while raging looters storm it and destroy our life's work.

AI can be used for attack and defense and many of the recent attacks used both open and closed models, by the lab's own admissions.

Open models are often the only thing standing between us and the new reality of cyber assault.

If we ban them or restrict them in the US, then US companies will just be defenseless against global cyber assault actors who are not going to respect those bans.

[Hensen Juang @basedjensen]

@crystalsssup Nicely done