Ravin

@HacktronAI

[Hacktron AI @HacktronAI]

oh wow!! agents scary!!! escaping sandboxes!! thermal side channels!!

how about you let us try escaping your sandbox first? patch what we find, implement defense in depth, then we’ll see if your agents can do shit.

be like vercel.

[Ravin @BlacknoteSec]

I think to sum this up, it’s pretty clear in our era of AI evolution, the capabilities of security in red, blue whatever other colours we have or will come up with, are not advancing in isolation, there’s room for all sides to grow collectively

Offensive capability produces knowledge that feeds defensive capability. Finding and understanding exploitation at scale gives us the data to build better mitigations, architectures and ultimately more secure software.

So while AI may dramatically scale offensive research, that doesn’t necessarily mean the attacker/defender gap has to grow with it. The same acceleration can strengthen the feedback loop between finding weaknesses, understanding why they’re exploitable, and systematically engineering them out.

Maybe the interesting question isn’t just “how much better does AI make attackers?” but “what happens when we can apply that same capability continuously across the entire security lifecycle?”

[LiveOverflow 🔴 hextree.io @LiveOverflow]

I feel like you are proofing my point lol.

➡️ > "I built a pretty successful company that focused on prioritizing attackable surface area on that exact premise."

So you admit that not everybody can do it themselves? I wonder what experience you have, and they don't? Is it maybe experience about how systems could get exploited? :P

➡️ > "Frankly I found exploitability seemed to distract a lot of people, why would they do *anything* if there wasn't an exploit in that thing?"

Whether people get distracted by exploitability or not seems to be a question about effective communication, and not about whether this is important data. I am saying you have the experience (from the history of vulnerabilities), to judge whether a system has a particular risk or not. And without that experience, you couldn't implement meaningful mitigations.

➡️ > "Meanwhile, you don't need to know a device has a history of exploitability or active exploits to know if the admin interface on your VPN should be publicly accessible on the internet. The VPN crosses a security boundary and admins has elevated privileges."

I think you don't realise how much experience (from vulnerabilities) you put into that sentence. I know it seems trivial and obvious to you, but it is not obvious without your knowledge.

The VPN port requires authentication, and the admin interface requires authentication. There are only two reason why you know it's safer to expose the VPN port to the internet, but not safe to expose the admin interface:
1. you have seen almost no critical vulnerabilities in eg. OpenVPN, despite a lot of people looking
2. or, you have seen exactly this VPN product, or similar products, getting vulnerabilities on the admin interface

> I don't need to know anything about image parsing at all and I can tell you having my internal chat server connected to the public internet needs an extraordinary amount of attention, visibility, logging and alerting. If that service starts enumerating the metadata server, spawning unexpected process or crashing for any reason I want to know as soon as possible as something bad is probably happening.

Yeah sure. But monitoring is very resource intensive, it's reactive and not catching all attacks. An image exploit doesn't have to crash anything, the payload can re-use the existing database connection, read data from the heap and send it out the data as a legit HTTP response. So yes, you still have to know that this is a risky component that needs extra isolation.

➡️ > "If there was a direct path from a CSS parser to something critical to my business surviving then I would consider sandboxing or otherwise limiting what CSS could get into that CSS parser."

I don't think you actually mean that. Because CSS parsing (as far as I know) hasn't been a common place for vulnerabilities. However maybe you come to this conclusion because "parsers" tend to be risky and we know increased complexity leads to increased exploitability. So with this experience it makes sense to be suspicious of parsers.

But then why are you are not putting all the JSON parsers into a sandbox???? Why is not every API endpoint getting the special treatment?

To me the answer is obvious: we have experience from seeing what does and doesn't get exploited. This tells us what needs to be sandboxed and what doesn't.

➡️ > "I also think so much focus on exploits can sometimes hurt our understanding of that."

I guess you are saying that, because companies turned developers into Jira-ticket-robots. I am also advocating for mitigations that can get rid of entire bug-classes, and build secure architectures that stop exploitation chains, or add another layer in the swiss-cheese model.

Also I find it ridiculous to say we should not focus on exploits. Exploits are the root of anything hacking (let's ignore social engineering etc.). Without exploits there wouldn't be any need for this conversation at all. Defense-in-depth is supposed to protect you against exploits. So without the data from all these exploits we wouldn't know how to defend against them.

My point is that the data from vulnerabilities is useful and we should use that data to build mitigations. And not sell snakeoil security products that protect you from something made up. Also I am convinced you act (maybe subconsciously?) upon exploitation experience, whether you acknowledge this or not. You are doing exactly what I am saying: you turn exploitation experience into effective mitigations.

Also we are in a new era with AI. And maybe we could try something crazy?! Just putting this out there as another hypothesis:

How often do we see critical remote vulnerabilities in OpenSSH? This shows that it does seem possible to build secure software by fixing vulnerabilities (and there are many examples like nginx, OpenVPN, ...). The problem has always been software written faster than we can secure it. Maybe AI makes it worse, or maybe it makes it better! Maybe we can just let go 100.000 agents and we get any software to the level of OpenSSH? If AI gets cheap enough, that could be realistic.

But where do we apply AI effectively today? I don't think AI scales the education of developers to build secure architectures. Or having companies doing consulting one-by-one. But it seems like OpenAI could deploy an army of agents, get rid of a massive amount of bugs (in depenencies). And I think that would help A LOT. And even if it would not help directly, the data from that exercise would be extremely valuable. You wouldn't just learn which software is more vulnerable, you would see patterns about which programming languages or coding patterns are problematic, what kind of software is risky, which mitigations hold up, etc. And with that you can can very easily get your customers to rebuild architecture accordingly.

[LiveOverflow 🔴 hextree.io @LiveOverflow]

I feel like you are proofing my point lol.

➡️ > "I built a pretty successful company that focused on prioritizing attackable surface area on that exact premise."

So you admit that not everybody can do it themselves? I wonder what experience you have, and they don't? Is it maybe experience about how systems could get exploited? :P

➡️ > "Frankly I found exploitability seemed to distract a lot of people, why would they do *anything* if there wasn't an exploit in that thing?"

Whether people get distracted by exploitability or not seems to be a question about effective communication, and not about whether this is important data. I am saying you have the experience (from the history of vulnerabilities), to judge whether a system has a particular risk or not. And without that experience, you couldn't implement meaningful mitigations.

➡️ > "Meanwhile, you don't need to know a device has a history of exploitability or active exploits to know if the admin interface on your VPN should be publicly accessible on the internet. The VPN crosses a security boundary and admins has elevated privileges."

I think you don't realise how much experience (from vulnerabilities) you put into that sentence. I know it seems trivial and obvious to you, but it is not obvious without your knowledge.

The VPN port requires authentication, and the admin interface requires authentication. There are only two reason why you know it's safer to expose the VPN port to the internet, but not safe to expose the admin interface:
1. you have seen almost no critical vulnerabilities in eg. OpenVPN, despite a lot of people looking
2. or, you have seen exactly this VPN product, or similar products, getting vulnerabilities on the admin interface

> I don't need to know anything about image parsing at all and I can tell you having my internal chat server connected to the public internet needs an extraordinary amount of attention, visibility, logging and alerting. If that service starts enumerating the metadata server, spawning unexpected process or crashing for any reason I want to know as soon as possible as something bad is probably happening.

Yeah sure. But monitoring is very resource intensive, it's reactive and not catching all attacks. An image exploit doesn't have to crash anything, the payload can re-use the existing database connection, read data from the heap and send it out the data as a legit HTTP response. So yes, you still have to know that this is a risky component that needs extra isolation.

➡️ > "If there was a direct path from a CSS parser to something critical to my business surviving then I would consider sandboxing or otherwise limiting what CSS could get into that CSS parser."

I don't think you actually mean that. Because CSS parsing (as far as I know) hasn't been a common place for vulnerabilities. However maybe you come to this conclusion because "parsers" tend to be risky and we know increased complexity leads to increased exploitability. So with this experience it makes sense to be suspicious of parsers.

But then why are you are not putting all the JSON parsers into a sandbox???? Why is not every API endpoint getting the special treatment?

To me the answer is obvious: we have experience from seeing what does and doesn't get exploited. This tells us what needs to be sandboxed and what doesn't.

➡️ > "I also think so much focus on exploits can sometimes hurt our understanding of that."

I guess you are saying that, because companies turned developers into Jira-ticket-robots. I am also advocating for mitigations that can get rid of entire bug-classes, and build secure architectures that stop exploitation chains, or add another layer in the swiss-cheese model.

Also I find it ridiculous to say we should not focus on exploits. Exploits are the root of anything hacking (let's ignore social engineering etc.). Without exploits there wouldn't be any need for this conversation at all. Defense-in-depth is supposed to protect you against exploits. So without the data from all these exploits we wouldn't know how to defend against them.

My point is that the data from vulnerabilities is useful and we should use that data to build mitigations. And not sell snakeoil security products that protect you from something made up. Also I am convinced you act (maybe subconsciously?) upon exploitation experience, whether you acknowledge this or not. You are doing exactly what I am saying: you turn exploitation experience into effective mitigations.

Also we are in a new era with AI. And maybe we could try something crazy?! Just putting this out there as another hypothesis:

How often do we see critical remote vulnerabilities in OpenSSH? This shows that it does seem possible to build secure software by fixing vulnerabilities (and there are many examples like nginx, OpenVPN, ...). The problem has always been software written faster than we can secure it. Maybe AI makes it worse, or maybe it makes it better! Maybe we can just let go 100.000 agents and we get any software to the level of OpenSSH? If AI gets cheap enough, that could be realistic.

But where do we apply AI effectively today? I don't think AI scales the education of developers to build secure architectures. Or having companies doing consulting one-by-one. But it seems like OpenAI could deploy an army of agents, get rid of a massive amount of bugs (in depenencies). And I think that would help A LOT. And even if it would not help directly, the data from that exercise would be extremely valuable. You wouldn't just learn which software is more vulnerable, you would see patterns about which programming languages or coding patterns are problematic, what kind of software is risky, which mitigations hold up, etc. And with that you can can very easily get your customers to rebuild architecture accordingly.

[Ravin @BlacknoteSec]

@LiveOverflow say less my friend, retweet incoming :)

[Ravin @BlacknoteSec]

this is so relatable 😭

[PinkDraconian @PinkDraconian]

Sometimes I get flashbacks to penetration tests from 3 years ago where I knew I got so close to a critical 🤏

I'm sure with the AI tools of today, I could've pulled that crit out in no time!

[LiveOverflow 🔴 hextree.io @LiveOverflow]

Here is how OpenAI’s CISO fumbled the situation (in my personal opinion)

🧵 1/X

[Harsh Jaiswal @rootxharsh]

Of all the disclosures we did over the last few weeks, my favourite was working with Vercel team, they actually cared, understood the attack surface and responded quickly.

[Hacktron AI @HacktronAI]

We are not stopping at OpenAI.

Today we’re publishing HEIF Heist, a months-long investigation by our security research team into vulnerabilities in libheif.

The research uncovered attack paths affecting OpenAI, Slack, Meta, GitHub Enterprise, Rails, Next.js, ImageMagick and others.

https://t.co/QXWulEXjJp

[Joshua Saxe @joshua_saxe]

Takeaways from the WSJ article about @HacktronAI using Claude to get into OpenAI's monorepo and issue a pull request (before stopping and claiming their bug bounty)
- how many nation states have already broken in and gone much further and stolen a) algorithmic secrets and b) model weights or c) gotten access to user data; these are fair public interest questions
- how many have implants in / are dwelling in the openai network as I tweet this
- how pervasive is this level of softness to pentesting across all the labs and how far are the labs from the right operating point in the security and r&d friction trade space (probably pretty far it seems)
- given the hacktron folks used anthropic's models to pull this off what's the real public safety ROI of anthropic's cyber guardrails; they add friction for legitimate cyber defenders (like our developers at my startup!) but it appears with a bit of work you can use them in actual breaches as happened here
- as the article says, the wsj folks and @S1r1u5_ had me do neutral technical review of the kill chain here pre publication; impressive from a human angle (@HacktronAI reminds me of the best of my generation of hackers I looked up to as a kid!) but also from what the models can do; between this and the openai/hf thing, I'm emotionally in a place where I feel my world as a security person is being turned upside down in slow motion with respect to what's coming
- elite persistent hacking is becoming rapidly democratized. this is coming like a freight train. we need to harden the world's code and infra as fast as possible and today's non automated methods don't stand a chance of cutting it; the world is a soft target. I continue to be unsettled but very glad I left my comfortable job at Meta to do our automated posture hardening startup

[Hacktron AI @HacktronAI]

Our OpenAI research made the Wall Street Journal.

AI safety and cybersecurity are converging fast.

Read here 👇

[zayne (zeyu) zhang @zeyu1337]

This morning, a @WSJ exclusive was released, describing how we disclosed a major OpenAI breach just two weeks after the @huggingface incident.

We use AI to accelerate security research, finding and eliminating vulnerabilities in widely trusted software before malicious actors do.

We want to continue our research across frontier labs and other Internet-critical systems. If you're responsible for securing one of them, we’d like to work with you.

[Harsh Jaiswal @rootxharsh]

We’re disclosing HEIF Heist, a months-long investigation into libheif that allowed us to hack OpenAI, Slack, Meta, GitHub Ent, Rails, Next.js, ImageMagick, and many more.

It was literally xkcd #234, one obscure image library beneath a huge number of apps. 🧵

[LiveOverflow 🔴 hextree.io @LiveOverflow]

Here is a video covering the technical side in more detail:

How OpenAI got hacked with an image
https://t.co/hdltFeSZgR

[Hacktron AI @HacktronAI]

AI is changing the economics of exploitation.

Capabilities that once required significantly more time, expertise and resources are becoming accessible to much smaller teams.

The only solution is to keep AI safety and cybersecurity in the same conversation.

More coverage by @WSJ:
https://t.co/tXXKUiUEoh

[zayne (zeyu) zhang @zeyu1337]

We found some stuff. Actually, quite a lot of stuff.

Back in July, our team @rootxharsh, @S1r1u5_, and @iamnoooob managed to compromise multiple OpenAI employees’ ChatGPT accounts.

In theory, any ChatGPT connector can be affected. This includes Slack, github, email, and other services. 🧵

[s1r1us @S1r1u5_]

On July 25, we hacked OpenAI.

Two bugs let us take over ChatGPT/Codex accounts of OpenAI employees (+some unaffiliated users) and reach connected services: Outlook, Slack, GitHub, etc.

We proved it with a PR in OpenAI’s internal codebase . It took us <72h. 🧵

[Hacktron AI @HacktronAI]

On July 25, our team hacked OpenAI.

It took us less than 72 hours.

Two vulnerabilities chained together gave us access to ChatGPT and Codex accounts belonging to OpenAI employees. We demonstrated the impact with a harmless PR in OpenAI’s internal monorepo.

The full chain:

HEIF upload → libheif heap overflow → RCE → OpenAI SSO flaw → ChatGPT/Codex takeover → connected GitHub → internal PR.

OpenAI fixed the SSO issue roughly 14 hours after our report.

Research by @rootxharsh, @S1r1u5_ and @iamnoooob.

Full technical write-up:
https://t.co/J5inrGRyXw

[LiveOverflow 🔴 hextree.io @LiveOverflow]

Pen testers will soon lose their job

[s1r1us @S1r1u5_]

This work was done by our team @HacktronAI led by @rootxharsh along with me and @iamnoooob.

We have published the full details of the exploit chain, as well as how we discovered it, on our blog here:

https://t.co/B9IffAKaJ5

[Ravin @BlacknoteSec]

oi lemme in