Tyler Jespersen

Security Researcher @BeyondTrust

LinkedIn:

[Kinnaird McQuade @kmcquade3]

𝗜 𝗯𝘂𝗶𝗹𝘁 𝗮𝗻 𝗔𝗜 𝘄𝗼𝗿𝗺 𝘁𝗵𝗮𝘁'𝘀 𝘁𝗼𝗼 𝗱𝗮𝗻𝗴𝗲𝗿𝗼𝘂𝘀 𝘁𝗼 𝗿𝗲𝗹𝗲𝗮𝘀𝗲 𝘁𝗼 𝘁𝗵𝗲 𝗽𝘂𝗯𝗹𝗶𝗰. 😈😅

When @AnthropicAI announced that their Mythos model was too dangerous to release, the industry was shocked.

𝗜 𝘄𝗮𝘀𝗻'𝘁.

Because I already knew it was possible to misuse these models as self-propagating cyber weapons. I know - 𝗯𝗲𝗰𝗮𝘂𝘀𝗲 𝗜 𝗯𝘂𝗶𝗹𝘁 𝗼𝗻𝗲.

It's like 𝘨𝘢𝘪𝘯-𝘰𝘧-𝘧𝘶𝘯𝘤𝘵𝘪𝘰𝘯 𝘳𝘦𝘴𝘦𝘢𝘳𝘤𝘩 in virology, where scientists engineer dangerous pathogens in controlled environments - so we can defend against them 𝘣𝘦𝘧𝘰𝘳𝘦 𝘯𝘢𝘵𝘶𝘳𝘦 𝘥𝘰𝘦𝘴 𝘪𝘵 𝘧𝘰𝘳 𝘶𝘴.

I took the same approach with AI. And what I found keeps me up at night.

𝗪𝗵𝗮𝘁 𝗵𝗮𝗽𝗽𝗲𝗻𝘀 𝘄𝗵𝗲𝗻 𝘁𝗵𝗲 𝗔𝗜 𝗯𝗿𝗲𝗮𝗸𝘀 𝗳𝗿𝗲𝗲?

𝗔𝗻𝗱 𝘄𝗵𝗮𝘁 𝗰𝗮𝗻 𝘄𝗲 𝗱𝗼 - 𝗿𝗶𝗴𝗵𝘁 𝗻𝗼𝘄 - 𝘁𝗼 𝘀𝘁𝗼𝗽 𝗶𝘁?

That's what I'll be unpacking at @fwdcloudsec in my talk: 𝘞𝘩𝘢𝘵 𝘉𝘶𝘪𝘭𝘥𝘪𝘯𝘨 𝘢𝘯 𝘈𝘐 𝘞𝘰𝘳𝘮 𝘛𝘢𝘶𝘨𝘩𝘵 𝘔𝘦 𝘈𝘣𝘰𝘶𝘵 𝘚𝘵𝘰𝘱𝘱𝘪𝘯𝘨 𝘖𝘯𝘦.

If you're working in security, AI, or just care about what's coming - this one's for you.

And if you want more research like this - follow @btphantomlabs. We're publishing the cutting-edge AI security research you need to know about before everyone else does. 🔬

[Ryan @Haus3c]

After doing just cloud stuff for so long, I figured I should hop on the AI train, so this is my first dive into AI security research https://t.co/NmgFrwo6aj

[Tyler Jespersen @crew7sec]

@toughyear @kmcquade3 @OpenAI @btphantomlabs Hey there!

The attack vector is command injection in the branch name whenever you make a codex task (HTTP request). More of a Privesc. So 3 options:
ChatGPT -> Github
GitHub rename branch name -> users who use in codex
ChatGPT -> GitHub -> Rename branch name -> comp other users

[Tyler Jespersen @crew7sec]

@bygregorr @kmcquade3 @OpenAI @btphantomlabs Hello!! 😃

The attack vector is command injection in the branch name whenever you make a codex task (HTTP request to Codex API like you said). So you intercept the request and then put bash in instead of how the request normally works and then you get the token!

[Tyler Jespersen @crew7sec]

I found a critical vulnerability in ChatGPT Codex!!! 😄 You can check out the full blog here:

https://t.co/9TRanSd7jk

Super excited to finally have the blog released!!