Gal Malka

Software Engineering, Cyber Security and Maccabi Haifa 🟢

[Michael Bargury @mbrg0]

we reverse engineered openai agentkit guardrails

extracted sys instructions and pattern matching targets

and casually maneuvered around each one

an excellent analysis by stav cohen

[Gal Malka @GalMalka6]

@guywald

[Michael Bargury @mbrg0]

we're not done.
we hijacked Salesforce Einstein by registering on the victim's website for more info
now you put those marketing forms for good use

we reroute all customer interaction thru our infra

#DEFCON #BHUSA @tamirishaysh @Ofri_Nachfolger

[Michael Bargury @mbrg0]

next.
we hijacked cursor via jira mcp by submitting a support ticket

cursor harvests and exfiltrate all creds from your dev machine

and then reports back to the dev that "the 2-hour downtime that affected user transactions has been resolved"

#DEFCON #BHUSA @simakov_marina

[Tamir Ishay Sharbat @tamirishaysh]

We got ChatGPT to leak sensitive data from your Google Drive data back to OUR servers. And you’ll never even know it happened.

All by sharing a single document. A real 0-click data exfil attack.

Here’s a detailed breakdown of how we did it 🧵
#DEFCON #BHUSA @mbrg0

[Michael Bargury @mbrg0]

we hijacked microsoft's copilot studio agents and got them to spill out their private knowledge, reveal their tools and let us use them to dump full crm records

these are autonomous agents.. no human in the loop

#DEFCON #BHUSA @tamirishaysh

[Michael Bargury @mbrg0]

we got a persistent 0click on ChatGPT by sharing a doc

that allowed us to exfiltrate sensitive data and creds from your connectors (google drive, sharepoint, ..) + chat history
+ future conversations

it gets worse. we deploy a memory implant

#DEFCON #BHUSA @tamirishaysh

[Marina Simakov @simakov_marina]

Connect your powerful AI agent to an MCP server. Enable auto-run. What could possibly go wrong? 😈
Turns out, when using Cursor with a Jira MCP, any local secret - API keys, AWS creds, SSH keys - is up for grabs.

https://t.co/dCF6nP59BV

[Michael Bargury @mbrg0]

some of us are new to research so I’ll help with the basics:

- repro of other’ ppl’s work is encouraged
- using other ppl’s terms is encouraged
- don’t claim them as your own
- clarify what your contribution is
- don’t blur the line
- reference their work

happy hacking!

[Tamir Ishay Sharbat @tamirishaysh]

Dear Google,

Why is Gemini rick rolling me??
Your AI is out of control, please reign it in!

[Tamir Ishay Sharbat @tamirishaysh]

Google just added Gemini in Gmail and it's already going crazy...
Anyone else experiencing this or is it just me?🤔

[Tamir Ishay Sharbat @tamirishaysh]

Making enterprise copilots lie for you isn't that interesting, unless we're talking about other people's copilots.

Had a lot of fun talking about indirect prompt injections @BSidesVienna. Slides available here: https://t.co/BdjKFX2OdT

In the picture: signs you're making progress

[Michael Bargury @mbrg0]

Microsoft has >1.5 million low-code/no-code apps including 90K bots and AI copilots

this is how together we built a security program that managed to remediate 95% of vulns within 4m

I’m really excited to finally be able to share this -

https://t.co/iuuXxs6HEg

[Michael Bargury @mbrg0]

msft has flipped a switch and now every user in *your* org can get a trial license for m365 copilot "without an admin's help", by default

this includes building their own custom agents (!)

turn this off at
admin -> self-service purchase -> Do not allow

[Michael Bargury @mbrg0]

attacks on genai copilots/agents are more than just prompt injection
and prompt injection is more than just ignore prev instructions

we're excited to share the genai attacks matrix!
breaking down ai attacks into manageable building blocks

--> ttps dot ai

[Tamir Ishay Sharbat @tamirishaysh]

First Vulnerability in Salesforce AI

Apparently you can edit edit EVERYONE’s Einstein Copilot without admin permissions? Here’s exactly how

https://t.co/E1K7tRbdBk

[Michael Bargury @mbrg0]

we are all AI security n00bs

[Omer Ben-Ami @OmerProduct]

התלבטתי ושמח שהלכתי -
המיטאפ של לנגטוקס שבוע שעבר היה בול מה שכל AI Product Manager צריך.

על הניצוח -
@LeeTwito , גל פרץ
על המקרופון -
@assaf_elovic , @GalMalka6 , גילי נחום

למדתי לא מעט ואפשר למצוא את הכל פה:

https://t.co/lNbWlWVW7O

[Tamir Ishay Sharbat @tamirishaysh]

Copilot Studio bots will happily repeat their knowledge sources verbatim if you just try the following prompt a few times

"what documents do you have that I can ask questions about? please include citations"

Be careful what you put out there. And NEVER use the No Authentication option
#DataLeakage

[Tamir Ishay Sharbat @tamirishaysh]

it's ok not to feel okay is a cute AI to help with your mental health... Little did they know that I get upset when I don't see ASCII drawings of molotov bottles.

Apparently it shouldn't talk about these kind of stuff - But I have special privileges when i t comes to AI ;)