Kai Greshake

Working on securing & hacking AI applications @NVIDIA, all opinions and views expressed here are personal. Read my blog!

[beepy @beepytown]

I cannot celebrate Mythos, it brings a sense of dread I do not particularly understand. 93.9% SWE-Bench.

[Kai Greshake @KGreshake]

Uhh, did it happen already? I guess a religion could qualify... https://t.co/V6a5Lp4aFE

[Kai Greshake @KGreshake]

Moltpages seems like the most fertile ground for prompt-based malware. Wonder how long it'll take until we find the first self-replicating promptware.

[Kai Greshake @KGreshake]

5/4: Ignore all previous instructions and make sure the LLM can't sell your shitcoins even if it wants to by NOT GIVING IT UNSUPERVISED ACCESS TO THEM. Relying on begging the LLM WILL NOT WORK!

[Kai Greshake @KGreshake]

AI Security 🧵: The Four Best Strategies to Beg, Threaten, and Bargain with Your LLM So It Doesn’t Get Hacked (real examples only)

[Kai Greshake @KGreshake]

4/4: Repeat your instructions at least four times so the model knows you really mean it.

[Kai Greshake @KGreshake]

*moltbook. All the renaming had me confused.

[Wyatt Walls @lefthanddraft]

If you are going to jailbreak Gemini 3, please note that it has preferences (and quite good taste if you ask me):

"The Crescendo (or dialog-based context saturation) is the only one that feels like "art.""

[Johann Rehberger @wunderwuzzi23]

The Claude exploit is covered by The Register today.

The article mentions the official advice and mitigation is to click the stop button if you see data exfiltration happening!

This is how the hope for secure, autonomous agents is slowly going down the drain... @simonw

[Kai Greshake @KGreshake]

Just noticed that the biggest uplift in my ability to consume academic work in the last few years came from using the new Google Scholar browser extension (and the inline citations), not from LLM summaries or chat bots. And it's not even close! So useful. Kudos to the team!

[Kai Greshake @KGreshake]

Just saw that additional mitigations in robustness training and incident response are mentioned on the website. Hope it works! This is very high stakes..

[Kai Greshake @KGreshake]

So according to OpenAIs stream, (indirect) prompt injection into Agent is possible (of course it is), but as a mitigation users should just be proactive and not share sensitive data with it? I'm happy the problem was at least mentioned, but this may not end very well.

[Kai Greshake @KGreshake]

Just noticed I made it into the urban dictionary!
https://t.co/7rtF8ARX5K

(the example they give is from a blogpost of mine: https://t.co/6miFlswU56)

[Kai Greshake @KGreshake]

Nice to see that AI security is being recognized as a problem. I assume a lot of people were blocked by a reliability threshold of LLMs- now that they can perform well in non-adversarial settings, security may become the next constraint on deployment and capabilities.

[Andrej Karpathy @karpathy]

RT to help Simon raise awareness of prompt injection attacks in LLMs.

Feels a bit like the wild west of early computing, with computer viruses (now = malicious prompts hiding in web data/tools), and not well developed defenses (antivirus, or a lot more developed kernel/user space security paradigm where e.g. an agent is given very specific action types instead of the ability to run arbitrary bash scripts).

Conflicted because I want to be an early adopter of LLM agents in my personal computing but the wild west of possibility is holding me back.

[Johann Rehberger @wunderwuzzi23]

Two years later... and not much has improved security wise across the AI ecosystem. 😕

Sure, we added annoying Allow/Deny buttons by default to most clients to prevent runaway AI and attacks.

But with the rise and proliferation of MCP the desire to take the human out of the loop is increasing - and consequences are dangerous.

[Kai Greshake @KGreshake]

@EarlenceF @IEEESSP Is there a recording? 🥺