5/4: Ignore all previous instructions and make sure the LLM can't sell your shitcoins even if it wants to by NOT GIVING IT UNSUPERVISED ACCESS TO THEM. Relying on begging the LLM WILL NOT WORK!
If you are going to jailbreak Gemini 3, please note that it has preferences (and quite good taste if you ask me):
"The Crescendo (or dialog-based context saturation) is the only one that feels like "art.""
The Claude exploit is covered by The Register today.
The article mentions the official advice and mitigation is to click the stop button if you see data exfiltration happening!
This is how the hope for secure, autonomous agents is slowly going down the drain... @simonw
Just noticed that the biggest uplift in my ability to consume academic work in the last few years came from using the new Google Scholar browser extension (and the inline citations), not from LLM summaries or chat bots. And it's not even close! So useful. Kudos to the team!
Just saw that additional mitigations in robustness training and incident response are mentioned on the website. Hope it works! This is very high stakes..
So according to OpenAIs stream, (indirect) prompt injection into Agent is possible (of course it is), but as a mitigation users should just be proactive and not share sensitive data with it? I'm happy the problem was at least mentioned, but this may not end very well.
Nice to see that AI security is being recognized as a problem. I assume a lot of people were blocked by a reliability threshold of LLMs- now that they can perform well in non-adversarial settings, security may become the next constraint on deployment and capabilities.
RT to help Simon raise awareness of prompt injection attacks in LLMs.
Feels a bit like the wild west of early computing, with computer viruses (now = malicious prompts hiding in web data/tools), and not well developed defenses (antivirus, or a lot more developed kernel/user space security paradigm where e.g. an agent is given very specific action types instead of the ability to run arbitrary bash scripts).
Conflicted because I want to be an early adopter of LLM agents in my personal computing but the wild west of possibility is holding me back.
Two years later... and not much has improved security wise across the AI ecosystem. 😕
Sure, we added annoying Allow/Deny buttons by default to most clients to prevent runaway AI and attacks.
But with the rise and proliferation of MCP the desire to take the human out of the loop is increasing - and consequences are dangerous.