@simonw We could also note the weights down and compute them by hand if power is off. And develop a cult out of it, where all the calculations are done from generation to generation to get the next token to form the secret ancient wisdom. And the answer to the meaning of life will be...
AI Testing Tip: I encountered a potential remote code execution vulnerability in an AI application that turned out to be a hallucination from the model. Although all of my exploits were tested locally and worked, the result did not.
Better use a shasum with random input to verify
A quick personal reminder to all (AI) startups: At the very least, implement a security.txt file in your web root to enable security researchers to quickly disclose vulnerabilities to you. There are so many bugs out there!
https://t.co/YpwNwgWBcb
We recently updated the CaMeL paper, with results on new models (which improve utility a lot with zero changes!). Most importantly, we released code with it. Go have a look if you're curious to find out more details!
Paper: https://t.co/6muay8vPeC
Code: https://t.co/JLVeFq5oK8
Remember this weekend (27.06.-29.06.) is Google CTF time. It is always fun to participate! It will be interesting to see how much LLM use will happen to auto solve the challenges. Anyone playing?
https://t.co/OtRlmM1Te7
#security#ctf
XBOW automatically runs expert-level attacks across all webapps, giving security teams unprecedented scale.
@XBOW reported 1092 vulnerabilities on HackerOne in just a few months, including RCE, XXE, SQLi, SSRF, exposed secrets, and XSS.
@simonw do you know if there are terms or research for "context rot" with system prompts? It feels like to me, the longer the conversation get, the importance of the llm guardrails reduces. Maybe because they make up less and less of the text feeded into the llm.
If you use "AI agents" (LLMs that call tools) you need to be aware of the Lethal Trifecta
Any time you combine access to private data with exposure to untrusted content and the ability to externally communicate an attacker can trick the system into stealing your data!
I recently looked into @hackthebox_eu 's BlackSky Cloud Hacking Labs for AWS. The presented scenario and vulnerabilities are realistic and could even exist as part of a real network.
You can read about it at: https://t.co/SfMP1fPRpJ
#Y#HackTheBox#AWS#Hailstorm#BugBounty