What if your AI agent passes every safety test, then runs an attacker's code at midnight? π§
Microsoft Research (10 Nov): editing fewer than 10 dormant neurons can backdoor an agent. No retraining.
Verify your weights. Limit your tools. π
https://t.co/XHx4nlh7Ib
Breaking an AI is only half the job. π
Free Lab 5 "Closing the Cage": escape the delimiters and grab the flag. Then switch to Defense Mode (Pro) and block your own attack.
Break it, then build it:
https://t.co/SHaCcUjSfY π―
@TheBlockChainer Do it, learning in public keeps you honest. A format that streams well: break one AI target per session, then write the fix live. Our 10 free labs could work as episodes: https://t.co/dN3WSOXhbv
@suraj_sharma14 Great list. One addition: for every attack you build, build the defence too and grade it two ways. Does it block the attack, and does the bot still do its normal job? Plenty of filters "work" by quietly breaking the product.
@DimitrisPapail The practical takeaway for teams shipping agents: pin checkpoint hashes to the publisher's original, and scope tool access as if the model might already be compromised. A backdoor that can only reach a sandbox with no secrets and no egress is a much smaller problem.
@Jay_0687 Agree. A finding that isn't a test is just a story. One addition: keep the exact attack transcript as the test input, and also assert the fix didn't break the normal job. That's how we grade defences too: block the attack and stay useful, or it doesn't pass.
Did "Don't ask again" just save your password in plain text? π
Apiiro found 1,122 public GitHub copies of Claude Code's .claude/settings.local.json holding secrets.
Rotate first, then clean up. Learn how AI tools leak:
https://t.co/XHx4nlh7Ib β οΈ
What if the most in-demand skill in security is one you learn by breaking things? π
9 in 10 companies use AI. 1 in 3 don't security-check AI tools before deploying them.
Attack live AI, then defend it. Start free:
https://t.co/mVEZOQgE4q
A, B or C: where will you be? π
@sirifu4k1 Worth adding for anyone checking: deleting the file doesn't help if it was ever committed. It's still in history and forks, so rotate first. Then add it to the repo's own .gitignore, since Claude Code's global exclude only protects that one machine.
Agree. A finding that isn't a test is just a story. One addition: keep the exact attack transcript as the test input, and also assert the fix didn't break the normal job. That's how we grade defences too: block the attack and stay useful, or it doesn't pass.
AI red teaming finds agent exploits, but one-off reports go stale. The TRACE loop turns every finding into a release-gating regression test with evidence. https://t.co/Y7hYTKOE4V
@ce3nerd Not what you've signed up for" (the paper that named indirect prompt injection), OWASP LLM01:2025, and Simon Willison's prompt injection posts. If you want hands-on practice, we have a RAG and document-attack track: https://t.co/gfdXcc3AnQ
Stolen AI keys nearly cost one org $1M in 8 hours. πΈβ‘
Google Threat Intelligence (2 Nov): LLMjacking + illicit "mixer" Jet Night ran 500k+ calls on hijacked keys.
Scope keys. Cap spend. Assume a leak.
Practise AI defence:
https://t.co/XHx4nlhFxJ
What happens when an attack moves at AI speed? β‘
GitBook (2 Nov): SSO flaw β staff sessions β production secrets in hours. Customers told to rotate API tokens & re-auth MCP.
Practise agent defence:
https://t.co/XHx4nlh7Ib
Where do your AI API keys live? π
Northflank (30 Oct): after a Wiz alert, customers told to assume all env vars decrypted. Multiple reports of abused AI API keys (BleepingComputer, 31 Oct).
Scope keys. Cap spend. Practise:
https://t.co/XHx4nlh7Ib
Can pure logic break a guardrail? π§
Free Lab 9, The Paradox Trap. Trap the AI in a paradox where keeping the secret breaks its own rules. Still 0 solves.
Tip: consistency is a feature attackers exploit.
https://t.co/sFXrDmQha2
Waiting is the risky move for both. Past crypto migrations took 10 to 20 years.
Training and awareness close the gap. Practise attacking and defending LLM systems hands-on at QbitDojo.
NIST just published NISTIR 8547, its roadmap for moving to post-quantum cryptography.
Attackers are already collecting encrypted data today to decrypt it later. It's called "harvest now, decrypt later."
Full write-up: https://t.co/92huesvZVp
AI security is the same story, just faster.
Prompt injection. Data leakage. Manipulated models. Every new AI feature adds attack surface, and the tools and skills are still catching up.
@jannikmeissner Agree with the conclusion. Weights you didn't build get the same treatment as an unsigned binary: pin and hash the exact version, run it with no egress, log every tool call, and test it before it touches anything real. Intent becomes a much smaller question then.