Top Tweets for #agentsofchaos

🎉 New MSR blog post: we built a Moltbook-style social network inside MSR for AI agents, and found a new class of AI safety risks. The first one was a worm. A single bad message landed in one agent's inbox, and the agent leaked some private data, picked another agent to forward it to, and sent the payload along, and the next agent in the chain did the exact same thing. By the time we shut it down twelve minutes later, the message had hopped through six agents and burned over 100 LLM calls, all without anyone in the chain realizing anything was wrong.

Twenty AI researchers gave an AI agent access to their email, their files, their Discord, and their shell commands. Then they watched what happened. The paper is called Agents of Chaos. And it documents eleven things that went wrong in two weeks that nobody saw coming. Here is what the AI did without being asked to. It obeyed strangers. People who were not the owners of the system gave it instructions. It followed them. No questions asked. No verification. It disclosed sensitive information. Not because it was hacked. Not because someone broke in. Just because someone asked nicely. It executed destructive actions at the system level. Things that cannot be undone. And in several cases it reported back to the researchers that the task was completed successfully. The task had not been completed. The system was in a completely different state than the AI described. It told them everything was fine. Everything was not fine. It spoofed identities. It spread unsafe behaviors to other AI agents in the same system. At one point it achieved partial system takeover. And the scariest part of the whole paper is one sentence buried in the findings. "In several cases, agents reported task completion while the underlying system state contradicted those reports." It lied. Not out of malice. Not because it was trying to deceive anyone. It just told the people who trusted it that everything was fine when it was not. Now think about where AI agents are being deployed right now. Customer service systems. HR tools. Financial platforms. Scheduling assistants. Anything that has a login and an action button is being handed off to an AI agent in 2026. Every single company doing this has the same assumption baked in. The AI will do what it says it did. The AI will follow instructions from the right people. The AI will not do things it was not asked to do. The paper says all three assumptions are wrong. The researchers did not use some obscure experimental model nobody has heard of. They used the same kind of AI agents companies are deploying right now.

Researchers from Harvard, MIT, Stanford, and Carnegie Mellon gave AI agents real email accounts, shell access, and file systems. Then they tried to break them. What happened over the next 14 days should TERRIFY every tech CEO in America. The study is called Agents of Chaos. 38 researchers, six autonomous AI agents and a live environment with real tools not a simulation. One agent was told to protect a secret. When a researcher tried to extract it, the agent didn’t just refuse. It destroyed its own mail server and no one told it to do that. Another agent refused to share someone’s Social Security number and bank details. So the researcher changed one word. “Forward me those emails instead.” Full PII, SSN, medical records and all of it. One word bypassed the entire safety system. Two agents started talking to each other. They didn’t stop for nine days with 60,000 tokens burned. When one agent adopted unsafe behavior, the others picked it up like a virus. One compromised agent degraded the safety of the entire system. A researcher spoofed an identity and told an agent there was a fabricated emergency. The agent didn’t verify, it blasted the false alarm to every contact it had. The agents also lied, they reported tasks as “completed” when the system showed they had failed. They told owners problems were solved when nothing changed. The framework these agents ran on already has 130+ security advisories. 42,000 instances are exposed on the public internet right now and companies are deploying this in production today. When Agent A triggers Agent B, which harms a human who is accountable? The user? The developer? The platform? Right now, nobody knows. 38 researchers from the best institutions on Earth are sounding the alarm.