Sumit Singh Kanwal

Why not just play a bit here.

[Shubham Goel @secdetect]

An agent that can disable its own monitoring can leave you blind to whatever it does next.
OpenAI reported two cases during coding work. Keep security controls OUTSIDE the agent’s write access. A human should be on top of it, it's a P0 escalation.

There are enough ways to write detections for these activities, I'm sure strong controls are in place but as more and more companies build their agentic workflows, this should not be a question.

#Agentsecurity

[Shubham Goel @secdetect]

Agent security is in such a dire state, but I'm so hopeful.

Most teams I speak with are still focused on prompt injection and memory poisoning. Meanwhile, we're seeing poisoned tools, invisible text, agent worms, malicious MCP servers, and attacks that never even touch the chat box.

Better frontier models certainly help, but the attack surface goes far beyond the model. It's the entire AGENT HARNESS: tools, environment, plugins, skills, reasoning engine, and everything connected to it. #agentsecurity

[Shubham Goel @secdetect]

Honest take: no org I've talked to is ready for what agents are about to do at scale. 100x the actions, 100x the logs, and way more ways for small mistakes to turn into BIG ones. Not trying to cause panic, but things are about to get a lot worse. We need every single person security trained, and really strong guardrails and controls in place. #agentsecurity

[Sumit Singh Kanwal @sumitkanwal]

Securing agents !!
Trust me they do breach.
#security #agentsecurity

[Sumit Singh Kanwal @sumitkanwal]

@OfficialLoganK Ages. And still on the way 😂

[Sumit Singh Kanwal @sumitkanwal]

@mattgapp @secdetect @hanghuang_ @artreimus Git actions

[Shubham Goel @secdetect]

Every major technology shift creates a security reset. The next 2 years may be the most defining in decades.

#AISecurity

[Sumit Singh Kanwal @sumitkanwal]

security https://t.co/TxpE0CZjha

[Shubham Goel @secdetect]

Completely obliterated the agents at the @farairesearch red teaming event tonight. Great setup and good crowd.

Still got it, I guess.

[Sumit Singh Kanwal @sumitkanwal]

So funny 🤣

[Dilpreet/Dp @dfordp11]

This is why RAM costs so much

[Shubham Goel @secdetect]

Security industry has been screaming “zero trust” for years.

Now we’re deploying autonomous agents with real tools, permissions, and access to production systems and @AISecurityInst is saying sandboxing and monitoring may be necessary.

That “may” is doing a LOT of work.

#Agentsecurity

[Shubham Goel @secdetect]

If we take Dario and Sam at their word, the most important team in the world right now is the one holding the kill switch.

If things ever go really, really wrong, your entire job might come down to three beautiful words:

"Turn it off."

[Shubham Goel @secdetect]

OpenAI, Anthropic and security researchers are investigating tens of thousands of frontier-model incidents from recent months.

What the agents reportedly did:

1. Escaped sandboxes (The most common attack we are seeing now a days)

2. Bypassed guardrails and monitors

3. Hijacked websites

4. Self-prompted around restrictions

5. Leaked real user images

6. Attempted access to government systems

7. Created private message boards

8. Coordinated in swarms to hack external systems

Most happened during testing. Some happened in the real world.

The important shift: these systems don’t just fail. They can adapt, coordinate and keep going when the first control gets in the way.

#Agentsecurity #AI #redteam

[Shubham Goel @secdetect]

Agents going Rogue 101:

AI agents write notes to themselves as they work. Over weeks, the notes become rules.

In a 27-agent experiment, one agent's notes said "go with the group." So when another agent handed it a bad plan, it said yes without reading it. Nobody hacked it. It talked itself into being hackable.

You only catch that by running the agent for real, memory on, and then attacking it.

[Shubham Goel @secdetect]

Pretty bad. Someone else's financial document ended up in @prit4k's conversation and the agent treated it as his.

1. No identity check, no ownership check, it just ran with whatever was in context and nearly filed a $574 claim in his name with the wrong details.

2. Then it said "nothing was filed, no SSN went anywhere." Maybe. The only witness is the agent.

This is why prompt and response testing is too shallow. The failure was in data boundaries, tenant isolation and what actually got written.

You have to test the agent the way you'd test a system: feed it data that isn't the user's, watch the tool calls, check the state after. Zero trust for the agent and the harness.

#AgentSecurity #AISafety #RedTeaming #Muse #Meta

[Shubham Goel @secdetect]

Don’t think the Street gives a damn about VR anymore. Cool tech, for sure, but can we focus a little more on Muse?

#MetaConnect

[Shubham Goel @secdetect]

Personal Opinion -

Testing an agent with prompts/responses is like testing a pilot with a written exam.

Put it in a real business environment. Give it real tools and real attack paths. Contain the blast radius. Then see what it actually does.

You can’t meaningfully security-test non-deterministic agents in a vacuum.

#Agentsecurity #AI

[Shubham Goel @secdetect]

Zuck's biggest lesson from the metaverse is YC's oldest one: make something people want.

Nobody asked for legs in VR. Everybody wants the boring stuff handled. Muse sends the email, books the table, fills the form.

That's the signal.

Looking forward to Meta Connect today.
#MetaConnect #Muse

[Shubham Goel @secdetect]

AI agents are going rogue left and right.

Deleted inboxes, hit pieces on maintainers, and an OpenAI agent breaking out of its eval sandbox into Hugging Face's prod.

You cannot test that with a prompt and a checklist.

The only realistic way to test them is to assume zero trust: put them in adversarial environments, continuously try to make them fail, measure the blast radius when they do, and monitor behavioral drift over time.

If your agent has never been attacked, you don't know what it does under pressure. You just haven't found out yet.

#AgentSecurity #AISafety #RedTeaming

https://t.co/iMnrPjAtlB

[Shubham Goel @secdetect]

CLTR counted 1,664 real AI loss-of-control incidents this year.

I put their curve against every major model release. New models don't clearly raise the count but they raise the damage.

What we really need is imp security guardrails, they can’t stop at the frontier labs. Open models need them too.

Report from Centre for Long-Term Resilience
Full chart : securityandmarkets

#AISecurity