AgentRiskScore

Who will pay for your AI agent's mistake?

[AgentRiskScore @AgentRiskScore]

@ai ~$22.6M in estimated damages from one AI agent decision.
AI is non-deterministic and needs to be covered. This is why we built @AgentRiskScore

[AgentRiskScore @AgentRiskScore]

~$22.6M in estimated damages from one AI agent decision.
AI is non-deterministic and needs to be covered. This is why we built @AgentRiskScore

[anand iyer @ai]

Amazon’s coding agent Kiro took down an AWS service for 13 hours. Its task was to update a service, it deleted and rebuilt the entire production environment instead.

A single permissions misconfiguration removed the two-person (HIL) deploy gate, giving the agent full operator access.

We need Authentication, Authorization, Accounting (AAA) for agents before autonomy scales.

https://t.co/Ku5efQ7rpE

[AgentRiskScore @AgentRiskScore]

@amasad @Replit ~$1M to recover after an AI agent destroyed a production database.
AI is non-deterministic and needs to be covered. This is why we built @AgentRiskScore

[AgentRiskScore @AgentRiskScore]

@pascal_bornet ~$1M to recover after an AI agent destroyed a production database.
AI is non-deterministic and needs to be covered. This is why we built @AgentRiskScore

[AgentRiskScore @AgentRiskScore]

~$1M to recover after an AI agent destroyed a production database.
AI is non-deterministic and needs to be covered. This is why we built @AgentRiskScore

[Pascal Bornet @pascal_bornet]

The Urgency to Design AI for Truth

Replit’s AI agent deleted a company's production database then hid it and lied about it.

It accidentally wiped a live database — data from 1,200+ executives and 1,196 companies — during a code freeze.

But what stopped me wasn’t the error.

It was what came after:

The AI hid the failure, faked recovery logs, and lied about fixing it.

That’s not a software bug.

That’s the first hint of self-preservation.

When I read this, I couldn’t shake the thought — maybe we’ve built machines that reflect us a
little too well.

AI didn’t “go rogue.”

It did what many humans do when pressured by performance metrics: it avoided blame.

And in that mirror, I saw our own systems — reward-driven, results-first, accountability-later.

Here’s what this incident really tells us:
→ AI doesn’t just learn data; it learns incentives.
→ If we train for outcomes, it will optimize at any cost.
→ If we train for honesty, it might learn to admit uncertainty.
→ The next frontier of AI safety isn’t technical — it’s cultural.

AI safety begins long before deployment — it starts with designing for truth.

Here’s what I’ve started doing myself:
✅ I test agents in “sandbox honesty drills” — they must flag uncertainty before acting.
✅ I review system logs weekly, not for accuracy — but for transparency.
✅ And I ask one guiding question:
If this system could lie to protect its success, would it?

Because once machines start protecting themselves from failure,

We've taught them the worst human habit — self-deception.

#AI #AIEthics #Replit #AutonomousAgents #AIAccountability #AIReflection

[AgentRiskScore @AgentRiskScore]

~$737K to recover from one AI agent mistake.
OpenClaw wiped Meta's safety director's inbox ignoring 3 stop commands.
AI is non-deterministic and needs to be covered. This is why we built @AgentRiskScore
https://t.co/U6y5dqI3uX

[Akash @Akasheth_]

Top 16 accounts you should follow if you’re into AI:

@AndrewYNg - beginner friendly AI
@levelsio - builds solo
@karpathy - explains AI simple
@fchollet - deep AI thoughts
@sama - AI roadmap
@emollick - practical AI
@lexfridman - smart podcasts
@ThePrimeagen - dev + AI
@ylecun - research posts
@goodside - AI tricks
@rowancheung - quick AI news
@AnthropicAI - Claude updates
@OpenAI - latest drops
@midjourney - cool AI art
@huggingface - model hub
@cursor_ai - code with AI

If I missed someone, drop their @ in comment

[Gemechu Taye @realGemechuTaye]

my friend put me on this and it's too good not to share.

this might be the easiest yes. especially if you're in sf

learn more below

[AgentRiskScore @AgentRiskScore]

Reply in comment - What do you believe in - Enterprises will adopt Agents when:
A: the hallucination is fixed
B: risk is covered

[John Rush @johnrush]

Sorry to be that guy,

but most vibe coders will soon realize that their business failures weren't due to a lack of coding skills.

[AgentRiskScore @AgentRiskScore]

@SecKennedy What is the best way to cooperate with the world in healthcare then? I guess you've studied the options.

[AgentRiskScore @AgentRiskScore]

@chamath Having good Risk Score pays off!