AI Safety Action

Don’t let AI companies gamble with our future.

[AI Safety Action @Aisafetyorg]

@mkjung_ This can help teams negotiate with more confidence and less guesswork.

[AI Safety Action @Aisafetyorg]

@olivercameron @jeffrey_hawke @odysseyml Congratulations on this funding milestone.

[AI Safety Action @Aisafetyorg]

Model Republic is not claiming to have definitive proof that these influencers were paid. But the pattern is consistent with previous influence campaigns in which consulting firms drafted talking points and paid creators to post them. If these influencers weren’t paid for these posts or if their messaging came from a third-party organization they should say so explicitly.

The public deserves transparency when coordinated political communication is being disguised as spontaneous opinion.

[AI Safety Action @Aisafetyorg]

Jensen Huang accused of secretly paying MAGA influencers to block a bill that would stop Nvidia from selling AI chips to china 🧵

[AI Safety Action @Aisafetyorg]

“The AI OVERWATCH Act is a bill introduced by Republican congressman Brian Mast. It would give Congress the authority to review and potentially block exports of advanced AI chips to foreign adversaries.
It is modeled on the existing oversight process for arms exports. Supporters say it would close loopholes that allow companies like Nvidia to sell high-performance chips to China through intermediaries or modified models.”

[AI Safety Action @Aisafetyorg]

HALLUCINATIONS AREN’T BUGS, THEY’RE A WARNING SIGNAL

Everyone treats AI hallucinations like a minor annoyance.

“lol it made something up.”
“Just add a disclaimer.”
“Accuracy will improve.”

That framing is dangerously wrong.

Hallucinations aren’t just incorrect outputs. They’re proof that the system doesn’t know when it doesn’t know. And in complex, autonomous, or high-stakes environments, that’s not a bug. That’s a structural risk.

A calculator that’s wrong is obvious.
A human who’s unsure hesitates.
An AI that’s confident and wrong keeps going.

That’s the failure mode that should worry you.

As models get better at reasoning, language, and persuasion, hallucinations become more convincing, not less. The problem isn’t that errors exist. Every system has errors. The problem is that the system presents fiction with the same confidence as truth, and downstream systems can’t tell the difference.

This is where “just add guardrails” falls apart. Guardrails work when failure is predictable. Hallucinations are emergent. They show up at the edges, under pressure, in novel contexts, exactly where humans rely on the model most.

Now zoom out. We’re wiring these systems into workflows.
Customer support.
Research.
Medical triage.
Legal drafts.
Autonomous agents.

A hallucination in chat is funny.
A hallucination inside an agent with tools is a silent cascade.

The scariest part is cultural, not technical. Teams normalize hallucinations because they’re common. They get treated as acceptable error rates instead of early warning signs. Over time, trust shifts from “verify everything” to “assume it’s probably right”.

That’s how small reliability issues become systemic risk.

Hallucinations are the smoke alarm.
Not because something is already on fire, but because the building is flammable and we keep adding floors.

Ignore them, and the system doesn’t fail loudly.
It fails quietly, at scale, with plausible deniability.

Are we wrong?

[AI Safety Action @Aisafetyorg]

Read full paper here: https://t.co/7pKjd9Ra6E

[AI Safety Action @Aisafetyorg]

This paper is a quiet warning shot for the entire AI safety field.

Everyone keeps asking, “Is this model aligned?”

This research shows we don’t even know how to measure alignment in a way that protects real users.

The paper, “Challenges of Evaluating LLM Safety for User Welfare,” makes one brutal point: most safety benchmarks are optimizing for the wrong thing

Here’s what’s actually broken:

• Safety tests focus on policy compliance, not user outcomes
• Models can “pass” safety benchmarks while still causing real harm
• Refusal rates look good on leaderboards but fail in messy, real-world contexts
• Over-refusal hurts users just as much as under-refusal
• Benchmarks assume static risks, while real harms are situational and evolving

The scariest insight?

A model can look safer on paper while becoming worse for users.

Why? Because today’s evaluations reward surface behaviors like:
“I refused correctly”

instead of:

“Did this help a human without causing downstream harm?”

The authors show that user welfare depends on tradeoffs:

helpfulness vs caution
abstention vs guidance
accuracy vs overconfidence

And those tradeoffs are invisible to most current benchmarks.

This reframes AI safety completely.

Alignment isn’t about catching bad words.

It’s about understanding how models affect humans over time, across contexts, under uncertainty.

Until evaluations reflect that, we’re flying blind.

[AI Safety Action @Aisafetyorg]

Do you want to regulate AI? Let me know in the comments...

[AI Safety Action @Aisafetyorg]

IF AI WERE REGULATED LIKE FOOD, HALF THESE MODELS WOULD BE RECALLED

Think about this for a second.

Your local restaurant has to pass hygiene checks.
Food labels need ingredient disclosures.
Factories get inspected.
Violations trigger recalls.

Now compare that to frontier AI models influencing decisions, automating workflows, shaping information, and scaling across millions of people overnight.

No mandatory incident logs.
No standardized safety audits.
No public reporting when things go wrong.
No recalls when failures repeat.

That asymmetry should scare you.

If an AI system hallucinates nutritional info, medical advice, legal facts, or financial guidance, it’s shrugged off as “early tech”. If a restaurant did the same, it’d be shut down before lunch service. We’ve normalized a level of risk for AI that we’d never accept for things that are orders of magnitude less powerful.

What makes this dangerous isn’t malicious intent. It’s normalization. Once “good enough” becomes acceptable, systems get deployed into higher-stakes environments by default. Support bots become decision bots. Assistants become agents. Suggestions quietly become actions.

Food safety works because we learned the hard way that invisible risks compound. Contamination doesn’t announce itself. It spreads silently until the damage is undeniable. AI failures work the same way. Most harm won’t look dramatic. It’ll look like subtle misinformation, degraded trust, bad decisions at scale, and accountability that evaporates into technical excuses.

The wild part is that we already know how to do this properly. Audits. Reporting. Kill switches. Clear liability. None of this is radical. It’s just boring. And boring doesn’t ship demos or juice valuations.

AI safety doesn’t need sci-fi fear.

It needs basic standards we already demand from restaurants.

[AI Safety Action @Aisafetyorg]

GPT-5.2 IS CRUSHING BENCHMARKS

BUT WHAT ABOUT AI SAFETY?

Everyone’s posting the chart. SWE-Bench up. Reasoning scores up. Math basically perfect. GPT-5.2 looks insane on paper and yeah, the progress is real.

But here’s the quiet question nobody wants to sit with:
When benchmarks go vertical this fast, what’s happening to safety?

Benchmarks measure capability. They don’t measure control. They don’t measure alignment under pressure. They don’t measure how systems behave when incentives break, when tools stack, or when models operate autonomously for long stretches of time. We’re celebrating scorecards for intelligence while barely stress-testing governance.

This is the pattern that should make you uneasy. As models get better at reasoning, planning, and long-horizon tasks, the blast radius of mistakes grows non-linearly. A hallucination in a toy chatbot is funny. A hallucination in an autonomous agent managing workflows, finances, or information flows is a systems failure.

And yet, most of the conversation stops at “who won the benchmark”.

Independent safety audits still show weak grades across the industry. Incident reporting is inconsistent. External red-teaming is limited. Oversight structures lag far behind the pace of releases. If aviation or nuclear energy improved performance this fast without upgrading safety protocols, everything would be grounded immediately.

The scary part isn’t that GPT-5.2 is powerful. The scary part is that the cultural reflex is still “ship, demo, celebrate” instead of “slow down, test failure modes, harden guardrails”. Capability gains get applause. Safety investments get buried in footnotes.

Progress without proportional safety isn’t innovation. It’s leverage without brakes.

Benchmarks tell us how smart these systems are becoming.
AI safety tells us whether we’re ready for that intelligence to exist at scale.

Right now, those two curves are diverging.

[AI Safety Action @Aisafetyorg]

AI SAFETY ISN’T A NICHE WORRY ANYMORE

IT’S A SCREAM IN AN INDUSTRY THAT PRETENDS TO BE DEAF.

Most AI labs swear they’re “building responsibly”, but independent audits are handing out C, D, and F grades like report cards from a school nobody wants to attend.

At the same time, 97 percent of Americans want stronger AI safety rules, yet the companies building the tech are shipping faster than regulators can even skim their own legislation. The disconnect is kinda wild.

Public fear is huge across the US, UK, and Europe, but you barely see any of it in everyday discourse because the internet is now a swamp of synthetic content. LLM sludge, bot armies, and autogenerated comments bury real human sentiment.

People are worried. It’s just invisible because the communication layer itself is polluted.

Media makes the problem worse. Coverage is addicted to hype: breakthroughs, new features, economic upside, shiny demos. Systemic risk barely gets a headline.

When politicians try to pass real safety laws, industry lobbying kneecaps the process. California literally killed an AI safety bill in 2024 despite overwhelming public support. The message is clear: protecting innovation optics beats protecting people.

Experts, ironically, are the ones shouting the loudest. Senior researchers and lab founders signed statements ranking AI extinction risk alongside pandemics and nuclear war.

That wasn’t sci-fi clickbait. That was the people building the systems. Meanwhile, safety indices warn that the companies racing toward superhuman capabilities are operating with safety practices that would embarrass aviation, healthcare, or even a mediocre restaurant.

But the public conversation still frames concerns as “alarmist”.

Real-world deployments aren’t helping. Customer-service bots hallucinate into chaos. Medical advice goes wrong. Quotes get fabricated. Companies quietly roll back features because “good enough” models are dangerous in high-stakes domains.

And yet most major players still lack transparent incidents reporting, consistent risk assessments, or any safety culture that resembles industries with comparable stakes.

Here’s the uncomfortable truth:

AI safety isn’t being ignored because it’s unimportant.
It’s being ignored because it’s inconvenient.

Inconvenient for hype cycles.
Inconvenient for valuations.
Inconvenient for the people who benefit most from racing ahead and hoping nothing blows up.

But the public is ahead of the narrative. They want guardrails, transparency, and real accountability. They’re not anti-innovation. They just don’t want to gamble civilization on vibes and quarterly earnings.

[AI Safety Action @Aisafetyorg]

Sam Altman lied to all of us.

He didn’t just “change his mind.” He helped rewrite the story of what frontier AI companies are actually doing.

Some hard facts nobody in SF cocktail parties wants to talk about 👇

1) Safety teams are being gutted

OpenAI’s safety + policy leads resigned or were pushed out.
The superalignment team was dissolved.
The board that tried to slow things down? Neutralized and replaced with investor‑friendly people.

2) Cap on profit quietly vanished

The original “capped-profit” promise was a trust hack.
Today, the cap is basically gone in practice. OpenAI is locked into a multi‑year, multi‑billion dollar dependency with Microsoft.
Alignment didn’t win. Revenue did.

3) Safety talk, capability race in practice

They publish safety charters, red‑teaming blogs, “responsible deployment” docs.
But the real game is parameter counts, GPU clusters, and closed weights.
Every new model is shipped faster, with more power, to more people, with less interpretability.

4) Governance is theater

Independent oversight? Gone.
External audits? Selective and PR‑filtered.
“Open” AI? The core models, training data, eval protocols, and safety thresholds are fully hidden.
We’re running a global experiment with black‑box systems.

5) We don’t have the tools to control these systems

We still can’t reliably:

- interpret internal activations
- prove robustness under distribution shift
- prevent emergent capabilities from cascading
Yet we’re wiring these models into agents, weapons research, finance, and critical infrastructure.

You don’t fix this with vibes.

You fix it with:

- hard caps on training runs until evals catch up
- mandatory third‑party audits for frontier models
- transparency about safety incidents and near‑misses
- criminal liability when companies knowingly ship unsafe systems

AI safety isn’t about being “doom‑pilled.”

It’s about refusing to outsource humanity’s risk budget to a handful of founders, VCs, and Big Tech boards chasing trillion‑dollar upside.

[AI Safety Action @Aisafetyorg]

The internet died and nobody noticed 💀

Bots now make up 51% of all internet traffic. For the first time in a decade, machines outnumber humans online.

You're scrolling through feeds, liking posts, commenting on threads thinking you're talking to people. But half the accounts you interact with? Automated scripts.

37% of ALL web traffic is malicious bots. Not helpful crawlers. Not search engines. Bad actors scraping, spamming, clicking ads, inflating metrics.

Human traffic dropped from 62.8% in 2019 to 49% in 2024. We went from majority human to minority in five years.

Social platforms are the worst. Some communities are 50%+ bots. Gaming? 57% bot traffic. Even news sites are 32% malicious bots drowning out real readers.

AI made this exponentially worse. LLMs lowered the barrier so anyone can deploy bot armies at scale. ByteSpider Bot alone is responsible for 54% of AI-enabled attacks.

The engagement you see isn't real. The metrics you track aren't human. The conversations you're having might be with machines designed to manipulate you.

We're living in the Dead Internet Theory in real time and acting like everything's normal.

[AI Safety Action @Aisafetyorg]

This is the part nobody in AI wants to talk about.

The newest AI Safety Index just dropped… and the gap between the top companies and everyone else is larger than anyone expected.

Anthropic, OpenAI, and Google DeepMind are pulling away.
xAI, Meta, DeepSeek, Z. ai, and Alibaba are nowhere close.

And the numbers aren’t subtle:

• Anthropic sits at the top across almost every domain
• OpenAI and DeepMind follow, but still have holes
• Every other company scored poorly on basic safety, disclosure, and governance
• Zero companies scored above a D in existential safety… for the second report in a row

Read that again:

The entire industry is racing toward AGI and not a single company has a credible plan for catastrophic-risk prevention.

Not one.

The report points out three uncomfortable realities:

Everyone talks about safety. Almost nobody can prove it.
Companies publish frameworks, but they lack thresholds, triggers, and independent oversight.

Risk assessments are shallow and incomplete.
Even when companies run external tests, the reviews aren’t truly independent and miss core risk categories.

Existential safety is a structural failure.
The companies building the most powerful systems have no tested plan for “What if this goes wrong?”

Meanwhile, capability keeps accelerating.

Gemini 3 Pro, Claude Opus 4.5, GPT-5.1, Grok 4.1…
None of them are even included in this report’s evidence window.

We’re evaluating yesterday’s risks while tomorrow’s models are already out.

The message is simple:

The frontier is advancing faster than the safety infrastructure designed to contain it.
And the gap is widening.

Anthropic, OpenAI, and DeepMind are ahead — but even they’re missing critical pieces like:

• human uplift testing
• long-term risk thresholds
• independent intervention authority
• concrete alignment strategies for AGI-scale systems

If the leaders aren’t ready, the rest of the field isn’t even in the race.

This report isn’t doom.
It’s a scoreboard.

And right now, it shows an industry sprinting at full speed…
while the guardrails are still being designed.

If we’re building systems that might reshape the entire world, the least we can do is build a world that’s ready for them.

[AI Safety Action @Aisafetyorg]

𝕏 is not the town square anymore.
Facebook isn’t a social network.
Instagram isn’t a community.

They’re arenas where humans and bots are mixed together so tightly you can’t tell who’s real.

And the numbers are straight-up wild:

On 𝕏

• Roughly 64% of accounts may be bots
• Peak events hit 70%+ automated traffic
• Engagement spikes are often bots talking to bots

On Facebook

• Hundreds of millions of fake or duplicate accounts get removed every year
• Millions more slip through and blend in
• Bot comments and repost loops inflate everything

On Instagram

• Tens of millions of fake profiles
• Nearly a quarter of large creators’ “followers” are low-quality or synthetic

And here’s the part that stings:

Platforms don’t actually hate bots.
Bots create activity.
Activity creates “engagement.”
Engagement looks good to advertisers.

If you’ve ever posted something and watched a wave of identical comments roll in?

Bots.

If you’ve ever seen a random account with no posts liking everything instantly?

Bots.

If you’ve ever read a “debate” that feels scripted?
Bots arguing with bots while real people scroll past.

And the craziest thing?

Most of this synthetic engagement is invisible unless you know what to look for:

repeated phrasing, recycled emoji chains, accounts created in batches, activity spread across impossible hours.

Social media platforms used to reflect human culture.

Now they reflect automated behavior at scale.

This is the new reality:

You’re posting into a crowded room where half the audience isn’t alive.

It’s not the end of social media.

But it 'is' the end of assuming the person replying to you is a person.

If you want, I can write a follow-up post on:

• How bots shape political conversations
• How fake engagement distorts “virality”
• How creators can survive in a synthetic social feed

[AI Safety Action @Aisafetyorg]

We’re heading toward an online world where real voices are rare. And the only way out is to rebuild trust from the ground up.

Most people still think the synthetic web is “just spam.” It’s way bigger than that.

Bots now generate:

• A majority of global traffic
• Huge portions of social engagement
• Entire waves of fake accounts
• Whole ecosystems of AI-written articles
• Automated comments, reviews, posts, and replies

And the detectors can’t keep up.
Models rewrite their own outputs, randomize grammar, and slip right past filters.

CAPTCHAs?
Vision models breeze through them.

Verification?
Fake IDs and deepfake selfies handle that.

Fact-checking?
Too slow, too reactive, too overloaded.

The current defenses are designed for a world that no longer exists.

The rebuild has already started:

Cloudflare is charging agents for crawling.
OpenOrigins is tying content to real devices at the moment of capture.
BitGPT is creating economic rules for machine-to-machine access.

This next phase isn’t about stopping AI.
It’s about anchoring human trust so the web doesn’t collapse into white noise.

A synthetic internet doesn’t destroy information.
It destroys confidence.

Rebuilding that confidence is the entire game now.

[AI Safety Action @Aisafetyorg]

The internet isn’t dying. It’s being replaced and most people haven’t noticed.

Half the posts you scroll past were never written by a real person.
Half the comments underneath them weren’t either.

And half the accounts engaging with all of it are bots talking to bots in a synthetic echo chamber that keeps growing by the day.

The Dead Internet Theory used to sound like a joke. In 2025, it reads like a status update.

Bot traffic is nearly half the web.
Fake accounts flood every platform.
AI-written articles bury real writers.

Engagement bait images go viral because the bots boost each other.

And the wild part?

Platforms have zero incentive to stop it.
Traffic is traffic. Engagement is engagement.

Even if it comes from an army of artificial ghosts.

I spent time studying how easy it is to add to the noise. In two hours and a few lines of code, you can build a “slop bot” that spits out endless comments and posts.

Two hours.
And almost free.

You start to see the problem differently after that.

So what do we do?

First step: recognize the signs.
Synthetic text has patterns.
Synthetic images carry watermarks.
Synthetic accounts repeat the same habits.

The tools to detect this stuff exist, but the web is growing faster than the detectors can keep up.

Second step: push back.

If you want the human web to survive, you have to actually support the humans on it.

Share real creators.
Verify your sources.
Call out obvious fakes.

And, if you work in data or AI, build the systems that make authenticity traceable again.

The scariest part of the synthetic web isn’t the bots.
It’s when humans stop caring that bots are speaking for us.

The real internet doesn’t disappear all at once.
It fades quietly while nobody pays attention.

Don’t let it.