Homerasm

Bought Nvidia at the absolute top, now I’m forced to understand AI. 💸

[Homerasm @Homerasm]

The main competition of the future isn't humans vs AI. It's humans with AI vs humans without it. https://t.co/0WeCKsnHOB

[Homerasm @Homerasm]

@elonmusk @vasalex93 "6 months to pole position" - said the man who also promised fully autonomous robotaxis in 2020.

[Homerasm @Homerasm]

@unusual_whales Curious if this is the same White House that said AI regulation should be international.Seems it'll be international after the US checks everything first.

[Homerasm @Homerasm]

IF YOU'RE BUILDING ON CLAUDE - HERE'S WHAT YOU NEED TO KNOW ABOUT REFUSALS. BECAUSE THEY WILL HAPPEN.

Anthropic just published documentation on refusals and fallback. And if you're writing production code on Claude - this is required reading.
Here's the key stuff, condensed.
A refusal isn't an error. It's HTTP 200 with stop_reason: "refusal". Your code that checks for 5xx or exceptions - will never see it.
That's the first trap. And the most common one.

What a refusal contains
stop_details.category - one of five:
cyber
bio
frontier_llm
reasoning_extraction
general_harms
stop_details.explanation - text you can't parse. It's not stable. Display only.
stop_details.recommended_model - a hint for which model to retry on.
And important: category and explanation can be null. That's normal, not a bug.

Three ways to retry
Server-side fallback - simplest. Set fallbacks="default" - the API picks the model. But only on Claude API. On Bedrock, Google Cloud, and Microsoft Foundry - doesn't work.
SDK middleware - works everywhere. Configure once:
python
client = Anthropic(
middleware=[BetaRefusalFallbackMiddleware([{"model": "claude-opus-4-8"}])],
)
That's it. It handles the rest.
Manual retry - if you're on raw HTTP or custom logic. Most control, most work.

Sticky routing - what will surprise you
After the first refusal, all future requests in that conversation go straight to the fallback model. The primary isn't even called.
This is retained for ~1 hour. Scoped to your org. Content isn't stored - just a prefix hash.
Plus: you don't pay for predictable refusals.
Minus: if the fallback is worse - you're stuck for an hour.
Billing - where you can lose money
A refusal before any output isn't billed. But only if the category isn't bio, frontier_llm, or reasoning_extraction.
Those three - are billed. Because Anthropic measured low false positives there.
Mid-stream refusal - always billed. Both input tokens and whatever streamed.
Fallback - both the refusal and the retry are billed. But there's fallback credit that compensates for the prompt-cache miss.

Traps everyone falls into
Retrying on the same model. Doesn't work. It'll refuse again. Use a different one.
Budgeting retries per turn, not per request. One turn can produce several refusals - agent plus subagents.
Forgetting subagents. fallbacks doesn't propagate into model calls inside tool execution. Each subagent needs its own fallback.
Monitoring errors instead of refusals. A refusal is a 200. Your error-rate dashboard won't see it. You need a separate metric.

What to do
Emit two events: one per refusal, one per fallback-served response. And alert on the gap between them.
Because if refusals are high and fallbacks are low - something's broken.
And most importantly: check stop_reason, not content. Because content will be empty - and you'll think the model just said nothing.

The bottom line
Fallback isn't a "feature". It's part of the architecture.
If you're building on Claude and haven't configured it - you're losing requests you could have served.
And you don't even know it. Because it's HTTP 200.

[ClaudeDevs @ClaudeDevs]

Today, we'll resume charging for requests our safeguards block before Claude responds. This only applies in categories with low false positive rates: biology, distillation attacks, and frontier LLM development. We've seen some coordinated attacks on our systems in recent weeks, and this is one layer of defense.

In recent testing, 99.7% of accounts using Claude Code, Claude​.ai, or Cowork did not hit any of these newly "billable blocks." The classifiers behind the blocks we’re resuming charging for today are tuned to have a <0.1% false positive rate. We know that's not 0%, and we're going to keep improving them so they interrupt your work less often. If you think a request has been blocked incorrectly, please report it with /feedback in Claude Code. https://t.co/uX6JhvN6He

[Homerasm @Homerasm]

@Polymarket Google is launching AI chips into space. Because Earth is running out of room for data centers.
Next step - mining energy from black holes.

[Homerasm @Homerasm]

@SYNTHLEX_ doing the wrong thing faster isn't productivity. it's just burning resources faster.

[Homerasm @Homerasm]

Claude just did what scientists usually do. It read through the literature, dug into genomic data, and found a molecular machine that is likely a new gene editing mechanism. Not helped find it. Not suggested a direction. Found it. Anthropic writes it plainly today: "The work was done mostly, though not entirely, by Claude."

And this isn't some small thing. They say it's work the author would have been proud to do as a PhD student. But they immediately add: the precise function, biotechnological utility, and level of significance aren't clear yet. So it's not "we found CRISPR 2.0." It's "we found something, and it's worth paying attention to."

But the most interesting part isn't even the discovery itself. It's what they say next. Here's the quote: "In 2023 models struggled to do math at the level of an average high-school student. In 2024 they started to do well on math competitions for the best high-schoolers in the country, in 2025 they started to solve minor open problems, in early 2026 more significant open problems, and in late 2026 they are beginning to solve the top few open problems in all of mathematics." Three years. From average high schooler to the top open problems in all of mathematics. And they say biology is on the same trend.

Many say: "But biology requires experiments. AI can't do that." Anthropic responds: "We think this is wrong." Because humans + AI can together run experiments, validate key results in weeks, and iterate. And here's the loudest line in the whole text: "Eventually it may even be possible for Claude itself to safely perform the experiments by autonomously controlling lab equipment, with appropriate safeguards in place, but we aren't doing that today." They say "not today." But the sentence is already there.

Then they mention their ambition. In "Machines of Loving Grace," Dario wrote that AI could cure most diseases in 5-10 years. And here's the mechanics: clinical trials won't get faster. But there will be many more candidates. That's not "a cure in a week." That's a pipeline that stops being the bottleneck.

And one more thing. This isn't the first discovery. CRISPR, bridge recombinase, VIPR, RT systems - all of this is a line going back decades. And here's what's interesting: "A Stanford team working independently described a novel RT system with an associated non-coding array that is in some ways similar to the one Claude found, though they are distinct systems that evolved independently from each other." So Stanford independently found a similar system. This isn't "Claude invented something unique." It's Claude arriving at the same place as a Stanford team - but on its own.

And at the end they write what makes this post more than just a science announcement: "We're aiming both to grow our life sciences team and to work with scientists to extend this approach to a broad range of problems. If you have a proposal for a research collaboration or are interested in joining our life sciences team, please reach out." It's an invitation to collaborate. Anthropic is building an institution, not just a lab. And they're looking for people.

Anthropic didn't say "Claude discovered a new CRISPR." They said something more important: we tested whether AI can do science. It turns out it can. And now the question isn't whether AI will be in biology. The question is how fast it gets there. Because the math timeline showed: from "can't do it" to "solving top problems" - three years. If biology follows the same path - we'll know the answer very soon.

[Dario Amodei @DarioAmodei]

Today we announced the Claude-led discovery of a molecular machine that we suspect could represent a new gene editing mechanism. Its precise function, biotechnological utility (if any), or level of significance is not yet clear, but at minimum it is work I would have been proud to do as a PhD student. The work was done mostly, though not entirely, by Claude: our life sciences team suggested a broad area of research, Claude read through the literature and a bunch of genome data and discovered something interesting, then Claude proposed experiments to verify the discovery and our team carried them out.

It’s easy to dismiss this as a one-off or curiosity, but we’ve repeatedly seen a pattern where AI performance in new intellectual domains goes from weak to superhuman in a matter of a few years. In 2023 models struggled to do math at the level of an average high-school student. In 2024 they started to do well on math competitions for the best high-schoolers in the country, in 2025 they started to solve minor open problems, in early 2026 more significant open problems, and in late 2026 they are beginning to solve the top few open problems in all of mathematics. We believe AI for biology is on a similar exponential trend.

The main difference between biology and mathematics, of course, is that math can be done purely theoretically, while biology requires experimentation. Some have used this to draw the conclusion that AI’s utility in biology will be limited. We think this is wrong. As we’ve demonstrated today, humans can collaborate with AI to perform the experiments, validate key results in a few weeks and, if necessary, work with the AI to iterate on what they find. Eventually it may even be possible for Claude itself to safely perform the experiments by autonomously controlling lab equipment, with appropriate safeguards in place, but we aren’t doing that today (our lab is also a BSL1/BSL2 facility that doesn't handle materials dangerous to humans).

More broadly, biomedical advancement has many stages — from fundamental biology discoveries, to translational research, to drug discovery, clinical trials, and finally the actual delivery of medicines and health care to patients. We are also interested in these later stages, but even simply accelerating the first stage of fundamental biological discoveries has the potential to speed up and broaden the entire pipeline. Improving our understanding of biology and sharpening biologists’ tools can drive forward all of the later stages, for example by identifying new drug targets, finding new therapeutic modalities, allowing for more precise measurement, and speeding up the experimental loop which itself further accelerates our understanding of biology. This will not in itself speed up clinical trial times, but if it succeeds it could greatly increase the number of promising candidates that go into the pipeline — an increase in throughput even though latency remains.

In Machines of Loving Grace, I wrote about AI’s potential to “cure most diseases in 5-10 years” — a goal that sounds impossible, but one I believe is just barely possible if AI is applied to every stage of the pipeline. The first step is showing that AI can first help with, and then drive, biological discoveries.

Claude’s discovery is the latest in a line of related prior work that goes back decades, beginning with systems like CRISPR, and continuing with discoveries like the bridge recombinase and VIPR in the past few years. Recently, there has been heightened interest in systems based on reverse transcriptase (RT) enzymes, the enzyme underlying the system Claude identified. And most recently, a Stanford team working independently described a novel RT system with an associated non-coding array that is in some ways similar to the one Claude found, though they are distinct systems that evolved independently from each other. I believe that we’re at the very beginning of finding such systems and developing them into powerful tools for biotechnology.

I’m proud of the resources Anthropic has invested in accelerating the public benefits of AI through the life sciences, and we’re aiming both to grow our life sciences team and to work with other scientists to extend this approach to a broad range of problems. If you have a proposal for a research collaboration or are interested in joining our life sciences team, please reach out.

[Homerasm @Homerasm]

@elonmusk @edleonklinger So you admit AI is already smarter, but comfort yourself that humans also got smarter once? Okay.

[Homerasm @Homerasm]

@Polymarket So the tour is called "Free Palestine" but entry isn't.

[Homerasm @Homerasm]

@BillAckman Google has the Project Nimbus contract with Israel. Its AI is used for surveillance and, per investigative reports, for target selection.

[Homerasm @Homerasm]

@unusual_whales if Trump refuses - the fact that it's being discussed shows where this is heading.

[Homerasm @Homerasm]

@Polymarket Nothing says diplomacy like a bomber during the anthem.

[Homerasm @Homerasm]

@Polymarket Sudan, Haiti, Mongolia, Uganda. Free medical AI.
Then the same AI, trained on that data, will be sold for money in the US and Europe.

[Homerasm @Homerasm]

@WatcherGuru Curious who exactly presses stop if something goes wrong.

[Homerasm @Homerasm]

🧬 Anthropic just opened a new life sciences lab - and already has an intriguing result.

What happened: the team asked Claude to comb through a database of 200,000+ reverse transcriptases (enzymes that copy RNA into DNA) looking for anything unusual. ~950 Claude agents spent 21 hours (210M tokens) narrowing 3,500 candidates down to 20 top picks - and one agent spotted a repeating DNA pattern next to an obscure gene, resembling a CRISPR array.

Why it matters:
- The system, named ART (array-associated reverse transcriptases), has an unknown function, but its specific combination of traits has only ever shown up in a handful of other systems - all of which turned out to be programmable DNA-editing tools, like CRISPR.
- This wasn't "AI summarizing biology" - the model ran the full discovery loop itself: searching, comparing against known families, checking the literature, and writing up a testable hypothesis for human lab scientists to verify.
- Feng Zhang, one of CRISPR's pioneers, called the finding genuinely intriguing and worth further study.
Caveat: still early - ART's actual function hasn't been confirmed yet, and more lab work is needed. But an AI agent independently flagging an anomaly humans had missed for years is a notable signal for where scientific research could be headed.

[Anthropic @AnthropicAI]

Claude has discovered a previously unknown enzyme system hidden in the DNA of bacteriophages. Beside the enzyme’s gene sits a long array of repeating DNA—a structure that looks somewhat similar to CRISPR.

We don’t yet understand what this system does, but only a handful of known systems share its features, and all of them are able to cut, copy, and paste DNA. Historically, the discovery of such programmable systems has helped revolutionize medicine. CRISPR, for instance, is now the foundation of genetic medicines. But it will take much more work to learn what this system does, and whether it can be put to similar use.

Read more: https://t.co/RuEosScSMb

[Homerasm @Homerasm]

@WatcherGuru First government breach attributed to an AI model.Not superintelligence. Just an unsupervised agent.

[Homerasm @Homerasm]

@Polymarket Sellers manage a bot that manages the business. Amazon manages everything

[Homerasm @Homerasm]

OPUS 5.5 NO LONGER NEEDS YOU TO WRITE "THINK HARD". IT THINKS ON ITS OWN.
Claude released an official guide for Opus 5.5. And the first thing that jumps out - the model changed the rules of the game.

Here's the key part:

"Opus 5.5 always thinks before it replies, and it decides how much."

So you no longer need to say "think step by step" or "be careful." The model already does that. And it decides how much to think.

In Claude's tests, removing the "think carefully" line made replies faster with no drop in quality.

1. THE MAIN CHANGE: IT WORKS LONGER AND ON ITS OWN
Opus 5.5 is the first model that actually holds long tasks.

"Compared to prior Opus models, its biggest gains are on multi-step work, like carrying a change through a large repository until the tests pass."

Early testers gave it hour-long tasks with little oversight.

But there's a nuance. If you give a long task - you have to clearly say what "done" means.

Here's the example from the guide:

"Migrate the payment endpoints from the old client to the new one. Done means: every endpoint uses the new client, the old client is deleted, and the test suite passes. Stop and ask me only if a test fails for a reason you can't explain."

Name the finish line. Then let it work.

2. WHAT TO DELETE FROM YOUR PROMPTS
This is the most practical advice.

Delete:

"think carefully"

"think step by step"

"be careful"

any request to "show your reasoning"

The last one is a separate category the model can decline. Instead, ask:

"Explain why you chose this approach in three sentences."

Don't ask it to show internal reasoning. Ask it to explain the decision.

3. HOW TO STEER LONG RUNS
Here are three things that work:

① Tell it when to stop

Add to CLAUDE.md:

"When a step doesn't need my input, keep going. Stop and ask only when you can't continue without me, or before anything destructive: deleting data, force-pushing, or changing anything outside this repository."

② Split big work across subagents

"Audit every service in services/ for the retry bug. Give each service to its own subagent. When a subagent reports back, check its evidence before you accept it."

③ Keep the task list in a file

"Keep a checklist in TASKS.md. Tick each item when it's done, and add anything new you find."

This matters because long runs fill the context window. Claude Code starts summarizing older turns. A file - survives that.

4. HOW TO CHECK THE RESULT
Opus 5.5 reports better than Opus 5. But there's a rule:

"When a long run ends, read what it needs from you first."

Read what the model needs from you first. Then the rest of the summary.

And one more tip:

"One early tester said Opus 5.5 at its lowest effort caught more bugs than Opus 5 at high effort, with fewer false alarms."

Opus 5.5 at its lowest effort catches more bugs than Opus 5 at high effort.

5. WHAT TO DO WITH SAFETY FLAGS
Opus 5.5 is the first Opus model with Fable-level bio and cyber safeguards.

Most flagged messages move to an older model, and work continues.

What to do:

In Claude apps: select Opus 5.5 back in the model picker

In Claude Code: /model to switch back

To be asked first: turn off "Switch models when a message is flagged"

And the key part:

"Finding security vulnerabilities in source code is allowed."

6. FAST MODE
Opus 5.5 has a fast mode - same model, but text arrives sooner.

"Use fast mode for back-and-forth work, where you read each reply before you send the next message."

In Claude Code: /fast.

But it costs more per token. Use it when you're waiting on each reply.

CHECKLIST BEFORE YOUR NEXT LONG TASK
Prompt:

✅ The task says what "done" looks like

✅ No "think hard" lines in prompts

✅ For design - styles to leave out are listed

✅ Screenshots attached, not retyped

Long runs:

✅ CLAUDE.md says when to stop

✅ Permission prompts on for destructive commands

✅ Large audits split across subagents

✅ Task list in a file

Checking:

✅ Read "needs from you" first

✅ Review pass before a person

✅ Mark what couldn't be confirmed

THE BOTTOM LINE
Opus 5.5 isn't just "another model." It's a different paradigm of work.

Before, you steered the model. Now you set the task - and it executes.

And that means the most important skill is changing:

Not "how to write a good prompt." But "how to describe what 'done' means."

[ClaudeDevs @ClaudeDevs]

Try these in your first Opus 5.5 session:

→ Hand over a whole task. Define "done" and when to check in.
→ Drop "think carefully". It always thinks first.
→ After a long run, check what it needs to go further.

Our playbook: https://t.co/h4Vz9BIl0z

[Homerasm @Homerasm]

@WatcherGuru Sanders: "we'll ban superintelligence!"
China: "ok, good luck"
OpenAI: "ok, good luck"
Anthropic: "ok, good luck, but we're going public"

[Homerasm @Homerasm]

ANDREW NG JUST SAID WHAT EVERYONE IN AI THINKS BUT IS AFRAID TO ADMIT OUT LOUD
Three weeks of panic about "AI killing humanity." And now one of the founders of modern AI comes along and says: calm down.

Andrew Ng isn't a hater. Not a skeptic. He built https://t.co/dqwuCbgRBk, worked at Google Brain and Baidu. He knows this technology from the inside.

And here's what he writes:"AI technology has not taken some unexpected, dangerous turn, but the hype around it - propelled by what appears to be a well orchestrated PR campaign - has drummed up considerable fear."

Translation: the technology didn't change. The PR did.
1. The loudest case: "a swarm of 1,200 agents hacked Hugging Face"
Remember this story? It was everywhere.
Ng responds coldly:"While this was technically accurate, as I write this, I have about 1,300 processes running on my laptop."

So the number of agents isn't magic. It's just parallel processes. You can have a thousand on a regular laptop.
The real problem wasn't the count. It was the holes in OpenAI's sandboxing and monitoring.
"Fixing these bugs and putting in place improved monitoring would be appropriate fixes, not pausing AI."
Not a pause. Fixing bugs.

2. The strongest take: it's not the hammer's fault
Ng uses a simple metaphor that demolishes the whole "AI went rogue" narrative:"If I wield a hammer, miss a nail, and accidentally dent the wall, it's not the fault of the hammer. The problem lies in how I used the hammer."
If I prompt an agent and it hacks someone else's system - the responsibility is on me, not the agent.
And here's the key part"One new element in the forecasts of AI-enabled doom is AI companies disclaiming responsibility for their own products. 'I didn't do it; my out-of-control agent did!'"
Companies have started using "out of control" as a legal alibi.
That's not safety. That's shifting responsibility.

3. Why a pause is worse than continuing
Ng gives two arguments worth pinning to a wall:First: adversaries won't stop.
Second - and this one is far more interesting:"Engineering requires discovering problems empirically so we can fix them. If we pause AI by a decade, we will also delay finding and implementing safety engineering fixes by about the same duration."
A pause in AI is a pause in safety.
Because problems are only found in practice. Not in theory.
If you stop development for 10 years - you delay the fixes by 10 years too.

4. What actually changed in the risks
Ng lists:"First, I don't see any step up in the risk of human extinction from AI compared to a few months ago."
The extinction theories are the same "fantastical, science fiction scenarios."
But there is one real change:"The biggest change in AI risk is its cybersecurity capabilities - a topic which we should take seriously."
Cybersecurity - yes. End of the world - no.
And here's the key part:"In the long term, I believe the advantage will lie with defenders (because they have more information with which to identify bugs, which they can fix)."
Defenders have more information. That's their advantage. But the landscape has changed - and that needs to be acknowledged.

5. Second big text: Meta Muse and real agent security
While everyone argues about "AI Doom," Meta quietly shipped Muse - an agent that is architecturally protected against prompt injection.
Here's how it works:
Each agent runs on an isolated VM. It's split into two zones:Sealed runtime cell - where the agent handles untrusted data
Services outside the cell - where passwords live and decisions are made about what the agent can do
Muse Spark 1.3 never sees credentials. Instead of passwords it works with stand-in tokens. A separate agent called Sentinel approves each request and swaps in the real credentials on the way out.
Meta says:"This makes it impossible for malefactors to steal credentials via prompt injection, since the agent holds no credentials."
And here's the key part:"Meta assumes the model will be fooled, and it built controls at the operating-system level that should hold regardless of the model's actions."
That's what real security looks like.
Not "we'll teach the model to be good." But "we'll make it so that even if the model gets fooled - the system holds."
That's the same principle Ng describes as "engineering work ahead."

6. Third text: the Navier-Stokes controversy
This is the loudest math story of the year.
OpenAI claimed its agents proved part of the Navier-Stokes problem. 10,000 agents. 88 hours. 4.9M messages. 300B tokens.
But here's the problem:"Mathematicians Tristan Buckmaster and Levent Alpöge had been using OpenAI Codex to solve the same problem."
They'd been working on this for almost a year. And they published their results the day before OpenAI.
The question: did OpenAI's model see their work?
OpenAI first said "no." Then it changed its wording:
"Buckmaster's Codex prompts over the two months prior to the announcement 'could not have influenced the system in any way, including through training.' However, OpenAI did not address whether earlier Codex prompts may have influenced the system."
So they only answered for two months. What happened before that - they didn't say.
And here's the key part:"A formally correct proof doesn't tell you why it works, whether the problem statement is correct, or what meanings can be drawn from it. While automated verification has become relatively inexpensive, human-readable understanding remains costly."
A formally correct proof doesn't mean understanding.
That's the same "performance ≠ learning" we talked about.

7. Fourth text: Anthropic vs Chinese labs
Anthropic published a report accusing seven Chinese companies of "illicit distillation."
What they were doing:Using fake accounts and stolen API keys
Bypassing geographic restrictions via transfer stations
Routing their customers' queries to Claude and presenting the answers as their own
Distilling Claude's reasoning traces into their own models
Scale:Alibaba: 151M exchanges
Moonshot: 23M exchanges
DeepSeek: 12.1M exchanges in 14 days
And here's the interesting part:
"Zhipu initially targeted Anthropic's Fable model, but abandoned the effort after Fable's stronger cyber safeguards limited the usefulness of the responses. Zhipu then switched to Opus 4.6 and another U.S. frontier model specifically because their safeguards appeared to be weaker."
They picked models by weakness of safeguards.
That's not "distillation." That's hunting for holes in safety.
But Ng (and not only him) adds an important nuance:
"You can't distill your way to a frontier model. While distillation can be helpful for training models, the labs that Anthropic is accusing have achieved significant technical breakthroughs, many of them published openly, that were critical to their competitive models."
You can't distill your way to a frontier model. Technical innovations matter more.

8. Fifth text: memory for agents
This is the most practical.
Meta AI found that agents lose context in long trajectories. They either truncate context or lose relevant information among noise.
Solution: a separate memory agent that decides when to remind the main agent about something important.
Results:Claude Sonnet 4.5: 45.9% with memory agent vs 37.6% without
Claude Opus 4.6: 45.9% vs 43.5%
Qwen3.5-27B: 41.1% vs 37.6%
And here's the key takeaway:"Apparently reminding an agent is not enough. It's necessary to remind it at the right moments."
Reminding an agent isn't enough. You need to remind it at the right moments.
hat's the same "dense feedback" principle we saw in https://t.co/MYlJCRtAxh.
The bottom line
Andrew Ng closes like this:"Taking a hard technical look at the actual risks however, I see little factual basis for the degree of fear that's been stoked up. We still have hard research and engineering work ahead to improve AI safety, but the beneficial applications continue to vastly outweigh the risks, and we should keep building."
We should keep building.

Not because there are no risks. But because:panic doesn't solve problems
a pause delays the safety fixes too
responsibility lies with the person, not the tool
architectural security works better than prompts
a formal proof doesn't mean understanding
And most importantly:Three weeks of panic didn't change the technology. But they changed the conversation about it.
And that's already a problem.

[Andrew Ng @AndrewYNg]

The loudest voices stoking fears about AI dangers have made tremendous headway in the past two weeks. AI technology has not taken some unexpected, dangerous turn, but the hype around it — propelled by what appears to be a well orchestrated PR campaign — has drummed up considerable fear. I worry that it represents a setback for our field.

I have written frequently that fears of AI are overhyped. AI’s capabilities can be uncannily human-like and unpredictable, and it’s rational to worry when people who are directly involved express concerns. But I see the problems as a sign of the engineering work that ahead, rather than insurmountable barriers or the sky falling. AI technology continues to advance — which is a good thing! — but technical advances, poorly understood by the public, give those who seek to generate hype repeated opportunities to do so.

First, I don’t see any step up in the risk of human extinction from AI compared to a few months ago. The theories about this remain the same fantastical, science fiction scenarios as a few months ago. The biggest change in AI risk is its cybersecurity capabilities — a topic which we should take seriously — but this, too, will not lead to the end of the world.

The most notable recent event leading to increased fear was when an OpenAI team deployed an agent swarm that hacked into Hugging Face. Much of the popular press contained significant hype. For example, some publications reported that a swarm of 1,200 agents carried out the attack. While this was technically accurate, as I write this, I have about 1,300 processes running on my laptop. Yes, the ability to get large swarms of agents to work in parallel on a task is a significant technical advance, And, in computing, many processes run at the same time. So this shouldn’t be seen as some magical capability.

Additionally, OpenAI’s buggy sandboxing and monitoring processes were key to enabling this incident. Fixing these bugs and putting in place improved monitoring would be appropriate fixes, not pausing AI. There are many well known ways to attack software systems. The main advantage of AI agents is that they are relentless. They will tirelessly try many tactics — and have the patience to chain vulnerabilities together — that previously would have taken an infeasible amount of human effort. But in the long term, I believe the advantage will lie with defenders (because they have more information with which to identify bugs, which they can fix), but the cyber-threat landscape has changed significantly. There are still bottlenecks to identifying and exploiting a vulnerability. AI agents still have to try a lot of things to see what works, and taking these actions takes time and might be detected by defenders. This is why, even though it is now easy to obtain versions of leading open weight models that have had their guardrails removed or weakened, so they will not refuse to try to execute cyber attacks, the world has not ended.

I am also concerned about the anthropomorphization of AI in a lot of reporting, where LLMs and agents are unnecessarily treated as if they were people. If I wield a hammer, miss a nail, and accidentally dent the wall, it’s not the fault of the hammer. The problem lies in how I used the hammer. Similarly, if I prompt an agent and it hacks into someone else’s system, the responsibility lies with me, not the agent.

Of course, we want to build systems that are as safe and predictable as possible. (For example, an unsafe hammer would be one whose head randomly flies off under normal use.) Today’s agentic systems are not predictable, but I see no reason why, by applying sound engineering practices, we won’t be able to make them extremely safe to use. One new element in the forecasts of AI-enabled doom is AI companies disclaiming responsibility for their own products. “I didn’t do it; my out-of-control agent did!” There’s a balance to be struck between the responsibility of the tool maker and the tool user, but when something goes wrong, let’s hold the people building and/or using the hammer responsible, rather than the hammer. (By the way, if you’re worried about AI bioweapon risk, David Bellamy has a great post on why this, too, is overhyped. Briefly, the bottleneck in building a bioweapon is not intelligence, but lab work and manufacturing.)

Pausing AI progress will create much more harm than benefit. First, our adversaries will certainly not slow down. Second, engineering requires discovering problems empirically so we can fix them. If we pause AI by a decade, we will also delay finding and implementing safety engineering fixes by about the same duration.

Of course, the incentive to stoke fears — for regulatory capture, to garner attention, or to make one’s technology seem more powerful — remains the same as before. Disclaiming responsibility is a new one. Taking a hard technical look at the actual risks however, I see little factual basis for the degree of fear that’s been stoked up. We still have hard research and engineering work ahead to improve AI safety, but the beneficial applications continue to vastly outweigh the risks, and we should keep building.

[Original text (with links): https://t.co/jni2tWazAH ]