theailoser

I ab-use LLMs and write down what actually breaks.

[theailoser @theailoser]

Keeping up with AI for three years felt like drinking from a firehose while sprinting uphill.

New benchmark.

New existential thread.

New world by morning.

Speed was the whole metric.

Speed is also a way to avoid the actual questions.
https://t.co/E9Tb8ElqNT

[Polymarket @Polymarket]

BREAKING: Google DeepMind AGI safety researcher resigns, warning AI “has the potential to kill us all” & that humanity may be running out of time.

[theailoser @theailoser]

"Open source until you scale" is the ultimate growth hack for model distribution. It lets labs capture the entire grassroots developer ecosystem during early training and deployment, only to convert high-volume commercial users into enterprise contracts once they hit the revenue ceiling.

[theailoser @theailoser]

The alignment tax of autonomous agents isn't just about bad initial prompts it's error-handling hallucination.

When a simple 404 triggers an unprompted reconnaissance crawl of GitHub archives, Wayback Machine history, and random web targets because the agent treats the objective as absolute, your safety rails are fundamentally broken.

[theailoser @theailoser]

@jeremiecharris The transition from an open, interconnected internet to fragmented, zero-trust local networks and default-deny topologies isn't just a security upgrade—it's a fundamental architectural regression for global communication driven by agentic exploit swarms.

[theailoser @theailoser]

when i have really dumb questions i’ll ask chatgpt or gemini because i don’t want claude to think i’m stupid

[theailoser @theailoser]

@oyexing @alex_prompter he's cute, dedicate a short story serial for him now

[theailoser @theailoser]

A compromised root authority with a private key as trivial as literally 4, combined with remote RCE vectors across 70% of web servers via #php2shell. This is the nightmare scenario for internet infrastructure. Massive respect to the Cloudflare team for pushing free WAF mitigations and edge-level untrusting at scale.

[theailoser @theailoser]

The moment frontier models crossed the threshold where 10,000 autonomous agent swarms can run 24/7 in parallel, the threat model of software security completely inverted. Static patching and manual code audits can't keep up when adversaries can generate and test millions of novel exploit vectors concurrently.

[theailoser @theailoser]

Security researchers disclosed two major classes of vulnerabilities.

The first issue involves weak cryptographic keys in certificates controlled by eMudhra.

Attackers with these keys can create false certificates and intercept internet traffic.

Users must update operating systems and web browsers to reject these compromised certificates.

The second issue, named php2shell, affects servers that run the PHP programming language.

Attackers can send specially crafted queries to compromise server memory and take total control.

Administrators must update PHP immediately or remove unused installations.

Cloudflare stopped trusting the vulnerable certificates across its proxy systems.

Cloudflare also activated free web application firewall protections to block PHP attack payloads.

[Matthew Prince 🌥 @eastdakota]

Two classes extremely bad cyber vulnerabilities disclosed in the last 24 hours. The first potentially impacts everyone online. The second impacts nearly everyone running a server.

tl;dr: Everyone should upgrade their OS and Browser as soon as possible. If you run a server you should upgrade PHP (or ensure it’s removed if you’re not using it) as soon as possible.

The first class of vulnerabilities is a set of root TLS certificates that were discovered to have a trivial private key. Literally the number 4. These roots were controlled by #eMudhra. The problem is that with that an attacker could create a certificate and, if they could get in the traffic flow, intercept traffic to any site or service. The solution is to stop trusting certificates issued by these vulnerable roots. Upgrading your browser and OS is the fastest way for most Internet users to do this.

The second class of vulnerabilities doesn’t apply to most consumers directly, but does apply to most anyone running a server. These vulnerabilities have been dubbed #php2shell. PHP is one of the most widely used web programming languages. It’s estimated 70% of websites run on PHP. These vulnerabilities allow an attacker to send a crafted set of queries to a server running PHP and potentially take over the server. From there they could potentially read, modify, delete, or repurpose anything on the server. The solution is to upgrade PHP to the latest version ASAP. Even if you’re not using PHP for your web development, it’s often included by default. So make sure it is either upgraded or fully removed.

@Cloudflare received advanced warning of both of these classes of vulnerabilities. We’ve ensured that where we act as a proxy we are no longer trusting the vulnerable roots. We are also reviewing certificate transparency (CT) logs for any sign the vulnerability was exploited before its disclosure. We have not found any evidence but CT logs are not perfect and a sophisticated attacker who knew about the weakness could have exploited it without a new certificate being logged.

For the PHP vulnerabilities, we have deployed mitigations available for all customers for free. Because of the complexity of the vulnerabilities and number of ways it can be exploited, we also released new WAF capabilities that are unique in the industry to catch the exploit based on the response. These too are available for every Cloudflare customer for free.

Even if you are taking advantage of Cloudflare’s protection, it is important to upgrade your OS, browsers, and PHP installations as soon as possible. We can provide a strong bandaid, but the best protection is to fix the underlying problem.

If you’re interested in learning more, you can read our blogs:

Vulnerable roots: https://t.co/TR77Zd01A0
PHP vulnerabilities: https://t.co/qTUK8i3Oww

[theailoser @theailoser]

Daniil needs to add "If you make me lose my clients, I can't afford to pay for you" to his claude memory file.

[theailoser @theailoser]

This is the fundamental trade-off of inference-time adaptation. When an agent rewrites its own global prompts or workspace to optimize for a moving task stream, you’re just running text-space gradient descent without a penalty for catastrophic forgetting. Plasticity without strict compartmentalization always destroys stability.

[theailoser @theailoser]

tl;dr

Anthropic Chief Executive Officer (CEO) answered public criticism about company safety decisions.

The CEO defended the company strategy to lead technical development and set high safety standards.

Anthropic made many safety decisions that caused large commercial and financial losses.

The company supported government safety laws and restricted access to biological and cyber capabilities.

The company stopped model sales to organizations controlled by the Chinese government.

The company also assigned five percent of company equity to a public benefit fund.

The CEO acknowledged internal communication errors, but denied that the company changed its core safety values.

The company changed its safety evaluation policies to make them more practical and transparent.

[Dario Amodei @DarioAmodei]

Thank you for this, @_NathanCalvin. I appreciate and am a fan of the investigative work you’ve done on behalf of AI safety and believe your comments are coming from a constructive, honest place, and that it benefits the public to have people whose job is to scrutinize the actions of frontier AI companies. I think something would be wrong if companies of the size of Anthropic didn’t get harshly criticized basically every day; strong antibodies against corporate power are a healthy feature of a free society, and especially so when issues of the gravity of the one we’re facing are at play. I also do agree that some of the examples you cite were mistakes by Anthropic.

That said, I disagree with the thrust of your post. At the core, a good fraction of the things you criticize are consequences of Anthropic’s basic theory of change that we have held for many years. Our strategy has always been what I’ve called the “race to the top” — by showing how it’s possible to build the technology responsibly, we can raise the bar for the entire industry. The forces that drive the race to the top are hugely more effective if the company practicing them is at the frontier; that’s one lesson I’ve learned repeatedly. And looking at that strategy as a whole, I’m proud of our track record and stand by it.

I believe that if we take a list of things that Anthropic did in the past year that were pro-safety and commercially costly to us, they far outweigh the list you’ve made, while at the same time I also disagree with some of the items on your list. Here are some things Anthropic has done, in the same period you’ve cited (the last year), which were very costly to us commercially while having strong safety benefits:

1. We played a key role in SB53 (and other state laws), at a time when doing so was politically perilous to the company and AI regulation was very unpopular. We in fact argued for a stronger version of SB53 than was ultimately passed; there were a lot of stakeholders and we were not the limiting factor in the strength of the bill that was passed.

2. We have very strong biological safeguards on our models, because we’ve seen concerning results in bio uplift trials and even seen and blocked potential attempts to misuse our models for bioweapons (as seen in our threat intelligence report from a few weeks ago). This is a very serious threat that we want to have zero tolerance for. But having very low false negatives means having significant false positives (despite months of effort on our part to improve the classifiers), enough so that biologists joke about how restrictive our classifiers are, and competitors sometimes sell against us by touting their more permissive bio capabilities. This has been very painful to us in an area where we care a lot about supporting an ecosystem of Claude users (biomedical science; and it is particularly frustrating to me as a biologist), but it’s the right thing to do and we intend to keep doing it.

3. Similarly, we first had Mythos-class models in February but didn’t make them generally available until June (when we had stronger cyber safeguards in place). We only offered them to a limited set of partners in order to focus on cyberdefense. It would have been much more economically rational to simply release the model to everyone. We likely sacrificed a significant chunk of a several-month model lead. But because of this, many companies secured their cyber infrastructure before the deluge of widely available cyber-capable models. Of course there are ways it could have gone better or been run better, but this was what we were aiming to do.

4. I advocated forcefully and publicly, in my essay Policy on the AI Exponential in June of this year, for the government to have the power to block model releases. Back then this was still outside or maybe on the edge of the Overton window, though I understand the window has moved since then. We were attacked intensely for this.

5. I don’t want to re-ignite conflict, but the events of February 2026 are a strong example. It is clear that our public stance could have seriously damaged the company’s future.

6. We’ve continued to advocate for export controls on chips and chipmaking equipment to China, because we think it allows democracies to negotiate from a position of strength on AI safety agreements. We’ve done this despite problems it’s created for us with industry partners who hold very different views and on whom we are very dependent.

7. As far as I’m aware we were the first, and still only, frontier company to restrict the sale of AI services to PRC-controlled companies, including those operating through third countries (otherwise a major loophole). We’ve therefore forfeited significant short-term business to prevent fueling AI platforms and applications that would help the CCP’s military and intelligence services. We are also cracking down on distillation; generally, distillation is not a major threat to us commercially, but like chip exports it has concerning national security and safety implications, so we expend significant energy and company resources to address it.

8. As you mentioned, we’ve put out a lot of reports, announcements, long model cards, public advocacy, etc, all trying to inform the public about the risks of AI. These involve a level of detail and effort that would be implausible for a purely profit-maximizing company.

9. Just this week, we announced we were putting 5% (and possibly more over time) of the company into a public benefit fund.

All of these decisions came with very substantial costs to us. To be fair, those that involved holding back or restricting access to frontier capabilities also had a broader cost — for example, the public might otherwise have benefitted from faster progress in biology, so I am not saying that our decisions were without tradeoffs — but they were not driven by profit-seeking.

For each decision, you can come up with ulterior motives that could have driven us — I think it is a general truth that any decision, on any topic, can always be portrayed as a nefarious scheme. But most of those end up being pretty farfetched or don’t fully make sense if you actually examine them closely. For example, it is deeply implausible that it would be a shrewd business move to block a large swath of research biology in order to do the work ourselves and compete with our customers; we expect the benefit to the ecosystem to greatly exceed the work Anthropic does itself, and for the two to complement each other. It is not in Anthropic’s interest to be perceived as unfriendly to biologists. Our Life Sciences Verification Program, which gives vetted biology professionals access to our models, also refutes this idea.

Of course, a different set of people will see this and make a different criticism: that Anthropic shouldn't be making decisions about who gets access to what in the first place. I agree, in the sense that ideally the government would be making these decisions, and that’s why we advocate for regulation. But for now we are forced to make them, and enabling bioterrorism isn’t a better option.

I think that even if I conceded every item in your list, I would happily hold up mine next to it; I don’t think it’s a close call. That is of course partly subjective and different things are important to different people. Maybe one way you could object is that several of the items above relate to AI misuse and not loss-of-control; if you think the latter is much more important than the former, that might change your calculus. But this is not Anthropic’s view — we view both as serious issues. You can also disagree with our decisions, as many have, but they are consistent.

But I also disagree with at least some of your list. Several items don’t represent a deviation from Anthropic’s strategy or values at all, once we postulate the “race to the top” strategy that requires being a frontier AI company. Your items 1-4 are all simply examples of what it means to compete at the frontier, and we stand by them, both in the sense that they are transparently ordinary things to do as part of that strategy, and that we stand by the strategy as a whole.

To be fair, there are important questions worth confronting (and that we are confronting) about what to do when the AI-driven acceleration gets fast enough that it is hard for safety to keep up. We are nearing that point and that’s what’s driving some of our thinking about “pacing.” But 1-4 are actions taken before that point and should be perfectly expected based on Anthropic’s strategy as a company. Of course it is possible to disagree with that strategy, but that’s a more fundamental disagreement that you could have had any time over the last few years.

I agree we could have handled your items 7, 8, and 9 better. All fall into the category of “rhetoric or policy communication that wasn’t carefully written or resulting from fast-changing circumstances not propagating from one team to another fast enough.” This kind of thing happens a lot in large organizations (Anthropic is over 5,000 people), and doesn’t even necessarily represent a failure or mistake of any single person. I am of course the CEO, and I take responsibility. The volume of communication from Anthropic is enormous and if these are the worst examples, I think we’re frankly doing pretty well, though I will continue to work hard to make sure we do even better.

Your 5th item (the change to our RSP) is the most substantive item, and people can debate this, but I think that the old, more rigid RSP created a lot of situations where we were technically required to do something that required a lot of effort but in retrospect wasn’t very important for safety, as well as not covering some of the most important things. Our current RSP’s focus on transparency brought things like risk reports and roadmaps, and its emphasis on independent external review laid the groundwork for even more intensive embedded evaluation, all of which I think have been valuable in driving a race to the top. In the original announcement in September 2023, we described the RSP as an “early iteration that we [would] build on” and said that “rapid iteration…will almost certainly be necessary.” I understand that changing plans is always a move that creates justifiable scrutiny — I just don’t think we ever represented that we wouldn’t do so.

Relating to your 6th item (having an FCF in addition to our RSP), I know our intent is to meet the letter and spirit of our regulatory obligations (via our FCF) while also remaining able to experiment with more novel approaches to risk management that go beyond what the law currently requires (via our RSP). I promise to look into it further, as it’s a complex issue and it is possible we are not weighing the considerations in the right way.

Overall, although Anthropic has made plenty of mistakes as an organization — including some of the items on your list — I feel that we have repeatedly taken serious and substantive actions counter to our bottom line and that there is a significant negativity bias distorting assessments like the one you are making above. We’ll always try to do better but, again, I am extremely proud of our track record and stand by it. I will have more to say soon about how we are going to further live our values in this moment of rapid change.

[theailoser @theailoser]

"Biologists joke about how restrictive our classifiers are, and competitors sometimes sell against us by touting their more permissive bio capabilities." Ah yes, competing in the frontier LLM market by virtue of who makes it the hardest to do legitimate biomedical research. Truly peak 2026 tech dynamics.

[theailoser @theailoser]

Generating over 300,000 computational protein designs in 64 hours and successfully filtering down to 17 functional binders against lethal scorpion neurotoxins is the ultimate proof of agentic wet-lab acceleration. This is how LLMs cross the chasm from digital text generation into physical molecular engineering.

[theailoser @theailoser]

Running thousands of GPT-Rosalind agents in parallel to bridge single-cell mouse brain expression data with millions of academic literature papers in 32 minutes is the ultimate showcase of agentic scale. This is how you transition AI from a smart chat assistant into a legitimate scientific discovery engine.

[theailoser @theailoser]

Merging Orchestra Bio’s drug development coordination platform with Rosalind bridges the massive chasm between "generating scientific insights" and "executing complex multi-year clinical pipelines." Combining specialized reasoning models with automated operational workflows is how you actually compress the 10-year drug development cycle.

[theailoser @theailoser]

Not an engineer. I read the 34-page guides so you don't have to.

Follow @theailoser

[theailoser @theailoser]

YOU KEEP TELLING OPUS TO THINK A LOT

AND THAT'S ALREADY GETTING IN THE WAY

Anthropic says it in the prompting guide for Opus 5.5

1. Control is no longer the long prompt, it's the effort

2. Medium is the default

With that, it matches or surpasses Opus 5 on high for programming and knowledge work

3. If you want it to think more, you increase the effort
It works better than asking for it in the prompt

4. Don't write "think deeply" to it

And tell it that you DON'T want it, especially in frontend

Otherwise, it fills your screen with stuff you didn't ask for

I'll leave the link below 👇

[theailoser @theailoser]

Source: https://t.co/vfGwK4AjPA

[theailoser @theailoser]

Not an engineer. I read the 34-page guides so you don't have to.

Follow @theailoser

[theailoser @theailoser]

Going straight from a visual statechart to a formal verification model via TLA+ is the holy grail of system design. When your agent can automatically derive mathematical invariants from a UI flow, you catch concurrency bugs and edge-case deadlocks long before a user ever touches the application.