@cafreiman We have an idiom for this in our society: "standing on the shoulders of giants." Being 100 ft in the air doesn't mean you are a 100 feet tall. We have to see what you are standing on top of to figure out your specific height. The inherited technology stack is one of those things
people in san francisco say “agi pilled” as a general term of art for describing people who reason about the future with any degree of analytical skill. it has been imo a good proxy for human capital generally
Banning Mythos represents a basic misunderstanding of AI cybersecurity
The evidence we have now points to defenders currently benefiting more from frontier models than attackers. Every security professional within any large, complex organization knows they are sitting on a mountain of security technical debt they are not resourced to dig out of, and is not surprised when penetration testers reliably make it through their defenses. Defenders are deeply, deeply bottlenecked by labor.
Attackers, who already have the advantage, have less comparative advantage to rush to AI adoption. While new zero-days can be of incremental help, they have reliable TTPs today, which is why today's AI capabilities haven't generated a discontinuous surge in attacks.
The hope of frontier AI is that it could finally resource defenders in a way that's commensurate with the size of their sand hill of security tech debt; for every human security engineer we might eventually field a hundred agents that find and fix bugs in code, identify misconfigurations and exposures, work alongside people to remediate them, and in time remediate them autonomously, finally reversing or equalizing the attacker/defender asymmetry.
Of course, realizing this will take an enormous amount of innovation from the security community itself, including from security engineers inside the thousands of CISO organizations around the world, the open source enthusiasts and the pink-haired attendees of defcon, the graduate students and professors publishing in this area, etc.; these folks need open access to great models.
I don't think what I'm saying is controversial to hands-on security folks who've lived through similar dual-use regulation pushes around crypto and pen-testing tools.
So how did we get here? I think one reason is the makeup of the AI policy conversation.
I've moonlighted in AI security policy meetings since ChatGPT shipped in 2022, and have observed an ecosystem of character types:
There is the sharp Georgetown grad who works for a national security think tank in Washington; the frontier-lab AI safety constituency, focused on the coarse-grained claim that scaling laws will deliver catastrophic cyber capabilities; the AI luminary, whose deep fluency in machine learning buys a currency that licenses them to opine on cybersecurity despite having thought very little about it and holding only a toy model of how the domain works. I'm friends with many of these folks and they're brilliant.
But there are too few people -- who I admire deeply -- who carry real cybersecurity backgrounds into these conversations, and who, predictably, tend to treat most of what I have said above as obvious. We need to inject far more people with deep, practical cybersecurity backgrounds into the spaces where the framings and metaphors get set.
I think this goes both ways. You can acknowledge that the precedent here is not good and still call out the naivety that invited it
Anthropic, of all companies, should have foreseen that loudly branding its model a natsec-grade cyber weapon while actively courting this administration would hand it a straightforward rationale to move against them. This admin in particular has already shown, repeatedly, the inclination to act opaquely and arbitrarily. Anthropic itself experienced that firsthand with the supply chain risk designation (but apparently took no lesson from it). People are right not to let that level of naivety slide
the gleeful “stupid games, stupid prizes” response to the Anthropic situation is wildly shortsighted.
you can dislike Anthropic’s choices and still recognize that opaque, sweeping government interventions against frontier models doesn’t bode well for any ai company.
Today I'm publishing a new essay, Policy on the AI Exponential. AI is progressing extremely fast—much faster than the policy process was built to handle. The essay lays out where I think the technology is now, and the action needed to close the gap: https://t.co/Lh6PWae178
You have to be humble even when pursuing excellence.
I think the arrogance with which Anthropic has pursued the latest release has universally landed poorly.
Given Dario’s post was crafted to dovetail with a number of Admin positions while reversing previous Anthropic doctrine (see eg the reversal on fully-autonomous weapons), I would be very surprised if they were completely blindsided by this
The US government, citing national security authorities, has issued an export control directive to suspend all access to Fable 5 and Mythos 5 by any foreign national, whether inside or outside the United States, including foreign national Anthropic employees.
The net effect of this order is that we must abruptly disable Fable 5 and Mythos 5 for all our customers to ensure compliance.
Access to all other Claude models is not affected.
We apologize for this disruption to our customers. We believe this is a misunderstanding and are working to restore access as soon as possible.
Read our full statement: https://t.co/bwn0sximKZ
I wrote about Anthropic CEO Dario Amodei, who may or may not believe all the scary things he says but definitely wants you to think he is special despite doing a lot of very ordinary CEO stuff
I do not want to do AI research that is reactive to what these companies are doing, or even what they're saying.
The entire field keeps chasing after product releases. Some spend more time reading marketing copy than their colleagues papers and I just... do not want to do that?
3/3 effective altruists surveyed at the function this evening think that the stealth nerfing thing that Anthropic is doing with Fable (and potentially also Opus without disclosing it) is gross and unethical, EAs shouldn’t catch strays for standard commercially crass behavior
@alexolegimas@AnthropicAI You give them too much good intention imo. It’s not that they released a model and it was good in some cases and bad in others, and they are fine-tuning the parameters. They wanted to see how much they could push the anti-competition and understood they crossed the line.