ujval

CS PhD student @ucberkeley (on sabbatical) based in nyc @cornell_tech || security, privacy, tech policy, cs&law etc || @berkeleyRDI @initc3org

[ujval @_ujval]

actually it's only called surveillance if it comes from the Surveillance province of China, otherwise it's just sparkling data collection

[des @dotnetschizo]

“why don’t you move to SF the people are incredible”
the people in SF:

[Matt Bruenig @MattBruenig]

@cafreiman We have an idiom for this in our society: "standing on the shoulders of giants." Being 100 ft in the air doesn't mean you are a 100 feet tall. We have to see what you are standing on top of to figure out your specific height. The inherited technology stack is one of those things

[SE Gyges @segyges]

anthropic asked to be regulated and now they are, the long version https://t.co/dmfl57jfhz

[ujval @_ujval]

sounds like an epistemic bubble

[roon @tszzl]

people in san francisco say “agi pilled” as a general term of art for describing people who reason about the future with any degree of analytical skill. it has been imo a good proxy for human capital generally

[ujval @_ujval]

“mythos-level” “mythos-class” “post-mythos”

I immediately discount anything that comes after these words

[Joshua Saxe @joshua_saxe]

Banning Mythos represents a basic misunderstanding of AI cybersecurity

The evidence we have now points to defenders currently benefiting more from frontier models than attackers. Every security professional within any large, complex organization knows they are sitting on a mountain of security technical debt they are not resourced to dig out of, and is not surprised when penetration testers reliably make it through their defenses. Defenders are deeply, deeply bottlenecked by labor.

Attackers, who already have the advantage, have less comparative advantage to rush to AI adoption. While new zero-days can be of incremental help, they have reliable TTPs today, which is why today's AI capabilities haven't generated a discontinuous surge in attacks.

The hope of frontier AI is that it could finally resource defenders in a way that's commensurate with the size of their sand hill of security tech debt; for every human security engineer we might eventually field a hundred agents that find and fix bugs in code, identify misconfigurations and exposures, work alongside people to remediate them, and in time remediate them autonomously, finally reversing or equalizing the attacker/defender asymmetry.

Of course, realizing this will take an enormous amount of innovation from the security community itself, including from security engineers inside the thousands of CISO organizations around the world, the open source enthusiasts and the pink-haired attendees of defcon, the graduate students and professors publishing in this area, etc.; these folks need open access to great models.

I don't think what I'm saying is controversial to hands-on security folks who've lived through similar dual-use regulation pushes around crypto and pen-testing tools.

So how did we get here? I think one reason is the makeup of the AI policy conversation.

I've moonlighted in AI security policy meetings since ChatGPT shipped in 2022, and have observed an ecosystem of character types:

There is the sharp Georgetown grad who works for a national security think tank in Washington; the frontier-lab AI safety constituency, focused on the coarse-grained claim that scaling laws will deliver catastrophic cyber capabilities; the AI luminary, whose deep fluency in machine learning buys a currency that licenses them to opine on cybersecurity despite having thought very little about it and holding only a toy model of how the domain works. I'm friends with many of these folks and they're brilliant.

But there are too few people -- who I admire deeply -- who carry real cybersecurity backgrounds into these conversations, and who, predictably, tend to treat most of what I have said above as obvious. We need to inject far more people with deep, practical cybersecurity backgrounds into the spaces where the framings and metaphors get set.

[ujval @_ujval]

I think this goes both ways. You can acknowledge that the precedent here is not good and still call out the naivety that invited it

Anthropic, of all companies, should have foreseen that loudly branding its model a natsec-grade cyber weapon while actively courting this administration would hand it a straightforward rationale to move against them. This admin in particular has already shown, repeatedly, the inclination to act opaquely and arbitrarily. Anthropic itself experienced that firsthand with the supply chain risk designation (but apparently took no lesson from it). People are right not to let that level of naivety slide

[Nat Purser @NatPurser]

the gleeful “stupid games, stupid prizes” response to the Anthropic situation is wildly shortsighted.

you can dislike Anthropic’s choices and still recognize that opaque, sweeping government interventions against frontier models doesn’t bode well for any ai company.

[Shubhendu Trivedi @_onionesque]

@AnthropicAI "We believe this is a misunderstanding" and "this is too dangerous for anyone else to use for frontier capabilities" are not compatible.

[ujval @_ujval]

this hits so different today

[Dario Amodei @DarioAmodei]

Today I'm publishing a new essay, Policy on the AI Exponential. AI is progressing extremely fast—much faster than the policy process was built to handle. The essay lays out where I think the technology is now, and the action needed to close the gap: https://t.co/Lh6PWae178

[Sara Hooker @sarahookr]

You have to be humble even when pursuing excellence.

I think the arrogance with which Anthropic has pursued the latest release has universally landed poorly.

[ujval @_ujval]

tired of tweeting about basically nothing but Anthropic this week. logging off even if it means skipping this saga. need to detox

[James Rosen-Birch ⚖️🕊️ @provisionalidea]

Given Dario’s post was crafted to dovetail with a number of Admin positions while reversing previous Anthropic doctrine (see eg the reversal on fully-autonomous weapons), I would be very surprised if they were completely blindsided by this

[ujval @_ujval]

everything is going according to plan

[Anthropic @AnthropicAI]

The US government, citing national security authorities, has issued an export control directive to suspend all access to Fable 5 and Mythos 5 by any foreign national, whether inside or outside the United States, including foreign national Anthropic employees.

The net effect of this order is that we must abruptly disable Fable 5 and Mythos 5 for all our customers to ensure compliance.

Access to all other Claude models is not affected.

We apologize for this disruption to our customers. We believe this is a misunderstanding and are working to restore access as soon as possible.

Read our full statement: https://t.co/bwn0sximKZ

[Kate Aronoff @KateAronoff]

I wrote about Anthropic CEO Dario Amodei, who may or may not believe all the scary things he says but definitely wants you to think he is special despite doing a lot of very ordinary CEO stuff

[Deb Raji @rajiinio]

I do not want to do AI research that is reactive to what these companies are doing, or even what they're saying.

The entire field keeps chasing after product releases. Some spend more time reading marketing copy than their colleagues papers and I just... do not want to do that?

[SE Gyges @segyges]

this is poor theory of mind. the majority of EAs are oblivious to questions of power. this makes all six of them that aren't bizarrely powerful

[ujval @_ujval]

They're not strays. Anthropic is run by EAs, staffed by EAs, and fluent in the EA language that dresses self-interest as caution

[mattparlmer 🪐 🌷 @mattparlmer]

3/3 effective altruists surveyed at the function this evening think that the stealth nerfing thing that Anthropic is doing with Fable (and potentially also Opus without disclosing it) is gross and unethical, EAs shouldn’t catch strays for standard commercially crass behavior

[Ravid Shwartz Ziv @ziv_ravid]

@alexolegimas @AnthropicAI You give them too much good intention imo. It’s not that they released a model and it was good in some cases and bad in others, and they are fine-tuning the parameters. They wanted to see how much they could push the anti-competition and understood they crossed the line.

[ujval @_ujval]

can someone inform me which direction we're updating timelines

[ujval @_ujval]

so what's the verdict on mythos from safteyists and boosters. was it everything you hoped for?

[ujval @_ujval]

many have set themselves up for disappointment. huge nothingburger

[ujval @_ujval]

checks out if you've been following his takes recently. not exactly surprising bedfellows

[molly taft @mollytaft]

didn’t expect to see matt yglesias thanked for editing and comments in the footnotes