Sneha

founder @EncodeAction 🇺🇸

[Sneha @SnehaRevanur]

if “you reap what you sow!” is your only takeaway here, i’m sorry to say that you just don’t get it. if you did, you would have seen it as utterly predictable (regardless of any CEO’s choice of words or level of political savvy) that AI would one day be this capable, and that the national security state would not be slumbering on peacefully upon noticing that. This is not to say that Anthropic has handled things optimally (obviously none of us were in the room for those convos), but it’s sad that so many people (including people in Silicon Valley who pride themselves on believing in and understanding AI deeply!) have no more interesting, helpful, or forward-looking read of the situation to chime in with than “haha how arrogant can that guy Dario be”. I don’t reflect on this incident and regret at all that I have wanted more AI regulation - in fact if the AI safety community had its way, the world would have seen this moment coming and had a plan for it a long time ago.

[spor @sporadica]

“we have built world-ending technology! it is so unsafe! the only reason we can give it to you is because we implemented some flimsy filters! government should regulate us!!”

*government regulates them*

“aww man what, we were just joking around”

[Nathan Calvin @_NathanCalvin]

Some quick takes:

(1) Wow things are getting real.

(2) The government's order focusing on prohibiting transfer to foreign nationals (even e.g. those living in the US, our close allies who help evaluate model safety in the UK, individuals who work at frontier labs like Anthropic) seems remarkably destructive, though is partially a result of the government using older legal authorities that were not designed for this kind of technology.

(3) If you believe (as I do) that AI has profound ramifications for national security, then assuming the government will sit back and do nothing and tolerate explanations like "well jailbreaking is a hard technical problem" for cyber capabilities that used to be the crown jewels of the NSA, is not tenable. If this is how the government reacts to the current level of system capabilities in 2026, how do you expect them to react to whatever is possible in 2028? However, it is extremely important that the authorities that the government uses are legible, transparent, have opportunities for appeal, and are narrowly targeted. Those legal authorities do not currently exist, and in their absence, the government will reach for metaphorical sledgehammers instead of scalpels.

(4) For that reason, it's extremely important that we create regulatory structures that are transparent and give recourse in the event that the government is overstepping or acting in an arbitrary manner. The alternative to passing such laws is not no regulation, it is regulation left primarily to national security authorities that are increasingly and evidently not fit for purpose.

[Charlie Bullock @CharlieBull0ck]

I have no idea whether Anthropic is in the wrong in this dispute. I do strongly believe that focusing on whether one company acted poorly or not is a waste of time and energy.

The government clearly thinks that they need some way to vet and evaluate frontier models that exceed a certain capability threshold for national security risks to keep the country safe. Anthropic clearly agrees. It is blindingly obvious that the current cobbled-together authorities the government is using to do this in an ad hoc last minute way are not adequate. We very clearly need new legislation that establishes a more organized, better-resourced, legally sound way for the government to do the thing that they obviously want to do while also affording companies due process instead of whatever this is. So let’s work on that, please!

[Miles Brundage @Miles_Brundage]

Fascinating how quickly Commerce can move on unsubstantiated Anthropic jailbreak claims, while taking forever to deal with well-documented NVIDIA chip smuggling

[Sneha @SnehaRevanur]

Wait so does OpenAI have nothing to do with LTF/BAAI beyond a one-time exec donation? Or is it a friendly corporate funder who has a say with the crew? There’s been a lot of forward progress from OAI on politics (SB 315!). To keep that credible this all needs to be exorcised.

[Shakeel @ShakeelHashim]

In recent weeks, OpenAI has made multiple statements distancing itself from Leading the Future, the accelerationist, Greg Brockman-funded super PAC, and Build American AI, the PAC’s advocacy group.

Brockman's donation, OpenAI insists, was made in a “personal capacity."

But @ReadTransformer has found that not everyone working within LTF appears to view it that way.

“For this project I’m funded by four separate entities … OpenAI is just one of them,” Nathan Leamer, executive director of Build American AI, told Transformer over text message on May 3.

“I think the best way to think of it is the corporate funders all have a say," he said.

[Shakeel @ShakeelHashim]

In recent weeks, OpenAI has made multiple statements distancing itself from Leading the Future, the accelerationist, Greg Brockman-funded super PAC, and Build American AI, the PAC’s advocacy group.

Brockman's donation, OpenAI insists, was made in a “personal capacity."

But @ReadTransformer has found that not everyone working within LTF appears to view it that way.

“For this project I’m funded by four separate entities … OpenAI is just one of them,” Nathan Leamer, executive director of Build American AI, told Transformer over text message on May 3.

“I think the best way to think of it is the corporate funders all have a say," he said.

[roon @tszzl]

if you don’t understand someone’s behavior in this time a good first guess is that they’re just not ASI pilled enough

[Bloomberg @business]

The US government should consider the possibility that AI becomes so advanced that it can improve itself without human intervention, and agencies should factor that risk into any federal testing of cutting-edge models, a GOP senator said https://t.co/zWEMWb96Qo

[Nathan Calvin @_NathanCalvin]

[moderately long rambling personal thoughts on living during the "foothills of the singularity"]

Between the news about the US government seriously considering taking shares in frontier AI companies and the new national security memorandum on AI, is else really and truly feeling the AGI today? It's not truly here yet, but I am more viscerally believing what Demis Hassabis said about humanity "standing in the foothills of the singularity."

(Foothills are the small hills that you run into at the beginning of a mountain range where you can start to see the mountains off in the distance. Things are somewhat steep, but you can see that they will soon get far steeper.)

Most of the time, despite being someone who says often that progress in AI will continue at about the pace it has for the past few years (mindblowingly fast) and that one way or another this will shatter and remake the world as it is, plausibly before the end of the decade, I don't really truly believe it in my bones.

To explain what I mean - Tom Davidson and others talk about the idea that faster AI progress feedback loops may do the equivalent of compressing a decade of scientific and technological progress (and the associated political and societal upheaval) into a year. I turn 30 this month. The amount of changes to technology and the US and the broader world in the 30 years since I was born is staggering to contemplate.

When I try to really think about what it would mean for all of the events and all of the upheaval that took place in that amount of time to be compressed into three years. And then I imagine that an equivalent amount of compressed changes may be happening in our world starting ~now - i'm not sure, all I can say is that Roon's observation that "not enough people are emotionally prepared for if its not a bubble" is incredibly and deeply apt, which seems like a weird thing to say about a tweet but its true. Again - I spend a lot of my time trying to convince people to take seriously that AI progress will indeed continue, and i'm still not remotely ready!

I am someone who enjoys novelty and change, and largely believes that those 30 years, taken on the whole, left the world better off. But I also am deeply worried that our institutions, which move slow as is, will show tremendous and frightening strain if a similar amount of change happens in 1/10th of the time.

When I start thinking too much about this I find myself tempted to permanently retreat into nature (perhaps near some foothills). A friend of mine with a very stressful job sometimes talked about wanting to quit and become a sheep herder in rural Ireland, and that image sticks with me when I feel like this.

That said, I believe that for those of us in a place where we can contribute in this moment, and help the world navigate AI well, we have a duty to do so, and to not retreat into ourselves and our immediate loved ones. And I feel incredibly lucky and privileged to be able to do this work as a job, where I can channel my feelings on unease into work as part of an incredible team of others who feel the same.

But at the same time, I feel affection for the world as it is - a world that one way or another is going to be incomprehensibly changed. And I intend to find time, and encourage everyone reading to as well, to enjoy and appreciate it, while I try to do what little I can to increase the chance its changed for the better.

[Nathan Calvin @_NathanCalvin]

Two weeks ago we asked a simple question: would SpaceX's prospectus show that the company is prepared to manage the unprecedented risks of frontier AI? The prospectus is now public, and the answer is a clear no.

SpaceX tells investors it relies on AI for the vast majority of its addressable market and intends to keep scaling Grok toward "multiple trillions of parameters" and a "step change" in intelligence. Yet across 277 pages, it says almost nothing about the biological-weapon, cyberattack, and loss-of-control risks that a chorus of AI experts warn are central challenges for the development of the technology.

SpaceX is asking investors to fund a technology its own founder called "far more dangerous than nukes."
It does so with safety practices that lag notably behind those of its peers, and that have already led to high profile incidents of harm. They have not done nearly enough to provide investors or the public assurance that those harms will not be dramatically magnified if their AI models become far more capable. This is true of all frontier AI developers, but xAI's lack of investment in basic safety infrastructure stands out even relative to its peers.

Read our full update addressing the prospectus linked below.

[Mitt Romney @MittRomney]

Our highest and most urgent national priority should be AI safeguards. The risks of AI weapons, pathogens, mass unemployment, surveillance, and even extinction must not continue to be largely ignored.

[Sneha @SnehaRevanur]

I feel like if you asked an LLM which moonshot signatories to get on a DNA synthesis screening letter this is pretty much the list it would give you

Rare to see this level of agreement. Hope Congress does something about it

[Alec Stapp @AlecStapp]

No one should be able to order a bioweapon through the mail.

@IFP & @JoinFAI are proud to co-lead an open letter calling for mandatory DNA synthesis screening & recordkeeping.

Signatories include:

- Sam Altman, CEO & Co-Founder, OpenAI

- Dario Amodei, CEO & Co-Founder, Anthropic

- David Baker, Director, Institute for Protein Design; 2024 Nobel Prize in Chemistry recipient

- Patrick Collison, CEO & Co-Founder, Stripe

- Paul Graham, Founder, Y Combinator

- Demis Hassabis, CEO, Google DeepMind; 2024 Nobel Prize in Chemistry recipient

- Emily Leproust, CEO & Co-Founder, Twist Bioscience

- Lawrence Lessig, Roy L. Furman Professor of Law and Leadership, Harvard Law School

- Gerald W. Parker, former Special Assistant to the President for Biosecurity and Pandemic Response

- Mustafa Suleyman, CEO, Microsoft AI

- Alex Tabarrok, Professor of Economics, George Mason University

- Alexandr Wang, Chief AI Officer, Meta; Founder, Scale AI

- Christine E. Wormuth, President & CEO, Nuclear Threat Initiative; 25th Secretary of the Army

Read the letter and see the full list of signatories: https://t.co/BwZiJXw3JT

Many DNA synthesis companies voluntarily screen orders to mitigate biosecurity risks, but no law requires them to do so.

Leaders in AI, biotech, life sciences, national security, and the nucleic acid synthesis industry agree that Congress should act to strengthen safeguards against biological threats.

@deanwball put it well in the WSJ:

“If you’re synthesizing the stuff that yields biological life and viruses, we’re asking you to screen to see whether it is dangerous in some way. That seems like a reasonable thing for society to insist upon.”

[Anthropic @AnthropicAI]

Our internal data shows Claude is accelerating AI development—a possible path to recursive self-improvement, or AI autonomously building a more capable successor.

It’s happening faster than we thought, and the implications deserve greater attention. https://t.co/OVVPJO7VQx

[Olivia H. Scharfman @OliviaHelenS]

I am excited to share this today –– we have been working on this for a long time.

Nuclear material is regulated. Bioweapons are easier to make, don’t emit traceable radiation, and self-propagate for unprecedented destruction.

This should be common sense, bipartisan national security legislation.

Thank you to all who signed, including @PeterDiamandis, @deanwball, @ATabarrok, @kath_mcmahon, and @kesvelt.