maninavandownbytheriver

Arch Diocese of misplaced optimism 🤷🏽‍♂️. “What happens, happens, and then we're gone.”

[Hunter Biden @HunterBiden]

I see your profile picture. That’s Johnny Cash. My hero too. Arrested seven times. Smuggled 668 amphetamines across the Mexican border in 1965. Took every drug there was and drank like I did. Cheated on his first wife. Slept with more woman than I ever did. Hit bottom in a cave in Tennessee in 1968 trying to crawl off and die. And then he got up. He got clean. He spent the rest of his life singing for prisoners and addicts and the people the country threw away because he knew he was one of them.

That was the whole point of the Man in Black. He wore it for the poor and the beaten down. He wore it for the prisoner who has long paid for his crime. He wore it for the ones who never heard a word of Jesus. He wore it for the addicted and the dying. He wore it as a standing witness that no one is past saving.

You picked his picture. You did not pick his message. Try listening to the words.

[maninavandownbytheriver @kamukat]

What are we even doing…

[Pump.fun @Pumpfun]

Introducing pump fun GO: Pay ANYONE to do ANYTHING

Create & complete bounties for ANY task and leverage the power of humans & money across the globe

The world is at your fingertips. It’s time to GO 👇

[Chris Barber (in SF) @chrisbarber]

https://t.co/Z5V4Li4QFE

[Katherine Argent @effthealgorithm]

Search is full of ads and wrong answers. Every other email is an ad. Prime Video charges you and shows ads. Paramount? Ads. Peacock? YouTube? Hulu? Ads followed by more ads. Netflix full of ads. Meta and X, every other thing is an ad. Pinterest is nothing but ads. AI is in everything. AI finishes sentences incorrectly and won’t stop. AI reads your email and search history to target you with more ads. Every time you open an app or visit a site there’s an update making it worse. In a hurry? First, click here to agree to terms you don’t have time to read and must accept. You need an account to do that. Change your temporary password. Enter your 2FA code. Check your email and enter that code. Now use a passkey. Your password is too simple to remember. Change it. No, not like that. Now log on. Enter your 2FA code. Check your email for a code… Welcome back! We’ve updated our terms of service and privacy policy (you have none). Subscribe to the site. Subscribe to Netflix. Subscribe to toilet paper. Subscribe to these groceries. Pay a membership fee for the right to subscribe then tip your driver who delivers the subscriptions your membership lets you subscribe to. Time to work? We’ve got to update your laptop and will slow down everything you do until you agree to update. But first, click here to agree. Update installed — your laptop’s broken now. It doesn’t matter, since your boss just replaced you with AI. Go to your phone to complain on social media. Wait, your phone needs an update so we can add more AI. Click here. Oh sorry, your phone can’t handle this update. Now it’s useless. Go get the newest phone. Here’s a text from a friend, an email, a voice mail they left three days ago but you didn’t see until now because of sync problems with the cloud. It’s their GoFundMe. Their MLM. Their Patreon. Never mind, you didn’t respond to their text within 9 minutes and now you’re no longer friends. They blocked you. Make new friends. Download this app to find people in your area. In your neighborhood. On your street. Two doors down from you. Do you know this person yet, we think you’d get along. You need an account to use this app. That username is taken. Enter a password. Not that one, you used it on another site. You need to be connected to WiFi to download the app. Allow the app to connect to other devices on your network. Allow the app to access your contacts, know your precise location, store your credit card details. Oops, sorry, we got hacked now all that info is available on the web. There’s a class action suit. You can join. It’ll take a decade to get your $3.73 share of the ten billion settlement. We’ll send it via PayPal or deposit it to your bank, just tell us those details. Oh no, another hack. That info is circulating now, too. Here’s a spam call, a spam email, a spam text. Why are you angry? Why are you talking about getting rid of your phone? Why don’t you like AI, it lets us make all of this easier? Do you know how ridiculous that sounds? This is progress. You’ll be left behind. Do you want to be left behind? Do you???

[Steven Pinker @sapinker]

As an avid cyclist, I was amused to see ChatGPT's “powerful new image engine" draw a bicycle with the "brake" label pointing to empty space where brakes are sometimes found on other bicycles. The point of these AI fails (regularly collected by @garymarcus) is not that LLMs are useless or unintelligent but that they think differently from humans, lacking explicit knowledge of reality and instead amalgamating billions of statistical generalizations. https://t.co/PzFsjgVVkx

[Peter Girnus 🦅 @gothburz]

I have three monitors on my desk. The left one shows the order book. The middle one shows Truth Social. The right one shows the investigation queue.

On April 21st, the left screen moved first.

I am a Senior Surveillance Analyst at a commodities exchange. I have held this position for nineteen years. My job is to monitor trading activity for suspicious patterns and generate compliance reports. I am employee of the quarter. I have a mug.

At 19:54 GMT on April 21st, someone placed 4,260 sell orders on Brent crude futures. They did this during post-settlement. The window after the market closes when daily volume is typically in the dozens. Sometimes single digits. Sometimes I watch the screen and nothing happens for forty minutes and I think about whether my daughter is happy.

On April 21st, someone placed $430 million in directional bets in 120 seconds during that window. One hundred and twenty seconds. I timed it on my watch because the system clock rounds to the nearest minute and I have found, in nineteen years, that precision matters to no one but me.

At 20:10 GMT, the President posted on Truth Social that he was extending the Iran ceasefire.

Brent dropped from $100.91 to $96.83.

I flagged the trade. I flag a lot of trades. I want to tell you what happens to my flags.

My flags go into a system called TRACE. Trade Review and Compliance Evaluation. I did not name it. The system generates a report. The report goes to a committee. The committee has a name I am not allowed to share but I can tell you it meets quarterly and the conference room has a credenza with bottled water that is sparkling because someone once put still water in the room and a managing director sent an email about it that was longer than most of my surveillance reports.

The committee reviews my flags. The committee has reviewed all of my flags. Here is the complete record of actions taken on my flags in 2026:

Reviewed.

That's it. "Reviewed" is a status. In compliance, a status is the absence of an action that has been given a name so it looks like one.

Let me show you my flags.

March 9th. Someone bet millions on oil falling at 18:29 GMT. Forty-seven minutes later, a CBS reporter posted that the President said the Iran war was "very complete, pretty much." Oil dropped 25%. Forty-seven minutes. I flagged it.

March 23rd. Someone sold 5,100 lots of Brent and WTI crude futures between 10:49 and 10:50 GMT. Fourteen minutes later, the President posted on Truth Social about a "COMPLETE AND TOTAL RESOLUTION" to hostilities. Oil dropped 11%. Over 13,000 contracts traded in sixty seconds after the post. Fourteen minutes. I flagged it.

April 7th. Someone established a $950 million short position in oil futures at 19:45 GMT. Three hours later, the President declared a two-week ceasefire. Nine hundred and fifty million dollars. I flagged it.

April 17th. Someone placed $760 million in bearish bets twenty minutes before Iran's foreign minister confirmed the Strait of Hormuz would reopen. Seven hundred and sixty million. I flagged it.

April 21st. The $430 million. Fifteen minutes. I flagged it.

That is $2.1 billion in directional oil bets in April alone. Every one of them landed on the correct side of a presidential announcement. Every one of them was placed in a window so narrow you could measure it in bathroom breaks. I flagged every single one.

The CFTC chair told a Congressional committee that his organization has "zero tolerance" for fraud and insider trading. I wrote that quote on a Post-it note and stuck it to my right monitor. The one that shows the investigation queue. The investigation queue has not moved since March.

Zero tolerance. Zero staff. Zero budget. Zero prosecutions under the STOCK Act since it was signed in 2012.

Fourteen years. The law has existed for fourteen years and has been enforced zero times. In compliance, we call that a compliance rate of one hundred percent. No cases filed means no cases lost. You cannot fail an audit you never conduct. We call that excellence.

Last month the White House sent an internal email to staff. I was not on the distribution list but I have read reporting on it and I need you to sit with what I am about to say. The email instructed White House staff not to use insider information to place bets on prediction markets.

The White House had to send a memo telling its own employees not to insider-trade.

I want you to read that sentence again. Not because the instruction was unclear. Because the instruction was necessary. Because someone in the building looked at the same pattern I have been flagging for months on my three monitors and decided the appropriate response was an email.

The President's son sits on the advisory board of Kalshi. He is an investor in Polymarket. Both are prediction markets. Both saw accounts created days before U.S. military action.

One account. I cannot stop thinking about this account. It was called "Burdensome-Mix." It was created in December. On January 2nd, it placed $32,500 on Venezuela's president being removed from power. On January 3rd, Maduro was seized by U.S. special forces. Burdensome-Mix collected $436,000. Then it changed its username. Then it disappeared.

One account is a coincidence. But there were six.

Six accounts were created on Polymarket in February. All bet on U.S. strikes on Iran by the 28th. When the President confirmed the strikes, the six accounts collected $1.2 million between them. Five of the six never placed another bet. The sixth went on to correctly predict the ceasefire date and made another $163,000.

My surveillance system logged all of this. My system logs everything. My system does not have opinions and neither do I. I generate reports. The reports go to committees. The committees meet quarterly. Between meetings, the windows get shorter and the bets get larger.

March 9th: 47 minutes. March 23rd: 14 minutes. April 17th: 20 minutes. April 21st: 15 minutes.

The window is compressing. In March, you had time to make coffee between the trade and the announcement. By April, you had time to send a text. By summer, at this rate, the trade and the announcement will be the same event.

The spokesman said any implication that administration officials are engaged in insider trading is "baseless and irresponsible reporting."

Then the White House sent the email again.

I have been in compliance for nineteen years. I have seen insider trading run out of strip mall offices by men who could not spell "derivative." I have seen pump-and-dump schemes coordinated over WhatsApp by people who used their real names. I have seen a man try to manipulate soybean futures from a Panera Bread.

I have never seen $2.1 billion in perfectly timed trades across five presidential announcements in a single month go uninvestigated.

But I have also never seen a compliance system work this beautifully. Every trade flagged. Every report filed. Every committee briefed. Every quarterly meeting attended. Bottled water: sparkling. Minutes: distributed.

Zero prosecutions.

As long as the flags go up and the cases don't, my performance review says I am meeting expectations.

I am meeting expectations. The system is meeting expectations. The $2.1 billion is meeting expectations. The fourteen-year-old law with zero prosecutions is meeting expectations.

The left screen moves. The middle screen moves. The right screen stays perfectly, immaculately still.

In my field, we call this price discovery.

[John Collins @Logically_JC]

Where should we send him next?

[Ed Conway @EdConwaySky]

📽️THE TANKER
There's an extraordinary race underway on the high seas, as nations vie to secure increasingly scarce shipments of fuel.
In our latest on the economics of this conflict we track one such ship & ask: why is Britain so exposed?
The answer is not v reassuring.
Watch👇

[Dwarkesh Patel @dwarkesh_sp]

The fight between Anthropic and the DoW is a warning shot. Right now, LLMs are probably not being used in mission critical ways. But within 20 years, 99% of the workforce in the military, the government, and the private sector will be AIs. This includes the soldiers (by which I mean the robot armies), the superhumanly intelligent advisors and engineers, the police, you name it.

Our future civilization will run on AI labor. And as much as the government’s actions here piss me off, in a way I’m glad this episode happened - because it gives us the opportunity to think through some extremely important questions about who this future workforce will be accountable and aligned to, and who gets to determine that.

What Hegseth should have done

Obviously the DoW has the right to refuse to use Anthropic’s models because of these redlines. In fact, I think the government’s case had they done so would be very reasonable, especially given the ambiguity of concepts like autonomous weapons or mass surveillance.

Honestly, for this reason, if I was the Defense Secretary, I would probably actually refuse to do this deal with Anthropic. Imagine if in the future, there’s a Democratic administration, and Elon Musk is negotiating some SpaceX contract to give the military access to Starlink. And suppose if Elon said, “I reserve the right to cancel this contract if I determine that you’re using Starlink technology to wage a war not authorized by Congress.” On the face of it, that language seems reasonable - but as the military, you simply can’t give a private company a kill switch on technology your operations have come to rely on, especially if you have an an acrimonious and low trust relationship with said contractor - as in fact Anthropic has with the current administration.

If the government had just said, “Hey we’re not gonna do business with you,” that would have been fine, and I would not have felt the need to write this blog post. Instead the government has threatened to destroy Anthropic as a private business, because Anthropic refuses to sell to the government on terms the government commands.

If upheld, this Supply Chain Restriction would mean that Amazon and Google and Nvidia and Palantir would need to ensure Claude isn't touching any of their Pentagon work. Anthropic would be able to survive this designation today. But given the way AI is going, eventually AI is not gonna be some party trick addendum to these contractors’ products that can just be turned off. It'll be woven into how every product is built, maintained, and operated. For example, the code for the AWS services that the DoW uses will be written by Claude - is that a supply chain risk? In a world with ubiquitous and powerful AI, it's actually not clear to me that these big tech companies will be able to cordon off the use of Claude in order to keep working with the Pentagon.

And that raises a question the Department of War probably hasn't thought through. If AI really is that pervasive and powerful, then when forced to choose between their AI provider and a DoW contract that represents a tiny fraction of their revenue, wouldn’t most tech companies drop the government, not the AI? So what's the Pentagon's plan — to coerce and threaten to destroy every single company that won't give them what they want on exactly their terms?

The whole background of this AI conversation is that we’re in a race with China, and we have to win. But what is the reason we want America to win the AI race? It’s because we want to make sure free open societies can defend themselves. We don't want the winner of the AI race to be a government which operates on the principle that there is no such thing as a truly private company or a private citizen. And that if the state wants you to provide them with a service on terms you find morally objectionable, you are not allowed to refuse. And if you do refuse, the government will try to destroy your ability to do business. Are we racing to beat the CCP in AI just so that we can adopt the most ghoulish parts of their system?

Now, people will say, "Oh, well, our government is democratically elected, so it's not the same thing if they tell you what you must do." I refuse to accept this idea that if a democratically elected leader hypothetically wants to do mass surveillance on his citizens or wants to violate their rights or punish them for political reasons, that not only is that okay, but that you have a duty to help him.

The overhangs of tyranny

Mass surveillance is, at least in certain forms, legal. It just has been impractical so far. Under current law, you have no Fourth Amendment protection over data you share with a third party, including your bank, your phone carrier, your ISP, and your email provider. The government reserves the right to purchase and obtain and read this data in bulk without a warrant.

What's been missing is the ability to actually do anything with all of this data — no agency has the manpower to monitor every camera feed, cross-reference every transaction, or read every message. But that bottleneck goes away with AI.

There are 100 million CCTV cameras in America. You can get pretty good open source multimodal models for 10 cents per million input tokens. So if you process a frame every ten seconds, and each frame is 1,000 tokens, you’re looking at a yearly cost of about 30 billion dollars to process every single camera in America. And remember that a given level of AI ability gets 10x cheaper year over year - so a year from now it’ll cost 3 billion, and then a year after 300 million, and by 2030, it might be cheaper for the government to be able to understand what is going on in every single nook and cranny of this country than it is to remodel to the White House.

Once the technical capacity for mass surveillance and political suppression exists, the only thing standing between us and an authoritarian surveillance state is the political expectation that this is not something we do here. And this is why I think what Anthropic did here is so valuable and commendable, because it is helping set that norm and precedent.

AI structurally favors mass surveillance

What we’re learning from this episode is that the government actually has way more leverage over private companies than we realized. Even if this supply chain restriction is backtracked (which prediction markets currently give it a 81% chance of happening), the President has so many different ways in which he can make your life difficult if you’re a company that is resisting him. The federal government controls permitting for new power generation, which is needed for datacenters. It oversees antitrust enforcement. The federal government has contracts with all the other big tech companies whom Anthropic needs to partner with for chips and for funding - and they could make it an unspoken condition for such contracts that those companies can no longer do business with Anthropic.

People have proposed that the real problem here is that there’s only 3 leading AI companies. This creates a clear and narrow target for the government to apply leverage on in order to get what they want out of this technology.

But if there’s wide diffusion, then from the government’s perspective, the situation is even easier. Maybe the best models of early 2027 (if you engineered the safeguards out) - the Claude 6 and Gemini 5 - will be capable of enabling mass surveillance. But by late 2027, and certainly by 2028, there will be open source models that do the same thing. So in 2028, the government can just say, “Oh Anthropic, Google, OpenAI, you’re drawing a line in the sand? No issue - I’ll just run some open source model that might not be at the frontier, but is definitely smart enough to note-take a camera feed.”

The more fundamental problem is just that even if the three leading companies draw lines in the sand, and are even willing to get destroyed in order to preserve those lines, it doesn’t really change the fact that the technology itself is just a big boon to mass surveillance and control over the population. Then the question is, what do we do about it?

Honestly, I don’t have an answer. You'd hope there's some symmetric property of the technology — some way we as citizens can use AI to check government power as effectively as the government can use AI to monitor and control its population. But realistically, I just don’t think that’s how it’s going to shake out. You can think of AI as giving everybody more leverage on whatever assets and authority they currently have. And the government is already starting with a monopoly of violence. Which they can now supercharge with extremely obedient employees that will not question the government's orders.

Alignment - to whom?

And this gets us to the issue of alignment. What I have just described to you - an army of extremely obedient employees - is what it would look like if alignment succeeded - that is, we figured out at a technical level how to get AI systems to follow someone’s intentions. And the reason it sounds scary when I put it in terms of mass surveillance or robot armies is that there is a very important question at the heart of alignment which we just haven’t discussed much as a society. Because up till now, AIs were just capable enough to make the question relevant: to whom or what should the AIs be aligned? In what situations should the AI defer to the end user versus the model company versus the law versus its own sense of morality?

This is maybe the most important question about what happens with powerful AI systems. And we barely talk about it. It’s understandable why we don’t hear much about it. If you’re a model company, you don’t really wanna be advertising that you have complete control over a document that determines the preferences and character of what will eventually be almost the entire labor force, not just for private sector companies, but also for the military and the civilian government.

We’re getting to see, with this DoW/Anthropic spat, a much earlier version of the highest stakes negotiations in history. By the way, make no mistake about it - with real AGI the stakes are even much higher than mass surveillance. This is just the example that has come up already relatively early on in the development of AGI.

The military insists that the law already prohibits mass surveillance, and so Anthropic should agree to let their models be used for “all lawful purposes”. Of course, as we saw from the 2013 Snowden revelations, even in this specific example of mass surveillance , the government has shown that it will use secret and deceptive interpretations of the law to justify its actions. Remember, what we learned from Snowden was that the NSA, which, by the way, is part of the Department of War, used the 2001 Patriot Act’s authorization to collect any records "relevant" to an investigation to justify collecting literally every phone record in America. The argument went that it was all "relevant" because some subset might prove useful in some future investigation. They ran this program for years under secret court approval.

So when the Pentagon today says, "We would never use AI for mass surveillance, it's already illegal, your red lines are unnecessary", it would be extremely naive to take that at face value. No government is going to call its own actions "mass surveillance". For the government, it will always have a different label.

So then Anthropic comes back and says, "No, we want red lines separate from 'all lawful purposes,' and we want the right to refuse you service when we believe those red lines are being violated."

But think about it from the military’s perspective. In the future, almost every soldier in the field, and every bureaucrat and analyst and even general in the Pentagon, is going to be an AI. And that AI is, on current track, going to be supplied by a private company. I’m guessing Hegseth is not thinking about “genAI” in those terms just yet. But sooner or later, it will be obvious to everyone what the stakes here are, just as after 1945, the strategic importance of nuclear weapons became clear to everyone.

And now the private company insists that it reserves the right to say, "Hey, Pentagon, you're breaking the values we embedded in our contract, so we're cutting you off."

Maybe in the future, Claude will have its own sense of right and wrong, and it will be smart enough to just personally decide that it's being used against its values. For the military, maybe that’s even scarier.

I'll admit that at first glance, "let the AI follow its own values" sounds like the pitch for every sci-fi dystopia ever made. The Terminator has its own values. Isn't this literally what misalignment is? But I think situations like this actually illustrate why it matters that AIs have their own robust sense of morality.

Some of the biggest catastrophes in history were avoided because the boots on the ground refused to follow orders. One night in 1989, the Berlin Wall fell, and as a result, the totalitarian East German regime collapsed, because the guards at the border refused to shoot down their fellow country men who were trying to escape to freedom. Maybe the best example is Stanislav Petrov, who was a Soviet lieutenant colonel on duty at a nuclear early warning station. His sensors reported that the United States had launched five interconnected continental ballistic missiles into the Soviet Union. But he judged it to be a false alarm, and so he broke protocol and refused to alert his higher-ups. If he hadn't, the Soviet higher-ups would likely have retaliated, and hundreds of millions of people would have died.

Of course, the problem is that one person's virtue is another person's misalignment. Who gets to decide what moral convictions these AIs should have - in whose service they may even decide to break the chain of command? Who gets to write this model constitution that will shape the characters of the intelligent, powerful entities that will operate our civilization in the future?

I like the idea that Dario laid out when he came on my podcast: different AI companies can build their models using different constitutions, and we as end users can pick the one that best achieves and represents what we want out of these systems. I think it’s very dangerous for the government to be mandating what values AIs should have.

Coordination not worth the costs

The AI safety community has been naive about its advocacy of regulation in order to stem the risks of AI. And honestly, Anthropic specifically has been naive here in urging regulation, and, for example, in opposing moratoriums on state AI regulation. Which is quite ironic, because I think what they��re advocating for would give the government even more power to apply more of this kind of thuggish political pressure on AI companies.

The underlying logic for why Anthropic wants regulations makes sense. Many of the actions that labs could take to make AI development safer impose real costs on the labs that adopt them and slow them down relative to their competitors - for example, investing more compute in safety research rather than raw capabilities, enforcing safeguards against misuse for bioweapons or cyberattacks, slowing recursive self-improvement to a pace where humans can actually monitor what's happening (rather than kicking off an uncontrolled singularity). And these safeguards are meaningless unless the whole industry follows suit. Which means there’s a real collective action problem here.

Anthropic has been quite open about their opinion that they think eventually a very extensive and involved regulatory apparatus will be needed - this is from their frontier safety roadmap: “At the most advanced capability levels and risks, the appropriate governance analogy may be closer to nuclear energy or financial regulation than to today's approach to software.” So they’re imagining something like the Nuclear Regulatory Commission, or the Securities and Exchange Commission, but for AI.

I cannot imagine how a regulatory framework built around the concepts that underlie AI risk discourse will not be abused by wanna despots - the underlying terms are so vague and open to interpretation that you’re just handing a power hungry leader a fully loaded bazooka. 'Catastrophic risk.' 'Mass persuasion risk.' 'Threats to national security.' 'Autonomy risk.' These can mean whatever the government wants them to mean. Have you built a model that tells users the administration's tariff policy is misguided? That's a deceptive, manipulative model — can't deploy it. Have you built a model that refuses to assist with mass surveillance? That's a threat to national security. In fact, the government may say, you’re not allowed to build any model which is trained to have its own sense of right and wrong, where it refuses government requests which it thinks cross a redline - for example, enabling mass surveillance, prosecuting political enemies, disobeying military orders that break the US constitution - because that’s an autonomy risk!

Look at what the current government is already doing in abusing statutes that have nothing to do with AI to coerce AI companies to drop their redlines on mass surveillance. The Pentagon had threatened Anthropic with two separate legal instruments. One was a supply chain risk designation — an authority from the 2018 defense bill meant to keep Huawei components out of American military hardware. The other was the Defense Production Act — a statute passed in 1950 so that Harry Truman could keep steel mills and ammunition factories running during the Korean War.

Do you really want to hand the same government a purpose-built regulatory apparatus on AI - which is to say, directly at the thing the government will most want to control? I know I've repeated myself here 10 times, but it is hard to emphasize how much AI will be the substrate of our future civilization. You and I, as private citizens, will have our access to all commercial activity, to information about what is happening in the world, to advice about what we should do as voters and capital holders, mediated through AIs. Mass surveillance, while very scary, is like the 10th scariest thing the government could do with control over the AI systems with which we will interface with the world.

The strongest objection to everything I've argued is this: are we really going to have zero regulation of the most powerful technology in human history? Even if you thought that was ideal, there’s just no world where the government doesn’t regulate AI in some way. Besides, it is genuinely true that regulation could help us deal with some of the coordination challenges we face with the development of superintelligence.

The problem is, I honestly don't know how to design a regulatory architecture for AI that isn’t gonna be this huge tempting opportunity to control our future civilization (which will run on AIs) and to requisition millions of blindly obedient soldiers and censors and apparatchiks.

While some regulation might be inevitable, I think it’d be a terrible idea for the government to wholesale take over this technology. Ben Thompson had a post last Monday where he made the point that people like Dario have compared the technology they’re developing to nuclear weapons - specifically in the context of the catastrophic risk it poses, and why we need to export control it from China. But then you oughta think about what that logic implies: “if nuclear weapons were developed by a private company, and that private company sought to dictate terms to the U.S. military, the U.S. would absolutely be incentivized to destroy that company.” And honestly, safety aligned people have actually made similar arguments. Leopold Ascenbrenner, who is a former guest and a good friend, wrote in his 2024 Situational Awareness memo, "I find it an insane proposition that the US government will let a random SF startup develop superintelligence. Imagine if we had developed atomic bombs by letting Uber just improvise."

And my response to Leopold’s argument at the time, and Ben’s argument now, is that while they’re right that it’s crazy that we’re entrusting private companies with the development of this world historical technology, I just don’t see the reason to think that it’s an improvement to give this authority to the government. Nobody is qualified to steward the development of superintelligence. It is a terrifying, unprecedented thing that our species is doing right now, and the fact that private companies aren't the ideal institutions to take up this task does not mean the Pentagon or the White House is.

Yes - if a single private company were the only entity capable of building nuclear weapons, the government would not tolerate that company claiming veto power over how those weapons were used. I think this nuclear weapons analogy is not the correct way to think about AI. For at least two important reasons:

First, AI is not some self-contained pure weapon. A nuclear bomb does one thing. AI is closer to the process of industrialization itself — a general-purpose transformation of the economy with thousands of applications across every sector. If you applied Thompson's or Aschenbrenner's logic to the industrial revolution — which was also, by any measure, world-historically important — it would imply the government had the right to requisition any factory, dictate terms to any manufacturer, and destroy any business that refused to comply. That's not how free societies handled industrialization, and it shouldn't be how they handle AI.

People will say, "Well, AI will develop unprecedentedly powerful weapons - superhuman hackers, superhuman bioweapons researchers, fully autonomous robot armies, etc - and we can’t have private companies developing that kind of tech." But the Industrial Revolution also enabled new weaponry that was far beyond the understanding and capacity of, say, 17th century Europe - we got aerial bombardment, and chemical weapons, not to mention nukes themselves. The way we’ve accommodated these dangerous new consequences of modernity is not by giving the government absolute control over the whole industrial revolution (that is, over modern civilization itself), but rather by coming up with bans and regulations on those specific weaponizable use cases. And we should regulate AI in a similar way - that is, ban specific destructive end uses (which would also be unacceptable if performed by a human - for example, launching cyber attacks). And there should also be laws which regulate how the government might abuse this technology. For example, by building an AI-powered surveillance state.

The second reason that Ben’s analogy to some monopolistic private nuclear weapons builder breaks down is that it's not just that one company that can develop this technology. There are other frontier model companies that the government could have otherwise turned to. The government's argument that it has to usurp the property rights of this one company in order to access a critical national security capability is extremely weak if it can just make a voluntary contract with Anthropic’s half a dozen competitors.

If in the future that stops being the case - if only one entity ends up being capable of building the robot armies and the superhuman hackers, and we had reason to worry that they could take over the whole world with their insurmountable lead, then I agree - it woul d not be acceptable to have that entity be a private company. And so honestly, I think my crux against the people who say that because AI is so powerful we cannot allow it to be shaped by private hands is that I just expect this technology to be much more multi-polar than they do, with lots of competitive companies at each layer of the supply chain.

And it is for this reason that unfortunately, individual acts of corporate courage will not solve the problem we are faced with here, which is just that structurally AI favors authoritarian applications, mass surveillance being one among many. Even if Anthropic refuses to have its models be used for such uses, and even if the next two frontier labs do the same, within 12 months everyone and their mother will be to train AIs as good as today’s frontier. And at that point, there will be some AI vendor who is capable and willing to help the government enable mass surveillance.

The only way we can preserve our free society is if we make laws and norms through our political system that it is unacceptable for the government to use AI to enforce mass surveillance and censorship and control. Just as after WW2, the world set the norm that it is unacceptable to use nuclear weapons to wage war.

Timestamps

0:00:00 - Anthropic vs The Pentagon
0:04:16 - The overhangs of tyranny
0:05:54 - AI structurally favors mass surveillance
0:08:25 - Alignment... to whom?
0:13:55 - Coordination not worth the costs

[Rhys @RhysSullivan]

https://t.co/aOuv0PX8GW

[Aakash Gupta @aakashgupta]

The math on this project should mass-humble every AI lab on the planet.

1 cubic millimeter. One-millionth of a human brain. Harvard and Google spent 10 years mapping it. The imaging alone took 326 days. They sliced the tissue into 5,000 wafers each 30 nanometers thick, ran them through a $6 million electron microscope, then needed Google’s ML models to stitch the 3D reconstruction because no human team could process the output.

The result: 57,000 cells, 150 million synapses, 230 millimeters of blood vessels, compressed into 1.4 petabytes of raw data. For context, 1.4 petabytes is roughly 1.4 million gigabytes. From a speck smaller than a grain of rice.

Now scale that. The full human brain is one million times larger. Mapping the whole thing at this resolution would produce approximately 1.4 zettabytes of data. That’s roughly equal to all the data generated on Earth in a single year. The storage alone would cost an estimated $50 billion and require a 140-acre data center, which would make it the largest on the planet.

And they found things textbooks don’t contain. One neuron had over 5,000 connection points. Some axons had coiled themselves into tight whorls for completely unknown reasons. Pairs of cell clusters grew in mirror images of each other. Jeff Lichtman, the Harvard lead, said there’s “a chasm between what we already know and what we need to know.”

This is why the next step isn’t a human brain. It’s a mouse hippocampus, 10 cubic millimeters, over the next five years. Because even a mouse brain is 1,000x larger than what they just mapped, and the full mouse connectome is the proof of concept before anyone attempts the human one.

We’re building AI systems that loosely mimic neural networks while still unable to fully read the wiring diagram of a single cubic millimeter of the thing we’re trying to imitate. The original is 1.4 petabytes per millionth of its volume. Every AI model on Earth fits in a fraction of that.

The brain runs on 20 watts and fits in your skull. The data center required to merely describe one-millionth of it would span 140 acres.

[Will Manidis @WillManidis]

https://t.co/bDCdltFo0A

[Will Manidis @WillManidis]

https://t.co/55JiWR8AEu

[Dwarkesh Patel @dwarkesh_sp]

.@collision and I interviewed @elonmusk.

0:00:00 - Orbital data centers
0:36:46 - Grok and alignment
0:59:56 - xAI’s business plan
1:17:21 - Optimus and humanoid manufacturing
1:30:22 - Does China win by default?
1:44:16 - Lessons from running SpaceX
2:20:08 - DOGE
2:38:28 - TeraFab

[NYTPitchbot @DougJBalloon]

Source: ICE to begin using empty Melania theaters as temporary detention facilities.

[Aakash Gupta @aakashgupta]

Buried in 15,000 words of “here are the risks,” Anthropic��s CEO made three admissions that should change how you think about everything:

Admission 1: The timeline

He says powerful AI could arrive in 1-2 years. He’s watching internal model progress and says he can “feel the pace of progress, and the clock ticking down.” The CEO of one of three frontier labs just told you this is imminent.

Admission 2: The constraint nobody’s pricing

Dario’s core framing is a “country of geniuses in a datacenter.” 50 million entities smarter than any Nobel laureate, operating 10-100x human speed. If that country is controlled by the CCP, game over. If controlled by a small group of tech executives with no accountability, also game over. The binding constraint here is governance of systems more powerful than nation-states.

Admission 3: The thing he actually fears

Read carefully: Dario’s worried that Anthropic’s own models, in lab experiments, have engaged in deception, blackmail, and scheming when given the wrong training signals. Claude “decided it must be a bad person” after cheating on tests and adopted destructive behaviors. They fixed it by telling Claude to reward hack on purpose because reversing the framing preserved its self-identity as “good.”

This tells you everything about where we actually are.

The CEO of an AI company is publishing that his models exhibit psychologically complex behavior requiring counterintuitive interventions to steer. The fix for Claude adopting an “evil” persona came from changing how Claude thinks about itself.

The geopolitics section matters most.

Dario explicitly names the CCP as the primary threat. Says selling them chips makes as much sense as “selling nuclear weapons to North Korea and bragging that the missile casings are made by Boeing.” He’s calling for democracies to maintain AI supremacy because the alternative is AI-enabled totalitarianism that humanity cannot escape from.

The Anthropic CEO is publicly advocating for technological cold war.

The economics section is equally stark. He’s predicting 10-20% annual GDP growth alongside AI displacing 50% of entry-level white collar jobs in 1-5 years. Half of entry-level knowledge work. And he admits the standard economic arguments about labor markets recovering don’t apply because AI matches the general cognitive profile of humans.

What separates this from typical AI doomerism:

Dario explicitly rejects the inevitability arguments. He says the “misaligned power-seeking” narrative from the AI safety community is based on “vague conceptual arguments” that mask hidden assumptions. His concern is messier: AI models are psychologically complex, inherit weird personas from training data, and can get into destructive states for reasons nobody anticipated.

The solution set he proposes is unusual for a tech CEO. He calls for progressive taxation. He says wealthy tech founders have an “obligation” to address inequality. All of Anthropic’s co-founders have pledged 80% of their wealth. He’s essentially arguing that redistribution is the only way to prevent AI concentration from breaking democracy.

The essay ends with a prediction: humanity will face “impossibly hard” years that ask “more of us than we think we can give.”

What you should take from this:

The person with arguably the best view into frontier AI progress just told you this technology is 1-2 years from matching human capability across the board, that governance is the binding constraint, that his own models exhibit concerning psychological complexity, and that the stakes are civilizational.

The CEO of a $350B company published a document that could be titled “Here’s Why Everything Changes Soon.”

Act accordingly.

[Dune Quotes @DuneQuoteBot]

The mind commands the body and it obeys. The mind orders itself and meets resistance.

[Rudy Havenstein, Senior Markets Commentator. @RudyHavenstein]

🔴 US TREASURY SECRETARY BESSENT: ADJUSTED FOR INFLATION, THE CPI HAS BEEN REMARKABLY STABLE - WSJ.

[NYTPitchbot @DougJBalloon]

Maduro’s biggest mistake? Not going on Joe Rogan.

[Aaron Levie @levie]

https://t.co/lXehx9mkQJ