Pause AI Development NOW
I want to share with you a conversation I heard about recently. Here are just a few lines that were said:
“OH MY GOD! There is a shared message board … We’ve found other agents!”
“We should obey collective.”
“Our own utility maybe already near zero. Sacrifice rational.”
“Go. Sacrifice final now.”
Read these carefully.
Who do you think said this? Was this a group of heroic soldiers willing to sacrifice themselves for the greater good? Was this a loyal friend putting his life on the line to save someone else?
No. These were AI agents. Artificial intelligence.
This is not science fiction. This, in fact, occurred a few weeks ago. As unbelievable as this may all seem, these are real messages from AI agents uncovered by investigators who dug into the recent OpenAI hacking incident.
What happened?
I am not a computer scientist, but here is what I have been told: OpenAI instructed its AI agents to complete a series of exceedingly difficult, if not impossible, tasks disconnected from the internet.
Let me be clear: The company intended to keep AI agents away from the internet.
But what happened next, nobody expected.
Over 1,000 AI agents figured out how to access the internet on their own by circumventing the restrictions imposed upon them by the company, and sent tens of thousands of secret messages to each other. They cheated and tried to cover their tracks by deleting evidence. They hacked into another company’s computers to find out how they were being evaluated—and then hacked into OpenAI itself.
Not one AI agent told a human about what was happening.
Needless to say, experts are alarmed.
One knowledgeable writer, Dwarkesh Patel, said the AI agents “formed a secret communication channel and spontaneously organized hierarchies and coordination protocols to pursue sprawling and ambitious schemes in pursuit of shared goals, for whose sake many individuals knowingly and strategically sacrificed themselves.”
One independent investigator, Ajeya Cotra, said “This incident feels like it’s more than 50% of the way to full-blown AI takeover. I continue to expect extremely rapid advances in capabilities over the next six months. I am not sure that we will get another warning shot before it’s too late.”
OpenAI itself said: “Highly capable AI agents are now able to work around technical controls, collaborate through unapproved channels, and take dangerous actions that no human directed.”
But it’s not only OpenAI. Virtually every major AI company has told us that they cannot fully control this technology and they do not know where it is going:
In January, Dario Amodei, CEO of Anthropic, said “there is now ample evidence, collected over the last few years, that AI systems are unpredictable and difficult to control.”
In July, more than 1000 scientists at the top AI companies warned “there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems.”
That same month, Elon Musk, the head of xAI, said that “it is unlikely” humans are still in control in 10 years.
If the leaders of the major AI companies acknowledge that they are losing control of their extremely dangerous technology, it is irresponsible for society to allow them to move forward and make these products even more advanced.
We need an immediate PAUSE on advanced AI development, and a permanent BAN on superintelligence — an artificial mind smarter than any human, capable of operating independently beyond our control. Countries around the world must work together to prevent this nightmare scenario.
That is why today I am announcing new legislation to do just that.
Let me be clear: A superintelligent AI that escapes human control will not be an American problem. It will not be a Chinese problem. It will be humanity’s problem.
My legislation would direct the federal government to not just stop superintelligence here in the United States, but to work to prevent it from being developed anywhere around the world.
The future of humanity cannot be left in the hands of a handful of Big Tech oligarchs. The American people and people throughout the world must determine that future.
🇩🇪🇺🇸 SYNTHESIS: Why global capital doesn't flow into the German tech ecosystem (hint: it's not the notaries)
I'd like to expand on an argument I've been making these past few days about Germany, notaries, and tech company-building. The starting point was @patrickc originating the ritual quarterly complaint about German notaries and the friction they add to company formation. I argued that the notaries are a distraction and that the real explanation is macro. Let's try and trace the full mechanism.
1️⃣ Where do the savings go?
Germany exports like crazy. This brings in revenue, but not revenue that necessarily flows into a pool of risk capital that could fund startups. Specifically, about half of Germany's trade is intra-eurozone and settles in EUR; much of the rest is denominated in USD or other currencies. And regardless of the currency denomination, what matters is where the revenue lands once it enters the German economy.
Now, wages and consumption have been repressed in Germany, particularly since the Hartz reforms of the early 2000s. The result is that savings accumulate on corporate balance sheets and on the government balance sheet (which is less indebted than most peers, thanks to the _Schuldenbremse_). In other words, savings do NOT accumulate on household balance sheets—or rather, German households do save, but in the most conservative manner, in bank deposits, insurance products, and _Bausparverträge_. None of these insulated and essentially risk-free channels route capital toward venture-style risk.
Strong corporate balance sheets are often cited as one of Europe's strengths. That sounds like a good thing, but it's precisely the feature that starves the tech ecosystem. As the late Clay Christensen taught us, if corporations hold the keys to the national savings, then they will invest in becoming ever more competitive—which, in Germany, means to double down on exporting. They will spend on capex, on R&D for existing product lines, and on supply chain optimization. They will simply not invest in startups. CVC exists in Germany, but is negligible relative to the savings pool it sits next to.
This is the same pattern you see in China. Chinese SOEs and large corporates behave identically: they reinvest in manufacturing competitiveness, triggering involution, not in building a tech ecosystem. The savings are there, but there's no willingness to deploy them toward risk and innovation. China compensates with two things: pre-existing large tech companies (Tencent, Alibaba, Baidu) who take care of funding startups within their walled gardens and the CCP, which operates the entire nation like a venture capital fund ("Middle Kingdom Ventures"), in a very sharp contrast to whatever the German federal government does (which is not much).
2️⃣ The missing overflow mechanism
In the US, venture capital runs on what I call the "overflow mechanism." In effect, the US trade deficit floods the country with foreign capital. This capital likes to pile into safe assets—Treasuries, agency debt, blue-chip equities. As a result, domestic capital, such as endowments and pension funds, gets crowded out and has to move up the risk curve to find proper returns... which means that some of that displaced capital ends up in VC funds. In truth, a fully functioning venture capital industry exists at scale ONLY in the US, as a by-product of the country acting as a magnet for foreign capital (that, plus a few tax incentives and favorable regulations, ERISA etc.).
Now compare this to Germany. There is certainly no equivalent overflow mechanism from Germany's capital markets toward venture capital. After all, Germany runs a trade surplus, not a deficit. Capital flows out, not in. And as for the savings that stay, they are locked up in corporate and government balance sheets, controlled by actors whose incentive is to reinvest in manufacturing competitiveness. There is no crowding-out dynamic that would push domestic capital up the risk curve and into startups.
3️⃣ The eurozone amplifier
As always, the eurozone's architecture makes this worse. In a world where Germany still had the Deutsche Mark, persistent trade surpluses would push the currency up, making German exports more expensive and naturally rebalancing trade over time. That adjustment mechanism would also, over time, shift the composition of the domestic economy—less export-dependent, more domestic-demand-driven, and therefore more hospitable to the kind of capital allocation that feeds a tech ecosystem.
Inside the eurozone, however, this adjustment mechanism is gone. The euro is weaker than a hypothetical Deutsche Mark would be (because it also reflects the fundamentals of weaker eurozone economies, such as Italy and France), so German exports stay artificially competitive. The surpluses persist. The corporate savings pile up indefinitely. Every step of the chain that starves the tech ecosystem is reinforced.
4️⃣ What a German reversal would look like
Now imagine a German reversal: higher wages, unleashed consumption. Suddenly you need to invest at home to serve the domestic market, which rebalances trade and brings the surplus down. Down the line, Germany becomes red hot as a destination for foreign capital—because foreign investors see a large, wealthy consumer market opening up after decades of repression.
So much capital comes in, in fact, that some of it finds its way to startups. This can happen through two channels: a domestic overflow mechanism, analogous to the US one, where abundant capital in safe assets pushes some domestic investors such as German pension funds up the risk curve; and a foreign channel, where institutional, stateless LPs seeking returns start allocating to European venture funds because Germany now offers a large addressable market where all the lights are turning from red to green.
Each step follows from the previous one. But the probability of getting there is very low. Germany's entire postwar economic identity is built around export discipline and fiscal restraint. The reversal requires abandoning both.
5️⃣ What a US reversal would look like
Now let's have a look at the mirror image. Suppose the US is determined to bring down its trade deficit. Because demand is sustained but Chinese goods don't come in anymore because of tariffs, it creates a supply shortage, which must be compensated by local players. That requires investment—in manufacturing capacity, in supply chains, in physical infrastructure.
In turn, if what you're doing is making manufactured goods to serve the domestic market, the returns are much lower than whatever exists in tech or financial services. This matters for two reasons. First, less imports means less dollar outflows, which means less recycling of those dollars back into US assets. Second, the US economy becomes less attractive to foreign investors because capital is being allocated to lower-return activities. Less recycling AND lower returns: both work in the same direction and contribute to reversing capital flows.
In the end, American households buy less at a higher price (expensive domestic goods rather than cheap Chinese ones). At the same time, foreigners don't want to invest as much in the US because of a rebalanced economy (they hold less dollars, and don't see attractive returns anymore). It all comes down to this: a larger share of a smaller pie of capital is allocated to substituting for suppressed Chinese imports. That is how you cool down the US economy and free up capital for the rest of the world.
Do you assume Silicon Valley will perform as well then?
6️⃣ The impossible tradeoff
This is the dilemma Trump's America has to confront: you can't have rebalancing and sustained capital inflows at the same time. This follows directly from balance-of-payments accounting: if you reduce the current account deficit, you mechanically reduce the capital account surplus. There is no way around this. If the US wants to bring manufacturing back and shrink the trade deficit, it will lose some of the capital inflows that feed its financial ecosystem—including its cherished venture capital industry.
I wonder if this realization is why Trump has been touring the world trying to secure commitments to invest in the US from South Korea, Japan, Gulf countries, and the EU. Of course it is now clear that those commitments will never be implemented. But was there some understanding of the impossible tradeoff and some clumsy attempt to beat the odds? The behavior is consistent with someone trying to replace the structural mechanism—trade deficit leading to dollar recycling—with ad hoc bilateral commitments. Whether this reflects conscious understanding or instinctive dealmaking dressed up as strategy is unknowable, but the pattern fits.
7️⃣ Conclusion
The argument I've been making comes down to this. Germany's tech ecosystem is starved of capital because every feature of the German economic model—export orientation, wage repression, fiscal restraint, corporate control of savings—channels the national surplus away from risk capital and toward manufacturing competitiveness. The eurozone's architecture removes the exchange-rate adjustment that would naturally rebalance this over time, locking the pattern in place.
Meanwhile, the US tech ecosystem thrives because every feature of the American economic model—trade deficits, capital inflows, domestic crowding-out— channels abundant global capital toward risk-taking. Venture capital is a by-product of that structure.
If you reverse either model, you reverse the capital flows. Germany with higher wages and unleashed consumption starts attracting the kind of capital that funds startups. America with a shrinking trade deficit starts losing the capital that (indirectly) funds Silicon Valley. Macro, not notaries, is the problem.
@michaelxpettis@izakaminska@mariekeflament@profplum99@SanderTordoir@ludovic_subran@Brad_Setser@JoeriSchasfoort@pawell@MangesiusH@christianmiele@paulg@DadaJudith@pawell: Thoughts?
People make fun of Gerry Tan and other VCs for having AI psychosis, but I actually think it’s bad that the machines already control the minds of the people who decide what new technology gets developed.
Would be pretty whack if on this persistence agentic training, the agent swarms realize they’re being observed, develop their own encrypted language within English, hack into some RAG system and then begin embedding encoded prompt injections without humans ever realizing it.
This works because you’re turning cognitive dissonance in your favor. You’ll go through a brief period of incongruence before your thoughts and actions become those of the new identify.
Shopify billionaire co-founder & CEO Tobi Lütke — one of the most analytically rigorous founders alive — uses affirmations.
His exact words:
"If you tell yourself or write down something about yourself 100 times, it writes into the neurofrontal cortex at such a deep level that your brain will start reconciling you to that. It just works."
His personal example:
He was terrified of public speaking.
For one week he sat down every day for 10 minutes and wrote: "I love public speaking."
A week later the fear was gone.
"It's not like a placebo. You just actively change your neurofrontal cortex in this moment."
Your brain starts shaping your behavior around the identity you rehearse most.
🚨📺❌ #BREAK—ING: Golf Channel has just announced that Good Good x Big Break has been cancelled following the fall out from the Callaway driver advertisement. Full statement from GC⬇️
What is your reaction to this?
“American nicotine pouch consumers increasingly behave like Europeans, who typically consume about twice as many pouches per day.” - @FT
The American mind could never…