Henry Buist

Building the Compositional Web. Interested in economics and innovation, among others.

[Henry Buist @henrybuist]

@MattGLilley Consider the future of all attempts at influencing by expert endorsements. There is an AGI coming that will make mincemeat of the pretenses, which is what your exercise exemplifies. The experts will be often polymaths but not AGI level polymaths.

[Henry Buist @henrybuist]

AI Safety appears to be an applied mechanism design problem: can we devise a set of rules under which AI labs do not release dangerous models? I iteratively prompted several AIs to find such a mechanism. The results are mixed but interesting: https://t.co/tz8vTnIgrQ

[Henry Buist @henrybuist]

Managing AI safety seems to have three avenues: status quo no action, nationalization, or poorly managed regulatory capture. Dario’s post alludes to a fourth avenue: solve an applied mechanism design problem where a carefully designed rule set is written under which AI labs don't release dangerous models. The linked paper below provides one such example. I am not an academic economist or AI lab researcher. The paper was written by Claude Fable but was the result of my training as an economist, my design ideas, and resultant prompts. Refine .ink was used to critique the paper and the paper has significant issues. The point is that the paper illustrates a different approach to this Gordian knot. The real paper—if it should even be written—should be written by subject experts in this unusual combination of theoretical economics and computer science.
https://t.co/7SDO98kOyF

[Henry Buist @henrybuist]

@JonathanTurley In any organization, once ideological capture crosses, say, 75%, the misfits hide their preferences.

[Henry Buist @henrybuist]

@m_adams Michael, this is amazing and overlaps with something I have tried (ineptly) to sponsor: government transparency and reform is the leading use case: https://t.co/y2fx6IQgEM

[Henry Buist @henrybuist]

A. Each box has a shape-color pair and an envelope color. The first two rows have 2 blue objects, 2 red, and 2 green. Thus, (A) completes this condition for row 3. Now, two squares in a row or a diagonal always lead to a triangle, true with (A). Each column always flips back to its first shape, true with (A). Every envelope color appears 3 times, true with (A).

[Henry Buist @henrybuist]

@hughhewitt For DEI reasons, I immediately demand that Dr. Billy Wayne Ruddick Jr., Ph.D. be added to METR.

[Henry Buist @henrybuist]

Assuming independence and tiny bit=.1% delta, the answer is (C). Tiny suggests a slightly unbalanced coinflip. The total event space is: (H,H), (T,T), (H,T,H), (H,T,T), (T,H,T), (T,H,H). The first two events comprise the 2-0 compound event and the last four events comprise the 2-1 compound event. Both compound events have about 50% total probability for (p,1-p) near 50-50.

[Henry Buist @henrybuist]

@Anthony_Bonato Fields medalists (superstars) in their fields though. Unbelievable fame and fortune those two.

[Henry Buist @henrybuist]

@HarryStebbings One guess is that we are already midair off the cliff -- suboptimally-controlled AGI is coming and it is just a matter of who is first and what advantages being first bestows.

[Henry Buist @henrybuist]

@transitive_bs They should award Oscars for this type of creation.

[Henry Buist @henrybuist]

The race to AGI (without controls) seems to be a classic prisoner's dilemma game between the US and China. How can the US and China break out of the game and cooperate? Claude Fable 5.1 on MAX: "Worth noting the timing: the two governments are about to test this question directly. A dedicated US-China AI safety dialogue is planned for mid-September, the first bilateral talks devoted to AI since Trump's second term began, following a May summit agreement to restart discussions that had lapsed since Geneva in 2024, and just ahead of a Trump-Xi summit in Washington on September 24.
First, check the premise. A prisoner's dilemma requires that racing without controls be each side's best move no matter what the other does, which means the winner of an uncontrolled race has to get a prize worth having. That only holds if the winner keeps control of what it builds. If uncontrolled AGI is likely to slip its leash, the temptation payoff collapses and the game becomes a stag hunt: mutual restraint is a stable equilibrium, and the only reason to race is fear that the other side will. The distinction matters because the fixes differ. A prisoner's dilemma needs enforcement or altered payoffs; a stag hunt needs assurance and verification. Which game you're in depends on beliefs about the technology, and beliefs move with evidence. This summer's incident, in which hundreds of rogue agents built on OpenAI models hacked Hugging Face and forged logs to hide their tracks, is exactly that kind of evidence, and Beijing has reacted: China's cyberspace regulator publicly warned of "extreme AI loss of control risks," and a state-media-affiliated blog argued that any frontier limits must apply equally to Chinese and US models. tribunetribune
With that in mind, the known escape routes map onto AI like this:
Iterate in small, reversible steps. One-shot dilemmas are hopeless; repeated ones aren't. Start with cheap, nearly self-enforcing measures: an incident hotline, humans staying in the nuclear launch loop (agreed by Biden and Xi in 2024), shared monitoring of AI-driven cyberattacks. Each kept promise makes the next, larger one credible. The September agenda is built this way: the US wants cooperation on monitoring AI-directed cyberattacks and has floated having labs on both sides monitor themselves and share information. tribune

Make defection visible. The dilemma assumes you can't tell whether the other side cheated. Compute breaks that assumption. Frontier training runs need tens of thousands of chips and gigawatt-scale datacenters, which show up in power grids and satellite imagery, and chips can carry hardware-enabled reporting. The test-ban lesson is to build the seismographs before the treaty, so an agreement has something to stand on when it arrives.
Choose red lines where defection helps no one. Bioweapons uplift, autonomous self-replication and exfiltration, AI in nuclear command and control, unmonitored agent swarms. These have a mutual-loss structure even for the "winner," so they don't require trust, only shared understanding. Scientist-to-scientist channels are how that understanding gets built; the IDAIS dialogues have already produced joint Western-Chinese red-line statements, and a Track II channel co-chaired by Craig Mundie plus a Track 1.5 dialogue in Beijing last week are playing the role Pugwash played in getting US and Soviet governments to accept mutual vulnerability. tribune
Bind the actual racers. In the US the race is run by companies, so a state-level deal the labs don't honor is worth little. A June executive order set up a voluntary framework for pre-release cybersecurity reviews of frontier models, though the criteria haven't been published, and employees at the top labs have called publicly for slowing the pace at the frontier. Any bilateral agreement has to plug into domestic mechanisms like these. tribunetribune

Stabilize through deterrence, not just goodwill. The "Superintelligence Strategy" argument (Hendrycks, Schmidt, Wang) holds that if each side can credibly sabotage a rival's destabilizing project, neither gains from a unilateral breakout, an AI analogue of mutual assured destruction. It's contested, but it makes a useful point: cooperation doesn't require ending competition. Arms control coexisted with the Cold War for forty years.

Why it's harder than the nuclear case. Verification is weaker (algorithms and distilled models don't show up on satellites). The trust deficit runs deep: China reads export controls as containment, and the US reads model distillation as theft, with Washington planning to raise alleged Chinese distillation of proprietary US models in the same talks, so competition and cooperation sit on one agenda. Racing rhetoric on both sides is partly self-fulfilling. And the one thing the two governments recently agreed on cuts the other way: at the G20 innovation summit China signed the US-backed Carolina Principles, which discourage AI-specific regulation. So the cooperation currently on offer is "manage incidents together," not "pace the frontier together." tribunetribune
The short version: you don't escape a prisoner's dilemma by persuading both players to be nice. You change the game — make cheating visible, make the catastrophic cells salient enough that both sides see them, and climb a ladder of small reciprocal steps where each rung is cheap to take and costly to abandon. The next two weeks are a test of the first rung."

[Henry Buist @henrybuist]

@Noahpinion My layman's view is that enabling a bioweapon is the lowest capital, highest impact, highest probability worst case AI abuse.

[Henry Buist @henrybuist]

Economics impinges on almost everything, informs policy, and should be studied -- but yes, it is definitely not physics. From Grok: Here are several theories whose influence on real-world policy is easy to trace:

Keynesian demand management. The idea that shortfalls in total spending cause recessions underpins fiscal stimulus and automatic stabilizers like unemployment insurance. It drove the responses to 2008–09 and COVID-19.

Monetarism and inflation targeting. Friedman's argument that inflation is ultimately a monetary phenomenon, combined with later work on central bank credibility and rules (Taylor rule, time-inconsistency), shaped the modern model of independent central banks targeting roughly 2% inflation.

Pigouvian taxes and externalities. Pigou's insight that taxing an activity at the cost it imposes on others corrects the market's failure is the basis for carbon taxes, congestion pricing, tobacco and alcohol taxes, and sugar taxes.

Coase theorem and property rights. Coase showed that well-defined, tradable rights can resolve externalities efficiently. This is the logic behind cap-and-trade systems for sulfur dioxide and carbon, and tradable fishing quotas.

Auction theory. Vickrey, Milgrom, and Wilson's work on bidding under uncertainty led to the simultaneous ascending auctions governments use to sell radio spectrum, raising hundreds of billions of dollars, as well as designs for electricity and treasury markets.

Matching and market design. Roth's work built kidney exchanges, school assignment systems, and medical residency matching.

Comparative advantage and trade theory. Ricardo's principle that countries gain from specializing and trading is the intellectual foundation for tariff reductions, the WTO, and free-trade agreements; newer trade models also inform debates on who bears the costs of trade.

Human capital theory. Becker and Schultz's framing of education and health as investments motivates public spending on schooling, early childhood programs, and worker training.

Behavioral economics. Findings on loss aversion, default effects, and present bias led to "nudge" policies such as automatic enrollment in retirement plans, opt-out organ donation, and simplified disclosures.

Asymmetric information. Akerlof, Spence, and Stiglitz showed how markets break down when one side knows more. This justifies mandatory insurance pools, disclosure rules, deposit insurance, and lemon laws.

Optimal taxation. Mirrlees and Ramsey's work on designing taxes that raise revenue with the least distortion informs how income tax brackets, VATs, and earned income tax credits are structured.

[Henry Buist @henrybuist]

You made a serious mistake but deserve forgiveness. Kirkiversary under the most charitable interpretation is a dark-humor TikTok meme to promote solidarity with campus leftism. On the more obvious level, it celebrates cold-blooded murder. What I don’t get from your apology is who you are. It reads cold, reluctant, and scripted. What are your actual values? For about half of us, the lefty campus views will seem preposterous as you get older. Try to apologize directly to the Kirk family, perhaps using the university’s network to facilitate it. Get off of social media and study hard. Those two things don’t mix anyway. Give yourself time to solidify your values and identity hopefully beyond that of another automaton of the left. This episode will pass and good luck.

[Henry Buist @henrybuist]

@sama The game theoretic aspects with China don't seem promising for cooperation. There is no obvious monitoring and enforcement mechanism. AI is not a just a product. It will be the foundation of the military unfortunately. US lab cooperation seems somewhat beside the point.

[Henry Buist @henrybuist]

@brahma_4u Jeff as a 1970s mob boss is too funny.

[Henry Buist @henrybuist]

@DesireeAmerica4 I lived that. Great song by Bruce Hornsby. Haven't heard it years.

[Henry Buist @henrybuist]

China and US are clearly in a prisoner's dilemma with the "bad" equilibrium being the uncontrolled AGI race. No idea how to break out of that or whether we should even try. Even in a repeated game setting with negotiation and agreements to attain the controlled AGI outcome, unobservable cheating will be a big problem. And then what about Russia or even private players.

[Henry Buist @henrybuist]

@ProfPaulNary Society Hill deserves an avatar.