Charles Krohn

Interested in open language models and free markets in intelligence and compute as forces for good

[Charles Krohn @op131csharpminr]

@TheZvi I riffed on this the other week:

https://t.co/izrN7OwAfC

At the time I thought that the Fed's independence was not so relevant to AI regulation, but that view seems dated now...

[Charles Krohn @op131csharpminr]

https://t.co/5hcMXMMh3I

[Charles Krohn @op131csharpminr]

What I find odd is that so many people who reject central planning in an economic context are so quick to jump to loss of control/gradual disempowerment framing when discussing the future more broadly. There’s no one in control of the economy, it’s not possible to control it, and even if it were, it wouldn’t be desirable. But despite this lack of control, market oriented systems allow for dramatically greater flourishing than control-based systems. These intuitions apply to AI safety too

[Charles Krohn @op131csharpminr]

I don’t love the “loss of control” or “gradual disempowerment” framings because I don’t think there’s a “we” that is currently in control in any meaningful sense. Instead we should be asking if AI increases human agency and ability to solve coordination problems, or decreases it, and much theorizing remains to be done

[Charles Krohn @op131csharpminr]

@tautologer Re: point (1) see https://t.co/NCif7NmHn8 . On point (2) see Helen Toner's writing on nonproliferation being the wrong approach to misuse

[Charles Krohn @op131csharpminr]

Depends on what you mean by "believe in RSI." In the very fastest takeoff scenarios, open models aren't relevant. But in slower (but still RSI) scenarios led by frontier labs, a vibrant open model ecosystem diffuses innovation, improves public understanding, and makes the top labs less central. I'm less concerned about safety because (1) the most dangerous uses come from frontier models with brand-new capabilities and not-totally-effective guardrails, and (2) nonproliferation is not a sustainable solution to misuse when open models continue to get better.

[Nathan Lambert @natolambert]

Banning open-source AI in any form would be a mistake. A general audience PSA with @kevinsxu on why open source upholds American values.

Managing frontier risks is hard, but reducing transparency, innovation, and education from kneecapping the open frontier would be worse.

[Charles Krohn @op131csharpminr]

So much policy is just about how much this administration likes you. This was Powell's problem, much moreso than his actual stance on rates, and it's Dario's problem too. I don't know what fixes it apart from a change of management or capitulation. Improvements to Fable's guardrails won't matter. Trump spent a year trying to fire Powell, supposedly because kept rates too high. But when rates seem set to rise again, he says "It's all right, whatever," because he's got his own guy in the seat. The actual policy doesn't matter.

https://t.co/sPblwwKB5L

[Charles Krohn @op131csharpminr]

Article here: https://t.co/izrN7OwAfC

[Charles Krohn @op131csharpminr]

https://t.co/5hcMXMMh3I

[Charles Krohn @op131csharpminr]

By now it's almost a cliche to argue that existing institutions aren't up to the task of regulating AI, and that something new is needed. I wrote a piece arguing that would-be institution designers can learn from the structure and organization of the Federal Reserve System: something unique among regulators in that it has real power, strong research capability, and carefully guarded independence from industry and the rest of government. Federal institution-building isn't the only solution here. We need new philanthropic institutions a la @nanransohoff, private governance a la @deanwball, and open research institutions a la @natolambert. But if you're going to build an effective regulator, the Fed offers some inspiration.

[Charles Krohn @op131csharpminr]

https://t.co/5hcMXMMh3I

[Charles Krohn @op131csharpminr]

Anthropic's anticompetitive announcement is a reminder of how all the top labs used to have a singular vision of ASI, which is long outdated and bad for the world. DeepMind initially had no competitors and they thought they could develop AGI on their own, in peace. OpenAI had the original "merge and assist" clause, thinking it was better if ASI was developed by a single entity. And now Anthropic is trying to make sure that they create the country of geniuses in a datacenter all by themselves. I am pleased that DeepMind has embraced a pluralist, liberal vision and I hope Anthropic eventually comes around.

[Charles Krohn @op131csharpminr]

Loved this and I think it highlights why the new Demis biography is so compelling. The guy has a tremendous work ethic, but it arises naturally from a desire to create and to know, rather than a desire to accumulate. It's not a coincidence that out of all the leading AI personalities, Demis is the one with the least utilitarian moral orientation.

[Charles Krohn @op131csharpminr]

My full submission is here:

https://t.co/r6xj7fBYXC

[Charles Krohn @op131csharpminr]

https://t.co/HMrXfMmmPG

[Charles Krohn @op131csharpminr]

I took a crack at this over the weekend. These are four excellent and challenging questions and I am excited to see the winners. I thought the third question was the most interesting and tractable. There are some sensible things that the OpenAI Foundation can do right now!

[Dwarkesh Patel @dwarkesh_sp]

$ 20k blog prize to answer some big questions about AI

The not-so-secret point of this whole contest is so that I can hire a research collaborator to think through questions like this hand in hand with me.

https://t.co/BD8YU5jtNd

[Charles Krohn @op131csharpminr]

I focused on two: a moonshot effort at independent frontier model pre-deployment evaluations, audits, red teaming, etc., and an effort to give away OpenAI shares to as many people as possible. The first addresses existential risk; the second helps share the gains from AI.

[Charles Krohn @op131csharpminr]

https://t.co/HMrXfMmmPG

[Charles Krohn @op131csharpminr]

This was a great article and a reminder that AI has a major problem delivering a positive vision for the next 3-10 years. "We'll bring you Utopia, but you have to go through hell first" is not at all compelling

[jasmine sun @jasminewsun]

Most people I know in AI think the median person is screwed, and they have no idea what to do about it.

I spent the last 3 months talking to dozens of researchers, economists, and policy experts about AI's impact on work; including reps from every frontier lab and several Congressional offices. Unfortunately, I was not reassured.

The AI industry is raising the alarm, but can't change course. These companies' core business model relies on the disruption they are warning about: their faith in full automation only makes them go faster.

Policymakers are waking up, but still paralyzed by data and debates. Econ wonks disagree on plenty, but even the limited scenario looks like a "painful transition" that will disempower millions of workers.

But an "underclass" is not inevitable, but rather a societal choice — and one we can and should stop. Instead of waiting for impact, we should start planning now to support workers through AI disruption. Whether policymakers can assuage concerns about economic security may determine if we get to reap AI's gains at all.

New from me for @NYTOpinion. I put a ton into researching what I think may be the biggest topic of the year, so hope you read it (gift link here!) https://t.co/NiGJpjyjzH

[Charles Krohn @op131csharpminr]

I grew up in Old Town, Alexandria and have many positive memories. Lots of kid friendly activities within walking distance: Windmill Park, the waterfront, Jones Point, the Torpedo Factory. King St metro isn't super convenient but at least it exists. Like many we outgrew our small place and later moved further out in VA, but for smaller or younger families I think it's a great place. When I recently visited, the appeal was still intact, though it's certainly more expensive than it used to be.

[Charles Krohn @op131csharpminr]

Legislation is slow and this is a highly personalized administration. There will be an overwhelming temptation to make safety decisions on a case by case basis. This is disappointing but unsurprising. Ad hoc informal regulation is a recipe for terrible outcomes

[The Wall Street Journal @WSJ]

Exclusive: The White House opposes a plan from Anthropic to expand access to its powerful artificial-intelligence model Mythos https://t.co/WmmlxMz9r5

[Charles Krohn @op131csharpminr]

This post reminded me of why METR, Redwood, Epoch, and a couple other organizations are so great. Safety work is too concentrated in the top labs, where there's danger of blurring into capabilities work, danger of complacency, and danger of being overly swayed by commercial motivations. It's important to have independent organizations doing research, monitoring progress, and improving public understanding. I wish there were more of them.