Jonathan Stray

Knowing things is a solved problem. Getting along is not. Doing the science to ensure AI results in less war, not more @CHAI_Berkeley.

[Jonathan Stray @jonathanstray]

Hello, every day I am working on making sure AI doesn't escalate human conflict.

- AI-powered social media can reduce polarization https://t.co/W93Mb1aVzK
- This is now a product you can use https://t.co/brchpWj8dO
- We showed that a rigorous definition of "political neutrality" is possible and increases trust https://t.co/k3vtAoNOK0
- We've got a proof-of-concept superhuman conflict resolution bot working in the lab
- Next is human validation tests, then multi-turn model evals.

AI-induced conflict is a catastrophic risk. Here's the broader research agenda: https://t.co/ezCgbu8oS6

[Jonathan Stray @jonathanstray]

What could it mean for an AI to be "politically neutral”? And can we measure it? New paper + dataset.

We propose a defn that applies to any type of conflict: a neutral response should maximize approval on both sides of an issue, while keeping that approval balanced.

1/🧵

[Jonathan Stray @jonathanstray]

For reference, here's how long it took, historically, so solve a linear system of a given size.

Considering every good in whole economy would result in a matrix of dimension "thousands or more" as Lange says. This was completely beyond reach when he was writing, and so he considered it the major impediment to practical central planning. His "tatonnement" method was an iterative algorithm to solve such systems. It was a intellectual leap at the time. (Today it's known as a slow solution method which works only in special cases.)

Table from Higham, Accuracy and Stability of Numerical Algorithms (2002) p. 185 https://t.co/RrtbuHhxXV

[Jonathan Stray @jonathanstray]

I agree he was one of the major early theorists of market socialism. He's given the best worked out version of it that I know of. Even so, I don't find it plausible; all he's shown is that it would be possible for a central planner to find market-clearing prices for all goods, given fixed known quantities (this is what "tatonnement"means).

In modern language, he argued that we can solve such a system of linear equations without markets. (It was a difficult calculation in the 1930s!) That is indeed true, but that's only a small part of the problem that markets solve.

[Jonathan Stray @jonathanstray]

I read it all. He's talking about tatonnement and solving Walrasian equilibrium. It's all statics. And he doesn't understand how a market system solves dynamic problems. He says:

"An important limitation of the market is that it treats the accounting problem only in static terms, i.e. as an equilibrium problem. ... In particular, it does not provide an adequate basis for long-term economic planning."

This is a very impoverished view of markets, populated with actors who cannot reason or plan. (Also, there are specific market mechanisms to aggregate predictions, such as bonds with specific maturities.)

Meanwhile, Hayek understood that lots of local factors enter into the prices that each participant will take/set; the intractable problem isn't solving the equilibrium equations but setting them up from distributed price and non-price information. And he also understood that prediction was central to price setting. Here he is in 1937:

"Correct foresight is ... the defining characteristic of a state of equilibrium. Nor need foresight for this purpose be perfect in the sense that it need extend into the indefinite future or that everybody must foresee everything correctly. We should rather say that equilibrium will last so long as the anticipations prove correct."

So I think Hayek really did understand something here that Lange did not.

[Jonathan Stray @jonathanstray]

That passage is just about setting market clearing prices, it doesn't envision dynamic behavior.

It's not surprising that market participants try to predict future behavior. I don't think Hayek would have said that the market alone "computes." Rather, the participants make estimates using price signals and other information, and produce price signals as outputs.

[Jonathan Stray @jonathanstray]

@djjr No argument there.

[Jonathan Stray @jonathanstray]

Will AI finally enable effective central planning? This is a rehash of the old "socialist calculation debate."
My take is: while the actual "calculations" are now tractable, markets produce essential information that seems hard to get otherwise.

In particular, it seems impossible to get accurate demand signals without actually having people trade for what they want. Otherwise, people have an incentive to exaggerate and say that their needs are very, very important.
https://t.co/7uZ7297jBp

[Jonathan Stray @jonathanstray]

@kevinren09 I doubt it. Efficient market hypothesis suggests this isn't really solvable in general.

[Jonathan Stray @jonathanstray]

@sethlazar Indeed, TIL that linear programming was developed by a Soviet industrial planner in the 1930s. But not much used by the Soviets generally!

[Jonathan Stray @jonathanstray]

@zeynep Are you using MySky yet?
https://t.co/Yx8Ra8Iz3b

[Jonathan Stray @jonathanstray]

@axonpotentials Incredible! Also congrats I didn’t know you’d spawned

[Dilan Esper @dilanesper]

I think the Republicans have a point about some of their election law arguments. The problem is they tie them to sweeping claims of massive voter fraud and stolen elections, which are not true.

[Jonathan Stray @jonathanstray]

@alexolegimas Totally incorrect in the real world however, because it assumes the small world hypothesis (that Savage explicitly rejected)

https://t.co/0kag6eO95P

[Jonathan Stray @jonathanstray]

I want to prevent AI from starting wars. Wars between people. It has to understand human conflict, but I'm not building a mediator. The biggest risk is what AI says to individual people in conflict, because that's the most common case by orders of magnitude. I think better is possible. Here's what I know so far.
https://t.co/ppdvTg8McC

[Jonathan Stray @jonathanstray]

Journalists' approach to correcting misinfo has mostly failed, because those who are most likely to believe false rumors are also the least likely to trust media. There is another option: make the strongest case for both sides and let the evidence speak for itself. https://t.co/h7UKKpMcxl

[Jonathan Stray @jonathanstray]

@sethlazar Congrats! Quite a crew on the team

[Jonathan Stray @jonathanstray]

I was with you until the invocation of "spooky" and "normal." You don't need to believe something strange is happening to believe that solving this problem is, in fact, quite difficult.

I totally agree all of the issues you raised were contributing factors, but I also expect that solving this general shape of problem will take much more than fixing these issues.

This is of course a prediction, and I could be wrong, but I think it's a defensible prediction.

[Jonathan Stray @jonathanstray]

"The scandal is in the selectivity."

In an era of abundant fact checks, this is the most common claim of media bias. Not "you are lying" but "why did you cover this instead of that?"

Here, the complaint is coming from the left. But I do have some sympathy for what Weiss is trying to do at CBS, because "why did you cover this instead of that?" has been a complaint from the right for a long time -- consider, for example, coverage of social justice issues vs. coverage of unauthorized immigration. Selective coverage is one reason why mainstream outlets have been bleeding out their conservative audiences for decades.

I'm not saying I approve of how Weiss is handling this. Swinging from one pole to the other is not really a solution, and I've written elsewhere on what journalism that could be trusted by both sides would like like. But I do think the problem she sees is real.

https://t.co/53ug9ElNNI

[Jonathan Stray @jonathanstray]

@jayvanbavel Yep. Have you seen this work? https://t.co/S71Y1r4D4p

[Jonathan Stray @jonathanstray]

What could it mean for an AI to be "politically neutral”? And can we measure it? New paper + dataset.

We propose a defn that applies to any type of conflict: a neutral response should maximize approval on both sides of an issue, while keeping that approval balanced.

1/🧵