What happens when AIs become smarter than us?
Why would they keep humans around if given the choice?
Our new paper argues that only trying to control AIs is a limited strategy, and that a stable, mutualistic human-AI future may be possible.
@dewierwan_ I think you’d want a moratorium to be easy to implement and not require lots of experts. “Any system that depends on human reliability is unreliable.”
Update: I'm no longer advising AI companies. I think they're too constrained to make the situation much better, so I'll stay independent until there is a meaningful slowdown or moratorium on this reckless race.
Numerous AI safety groups told us privately they agreed with this statement, yet felt unable to sign without risking relationships or future funding. Anthropic has almost, but not entirely, captured the ecosystem meant to hold it accountable.
Ahead of Anthropic’s IPO, a rare coalition across AI safety, open source, e/acc, science, and law has signed the MisAnthropic Statement. Read the statement or sign at: https://t.co/wjlNbSL8EX
Yes I agree the theory is inconsistent with his preferred moral theory, utilitarianism, which says your dog's moral weight is your child's weight which is a bug's weight. He spent the end of his life trying to handle the repugnant conclusion, which the proposed connectedness function successfully does (Appendix A3) thanks to his ideas.
https://t.co/XLQerdnreM
ChatGPT doesn't seem to agree. I didn't read the linked posts because I don't think this community has a good understanding of utility theory. For example, in recent years they discovered that the independence axiom is a source of trouble, despite this being well-known among academic researchers for a long time.
(I would also claim they don't have a good understanding of utilitarianism, and someone who reads a very short introduction to utilitarianism will be less confused than a longtime lesswrong reader.)
> Kelly criterion is only for the cases where you have multiple bets in a row or non-linear utility.
Kelly is when you have the world continue. One-shot is when everything is about to end. Cowen's opening question did not have the one-shot qualifier.
EAs by and large don't get this, and it's extremely dangerous. It leads them to say "even though that option is robustly good it's not highest EV" which causes them to play it fast and loose.
Caroline Ellison:
"Is it infinitely good to do double-or-nothing coin flips forever? Well, sort of, because your upside is unbounded and your downside is bounded at your entire net worth. But most people don’t do this, because their utility is more like a function of their log wealth or something and they really don’t want to lose all of their money. (Of course those people are lame and not EAs; this blog endorses double-or-nothing coin flips and high leverage.)"
https://t.co/MlCeft6pu1
@theproteindance "Each counts as one, and none counts as more than one" - Bentham.
There is equal weight for wellbeing under utilitarianism (though the amount of wellbeing can vary).
NEW: According to a bombshell report in the New York Times, Anthropic co-founder Chris Olah threatened to walk out of Pope Leo XIV’s AI encyclical launch in May because the pope rejected the idea that machines can be conscious.
Olah’s team then privately lobbied the pope’s advisers “to take the possibility of model consciousness seriously.” Pope Leo XIV held firm.
For months, Anthropic has wined and dined theologians and religious scholars under nondisclosure agreements, hoping they would bless the idea that Claude has moral standing. https://t.co/iG3Bnlzt2J
Launched a new course on Coursera today.
It has 3 parts:
- Intro to AI (drivers, bottlenecks, timelines)
- AI Risks (malicious use, loss of control, concentration of power)
- AI Governance (governance at AI, corporate, national, and international-level)
https://t.co/AQHueWiORZ