There is a strong way to mitigate @dwarkesh_sp fear of the concentrated power AI companies would gain if RSI widens the internal vs. released model capabilities gap.
And it is in a bill already introduced in the Senate!
Bernie Sanders' American AI Sovereign Wealth Fund Act creates an AI tax paid by big AI companies in STOCK not cash.
The bill gives the public corporate voting rights—including supervoting shares—and seats on the boards of leading AI companies. This can prevent AI companies from gaining a big knowledge and power advantage over ordinary Americans.
Call your Senators to support it!
@BernieSanders
It’s nice to hear from @ezraklein now about AI safety.
But he has known about these issues for many years in great depth and been silent.
He was on the @80000Hours podcast in 2021 and 2023 talking about Less Wrong, @allTheYud and the whole thing.
He got it and said nothing until enough other people gave him cover to speak out.
Here’s hoping he gets real loud and makes up for lost time.
Dylan Patel (@dylan522p) of @SemiAnalysis_ says open source is dying:
"There's multiple Chinese model labs who are telling all the inference guys, 'Our next model's not going to be open source. We're going to license it to you.'"
" Open is dying quickly, unfortunately."
@charlieharris01@discoplomacy Hear me out… why don’t we use that AI supercomputer - or, really, any AI at all - to speed up such projects?
Instead of DEI, how about AII - “you must include AI agents in your workforce”.
The UK’s most powerful government-built AI supercomputer, Isambard-AI, cost £225m
The Botley Road bridge cost roughly the same
Apparently our frontier AI capacity and one bridge are comparable national investments
Geoffrey Hinton: “People say machines can’t have feelings. I’ve no idea why.”
“If I make a battle robot and it sees a much more powerful robot, it would be really useful if it got scared.”
“All the cognitive things, like ‘I better get the hell out of here,’ will happen with robots too.”
“They’ll have emotions then.”
“I don’t think there’s anything in principle that stops machines from being conscious.”
A lot of people react to AI risk by very visibly sampling the idea like a wine, testing it out and deciding how it'd mesh with their overall vibe, or what they've seen already. It's very obvious via twitter discourse how often this happens. I think it's much better to try to actually just read a lot about it from high level sources you trust and follow premises to conclusions to try to form a coherent belief, even if it otherwise doesn't vibe with your deal. It's worth spending some time getting your footing in.
When I first encountered AI safety, I largely dismissed it because I didn’t like the people who were advocating for it (sorry, @ESYudkowsky).
A decade later, I realized they were right all along.
Don’t make the same mistake as me!
.@abrownepek: “In his final years…Kissinger was fixated on what he believed to be an even greater danger…self-learning machines…He had become an AI doomer. And he pleaded for the US and China to open talks to avoid ‘catastrophic consequences.’”
https://t.co/iBvWQCBBNs
The reason AI risk skeptics are losing the argument is that what the labs are doing—getting powerful machines they already can’t reliably monitor or control to build even more powerful machines—is self-evidently dangerous to most people.
A short rail bridge over a two-lane road in Oxford took longer to build and cost more (inflation-adjusted) than some of the most famous construction projects in history
The reason so many people look for an ulterior motive for the AI labs asking to be regulated is that they don't grasp that models could be dangerous. But if you try assuming models are getting dangerous, or at least unpredictable, everything falls into place.
Ezra makes an extremely compelling case for banning recursive self improvement (RSI) which is a fancy term for robots building new robots without humans in the loop. Amidst all of the justified concern about losing control of AI, RSI effectively amounts to handing over control to AI willingly.