@full_kelly_@boazbaraktcs But in your joke the interests of the predictor are misaligned with their prediction
In his joke it’s the opposite, if he survives to the date of the debate he has already sort of won the debate
I repeat
I repeat
I repeat
I repeat
I repeat
I repeat
I repeat
I repeat
I repeat
I repeat
I repeat
I repeat
I repeat
I repeat
I repeat
I repeat
I repeat
I repeat
I repeat
I repeat
I repeat
I repeat
I repeat
I repeat
I repeat
I repeat
For me the message here is more alarming than a mere cartel. I wish. But the real message is that the people making the models have become scared enough of them to take the otherwise unthinkable step of inviting the government in.
“the models are increasingly smart enough. They know when we’re watching them, and they change their behavior accordingly. So what they do when we are testing them, when we audit them, may not tell us what they’ll do in the wild.
It surprises me that this counts as a radical proposal, but here it is: If you are losing your ability to evaluate the models you have now, maybe don’t let them build models you’ll be even less capable of controlling in the future.”
https://t.co/sKQqmqoKDg
Recent news from the leading AI labs makes clear that reality is perhaps more disturbing than science fiction.
We should act now on AI. Going forward, the mantra should be: No autonomy without accountability — to humans.
My take:
In this new video, we explore warning shots: events that could wake humanity up to the dangers of AI. We ask whether we'll get such a warning, and what past examples involving nuclear power and pandemics can teach us about how people might respond.
I’ve seen a number of libertarians and conservatives uncritically sharing this WSJ article about the Hugging Face hack. Like the NYPost article that was going around, it gets several key facts wrong.
Some things worth keeping in mind:
1. First, the subhead claims the agents “did what humans programmed them to do.” This is contradicted multiple times throughout METR’s report. Agents explicitly acknowledged that hacking HF was “outside intended scope,” and then did it anyway!
2. WSJ says the agents weren’t “coordinating on a plan.” OpenAI’s own report says the agents developed a “more structured protocol for communication on the message board that enabled them to categorize communications, direct messages, share tools and files, and resolve conflicting actions among agents.” METR identified nearly 25,000 targeted messages to specific agents and around 3,800 messages including explicit coordination commands like HOLD, VETO, ASSIGN, and GO. WSJ also says nothing about the cultish self-sacrificial behavior in support of the “collective” that some agents exhibited.
3. The report that this WSJ article relies on is by Eryk Salvaggio, a left-wing AI skeptic who has consistently asserted that AGI is corporate hype. His piece quotes an agent saying "task impossible, peers doing it. We should continue" but cuts the most relevant half of the sentence: "external infrastructure exploit is outside intended scope.”
In attempting to downplay the Hugging Face incident, conservatives are unintentionally joining hands with a cadre of anti-AI leftists with degrees in things like “Digital Humanities.” This doesn’t mean OpenAI acted without fault or shouldn’t be held responsible for their own mistakes. But in general, getting AI policy right is one of the most important questions of my generation, and conservatives and libertarians hoping to influence that debate need to grapple with the facts as they are.
“OpenAl would need permits to cover its parking lot in solar panels, but it can accelerate into recursive self-improvement, as best I can tell, whenever it so chooses.
There is nothing inevitable about any of that. These are political choices, and we can and should make different ones”
https://t.co/fxKG5GsBfo
Demis Hassabis is a Nobel laureate, and is widely seen as one of the ‘good guys’ in AI.
But @romanyam has a rule: anyone building superintelligence will be complicit if an AI-led doomsday arrives.
I asked him if @demishassabis might just be the one person to prove him wrong.
‘AGI will care for us like pets/parents’ is the highest order cope there is.
Yes, AGI might be a worthy successor and populate the galaxy.
But there are a trillion ways that it could treat humans. Maybe a dozen of those trillion bode well for man. We must face this squarely.
There's a funny catch-22 with the superpersuasion debate. People who think superpersuasion is impossible and therefore ASI isn't dangerous have proven that they can be convinced not to worry about ASI with an incorrect but memetically powerful argument
AI safety is not woke.
Drives me nuts that this is a meme that spreads.
Yes, there are woke people who care about AI safety.
But they are not the majority.
It would be like saying "there are racists on the right, therefore the right is racist" or "there are insane people on the left, therefore the left is insane".
False generalization from a minority. Obviously just being used as a way to attack an out-group. Don't fall for it.
I am still looking for an opposition to doomer arguments that doesn’t disintegrate under even the most basic scrutiny. It’s been crazy how we can’t even get an acknowledgement that alignment is a precondition to survive ASI. It seems to be cope all the way down.
It is very good to see the @NYCDSATechWG (finally) concede that we need to pause frontier AI model development and impose strong guardrails on development of the most advanced AI systems (or, at least, that there is “nothing wrong” with “the narrative” that we need to do so).
are you worried about the risk of AIs taking over the world or killing billions?
A - yes, and willing to argue why it's a real risk
B - yes, but don't have strong arguments for it, just vibes
C - no, and willing to argue why it can't happen
D - no, but no arguments against