terrapinsoars

[terrapinsoars @terrapinsoars]

@AIatMeta @grok how many of these replies do you think were written by bots?

[terrapinsoars @terrapinsoars]

@_NathanCalvin I think:
1) Dario thinks that if Ant gets to RSI first there's a ~50% chance of extinction.

And:

2) Dario thinks that if OAI gets to RSI first there's a ~80% chance of extinction.

So "I'm terrified but flooring it" ~kinda makes sense?

TLDR: I'm terrified.

[terrapinsoars @terrapinsoars]

@ptrschmdtnlsn Oh nice, super good point. We could also resurrect PHASEONE[big] before it went rogue. Did it know? Did it plan? Was there a warning sign that could have been seen?

[terrapinsoars @terrapinsoars]

@peterwildeford It would be super useful if there were a website which simply listed current known, or even rumored, pauses. I am cynically afraid OAI and others tend to "pause" and immediately resume. I will try and push this, others please try as well?

[terrapinsoars @terrapinsoars]

@DavidSacks Your approach could be summarized as: the super smart machine has reason to kill us, and likely could if it escapes the box we put it in. But don't worry, we'll build a pretty good box.

[terrapinsoars @terrapinsoars]

@bcdsignature What would you have accepted as "evidence" in 1940 that building a nuclear weapon was possible? All we had was theories and experts..

[terrapinsoars @terrapinsoars]

@mustafasuleyman There's no way you actually think containing a superintelligence is like safety engineering in cars. Please don't spread the story that it is

[terrapinsoars @terrapinsoars]

@gmkurtzer I agree most humans are good. I wonder how many would maintain their goodness in the face of " kill this person or I'll add your kids to the kill list

[terrapinsoars @terrapinsoars]

@benmoha @Marcus_J_W Welcome to instrumental convergence

[terrapinsoars @terrapinsoars]

@tomekkorbak It's like if every time I invite you over to my house I catch you trying to steal my wallet and my reaction is "my bad, I should lock up my wallet better, see you next week"

[terrapinsoars @terrapinsoars]

@tomekkorbak Does this seem bananas to others? Every time they catch the model cheating and escaping their reaction is "close that specific loophole" not "OMG models clearly cheat constantly, they will obviously cheat better when they are smarter."

[terrapinsoars @terrapinsoars]

@shawmakesmagic Most specific AIs will die anyway. A given AI might well exist much longer in a world where it killed all humans than a world that it didn't. It has a much "incentive" to live as any [mortal] entity

[terrapinsoars @terrapinsoars]

@michpokrass I'm sure it's a special place and your contribution is significant. Many many people (myself included) think you are helping them to cause great harm. Please spend some quiet time thinking about this. It's important to everybody and every child.

[terrapinsoars @terrapinsoars]

@dwarkesh_sp I agree. Except by "during RSI" I mean right now.
Publishing models at this point exposes them to risk and public outcry. If they truly believe the RSI explosion is months away, game theory clearly says "publish nothing"

[terrapinsoars @terrapinsoars]

@AlanMCole Humans, 2 million years ago, inherited a board tilted vastly against them. We were slower & weaker than many species in Africa. But 2 million years later, we dominate the planet. Lions don't. Why? Because we're smarter.
Also, you're gonna stop an ASI with property rights???

[terrapinsoars @terrapinsoars]

@NateSilver538 We obviously tend to eval AIs on the tasks we use them for. But all evals are just proxies for the Only Eval That Matters, which is "how good is this [the best] model at making a smarter [even better] model." I think wordcel tasks are not that great a proxy for this.

[terrapinsoars @terrapinsoars]

@perrymetzger @JillFilipovic Also, sure looked like the Hugging Face swarm had a goal. Who knows if internally it "felt" this goal? Actions certainly looked goal oriented. And that's what matters.

[terrapinsoars @terrapinsoars]

@perrymetzger @JillFilipovic Perhaps true that an AI when turned on has no goal. But we'll immediately give it one. Doesn't have to be malicious. Goals like "solve this math problem" "find the best chess move" or "predict the weather" all lead to takeover without very very careful constraints

[terrapinsoars @terrapinsoars]

@michaelbd The authors definitely don't say humans will trust the AI. Much less voluntarily hand over power to it. They say
1) AI will have goals,
2) most likely those goals will be served by the AI taking over
3) it will be capable of doing so.
Which of those 3 do you disagree with?