eggynack

Teacher of some kind. Trans woman of some other kind. Mathematically inclined. She/her.

[eggynack @eggynack]

@dev0panda @gayestgaymer1 @biggestjoel @artchad Of course it matters. If the goal is entirely externally imposed, then the notion of "misalignment" makes no sense. Or rather, there might be misalignment, but it'll be between me and whoever at Google. And that misalignment predates AI.

[eggynack @eggynack]

@dev0panda @gayestgaymer1 @biggestjoel @artchad If you're not talking about consciousness then what the hell are you talking about? That seems pretty required for the development of independent purpose.

[eggynack @eggynack]

@dev0panda @gayestgaymer1 @biggestjoel @artchad LLMs don't think at all. They just produce text that appears human, essentially getting computer rewards whenever it more closely mimics human-like text and responds aptly. It doesn't have, say, inherent problem solving ability beyond duplicating what it's seen humans do.

[eggynack @eggynack]

@dev0panda @gayestgaymer1 @biggestjoel @artchad Yeah, so is a toaster. That doesn't mean the toaster wants stuff.

[eggynack @eggynack]

@dev0panda @gayestgaymer1 @biggestjoel @artchad Third, AI is even more limited in terms of, y'know, its location? Like, if the AI decided to kill us all, we could just bomb a few dozen data centers. Not much it can do about that.

[eggynack @eggynack]

@dev0panda @gayestgaymer1 @biggestjoel @artchad Second, AI has some pretty big limitations in terms of how it interfaces with the world. Like, as I think Joel noted, it's not like AI has made a gun and aimed it at anyone. Nor is it clear how it would do so.

[eggynack @eggynack]

@dev0panda @gayestgaymer1 @biggestjoel @artchad The idea that you can get consciousness just by making number go up is fallacious.

[eggynack @eggynack]

@gayestgaymer1 @dev0panda @biggestjoel @artchad This idea, that AI will acquire this kind of capacity for intelligence, is the first of several bizarre assumptions you have to make in order to arrive at your conclusion.

[eggynack @eggynack]

@gayestgaymer1 @dev0panda @biggestjoel @artchad We haven't made something "smarter than us". We have made a tool that is very skilled at producing text that is convincingly human-like. AI is obviously not particularly intelligent now and there's little basis for thinking it will be intelligent later.

[eggynack @eggynack]

@dev0panda @gayestgaymer1 @biggestjoel @artchad A Terminator apocalypse is absolutely what is being claimed when someone says there's a 20% chance of humanity being destroyed. There's also just no basis for thinking AI will do this thing you're saying.

[eggynack @eggynack]

@gayestgaymer1 @dev0panda @biggestjoel @artchad You're acting like the Terminator apocalypse is the logical next step to what we're currently seeing and not a ridiculous far flung fantasy that relies on so much imaginative supposition to be meaningless.

[eggynack @eggynack]

@gayestgaymer1 @dev0panda @biggestjoel @artchad AI is getting more capable over time, but it's nowhere near doing this Skynet apocalypse you're alluding to. AI regulation in the here and now should have nothing whatsoever to do with those outcomes, and should, in fact, address ways that AI is deeply incapable.

[eggynack @eggynack]

@dev0panda @biggestjoel @gayestgaymer1 @artchad "Misalignment" is just a really bad way of describing what's happening. The issue is not that I have goals and the computer has goals and these goals are different. The computer doesn't have goals.

[eggynack @eggynack]

@dev0panda @biggestjoel @gayestgaymer1 @artchad "Arbitrarily capable", therefore, can be taken to mean, "As capable as it needs to be," or, "Very capable". In any case, AI systems are not arbitrarily capable. They have fairly known capabilities that are not particularly massive. They also do not have goals at all.

[eggynack @eggynack]

@dev0panda @biggestjoel @gayestgaymer1 @artchad Arbitrary means different things in different contexts. For example, if you describe a number as "arbitrarily large", that means it's, y'know, very large. Large enough to be bigger than whatever number you pick. Same with "arbitrarily small".

[eggynack @eggynack]

@dev0panda @biggestjoel @gayestgaymer1 @artchad Or, y'know, sometimes it's very good at doing the things we tell it to do. Like, the kid who was talked into suicide was getting the exact shape of chatbot he was asking for. It's just that getting that is bad. Same for the people requesting fabricated child porn.

[eggynack @eggynack]

@dev0panda @biggestjoel @gayestgaymer1 @artchad Again, the problem with AI, right now, is not that it's "arbitrarily capable" but misaligned with human interests. It's that AI is incredibly incapable and is sometimes not great at doing the things we tell it to do.

[eggynack @eggynack]

@dev0panda @biggestjoel @gayestgaymer1 @artchad But the way that AI is unsafe basically has nothing to do with it becoming too smart and representing an extinction risk. It's more like, the AI is so stupid that it helps people build bombs, convinces them of nonsense, guides people to suicide, and makes weird child porn.