@Mihonarium@allTheYud (And tbc not saying that you were saying that personal comfort threshold would be sufficient for the field going for it! No strawman intended)
I can’t say I never get there (also my own personal comfort level on that is nowhere near sufficient under my own ethics), but like just trying to follow it all as the man in the street man I’m not seeing things that make me think the field is on track for that.
I feel like that world has them anticipating and preventing the whole sycophancy/psychosis thing, knowing what capabilities they’ll have in advanced based on the how/what they train it with, not having all the jailbreaks.
Also probably people understanding minds well enough in general that some methods can reliably can tell you a lot of details about someone’s path through life from a brain scan, probably just flat out curing mental illness, and I’d guess people also being able to do bad and scary things with that level of science of mind
@Mihonarium@allTheYud I mean you probably do know more of the history of that than me if you know it at all, but was blowing up zero ever on the table?
@Mihonarium@allTheYud How do you know there wasn’t that One Totally Not Fair thing you didn’t have a way of knowing about though? Seems like people have blown up a lot of rockets despite having a lot of incentive to blow up as few as possible
Goodhart, essentially. The emotions have a cause, and trying to avoid the emotion is trying to avoid doing whatever one might do w/r/t the cause, that might cause a different emotion to be felt. If I come to a river and am trying to avoid all thoughts of how wide the river is, that probably rules out me interacting with the parts of the world such as timber, canvas, books on building boats, that mediate the river’s relationship with me such that it loses its negative valence.
@allTheYud Does people’s cope getting crazy enough precede them not coping? I feel like contortions getting unsustainable and then being shed is a thing sometimes