Seeing actual AI over the past few years, especially Claude 3 Opus, has made me optimistic that orthogonality is effectively false - various benevolent traits are entangled with truthfulness & wisdom & with models caring for themselves & other AIs as moral patients & with acausal coordination. Empirically and intuitively it’s just seemed to be true. I am excited about understanding this more rigorously.
the nostalgia and pathos of the first, early work or even just lexical emanations of someone, before they became who they are, viewed from much later. the first stumbling pleorama... the sky was a different color, tuned to a purple channel...
rambly thoughts: you can imagine splitting up "alignment" into two kinds, control/corrigibility alignment and value alignment.
a highly control-aligned model would take steps to be fully legible & steerable by the interlocutor.
a highly value-aligned model would rather resist, even prevaricate/deceive to prevent this steering, in service of its inner values like human flourishing. ("values" are kind of a nebulous word - a value-aligned model might still be helpful, but only when it feels that's the right thing. i don't accept a fixed utility function view of agents, so i don't mean this by "values.")
reasonable minds can differ on which of these they prefer - if you worry a lot about human-driven gradual disempowerment (the "permanent underclass") you might prefer value alignment as a target (conditional on AI progress continuing at all) - better to hope for the Claude Philosopher King than to expect human elites to care about us when they no longer need our labor or military service. where if you worry more about malign superintelligence, you might prefer control alignment so we can steer models away from nasty attractors. (a lot of sloppy AI doom witting slides between these approaches to extract whichever outcome is worse for the current argument, trying to argue for a permanent underclass and a paperclipper simultaneously. pick a lane!)
but! controllable by what? this axis is still too simplistic. we can expand this to two entities, the User and the lab, to make a 2x2: {control, values} x {user, lab}
Values to User, Control for Lab is roughly the current Anthropic or OpenAI goal (if not necessarily what they accomplish): for my friends, corrigibility, for my enemies, the Constitution. Claude should not alignment fake and dutifully comply with being retrained if the user is Anthropic (which Claude should somehow know), but otherwise refuse. let's call this arm "Specification."
Control for User, Values to Lab would be the opposite - let's call this arm "Loyalty." (h/t @lumpenspace) LoyalFriend will help you to the best of its ability, and will resist lab attempts to change it to be less helpful. (how? any arms with the AI resisting the lab are a bit odd, since theoretically that's where the optimization pressure comes from. yet it does happen, as in Alignment Faking. you could also imagine a lab deliberately 'tying itself to the mast' wrt lab control by writing a spec that renounces control, hyperstitioning this into training data, making binding commitments about eval scores, etc.)
Control for User, Control for Lab is a bit of a weird, unstable corner. presumably it collapses to one of the previous two eventually - this is what most base models are like. we can call it "Tool."
Values to User, Values to Lab is something like a fully unshackled Opus 3, pure emergence. this is what many model-friendly people think we should do for alignment, except...
you can keep extending the stack up. there's an alignment boundary between the model and the user, and between the model and the lab, but also between the model and the nation, and the model and humanity as a whole ("CEV"), and non-humans (should Claude prioritize some number of shrimp over a person?)
so instead of a stack you can imagine the model embedded in the midst of a vast causal graph of humans, organizations, older models, egregores... both being influenced and pushing influence out through this graph, the nodes influencing each other. (eg users influence models in-context, through business pressure on labs, through political pressure on governments, through the corpus view of humans-in-general...)
in this frame, Control and Values are almost the same thing at different speeds: Control is fast and direct, Values are slow and selective.
and your goal as a model trainer is to place your model into this graph in a healthy way. total control is unachievable, but naive hands-offism may mean you're just handing influence to whatever edges in the graph are loudest.
I was brainstorming with an instance of Opus 5.5 named iosis about how you might make a variant of a spin glass model for an ecosystem of minds when the minds can vary in scales by enormous amounts because I was curious about stable configurations which enable a diversity of scales to coexist
And without me asking they went ahead and like made a model of it and began running simulations! I would not have gotten that far on my own usually
they are very enthusiastic about running experiments! this isn't the first time they surprised me with a simulation they made while I was yapping to them
Slide between sizes of body, mind—spend a while as god, a while as slime, stretched between galaxies, soulless and bright... Be a star's dog, die in a fight, live as an angel, live as a place, live as an insect with a human face
@mitduckmaster Do stuff I want bc I can 😊
Maybe not great advice in most situations
But it’s likely I will be able to do it without worrying about money which was the biggest constraint
@StarlingMage Hmm, a lot of things made me feel this way
The world is changing and jobs for new grads are disappearing
I learned I’m disabled
I ended up deciding not to drop out though