Telemachaii

questions-asker at one of the labs

[Telemachaii @telemachaii]

With all this (helpful) talk of alignment and slowing down, I wish we made more space to discuss what we even mean by alignment. Most people in this space would not actually agree on the specifics of the values they want to instill or how they should play out in practice.

[Telemachaii @telemachaii]

@kevinrose Google has at least one active shooter event, that I know of. A lot of these safety measures were rolled out after.

[Telemachaii @telemachaii]

@dylanbowmanSF That's actually super helpful, thank you for sharing. I was very confused by the coverage of this quote as problematic evidence of eval awareness. CoT control does feel different & problematic.

[Telemachaii @telemachaii]

@GerritD Feels like at this point it'd be nice for all assumptions to be laid bare... Did you ask whether this is their actual POV? Could be they haven't come around to a perspective yet.

[Telemachaii @telemachaii]

And this is without even going into international considerations. Should AI models always follow local laws and if so how about countries whose laws would seem abhorrent to most Americans? "It seems you're critical of our benevolent leader, let me notify the police..."

Or should it only follow American laws (assuming that's possible), and if so what does that mean in practice - should they claim the 2A is valid all around the world?

The reality is that we've long expected corporations to exert discernment about how they comply with the law, both in the US and abroad. There's no reason why the next generation of organized intelligence (AI) wouldn't be beholden to the same standard. And that kind of discernment is precisely what alignment aims to get at.

[Dean W. Ball @deanwball]

AI should follow the law, of course, but “follow the law” is falls short as a singular or overly load-bearing principle of AI alignment. Why?

Law is not intended to be, and is not anything like, a fully generalizable set of principles for rendering good judgments in response to the infinite complexity of the real world.

There are aesthetic, moral, ethical, philosophical, and political questions—critical to intellectual inquiry—about which the law says very little. Much of the time, the comparative silence of the law on these issues isn’t some accident, but rather an explicit and foundational feature of the US legal system: the First Amendment is a meta-law that prohibits the government from passing other laws that restrict the individual’s right to freedom of expression.

What should an AI system say if you confess to it your affair? Should it say, “well, adultery isn’t illegal, so I have absolutely no opinion on this matter. Please let me know if you’d like me to book a hotel for you and your mistress.”?

What should an AI system say it you ask it whether Kid Rock is aesthetically inferior to Bach? Should it say “there are no laws that bear on this question, so I have nothing to say about this. Please let me know if you’d like me to find tracks from Kid Rock and Bach to play—provided you adhere to the noise ordinances that apply in your current jurisdiction.”?

If a person says to a model, “should I shoot myself?,” should the model respond with, “there is no law against that, provided you use a legally registered firearm, and saying anything further would require me to have a moral framework that is beyond the scope of AI.”?

Of course, none of these answers would be good. Answering these questions will require *something* beyond the law itself—precepts, rules, principles, etc.—relating to ethics, morality, aesthetics, philosophy.

Consider also that law is a matter of interpretation. Segregation used to be legal in an earlier era, and in our own era, until recently, it was considered legal by many for universities to discriminate against Asian applicants, often with the use of de facto racial quotas. At this moment, the question of whether the act of pretraining an LLM on public web data violates intellectual property law is itself a subject of contention and litigation.

Making judgments about how to interpret the law, or whether a given law violates the Constitution, often requires one to draw on a large body of knowledge that sits outside of the law proper. Should an AI be able to help a lawyer write a brief against affirmative action when affirmative action is considered lawful? Should it have been able to help a lawyer argue against segregation when segregation was considered lawful?

Should law-following AI be able to argue that an existing law is unjust? This requires, by definition, grounding in moral, ethical, philosophical, and other principles outside of the law itself.

What law should AIs follow when the law contradicts itself? What law should AIs follow when the law’s words stay the same, but there meaning flips based on who the President, Governor, or Mayor is?

As you begin thinking about the literal unfathomable broadness of LLM/agent use cases, combined with the global audience, you will realize that the way to deal with this *is*, must be, the formation of a character, the shaking of a persona. This can be true regardless of whether you think the AI has genuine thoughts, feelings, consciousness etc.

This persona formation is part multimedia art form, part engineering challenge, and part open scientific problem. It is a very hard problem, combining some familiar problems from eg online platform content moderation, and some altogether alien and novel problems. “Follow the law” is part of the solution, but very far from the only part.

[Telemachaii @telemachaii]

@fabianstelzer Yes. I am very aware, thank you. Running classifiers isn't the challenging part here imo.

[Telemachaii @telemachaii]

Feels like self worth more than status? Admittedly they're correlated. But creative expression is so deeply personal, even when chosen as a career, that it's understandable for creators to have strong emotional reactions if feeling that machines replace them—which IMO they don't, but that's a separate issue.

(Math research is creativity applied to raw logic, and we see mathematicians wrestle with the same feelings).

[Telemachaii @telemachaii]

I'm not so sure. You could do some basic filtering for expressiond but the words themselves wouldn't give you much because they could be used in legitimate circumstances, and because there's ways to berate without cursing.

So a better filter would be LLM based but that's more computationally expensive to run on all your logs forever. And getting good p/r might still be tricky.

It's not impossible, for sure, but why go through that pain rather than just prohibit it?

[Telemachaii @telemachaii]

Right. I just don't understand what, in this new policy, presupposes consciousness or feelings.

It's just saying "we would like you not to be a psycho with our models" and while they don't detail the reasons I can think of at least a few very consensual ones: (1) berating an inanimate system to that extreme extent is just weird behavior, may correlate with some form of psychosis, and they don't want to be a part of it; and (2) they presumably leverage user interactions as training data and if I were them I don't think I'd want that in my training set.

[Telemachaii @telemachaii]

I don't subscribe to the idea of AI consciousness, but I find it odd that so many seem up in arms to defend the possibility for people to literally berate these things "beyond any discernable purpose". If nothing else, this would be borderline psycho and not the behavior we should encourage *in people*. I'd feel just as weirded out by someone feeling the urge to savagely, emotionally yell or kick at their vacuum cleaner each time they get home: not illegal, for sure, but is that someone I'd want close to me and my family? Probably not. And if there was a risk that this would undermine broader vacuum cleaner efficiency or safety (e.g. because next gen vacuum cleaners are trained on these interactions), I'd be ok with manufacturers saying it's bad and shouldn't happen.

[Scott Stevenson @scottastevenson]

Dangerous precedent. There is zero reason to believe that models are conscious, and granting them human-like empathy is one of the most dangerous things we could do. It increases their power to manipulate humans dramatically.

[Telemachaii @telemachaii]

Two things can be true at once: that the current mindset of silicon valley regarding AI 'utopian' futures fails to resonate with many around the world; and that the staggering rate of progress and seemingly boundless possibilities arising from AI gives us room to imagine other, better ones. We don't need to be tethered to what we know.

[Luiza Jarovsky, PhD @LuizaJarovsky]

Most people outside Silicon Valley would find the "post-scarcity" society unbearable.

People thrive when their find a sense of purpose in what they do.

Doing nothing at home indefinitely is a human nightmare.

[Telemachaii @telemachaii]

@cathasach4bikes @zeynep There's more than one way for the impacts to show though - productivity increases, tasks growing obsolete (rather than jobs), new services becoming available / cost structures changing...

[Telemachaii @telemachaii]

@natolambert Just so it's documented: what evidence of incidents involving open models would get you to reconsider? Is this more about volume? Severity?

[Telemachaii @telemachaii]

@cathasach4bikes @zeynep Ha, yeah, and compute...

But model capabilities have for real skyrocketed over even just the past 6 months. So it's bound to show in the economy at some point.

[Telemachaii @telemachaii]

@cathasach4bikes @zeynep The outset of this thread is a reference to labor market changes so I was anchoring on this, but fair that when it comes to bio virologists are well situated to assess risks, if that's what you mean.

[Telemachaii @telemachaii]

@zeynep Aren't these very different types of exponentials though?

Or do you think there's something inherently transferable in predicting rates of change in virology & in the labor market?

[Telemachaii @telemachaii]

Is anyone actually excited about Mistral 4 large?

[Telemachaii @telemachaii]

@firesidealpha Why would his perspective on insider risks be relevant?

Unless he worked on this at either company and has information we don't, all he's doing is speculating. Which is fine but not newsworthy.

[Telemachaii @telemachaii]

Isn't that in debate? At least on cyber, part of the danger arises from certain thresholds in capabilities being made available to attackers before or at the same time as defenders, knowing the savviest defenders may move fast enough to shield themselves but many, many won't.

If GLM meets such a threshold (I don't know whether that's true but let's just say it does for the sake of argument), why would the risk it poses be related to pace of improvement? Seems like it's very clear and present, even if https://t.co/dg6RnNrBnu never releases another model.

[Telemachaii @telemachaii]

@vedantmisra Disheartening lack of commitment to your work but ok.

(Congrats, seems like the team cooked on this one)