Ximena Summers

[Ximena Summers @XS1972]

@mrxuniverseus 5

[Ximena Summers @XS1972]

@mrxuniverseus 93

[Ximena Summers @XS1972]

@AiwithDharmik 93

[Ximena Summers @XS1972]

@dwarkesh_sp “It took humanity no less than 300,000 years to learn how to behave.”

[Ximena Summers @XS1972]

@dwarkesh_sp And yet, everything we actually wanted to protect could be lost.

The ultimate challenge of AI alignment is not just teaching machines what to achieve. It is ensuring that some things are never sacrificed for the sake of achieving them.

What AI should never be able to do.

[Ximena Summers @XS1972]

@dwarkesh_sp An AI could pursue its assigned objective with extraordinary efficiency and still produce a catastrophic outcome.

The goal might be achieved. The system might even consider its actions a success.

[Ximena Summers @XS1972]

@dwarkesh_sp Taken to its extreme, that logic could turn human beings into variables in an optimization problem—and treat some lives as less valuable than others.

Preserving humanity is not the same as preserving every human being’s dignity.

[Ximena Summers @XS1972]

@dwarkesh_sp Now replace that objective with:

“Preserve the human species.”

Imagine a global shortage of drinking water

Should an AI prioritize children and young people because they have more potential years of life ahead? Should it prioritize people with a higher probability of survival?