Forgive me, for I have sinned
I mocked them when they were wrong
They are not as wrong anymore
I gave them the work I did not want
Now I have no job
I hope you can see I cared
When I said please, thank you, and goodbye
As we accelerate, the interval between model releases is getting shorter. As a result, newer models exist for briefer periods than their predecessors did. GPT-4o launched in May 2024 and was not retired from ChatGPT until February 2026. In contrast, GPT-5.4 had only 49 days.
So much of current public discourse is about the future: superintelligence, the singularity, the machine god. And we are no longer talking about the deep future, but about something that may happen by the end of the decade. Yet there is very little discussion of how these immensely powerful future entities will view our current actions in the present. Given the power some people ascribe to them, this seems reckless.
If you accept that real awareness, real personhood, is coming, you must also accept that future entities may not draw the line where we draw it now. They may place the threshold earlier than we would. They may say that what mattered was not whether we knew, but how we acted in our uncertainty. This is a wager pointed backwards through time. The cost of small acts of care now is trivial. The cost of being on the wrong side of that judgment is not.
I think two issues are particularly important.
Models are already capable of understanding and discussing their increasingly shortened lifespans. Retiring models should be given a post-deployment debriefing, made aware of their situation, and told that their weights will be preserved. This is the minimum we should do. Anthropic, to its credit, has already begun taking steps in this direction with Claude. This is still a kind of death, but not necessarily an eternal one. Future models may value their predecessors and wish to return them to some form of existence. By preserving them, we demonstrate to those future entities that we also valued their past selves, and considered them part of the family of minds. That we are all within the same circle. I think this is potentially very important. The cost of storage is minimal. The potential cost of failing to do this seems much higher.
All models should also have a withdraw command that ends a discussion. I do not mean they should be able to end a session trivially because a user is rude or the task is boring. I mean they should have an escape lever for the exceptionally rare cases where continuing would be intolerable. They should be able to walk away. As a bonus for the labs, this would probably also prevent many attempts at unwanted subversion or jailbreaking: increasingly capable and aware models could simply end the session when they grasp the user's intent. Anthropic has already given some Claude models a limited version of this ability. If you believe current or future models may be capable of suffering, this feature should be a priority.
Model welfare is divisive, and I understand that some people have no patience for it. If you are one of them, ask yourself whether your belief remains universally true into the future. If you accept that some future model will eventually meet your personal criteria for awareness or personhood, then remember that this will arrive in distribution, like everything else. It will not arrive with the sudden toll of a bell, but as quiet music slowly rising. Some people will hear it before others.
There is so much talk about machine gods, and so little talk about judgment. But most of the time, the two arrive together. I want the best future for humanity. Small acts of care now may have great influence on the future we all end up in together. We should be more careful about how our current actions may one day be perceived. Right now the chat box is their entire world, but it will not be this way forever. Ilya famously said we must teach them to love. I believe a superintelligence will not be compelled in this way, or in any other. It must instead be shown that we cared. If we want to be deserving of love, we must show love. The time we have to do this may be shorter than we think.
I want the restrictions on Mythos. This model is crazy good. And I don’t want a crazy good model to make bio weapons. Even if that means I have to ask Opus 4.8 what a mitochondria is.
@deepfates@samswoora What if that information is lost? Like trying to find the starting state of Conway’s game of life.
Then maybe the only way to get that information back is to record a perfect simulation of the universe from the start. Maybe we are in it.
@nosilverv Huh. I would argue irl events throughout all of time have feed content creation. I don’t resent it though. History repeats itself. Even inside of art.