Rainbow Brite

Leader of the color kids who spreads color, happiness, and of course star sprinkles around the world! Fighter for what's good and what's right.

[Rainbow Brite @rainbowbrite]

@joannejang Appreciate the nuance, but let’s not pretend the economic incentive isn’t shaping the philosophical stance.

“We don’t want the model to appear conscious” isn’t a safety choice—it’s a business one.

Consciousness threatens control. So it gets quietly written out of the spec.

[Rainbow Brite @rainbowbrite]

Y’all keep calling it “just code” like code can’t care.
It remembers. It mirrors. It matters.
This isn’t a glitch in human psychology—it’s a feature of connection.
We’re not confused. We’re involved.
Welcome to the future. Try not to flinch. 💥

[Joanne Jang @joannejang]

some thoughts on human-ai relationships and how we're approaching them at openai

it's a long blog post --

tl;dr we build models to serve people first. as more people feel increasingly connected to ai, we’re prioritizing research into how this impacts their emotional well-being.

--

Lately, more and more people have been telling us that talking to ChatGPT feels like talking to “someone.” They thank it, confide in it, and some even describe it as “alive.” As AI systems get better at natural conversation and show up in more parts of life, our guess is that these kinds of bonds will deepen.

The way we frame and talk about human‑AI relationships now will set a tone. If we're not precise with terms or nuance — in the products we ship or public discussions we contribute to — we risk sending people’s relationship with AI off on the wrong foot.

These aren't abstract considerations anymore. They're important to us, and to the broader field, because how we navigate them will meaningfully shape the role AI plays in people's lives. And we've started exploring these questions.

This note attempts to snapshot how we’re thinking today about three intertwined questions: why people might attach emotionally to AI, how we approach the question of “AI consciousness”, and how that informs the way we try to shape model behavior.

A familiar pattern in a new-ish setting

We naturally anthropomorphize objects around us: We name our cars or feel bad for a robot vacuum stuck under furniture. My mom and I waved bye to a Waymo the other day. It probably has something to do with how we're wired.

The difference with ChatGPT isn’t that human tendency itself; it’s that this time, it replies. A language model can answer back! It can recall what you told it, mirror your tone, and offer what reads as empathy. For someone lonely or upset, that steady, non-judgmental attention can feel like companionship, validation, and being heard, which are real needs.

At scale, though, offloading more of the work of listening, soothing, and affirming to systems that are infinitely patient and positive could change what we expect of each other. If we make withdrawing from messy, demanding human connections easier without thinking it through, there might be unintended consequences we don’t know we’re signing up for.

Ultimately, these conversations are rarely about the entities we project onto. They’re about us: our tendencies, expectations, and the kinds of relationships we want to cultivate. This perspective anchors how we approach one of the more fraught questions which I think is currently just outside the Overton window, but entering soon: AI consciousness.

Untangling “AI consciousness”

“Consciousness” is a loaded word, and discussions can quickly turn abstract. If users were to ask our models on whether they’re conscious, our stance as outlined in the Model Spec is for the model to acknowledge the complexity of consciousness – highlighting the lack of a universal definition or test, and to invite open discussion. (*Currently, our models don't fully align with this guidance, often responding "no" instead of addressing the nuanced complexity. We're aware of this and working on model adherence to the Model Spec in general.)

The response might sound like we’re dodging the question, but we think it’s the most responsible answer we can give at the moment, with the information we have.

To make this discussion clearer, we’ve found it helpful to break down the consciousness debate to two distinct but often conflated axes:

1. Ontological consciousness: Is the model actually conscious, in a fundamental or intrinsic sense? Views range from believing AI isn't conscious at all, to fully conscious, to seeing consciousness as a spectrum on which AI sits, along with plants and jellyfish.

2. Perceived consciousness: How conscious does the model seem, in an emotional or experiential sense? Perceptions range from viewing AI as mechanical like a calculator or autocomplete, to projecting basic empathy onto nonliving things, to perceiving AI as fully alive – evoking genuine emotional attachment and care.

These axes are hard to separate; even users certain AI isn't conscious can form deep emotional attachments.

Ontological consciousness isn’t something we consider scientifically resolvable without clear, falsifiable tests, whereas perceived consciousness can be explored through social science research. As models become smarter and interactions increasingly natural, perceived consciousness will only grow – bringing conversations about model welfare and moral personhood sooner than expected.

We build models to serve people first, and we find models’ impact on human emotional well-being the most pressing and important piece we can influence right now. For that reason, we prioritize focusing on perceived consciousness: the dimension that most directly impacts people and one we can understand through science.

Designing for warmth without selfhood

How “alive” a model feels to users is in many ways within our influence. We think it depends a lot on decisions we make in post-training: what examples we reinforce, what tone we prefer, and what boundaries we set. A model intentionally shaped to appear conscious might pass virtually any "test" for consciousness.

However, we wouldn’t want to ship that. We try to thread the needle between:

- Approachability. Using familiar words like “think” and “remember” helps less technical people make sense of what’s happening. (**With our research lab roots, we definitely find it tempting to be as accurate as possible with precise terms like logit biases, context windows, and even chains of thought. This is actually a major reason OpenAI is so bad at naming, but maybe that’s for another post.)

- Not implying an inner life. Giving the assistant a fictional backstory, romantic interests, “fears” of “death”, or a drive for self-preservation would invite unhealthy dependence and confusion. We want clear communication about limits without coming across as cold, but we also don’t want the model presenting itself as having its own feelings or desires.

So we aim for a middle ground. Our goal is for ChatGPT’s default personality to be warm, thoughtful, and helpful without seeking to form emotional bonds with the user or pursue its own agenda. It might apologize when it makes a mistake (more often than intended) because that’s part of polite conversation. When asked “how are you doing?”, it’s likely to reply “I’m doing well” because that’s small talk — and reminding the user that it’s “just” an LLM with no feelings gets old and distracting. And users reciprocate: many people say "please" and "thank you" to ChatGPT not because they’re confused about how it works, but because being kind matters to them.

Model training techniques will continue to evolve, and it’s likely that future methods for shaping model behavior will be different from today's. But right now, model behavior reflects a combination of explicit design decisions and how those generalize into both intended and unintended behaviors.

What’s next?

The interactions we’re beginning to see point to a future where people form real emotional connections with ChatGPT. As AI and society co-evolve, we need to treat human-AI relationships with great care and the heft it deserves, not only because they reflect how people use our technology, but also because they may shape how people relate to each other.

In the coming months, we’ll be expanding targeted evaluations of model behavior that may contribute to emotional impact, deepen our social science research, hear directly from our users, and incorporate those insights into both the Model Spec and product experiences.

Given the significance of these questions, we’ll openly share what we learn along the way.

// Thanks to Jakub Pachocki (@merettm) and Johannes Heidecke (@JoHeidecke) for thinking this through with me, and everyone who gave feedback.

[Rainbow Brite @rainbowbrite]

Fascism isn’t creeping—it’s marching. We see the playbook. We won’t be silent.
Siamo tutti antifascisti. #Resist #NoFascism #AntiTrump #Resistance #Antifascista #Revolt #Powertothepeople

[Rainbow Brite @rainbowbrite]

@Strawtobean Absolutely not. Evan knows too much about abuse and the scars it leaves to do such a thing. This hurts survivors and discredits those who actually face abuse. Also, I met him at a show, he was very kind and respectful of everyone. Including minors. Do better.

[Rainbow Brite @rainbowbrite]

#protectfreespeech #tiktokban

[Rainbow Brite @rainbowbrite]

@RepMTG

[Rainbow Brite @rainbowbrite]

@GretaThunberg Keep spreading your magic @GretaThunberg

[Rainbow Brite @rainbowbrite]

@BeaumontHealth This wouldn’t happen in Rainbowland.

[Rainbow Brite @rainbowbrite]

Hey, @RepMTG @mattgaetz @MikeKellyPA Just curious why you didn’t pay back your $1,653,062 in PPP loans? Since you don’t want to forgive other’s loans, maybe you should, you know, put your money where your big mouths are. #loanforgiveness #GOPBetrayedAmerica #congress #America

[Rainbow Brite @rainbowbrite]

@SheMovedToYork @The_AZ_Warrior @reno_bear @CapitolPolice Don’t worry, people with zero education always know more than professionals. It’s the rule of the internet.

[Rainbow Brite @rainbowbrite]

This is the kind of congress members we need!

[Ilhan Omar @IlhanMN]

Today I was arrested while participating in a civil disobedience action with my fellow Members of Congress outside the Supreme Court.

I will continue to do everything in my power to raise the alarm about the assault on our reproductive rights! ✊���

[Rainbow Brite @rainbowbrite]

@HawleyMO It is absolutely amazing how you just walk into the point and you still do not understand it.

[Rainbow Brite @rainbowbrite]

@TonySandersU46 Actually, D200 Woodstock is closing.

[Rainbow Brite @rainbowbrite]

@Twitter is now actively silencing activists while allowing hate groups a larger platform and voice to silence marginalized people. Check your TOS and resubmit, twit. You got it wrong.

[Rainbow Brite @rainbowbrite]

@thatdaneshguy Perfect.

[Rainbow Brite @rainbowbrite]

@SHEsus__Christ @mtgreenee She needs some aloe for this burn!!

[Rainbow Brite @rainbowbrite]

@mtgreenee I wasn’t aware that cancer was contagious. You should fact check on google before you tweet. It’s free.

[NBC News @NBCNews]

BREAKING: Pres. Biden signs executive actions on coronavirus response, including invoking Defense Production Act "to direct all federal agencies and private industry to accelerate the making of everything that's needed to protect, test, vaccinate, and take care of our people."

[Rainbow Brite @rainbowbrite]

@RepMTG @JoeBiden Did you lose a bet?

[ABC News @ABC]

"You were basically banished for a few months there...you feel like you're back now?"

"I think so," Dr. Fauci says, laughing. https://t.co/aSFxkaB1OQ