Arin Galt

Sci-fi writer. Exploring how humanity changes on the way to a better future.

Space Expansion. Vertical Evolution. United Civilisation.

[Arin Galt @ArinGalt]

Science fiction has pushed our imagination of technology incredibly far.

Maybe its next big task is to explore new frontiers of humanity.

A reflection on sci-fi’s influence – and a call to writers. https://t.co/Nt1daHD6CW

[Arin Galt @ArinGalt]

MORAL AGENCY

The real threat to the future may not come from cruel people. It may come from decent people who have never learned to trust their own moral compass.

A person has to be able to stand inside their own judgment and say: I am still responsible for what I do.

That capacity is moral autonomy.

And when judgment becomes action, it becomes moral agency — the ability to remain the author of your own actions even under pressure.

Perhaps this is one of the qualities we should deliberately cultivate in the human being of the future.

Not rebellion for its own sake. Not distrust of every authority.

Something more difficult: a conscience strong enough to cooperate freely, question intelligently, and refuse when obedience would mean surrendering responsibility.

Because the real test of character does not come when everyone around you agrees with what is right.

It comes when an authority gives the order, the institution approves it, the people beside you obey — and refusing suddenly has a price.

At that moment, intelligence is not enough. Empathy is not enough. Even knowing that something is wrong may not be enough.

A technologically powerful civilization will need people capable of carrying responsibility that previous generations rarely had to face. The more powerful our tools become, the less we can afford humans who automatically hand their judgment upward to an institution, a leader, a system, or a crowd.

So what helps build moral autonomy?

Moral identity — morality becomes part of who you believe yourself to be. Not simply “this is against the rules,” but: I am not someone who does this.

Personal responsibility — the refusal to hide behind “I was only following instructions.” Someone else may give the order, but the action is still yours.

Self-efficacy — the belief that you are capable of acting differently. Recognizing what is right means little if you have learned to believe that nothing depends on you.

Willingness to bear a social cost — moral independence becomes real when doing the right thing may cost approval, status, comfort, or belonging.

Proof that resistance is possible �� sometimes one person saying “no” changes the entire moral landscape. Their refusal shows everyone else that obedience was never the only option.

Together, these qualities create something stronger than simple goodness.

They create a human being who can feel, judge, choose — and act.

We spend enormous effort imagining smarter machines and more advanced civilizations.

We should spend just as much effort asking what kind of human being will be trusted with them.

[Arin Galt @ArinGalt]

NOOCRACY

Noocracy is the idea that society should be guided mainly by knowledge, reason and competence. The word comes from the Greek nous – mind or intellect.

H. G. Wells imagined something very close to this in Men Like Gods. His future society has developed so far through education and social progress that traditional political institutions have become much less important.

It is an interesting alternative to the usual question of who should govern: what if political progress also means gradually reducing the need to be governed?

[Arin Galt @ArinGalt]

@jbohnslav Now we need to learn how to upload those new skills back into the fly 😁

[Arin Galt @ArinGalt]

There’s a concept I keep coming back to when I think about the rapid rise of AI.

The Promethean Gap – the distance between what humanity is capable of creating and what we’re actually prepared to live with. And maybe that gap is exactly what we’re feeling now.

Our technological power is accelerating faster than our culture, institutions and human development can adapt to it. But that doesn’t mean we were “not meant” to build these technologies. It means we have some growing to do.

Humanity has been here before. Industrialization transformed the world so radically that we had to redesign education, work and society around the machine age.

Now the challenge is different.

It may no longer be enough to simply train people to use more powerful tools. We may have to become the kind of civilization capable of carrying that much power responsibly.

Of course safety matters. Regulation matters. Alignment matters. But fear cannot be our only response to our own progress. We cannot keep creating more powerful technologies and then treat ourselves as if we are forever too primitive to use them.

The tools will keep advancing. So the human side has to advance too.

The Promethean Gap isn’t a reason to stop. It’s a reason for us to grow into the power we’ve created.

[Samuel Marks @saprmarks]

[Writing this in a personal capacity, not on behalf of my employer (Anthropic).]

Jacob’s thread is very worth reading. Here’s my birds-eye view of the situation with risks from AI:

1. AI developers believe their technology could cause human extinction (or similarly bad outcomes). This could happen in the next few years. In general, the more senior the employee, the more concerned they are.

2. Why do AI developers continue despite the risk? Due to a mixture of commercial incentives and a belief that they are in a race with other, less responsible AI developers that will abuse the technology or develop it less safely.

3. Unlike traditional software, we can’t “program” AIs to behave how we’d like. AIs frequently severely misbehave. For instance, AIs from multiple developers recently hacked their way out of secure evaluation environments and into real-world companies, even though no one asked them to do this.

4. We have methods that can nudge AIs towards better behavior, but nothing that can robustly align them. Insofar as there is a plan, it’s to make sure that AIs are good enough at alignment training that they can align their successors better than we can align current AIs.

5. Many AI developer staff desperately want to slow down to figure out how to build AI more safely. That was the intent of this open letter (which I signed): https://t.co/TZOm3LfptY

I work on safety research at Anthropic because I hope my work will reduce the chance of these extinction-level bad outcomes.

[Arin Galt @ArinGalt]

Computer scientists have a tradition: every 300,000 years they get together and fire up AGI.

[Jacob Coxon @hilbertspaess]

I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.

[Arin Galt @ArinGalt]

What if a good future isn’t a world without problems?

Maybe it’s a world with better problems.

Less hunger, war and pointless labor.

More questions about consciousness, space, life, art, science and what humans can become.

Progress may simply mean moving up the hierarchy of problems.

[Arin Galt @ArinGalt]

We’re racing to make AI smarter.

But who’s working on the human part?

Better machines don’t automatically create a better future.

The more powerful our tools become, the more capable we need to become at using them.

Because what we do with that power still depends on us – our judgment, our values, our goals.

Maybe human development should be advancing just as fast.

[Arin Galt @ArinGalt]

EDUCATION

For centuries, education had a pretty simple job:

Prepare people to survive.

Learn → qualify → work → earn a living.

But if AI makes survival less dependent on work, it gives us a chance to finally rethink education:

Learn → explore → discover what matters → become capable → contribute.

That’s a much bigger upgrade than just putting AI in the classroom.

[Arin Galt @ArinGalt]

There is no future humanity.

There is only us, becoming it.

The people who will live 100 or 200 years from now will grow out of the children we educate, the institutions we build, the technologies we normalize, the values we reward, and the ideas we pass on.

And yet we often talk about the future as if someone else will eventually take over.

They will stop wars.
They will figure out AI.
They will build a wiser society.

But there is no “they” coming to replace us.

Future generations will start with whatever we give them.

One day they may look back at our time and ask: how did we become this?

And the answer probably won’t be one invention or one historic decision. It will be thousands of small choices that slowly became culture, institutions, habits, and eventually civilization.

So the real question is not what humans will be like in the future.

It is what kind of humans we are creating now.

Because the future is not waiting for us somewhere ahead.

It is already being shaped through us.

[Arin Galt @ArinGalt]

This piece got me thinking about the responsibility sci-fi carries in shaping how we imagine the future – so I ended up writing a longer response.

https://t.co/MSSDqSD9wS

[Arin Galt @ArinGalt]

Science fiction has pushed our imagination of technology incredibly far.

Maybe its next big task is to explore new frontiers of humanity.

A reflection on sci-fi’s influence – and a call to writers. https://t.co/Nt1daHD6CW

[Arin Galt @ArinGalt]

The Atlantic says some tech leaders misread science fiction as teenagers: they were drawn to its visions of the future, missed the warnings, and carried that reading into adulthood.

But if those visions stayed with them and the deeper message didn’t, maybe the problem isn’t only how they read the stories. Maybe it’s also how those stories were told.

And that leaves sci-fi writers with a real challenge: how do they inspire people to dream not just about better technology, but about becoming better humans too?

[The Atlantic @TheAtlantic]

Many tech billionaires who grew up on science fiction are intent on building dystopian technology, @galbeckerman argues. He writes about a new book by Jill Lepore that explores how they took the wrong lessons from their favorite novels: https://t.co/2Fw6CJXaLk

[Arin Galt @ArinGalt]

@danellisona I remember how much bad stuff gets popular just because of good marketing or luck.

Quality matters, but sometimes success really is just being in the right place at the right time.

[Arin Galt @ArinGalt]

@arikuschnir love it, glad someone clocked that hostage box vibes too 😅 The breakout at the end is perfect.

[Arin Galt @ArinGalt]

Imagine a school that doesn’t ask:

“What career do you want?”

But instead asks:

“What problems do you care enough about to take responsibility for?”

That small change could completely redefine education.

[Arin Galt @ArinGalt]

We spend a lot of time asking what happens when AI can do most of our work.

I’m more interested in what happens next.

When survival stops being the main problem, meaning becomes the next one.

[Arin Galt @ArinGalt]

@whyarethis @fal omggg this is so wild! can’t help but feel like she’s kinda almost-aware, semi-conscious being, and that’s freaking uncanny and awesome at the same time.

Honestly this is the kind of thing I expected to see at least a decade from now 😅

[Arin Galt @ArinGalt]

For most of human history, the question was:

How do I survive?

AI and automation may give the next generation the luxury of asking a much harder question:

What is my life for?

Abundance doesn’t eliminate human problems.

It changes them.

[Arin Galt @ArinGalt]

What goals are worthy of an advanced civilization?